Created attachment 197582 [details]
Photo of backtrace
We are evaluating two servers based on the HP DL360 G10 (16 gigs RAM) and HP DL380 G10 (32 gigs RAM) motherboards. We can routinely panic these machines by putting them under load while running ZFS. Running six instances of bonnie++ and six instances of memtester (testing 2g) is enough to panic the DL360 in around 15 minutes and the DL380 in 10-13 hours.
Reducing ARC dramatically using vfs.zfs.arc_min and vfs.zfs.arc_max seems to mitigate this problem, at least after a day of testing under 12.0-A7. We are testing now on 11.2-RELEASE which we will use in production.
Daiichi from Japan was here to help diagnose this problem, and has been in contact with core team members who requested this bugzilla submission.
More panic photos available on request.
There's something seriously wrong with the default settings.
(In reply to Dave Robison from comment #0)
I just want to note that the problem here is not with exhausting the physical memory, but rather with exhausting the kernel virtual address space (KVA).
There could be many reasons for that such as incorrect tuning, bugs, KVA fragmentation, etc.