Bug 246216

Summary: ZFS sometimes causes a zio_execute()-related stack overflow panic not long after boot
Product: Base System Reporter: sigsys
Component: kernAssignee: freebsd-fs (Nobody) <fs>
Status: New ---    
Severity: Affects Only Me Keywords: crash
Priority: ---    
Version: 12.1-STABLE   
Hardware: Any   
OS: Any   

Description sigsys 2020-05-05 08:22:56 UTC
After updating a 12-STABLE machine from r358905 to r360653, I sometimes get panics not long after boot.  Seems like it might be happening about half of the time.  The system seems stable (so far) if it doesn't crash not long after boot.  The problem might have been there before that upate but I just didn't notice it because I got lucky.

[59] 
[59] Fatal double fault
[59] rip 0xffffffff825bbbd9 rsp 0xfffffe00d6db2f60 rbp 0xfffffe00d6db3450
[59] rax 0xfffff8006c998000 rdx 0xffffffff81b1b380 rbx 0xfffff80b2a0d2418
[59] rcx 0x3b rsi 0xfffff8006c998740 rdi 0x1ed5
[59] r8 0xfffff80bd6355668 r9 0 r10 0xfffff80bd6355250
[59] r11 0 r12 0xfffff8006c80b000 r13 0xfffff8006c9985b8
[59] r14 0xfffffe00c33fd000 r15 0xfffff8006c9985b8 rflags 0x10292
[59] cs 0x20 ss 0x28 ds 0x3b es 0x3b fs 0x13 gs 0x1b
[59] fsbase 0x8006828d0 gsbase 0xffffffff82a19000 kgsbase 0
[59] cpuid = 9; apic id = 09
[59] panic: double fault
[59] cpuid = 9
[59] time = 1588664508
[59] KDB: stack backtrace:
[59] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00041fcdc0
[59] vpanic() at vpanic+0x17b/frame 0xfffffe00041fce10
[59] panic() at panic+0x43/frame 0xfffffe00041fce70
[59] dblfault_handler() at dblfault_handler+0x1d1/frame 0xfffffe00041fcf30
[59] Xdblfault() at Xdblfault+0xd2/frame 0xfffffe00041fcf30
[59] --- trap 0x17, rip = 0xffffffff825bbbd9, rsp = 0xfffffe00d6db2f60, rbp = 0xfffffe00d6db3450 ---
[59] vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x29/frame 0xfffffe00d6db3450
[59] vdev_queue_io_done() at vdev_queue_io_done+0xb8/frame 0xfffffe00d6db34a0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db34e0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3530
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db3590
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db35e0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db3630
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db3670
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db36c0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db3720
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3770
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db37c0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db3800
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3850
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db38b0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3900
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db3950
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db3990
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db39e0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db3a40
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3a90
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db3ae0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db3b20
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3b70
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db3bd0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3c20
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db3c70
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db3cb0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3d00
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db3d60
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3db0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db3e00
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db3e40
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3e90
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db3ef0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db3f40
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db3f90
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db3fd0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4020
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db4080
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db40d0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db4120
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4160
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db41b0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db4210
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4260
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db42b0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db42f0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4340
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db43a0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db43f0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db4440
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4480
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db44d0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db4530
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4580
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db45d0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4610
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4660
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db46c0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4710
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db4760
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db47a0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db47f0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db4850
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db48a0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db48f0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4930
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4980
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db49e0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4a30
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db4a80
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4ac0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4b10
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db4b70
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4bc0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db4c10
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4c50
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4ca0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db4d00
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4d50
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db4da0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4de0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4e30
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db4e90
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4ee0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db4f30
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db4f70
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db4fc0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5020
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5070
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db50c0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5100
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5150
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db51b0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5200
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5250
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5290
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db52e0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5340
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5390
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db53e0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5420
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5470
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db54d0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5520
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5570
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db55b0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5600
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5660
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db56b0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5700
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5740
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5790
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db57f0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5840
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5890
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db58d0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5920
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5980
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db59d0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5a20
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5a60
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5ab0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5b10
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5b60
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5bb0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5bf0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5c40
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5ca0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5cf0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5d40
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5d80
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5dd0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5e30
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5e80
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db5ed0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db5f10
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db5f60
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db5fc0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6010
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db6060
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db60a0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db60f0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6150
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db61a0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db61f0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db6230
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6280
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db62e0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6330
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db6380
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db63c0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6410
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6470
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db64c0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db6510
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db6550
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db65a0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6600
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6650
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db66a0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db66e0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6730
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6790
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db67e0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db6830
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db6870
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db68c0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6920
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6970
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db69c0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db6a00
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6a50
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6ab0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6b00
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db6b50
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db6b90
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6be0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6c40
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6c90
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db6ce0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db6d20
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6d70
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6dd0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6e20
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db6e70
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db6eb0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6f00
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db6f60
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db6fb0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7000
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7040
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7090
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db70f0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7140
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7190
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db71d0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7220
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db7280
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db72d0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7320
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7360
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db73b0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db7410
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7460
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db74b0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db74f0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7540
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db75a0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db75f0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7640
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7680
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db76d0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db7730
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7780
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db77d0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7810
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7860
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db78c0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7910
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7960
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db79a0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db79f0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db7a50
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7aa0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7af0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7b30
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7b80
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db7be0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7c30
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7c80
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7cc0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7d10
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db7d70
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7dc0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7e10
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7e50
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7ea0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db7f00
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db7f50
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db7fa0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db7fe0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8030
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db8090
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db80e0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db8130
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db8170
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db81c0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db8220
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8270
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db82c0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db8300
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8350
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db83b0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8400
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db8450
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db8490
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db84e0
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db8540
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8590
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db85e0
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db8620
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8670
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db86d0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8720
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db8770
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db87b0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8800
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db8860
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db88b0
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db8900
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db8940
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8990
[59] zio_vdev_io_start() at zio_vdev_io_start+0x2a6/frame 0xfffffe00d6db89f0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8a40
[59] vdev_queue_io_done() at vdev_queue_io_done+0x100/frame 0xfffffe00d6db8a90
[59] zio_vdev_io_done() at zio_vdev_io_done+0x14e/frame 0xfffffe00d6db8ad0
[59] zio_execute() at zio_execute+0xac/frame 0xfffffe00d6db8b20
[59] taskqueue_run_locked() at taskqueue_run_locked+0x144/frame 0xfffffe00d6db8b80
[59] taskqueue_thread_loop() at taskqueue_thread_loop+0xb6/frame 0xfffffe00d6db8bb0
[59] fork_exit() at fork_exit+0x7e/frame 0xfffffe00d6db8bf0
[59] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d6db8bf0
[59] --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
[59] KDB: enter: panic
Comment 1 sigsys 2020-05-05 09:00:54 UTC
After some testing, it seems to only panic when it tries to import all of its pools on boot.  This computer has 3 pools: a main 2-way SSD mirror pool it boots from, a small 1 SSD pool (SSD also has a swap partition on it) and a 2-way HDD mirror pool.  Preventing the HDD pool from being imported automatically on boot prevents the panics.  And importing it manually a little bit after booting also does not panic.  Also, there are no kernel error messages at any point (even in the cases where it does panic (well, apart from the panic)) and nothing bad in zpool status.