Created attachment 184595 [details] Full crash log I upgraded my test vm from 11.1-RC2 to 11.1-RC3, and the system crashs during system boot. Environment: rMBP, OS X 10.9.5, VMware Fusion, 2 process core, 512MB memeory, 2G virtual hard disk for system install and two virtual hard disks for data, bridged VMXNET3 nic System is installed with ZFS root. The /boot/loader.conf is as following, kern.peom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" vfs.zfs.min_auto_ashift=12 zfs_load="YES" hw.pci.honor_msi_blacklist="0" hw.vmx.txnqueue="4" hw.vmx.rxnqueue="4" Crash log: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 02 fault virtual address = 0x18 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff808eac3a stack pointer = 0x28:0xfffffe001d3f2000 frame pointer = 0x28:0xfffffe001d3f2050 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 4 (doneq0) trap number = 12 panic: page fault cpuid = 1 KDB: stack backtrace: #0 0xffffffff80aada97 at kdb_backtrace+0x67 #1 0xffffffff80a6bb76 at vpanic+0x186 #2 0xffffffff80a6b9e3 at panic+0x43 #3 0xffffffff80edf832 at trap_fatal+0x322 #4 0xffffffff80edf889 at trap_pfault+0x49 #5 0xffffffff80edf0c6 at trap+0x286 #6 0xffffffff80ec3641 at calltrap+0x8 #7 0xffffffff808f26d8 at vt_flush+0x398 #8 0xffffffff80ac09cf at termcn_cnputc+0xff #9 0xffffffff80a100ed at cnputc+0x7d #10 0xffffffff80a103f8 at cnputs+0xb8 #11 0xffffffff80ab36ad at putchar+0x14d #12 0xffffffff80ab323d at kvprintf+0x103d #13 0xffffffff80ab3ea7 at vprintf+0x87 #14 0xffffffff80ab3e13 at printf+0x43 #15 0xffffffff80307c52 at xpt_announce_periph+0x62 #16 0xffffffff8032e089 at cddone+0x249 #17 0xffffffff8030dbe7 at xpt_done_process+0x677 Full crash log is attached. Anyone has a clue about this panic?
Same bugs for 11.1-RELEASE, need set affects version to 11.0-RELEASE ?
This affects me as well. I just tried upgrading a FreeBSD system from 11.0 to 11.1. The system would panic after rebooting. Strange thing though: After the 2nd panic, the kernel actually booted. Environment is VMware ESXi 6.0. Kernel backtrace: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 02 fault virtual address = 0x3c fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff808eac3a stack pointer = 0x28:0xfffffe03e6d14f70 frame pointer = 0x28:0xfffffe03e6d14fc0 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 4 (doneq0) trap number = 12 panic: page fault cpuid = 1 KDB: stack backtrace: #0 0xffffffff80aada97 at kdb_backtrace+0x67 #1 0xffffffff80a6bb76 at vpanic+0x186 #2 0xffffffff80a6b9e3 at panic+0x43 #3 0xffffffff80edf832 at trap_fatal+0x322 #4 0xffffffff80edf889 at trap_pfault+0x49 #5 0xffffffff80edf0c6 at trap+0x286 #6 0xffffffff80ec3641 at calltrap+0x8 #7 0xffffffff808f26d8 at vt_flush+0x398 #8 0xffffffff80ac09cf at termcn_cnputc+0xff #9 0xffffffff80a100ed at cnputc+0x7d #10 0xffffffff80a103f8 at cnputs+0xb8 #11 0xffffffff80ab36ad at putchar+0x14d #12 0xffffffff80ab323d at kvprintf+0x103d #13 0xffffffff80ab3ea7 at vprintf+0x87 #14 0xffffffff80ab3e13 at printf+0x43 #15 0xffffffff80322e24 at scsi_print_inquiry+0x144 #16 0xffffffff80307c9e at xpt_announce_periph+0xae #17 0xffffffff8033bb48 at dadone+0x16f8 Uptime: 3s
I have same issues on ESXi 6.5 , on VM reboot some times randomly crashes one time / several times with Fatal trap 12: page fault while in kernel mode then boots normally .
os version : FreeBSD hyperyon 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 log cd0: <NECVMWar VMware SATA CD00 1.00> Removable CD-ROM SCSI device kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x11c fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff808eac3a stack pointer = 0x28:0xfffffe011af73f80 frame pointer = 0x28:0xfffffe011af73fd0 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 4 (doneq1) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff80aada97 at kdb_backtrace+0x67 #1 0xffffffff80a6bb76 at vpanic+0x186 #2 0xffffffff80a6b9e3 at panic+0x43 #3 0xffffffff80edf832 at trap_fatal+0x322 #4 0xffffffff80edf889 at trap_pfault+0x49 #5 0xffffffff80edf0c6 at trap+0x286 #6 0xffffffff80ec3641 at calltrap+0x8 #7 0xffffffff808f26d8 at vt_flush+0x398 #8 0xffffffff80ac09cf at termcn_cnputc+0xff #9 0xffffffff80a100ed at cnputc+0x7d #10 0xffffffff80a103f8 at cnputs+0xb8 #11 0xffffffff80ab36ad at putchar+0x14d #12 0xffffffff80ab323d at kvprintf+0x103d #13 0xffffffff80ab3ea7 at vprintf+0x87 #14 0xffffffff80ab3e13 at printf+0x43 #15 0xffffffff80322e24 at scsi_print_inquiry+0x144 #16 0xffffffff80307c9e at xpt_announce_periph+0xae #17 0xffffffff8032e089 at cddone+0x249 Uptime: 3s
Umm... I set up a new vm, repeat rebooting it with 11.1-RC2 installing ISO, and it also crashes randomly.
Can we get a crashdump or text dump for one of these panics please?
(In reply to Josh Paetzel from comment #6) The system reboot instantly when crash, not leaving any vmcores in /var/crash on reboot. I've set dumpdev=AUTO in rc.conf, and swapinfo shows the dump device is /dev/da0p2. @Josh Paetzel, how can I get a crashdump then?
So what's happening is you are getting the kernel panic early before any userland config for dumping is happening. You'll need to install a kernel on this machine with "options DDB" in the kernel config file. That way when it panics you will get dropped to a ddb prompt and can dump manually.
Created attachment 185839 [details] DDB prompt I've got DDB prompt now, but it seems not doing a dump. Am I wrong? @Josh Paetzel
Try typing "dump" without the quotes. I'm traveling today and will look at this tonight.
(In reply to Josh Paetzel from comment #10) Thanks for that. Hmm, I need config dump device first... ```` Stopped at vga_bitblt_text+0x14a: movl (%rax,%rcx,4),%r15d db> bt Tracing pid 4 tid 100030 td 0xfffff800032bc560 vga_bitblt_text() at vga_bitblt_text+0x14a/frame 0xfffffe001d3dafd0 vt_flush() at vt_flush+0x398/frame 0xfffffe001d3db020 termcn_cnputc() at termcn_cnputc+0xff/frame 0xfffffe001d3db060 cnputc() at cnputc+0x7d/frame 0xfffffe001d3db090 cnputs() at cnputs+0xb8/frame 0xfffffe001d3db0b0 putchar() at putchar+0x14d/frame 0xfffffe001d3db130 kvprintf() at kvprintf+0x103d/frame 0xfffffe001d3db220 vprintf() at vprintf+0x87/frame 0xfffffe001d3db2f0 printf() at printf+0x43/frame 0xfffffe001d3db350 scsi_print_inquiry() at scsi_print_inquiry+0x144/frame 0xfffffe001d3db3f0 xpt_announce_periph() at xpt_announce_periph+0xae/frame 0xfffffe001d3db420 cddone() at cddone+0x249/frame 0xfffffe001d3db9e0 xpt_done_process() at xpt_done_process+0x677/frame 0xfffffe001d3dba20 xpt_done_td() at xpt_done_td+0x196/frame 0xfffffe001d3dba70 fork_exit() at fork_exit+0x85/frame 0xfffffe001d3dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe001d3dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db> dump Cannot dump: no dump device specified. db> ````
I've set `dumpdev="/dev/da0p2"` in /boot/loader.conf as per loader(8), but it seems not take effect when I do dumping in DDB prompt. It's so weird...
FWIW, I tried this with VMware ESXi, 6.5.0, 5310538 and 11.1-RELEASE-p1 today and was not able to reproduce the problem. Were there any significant kernel changes between 11.1-RELEASE and 11.1-RELEASE-p1 that might have resolved the problem already? I tried booting the VM dozens of times and was not able to get it to panic. :-/
(In reply to ferdinand.goldmann from comment #13) You need try another dozens of times, :) Changes between 11.1-RELEASE and 11.1-RELEASE-p111.1-RELEASE-p1 seems not related to this bug. https://github.com/freebsd/freebsd/compare/release/11.1.0...releng/11.1
I'm getting this with DDB, hope it helps. :) @Josh Paetzel db> bt Tracing pid 4 tid 100030 td 0xfffff800032bc560 vga_bitblt_text() at vga_bitblt_text+0x14a/frame 0xfffffe001d3db050 vt_flush() at vt_flush+0x398/frame 0xfffffe001d3db0a0 termcn_cnputc() at termcn_cnputc+0xff/frame 0xfffffe001d3db0e0 cnputc() at cnputc+0x7d/frame 0xfffffe001d3db110 cnputs() at cnputs+0xb8/frame 0xfffffe001d3db130 putchar() at putchar+0x14d/frame 0xfffffe001d3db1b0 kvprintf() at kvprintf+0x103d/frame 0xfffffe001d3db2a0 vprintf() at vprintf+0x87/frame 0xfffffe001d3db370 printf() at printf+0x43/frame 0xfffffe001d3db3d0 xpt_announce_periph() at xpt_announce_periph+0x62/frame 0xfffffe001d3db420 cddone() at cddone+0x249/frame 0xfffffe001d3db9e0 xpt_done_process() at xpt_done_process+0x677/frame 0xfffffe001d3dba20 xpt_done_td() at xpt_done_td+0x196/frame 0xfffffe001d3dba70 fork_exit() at fork_exit+0x85/frame 0xfffffe001d3dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe001d3dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- db> show register cs 0x20 ds 0x3b es 0x3b fs 0x13 gs 0x28 ss 0x28 rax 0 rcx 0x20 rdx 0xdd rbx 0xfffffe001d3db070 rsp 0xfffffe001d3db000 rbp 0xfffffe001d3db050 rsi 0x15 rdi 0 r8 0x63 r9 0xfffffe001d3db070 r10 0x12c r11 0 r12 0xffffffff81b48fc8 vga_conssoftc r13 0x20 r14 0xffffffff81970cc0 vt_conswindow r15 0x1c200043 rip 0xffffffff808f57ba vga_bitblt_text+0x14a rflags 0x10046 vga_bitblt_text+0x14a: movl (%rax,%rcx,4),%r15d db> x/ia vga_bitblt_text,0x14a vga_bitblt_text: pushq %rbp vga_bitblt_text+0x1: movq %rsp,%rbp vga_bitblt_text+0x4: pushq %r15 vga_bitblt_text+0x6: pushq %r14 vga_bitblt_text+0x8: pushq %r13 vga_bitblt_text+0xa: pushq %r12 vga_bitblt_text+0xc: pushq %rbx vga_bitblt_text+0xd: subq $0x28,%rsp vga_bitblt_text+0x11: movq %rdx,%rbx vga_bitblt_text+0x14: movq %rsi,%r14 vga_bitblt_text+0x17: movq %rdi,%r12 vga_bitblt_text+0x1a: testb $0x1,0x14c(%r12) vga_bitblt_text+0x23: jnz vga_bitblt_text+0xe3 vga_bitblt_text+0x29: movq 0x70(%r14),%rcx vga_bitblt_text+0x2d: movzwl (%rbx),%r15d vga_bitblt_text+0x31: movl 0x28(%rcx),%eax vga_bitblt_text+0x34: movq %rcx,0xffffffffffffffc8(%rbp) vga_bitblt_text+0x38: movl 0x2c(%rcx),%r8d vga_bitblt_text+0x3c: movzwl 0x7a(%r14),%edi vga_bitblt_text+0x41: imull %eax,%r15d vga_bitblt_text+0x45: movzwl 0x78(%r14),%edx vga_bitblt_text+0x4a: addl %edx,%r15d vga_bitblt_text+0x4d: movzwl 0x6(%rbx),%esi vga_bitblt_text+0x51: movzwl 0x4(%rbx),%ecx vga_bitblt_text+0x55: imull %r8d,%esi vga_bitblt_text+0x59: leal 0x7(%rdi,%rsi,1),%r13d vga_bitblt_text+0x5e: andl $-0x8,%r13d vga_bitblt_text+0x62: imull %eax,%ecx vga_bitblt_text+0x65: addl %edx,%ecx vga_bitblt_text+0x67: movzwl 0x7e(%r14),%edx vga_bitblt_text+0x6c: cmpl %edx,%r13d vga_bitblt_text+0x6f: cmovnb %edx,%r13d vga_bitblt_text+0x73: movzwl 0x7c(%r14),%edx vga_bitblt_text+0x78: cmpl %edx,%ecx vga_bitblt_text+0x7a: cmovnb %edx,%ecx vga_bitblt_text+0x7d: cmpl %ecx,%r15d vga_bitblt_text+0x80: jnb vga_bitblt_text+0x25c vga_bitblt_text+0x86: movzwl 0x2(%rbx),%edx vga_bitblt_text+0x8a: imull %edx,%r8d vga_bitblt_text+0x8e: addl %r8d,%edi vga_bitblt_text+0x91: andl $-0x8,%edi vga_bitblt_text+0x94: movq %rdi,0xffffffffffffffb8(%rbp) vga_bitblt_text+0x98: movl %ecx,0xffffffffffffffc0(%rbp) vga_bitblt_text+0x9b: nopl vga_bitblt_text+0xa0: cmpl %r13d,%edi vga_bitblt_text+0xa3: jnb vga_bitblt_text+0xd6 vga_bitblt_text+0xa5: movl %edi,%ebx vga_bitblt_text+0xa7: nopw vga_bitblt_text+0xb0: movq %r12,%rdi vga_bitblt_text+0xb3: movq %r14,%rsi vga_bitblt_text+0xb6: movl %ebx,%edx vga_bitblt_text+0xb8: movl %r15d,%ecx vga_bitblt_text+0xbb: call vga_bitblt_one_text_pixels_block vga_bitblt_text+0xc0: addl $0x8,%ebx vga_bitblt_text+0xc3: cmpl %r13d,%ebx vga_bitblt_text+0xc6: jb vga_bitblt_text+0xb0 vga_bitblt_text+0xc8: movq 0xffffffffffffffc8(%rbp),%rax vga_bitblt_text+0xcc: movl 0x28(%rax),%eax vga_bitblt_text+0xcf: movq 0xffffffffffffffb8(%rbp),%rdi vga_bitblt_text+0xd3: movl 0xffffffffffffffc0(%rbp),%ecx vga_bitblt_text+0xd6: addl %eax,%r15d vga_bitblt_text+0xd9: cmpl %ecx,%r15d vga_bitblt_text+0xdc: jb vga_bitblt_text+0xa0 vga_bitblt_text+0xde: jmp vga_bitblt_text+0x25c vga_bitblt_text+0xe3: movzwl (%rbx),%eax vga_bitblt_text+0xe6: movzwl 0x4(%rbx),%ecx vga_bitblt_text+0xea: movl %eax,0xffffffffffffffc8(%rbp) vga_bitblt_text+0xed: cmpw %cx,%ax vga_bitblt_text+0xf0: jnb vga_bitblt_text+0x25c vga_bitblt_text+0xf6: movq 0x88(%r12),%r12 vga_bitblt_text+0xfe: leaq 0x10(%r14),%rax vga_bitblt_text+0x102: movq %rax,0xffffffffffffffc0(%rbp) vga_bitblt_text+0x106: movw 0x6(%rbx),%ax vga_bitblt_text+0x10a: nopw vga_bitblt_text+0x110: movzwl 0x2(%rbx),%r13d vga_bitblt_text+0x115: cmpw %ax,%r13w vga_bitblt_text+0x119: jnb vga_bitblt_text+0x249 vga_bitblt_text+0x11f: movl 0xffffffffffffffc8(%rbp),%eax vga_bitblt_text+0x122: shll $0x4,%eax vga_bitblt_text+0x125: leal (%rax,%rax,4),%eax vga_bitblt_text+0x128: movq %rax,0xffffffffffffffb8(%rbp) vga_bitblt_text+0x12c: nopl vga_bitblt_text+0x130: movq 0x68(%r14),%rcx vga_bitblt_text+0x134: movl 0x3c(%r14),%eax vga_bitblt_text+0x138: movl 0xffffffffffffffc8(%rbp),%esi vga_bitblt_text+0x13b: addl %esi,%eax vga_bitblt_text+0x13d: xorl %edx,%edx vga_bitblt_text+0x13f: divl 0x38(%r14),%eax vga_bitblt_text+0x143: movq (%rcx,%rdx,8),%rax vga_bitblt_text+0x147: movl %r13d,%ecx vga_bitblt_text+0x14a: movl (%rax,%rcx,4),%r15d vga_bitblt_text+0x14e: movq 0xffffffffffffffc0(%rbp),%rdi vga_bitblt_text+0x152: movl %r13d,%edx vga_bitblt_text+0x155: call vtbuf_iscursor vga_bitblt_text+0x15a: movl %r15d,%edi vga_bitblt_text+0x15d: movl %eax,%esi vga_bitblt_text+0x15f: leaq 0xffffffffffffffd6(%rbp),%rdx vga_bitblt_text+0x163: leaq 0xffffffffffffffd7(%rbp),%rcx vga_bitblt_text+0x167: call vt_determine_colors vga_bitblt_text+0x16c: movl %r15d,%r8d vga_bitblt_text+0x16f: andl $0x1fffff,%r8d vga_bitblt_text+0x176: leal 0xffffffffffffffe0(%r8),%ecx vga_bitblt_text+0x17a: movb $0x3f,%rax vga_bitblt_text+0x17c: cmpl $0x264c,%ecx vga_bitblt_text+0x182: jnbe vga_bitblt_text+0x1e3 vga_bitblt_text+0x184: movq %rbx,%r9 vga_bitblt_text+0x187: xorl %esi,%esi vga_bitblt_text+0x189: movl $0xa8,%rdx vga_bitblt_text+0x18e: nop vga_bitblt_text+0x190: leal (%rdx,%rsi,1),%ecx vga_bitblt_text+0x193: movl %ecx,%edi vga_bitblt_text+0x195: shrl $0x1f,%edi vga_bitblt_text+0x198: addl %ecx,%edi vga_bitblt_text+0x19a: sarl $1,%edi vga_bitblt_text+0x19c: movslq %edi,%rbx vga_bitblt_text+0x19f: movzwl cp437table(,%rbx,4),%ecx vga_bitblt_text+0x1a7: cmpl %ecx,%r8d vga_bitblt_text+0x1aa: jnb vga_bitblt_text+0x1b5 vga_bitblt_text+0x1ac: leal 0xffffffffffffffff(%rdi),%edx vga_bitblt_text+0x1af: cmpl %esi,%edi vga_bitblt_text+0x1b1: jnle vga_bitblt_text+0x190 vga_bitblt_text+0x1b3: jmp vga_bitblt_text+0x1e0 vga_bitblt_text+0x1b5: movzbl cp437table+0x3(,%rbx,4),%esi vga_bitblt_text+0x1bd: addl %ecx,%esi vga_bitblt_text+0x1bf: cmpl %r8d,%esi vga_bitblt_text+0x1c2: jnb vga_bitblt_text+0x1cd vga_bitblt_text+0x1c4: leal 0x1(%rdi),%esi vga_bitblt_text+0x1c7: cmpl %edi,%edx vga_bitblt_text+0x1c9: jnle vga_bitblt_text+0x190 vga_bitblt_text+0x1cb: jmp vga_bitblt_text+0x1e0 vga_bitblt_text+0x1cd: subl %ecx,%r15d vga_bitblt_text+0x1d0: movzbl cp437table+0x2(,%rbx,4),%eax vga_bitblt_text+0x1d8: addl %r15d,%eax vga_bitblt_text+0x1db: nopl vga_bitblt_text+0x1e0: movq %r9,%rbx vga_bitblt_text+0x1e3: movb 0xffffffffffffffd7(%rbp),%cl vga_bitblt_text+0x1e6: shlb $0x4,%cl vga_bitblt_text+0x1e9: orb 0xffffffffffffffd6(%rbp),%cl vga_bitblt_text+0x1ec: movq 0xffffffffffffffb8(%rbp),%rdx vga_bitblt_text+0x1f0: leal (%r13,%rdx,1),%esi vga_bitblt_text+0x1f5: addl %esi,%esi vga_bitblt_text+0x1f7: movq 0x8(%r12),%rdx vga_bitblt_text+0x1fc: addq %rsi,%rdx vga_bitblt_text+0x1ff: cmpq $0,(%r12) vga_bitblt_text+0x204: jz vga_bitblt_text+0x210 vga_bitblt_text+0x206: movb %al,(%rdx) vga_bitblt_text+0x208: jmp vga_bitblt_text+0x211 vga_bitblt_text+0x20a: nopw vga_bitblt_text+0x210: outb %al,%dx vga_bitblt_text+0x211: orl $0x1,%esi vga_bitblt_text+0x214: addq 0x8(%r12),%rsi vga_bitblt_text+0x219: cmpq $0,(%r12) vga_bitblt_text+0x21e: jz vga_bitblt_text+0x230 vga_bitblt_text+0x220: movb %cl,(%rsi) vga_bitblt_text+0x222: jmp vga_bitblt_text+0x235 vga_bitblt_text+0x224: nopw vga_bitblt_text+0x230: movl %ecx,%eax vga_bitblt_text+0x232: movl %esi,%edx vga_bitblt_text+0x234: outb %al,%dx vga_bitblt_text+0x235: incl %r13d vga_bitblt_text+0x238: movzwl 0x6(%rbx),%eax vga_bitblt_text+0x23c: cmpl %eax,%r13d vga_bitblt_text+0x23f: jb vga_bitblt_text+0x130 vga_bitblt_text+0x245: movzwl 0x4(%rbx),%ecx vga_bitblt_text+0x249: movl 0xffffffffffffffc8(%rbp),%esi vga_bitblt_text+0x24c: incl %esi vga_bitblt_text+0x24e: movzwl %cx,%edx vga_bitblt_text+0x251: movl %esi,0xffffffffffffffc8(%rbp) vga_bitblt_text+0x254: cmpl %edx,%esi vga_bitblt_text+0x256: jb vga_bitblt_text+0x110 vga_bitblt_text+0x25c: addq $0x28,%rsp vga_bitblt_text+0x260: popq %rbx vga_bitblt_text+0x261: popq %r12 vga_bitblt_text+0x263: popq %r13 vga_bitblt_text+0x265: popq %r14 vga_bitblt_text+0x267: popq %r15 vga_bitblt_text+0x269: popq %rbp vga_bitblt_text+0x26a: ret vga_bitblt_text+0x26b: nopl
This wasn't resolved in -p1; I just experienced it myself on an ESXi 6.5.0U1 host. It doesn't happen in any deterministic fashion, so far as I can see: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 04 fault virtual address = 0x114 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff808eac3a stack pointer = 0x28:0xfffffe022f438000 frame pointer = 0x28:0xfffffe022f438050 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 4 (doneq0) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff80aada97 at kdb_backtrace+0x67 #1 0xffffffff80a6bb76 at vpanic+0x186 #2 0xffffffff80a6b9e3 at panic+0x43 #3 0xffffffff80edf832 at trap_fatal+0x322 #4 0xffffffff80edf889 at trap_pfault+0x49 #5 0xffffffff80edf0c6 at trap+0x286 #6 0xffffffff80ec3641 at calltrap+0x8 #7 0xffffffff808f26d8 at vt_flush+0x398 #8 0xffffffff80ac09cf at termcn_cnputc+0xff #9 0xffffffff80a100ed at cnputc+0x7d #10 0xffffffff80a103f8 at cnputs+0xb8 #11 0xffffffff80ab36ad at putchar+0x14d #12 0xffffffff80ab323d at kvprintf+0x103d #13 0xffffffff80ab3ea7 at vprintf+0x87 #14 0xffffffff80ab3e13 at printf+0x43 #15 0xffffffff80307c52 at xpt_announce_periph+0x62 #16 0xffffffff8032e089 at cddone+0x249 #17 0xffffffff8030dbe7 at xpt_done_process+0x677 Uptime: 2s
In my environment I can avoid it by setting the "kern.cam.scsi_delay" option. /boot/loader.conf: kern.cam.scsi_delay="5000" OS Version: # uname -v FreeBSD 11.1-RELEASE-p4 #0: Tue Nov 14 06:12:40 UTC 2017 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC # freebsd-version 11.1-RELEASE-p5 Environment: VMware ESXi 6.0U3 on HP DL380 G7
(In reply to Toshimichi Masubuchi from comment #17) If there is no effect with the kern.cam.scsi_delay option, try the kern.cam.boot_delay option. /boot/loader.conf: kern.cam.boot_delay="5000" or kern.cam.boot_delay="10000"
I'm observing this as well with 11.1-RELEASE-p6 on ESXi 6.5.0U1. Will try the loader.conf settings next...
is the mitigation working for anyone else?
I can confirm that this is still happening on ESXi 6.7.0 #1 SMP Release build-11675023 with FreeBSD 11.2-RELEASE. Did anyone confirm if those loader.conf tuneables helped?
I never heard back on whether the loader tunables helped or not. cddone() is in the callstack of every crash dump I saw. You can mitigate that by removing the CD hardware from the VM. (The virtual CD is the only device that causes that code path to be run)
This duplicate with https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217282, fixed in stable/11 by MFC r314116.
*** This bug has been marked as a duplicate of bug 217282 ***