Created attachment 219438 [details] kernel dump text I'm trying to grub-bhyve the OEL8 boot disk. I initiate it as: grub-bhyve -m /home/warlock/pit/cfg/root/oel8-device.map -r cd0 -M 4096M oel8 The device map file has this: (hd0) /dev/zvol/zaux/oel8 (cd0) /zroot/stash/iso/oel8.2_x64_boot.iso Those both obviously use ZFS resources. To cause the panic, I just type this: grub> ls The ISO is Oracle's version of RedHat 8. If it happens to be the bad actor you can probably grab your own copy: SHA1 (oel8.2_x64_boot.iso) = 54b1094367a80893167ad8cec37e9be638503917 That is a renamed V996905-01.iso, Hopefully you won't need to grab it yourself (free, but they make you jump through hoops). The zdev was just created like this: zfs create -V64G -o volmode=dev zaux/oel8 I've attached the crash dump text, but the short version is below. The "dirty" part is the r367433 pre-patch on top of r367430. FreeBSD ouroboros.phouka.net 13.0-CURRENT FreeBSD 13.0-CURRENT #226 r367430+999604acfd94-c272718(master)-dirty: Fri Nov 6 12:56:43 PST 2020 warlock@ouroboros.phouka.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 ... Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 02 fault virtual address = 0x28 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff82895808 stack pointer = 0x28:0xfffffe00e0927550 frame pointer = 0x28:0xfffffe00e09275b0 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 3307 (grub-bhyve) trap number = 12 panic: page fault cpuid = 1 time = 1604786502 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00e0927200 vpanic() at vpanic+0x182/frame 0xfffffe00e0927250 panic() at panic+0x43/frame 0xfffffe00e09272b0 trap_fatal() at trap_fatal+0x387/frame 0xfffffe00e0927310 trap_pfault() at trap_pfault+0x97/frame 0xfffffe00e0927370 trap() at trap+0x2ab/frame 0xfffffe00e0927480 calltrap() at calltrap+0x8/frame 0xfffffe00e0927480 --- trap 0xc, rip = 0xffffffff82895808, rsp = 0xfffffe00e0927550, rbp = 0xfffffe00e09275b0 --- zil_async_to_sync() at zil_async_to_sync+0x18/frame 0xfffffe00e09275b0 zvol_cdev_open() at zvol_cdev_open+0x322/frame 0xfffffe00e09275f0 devfs_open() at devfs_open+0x12f/frame 0xfffffe00e0927660 VOP_OPEN_APV() at VOP_OPEN_APV+0x35/frame 0xfffffe00e0927680 vn_open_vnode() at vn_open_vnode+0x19a/frame 0xfffffe00e0927720 vn_open_cred() at vn_open_cred+0x3d5/frame 0xfffffe00e0927870 kern_openat() at kern_openat+0x263/frame 0xfffffe00e09279c0 amd64_syscall() at amd64_syscall+0x131/frame 0xfffffe00e0927af0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00e0927af0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x8009ead4a, rsp = 0x7fffffffdea8, rbp = 0x7fffffffdf20 --- KDB: enter: panic Uptime: 16h31m34s Dumping 4205 out of 32633 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
I think this is not a virtualization problem. It's looks like a ZFS issue. https://github.com/openzfs/zfs/pull/11152 Here, the same trace: https://drive.google.com/file/d/1-dvj8eoUNqRWL5mcVtH5Px4NbrhLfzML/view https://github.com/openzfs/zfs/commit/ae37ceadaa2a8cf09fbf1a9baafaa6dc6e24318a
A commit references this bug: Author: oshogbo Date: Sun Nov 8 14:08:01 UTC 2020 New revision: 367487 URL: https://svnweb.freebsd.org/changeset/base/367487 Log: Check if the ZVOL has been written before calling zil_async_to_sync. The ZIL will be opened on the first write, not earlier. Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Mariusz Zaborski <oshogbo@vexillium.org> OpenZFS Pull Request: https://github.com/openzfs/zfs/pull/11152 PR: 250934 Changes: head/sys/contrib/openzfs/module/os/freebsd/zfs/zvol_os.c
Could you please verify if commit resolved your issue?
Crunching now. I usually do a full buildworld/buildkernel which isn't the fastest way to compile the kernel. Do you guys (kernel developers) find faster compile options (like using -DKERNFAST, no-clean options, etc) return adequate results? I'm happy wasting my computer's time, but don't want to waste YOUR time. :D
Almost always, -DNO_CLEAN works well.
The patch works! grub> ls (hd0) (cd0) (cd0,msdos2) (host)
I'm pleased that this helps.