Bug 249901

Summary: 12.2-BETA3: Panic in zfs_ctldir.c:358, zfsctl_create+0x111
Product: Base System Reporter: Joshua Kinard <freebsd>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed Not A Bug    
Severity: Affects Only Me    
Priority: ---    
Version: Unspecified   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Customized kernel config for 12.2-BETA3 none

Description Joshua Kinard 2020-09-26 02:30:53 UTC
Testing 12.2-BETA3 out, I get a warning spammed to the console a few hundred times, followed swiftly by a panic and a reboot.  Note: custom kernel with ZFS module compiled in.

<<normal boot sequence>>
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
sysctl_warn_reuse: can't re-use a leaf (kstat.zfs.misc.arcstats)
<<repeats another few hundred times>>
<<resuming what looks like a normal boot>>
Root mount waiting for: usbus1 usbus3
ugen3.2: <Chicony electronics Co.,Ltd. HP Truevision HD> at usbus3
panic: solaris assert: VFS_ROOT(zfsvfs->z_vfs, LK_EXCLUSIVE, &rvp) == 0, file:
../../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c, line: 358
cpuid = 0
time = 4
KDB: stack backtrace
#0 0xffffffff808a2345 at kdb_backtrace+0x65
#1 0xffffffff80857fe5 at vpanic+0x125
#2 0xffffffff80857eb3 at panic+0x43
#3 0xffffffff8036fcba at assfail+0x1a
#4 0xffffffff80438a71 at zfsctl_create+0x111
#5 0xffffffff8044f768 at zfs_mount+0x718
#6 0xffffffff80918fc5 at vfs_domount+0xb55
#7 0xffffffff80917dc7 at vfs_donmount+0x937
#8 0xffffffff8091b76d at kernel_mount+0x3d
#9 0xffffffff8091dd91 at parse_mount+0x491
#10 0xffffffff8091c1c5 at vfs_mountroot+0x545
#11 0xffffffff807f3f17 at start_init+0x27
#12 0xffffffff8081b0dd at fork_exit+0x7d
#13 0xffffffff80ba6ede at fork_trampoline+0xe
Uptime: 4s
<<reboots>>
Comment 1 Joshua Kinard 2020-09-26 02:33:22 UTC
Created attachment 218304 [details]
Customized kernel config for 12.2-BETA3
Comment 2 Joshua Kinard 2020-09-26 02:36:39 UTC
FWIW, 12.2-BETA2 worked fine.  I suspect the culprit is this change
(from https://lists.freebsd.org/pipermail/freebsd-stable/2020-September/092727.html):

    o Read/write kstats for ZFS datasets had been added from OpenZFS.

12.2-BETA3 GENERIC boots fine, so I suspect there's a bug when ZFS is compiled into the kernel versus the default of it being a loadable module.
Comment 3 Joshua Kinard 2020-09-30 03:00:41 UTC
Self-closing.  I initially didn't do a 'make clean' in the kernel build directory from 12.2-BETA2, so some extraneous code modules must've gotten sucked in that didn't belong.....or something.  After cleaning the build directory out and rebuilding the 12.2-BETA3 kernel, it boots fine now.