The system hangs when init chroot to a unionfs directory. This is a regression as 7-Beta did not have this problem Fix: Patch attached with submission follows: How-To-Repeat: Add the following to loader.conf: init_script="chroot" init_chroot="/base" Have the chroot script do the following: # mount -t unionfs -o noatime -o copymode=transparent /tmpfs /base {where /tmpfs is a suitable unionable filesystem, such as ufs} (please see attached for the output [via serial] of the boot [with the chroot file content])
Responsible Changed From-To: freebsd-bugs->daichi Over to maintainer.
For some strange reason some of the log got cut of when posting, where is the rest if it, also having changed the chroot script a bit I get the attached log (this time the kernel is booting off a md based iso9660 image vs previously CD base iso9660 image): WARNING: opening backing store: /base.uzip readonly md0.uzip: 26993 x 8192 blocks GEOM_LABEL: Label for provider md0.uzip is iso9660/DragonBSD. . Allocating temporary filesystem:lock order reversal: (sleepable after non-sleepable) 1st 0xc6454e10 vnode interlock (vnode interlock) @ /usr/src/sys/kern/vfs_hash.c:79 2nd 0xc6454df4 isofs (isofs) @ /usr/src/sys/kern/vfs_subr.c:2044 KDB: stack backtrace: db_trace_self_wrapper(c0b228e2,f52b67ec,c07d16c8,4,c0b1e285,...) at db_trace_self_wrapper+0x26 kdb_backtrace(4,c0b1e285,c0d9b9c0,c60763f0,f52b6844,...) at kdb_backtrace+0x29 _witness_debugger(c0b251ac,c6454df4,c0b19f3b,c60763f0,c0b2bf05,...) at _witness_debugger+0x1e witness_checkorder(c6454df4,9,c0b2bf05,7fc,1,...) at witness_checkorder+0x7ad __lockmgr_args(c6454df4,80100,c6454e10,0,0,0,c0b2bf05,7fc) at __lockmgr_args+0x75b vop_stdlock(f52b693c,c0b2b4f4,c0b18ea8,80100,c6454d9c,...) at vop_stdlock+0x5c VOP_LOCK1_APV(c0bf3d40,f52b693c,c0c2f4a0,c6454d9c,80100,...) at VOP_LOCK1_APV+0xab _vn_lock(c6454d9c,80100,c0b2bf05,7fc,4,...) at _vn_lock+0x5e vget(c6454d9c,80100,c6612af0,50,0,...) at vget+0xc1 vfs_hash_get(c6585a80,12800,80000,c6612af0,f52b6a8c,...) at vfs_hash_get+0xdf cd9660_vget_internal(c6585a80,12800,80000,f52b6a8c,1,c658a8a8) at cd9660_vget_internal+0x44 cd9660_root(c6585a80,80000,f52b6a8c,c6612af0,f52b6ba0,...) at cd9660_root+0x4e lookup(f52b6b88,c0b2b610,d8,c0,c656cd2c,...) at lookup+0x89f namei(f52b6b88,0,f52b6c4b,f52b6c44,c0b20389,...) at namei+0x430 kern_statat(c6612af0,200,ffffff9c,bfbfe3d0,0,...) at kern_statat+0x5f kern_lstat(c6612af0,bfbfe3d0,0,f52b6c1c,c0d9b9c0,...) at kern_lstat+0x36 lstat(c6612af0,f52b6cf8,8,f52b6d2c,c0bf5c90,...) at lstat+0x2b syscall(f52b6d38) at syscall+0x27b Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (190, FreeBSD ELF32, lstat), eip = 0x8309157, esp = 0xbfbfd4cc, ebp = 0xbfbfe178 --- . Overlaying temporary filesystem over base:. umount: unmount of /dev failed: Device busy Chroot to base... lock order reversal: 1st 0xc66368b8 isofs (isofs) @ /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vnops.c:1709 2nd 0xc65d2164 ufs (ufs) @ /usr/src/sys/kern/vfs_lookup.c:852 KDB: stack backtrace: db_trace_self_wrapper(c0b228e2,c5fba5cc,c07d16c8,4,c0b1e285,...) at db_trace_self_wrapper+0x26 kdb_backtrace(4,c0b1e285,c0d9b9c0,c6076250,c5fba624,...) at kdb_backtrace+0x29 _witness_debugger(c0b251ac,c65d2164,c0b19792,c6076250,c0b2b610,...) at _witness_debugger+0x1e witness_checkorder(c65d2164,9,c0b2b610,354,c5fba654,...) at witness_checkorder+0x7ad __lockmgr_args(c65d2164,80400,c65d2180,0,0,...) at __lockmgr_args+0x75b ffs_lock(c5fba728,c60710a8,c60cadc4,80400,c65d210c,...) at ffs_lock+0x7d VOP_LOCK1_APV(c0c175c0,c5fba728,c0c2f4a0,c65d210c,80400,...) at VOP_LOCK1_APV+0xab _vn_lock(c65d210c,80400,c0b2b610,354,13f,...) at _vn_lock+0x5e relookup(c65d210c,c5fba8e4,c5fba8a8,c656db00,c5fba8e4,...) at relookup+0x64 unionfs_relookup(c5fba8a8,c60cad20,c6616001,3,1,...) at unionfs_relookup+0xd6 unionfs_mkshadowdir(c6545620,c65d210c,c656db00,c5fbabb4,c60cad20,...) at unionfs_mkshadowdir+0x18b unionfs_lookup(c5fba9fc,c659ed9c,c5fbabb4,c659ed9c,c5fbaa1c,...) at unionfs_lookup+0x930 VOP_CACHEDLOOKUP_APV(c0f2cd60,c5fba9fc,c5fbabb4,c5fbaba0,c0c2f140,...) at VOP_CACHEDLOOKUP_APV+0xa0 vfs_cache_lookup(c5fbaa7c,c5fbaa7c,0,80000,c659ed9c,...) at vfs_cache_lookup+0xc3 VOP_LOOKUP_APV(c0f2cd60,c5fbaa7c,c0b2b610,1b0,c5fbaba0,...) at VOP_LOOKUP_APV+0xaa lookup(c5fbab88,0,c0b2b610,e9,c608cc2c,...) at lookup+0x567 namei(c5fbab88,c5fbab28,60,921,c60cad20,...) at namei+0x430 kern_statat(c60cad20,0,ffffff9c,834fd7a,0,...) at kern_statat+0x5f kern_stat(c60cad20,834fd7a,0,c5fbac1c,100ff01,...) at kern_stat+0x36 stat(c60cad20,c5fbacf8,8,c0b25d4c,c0bf5c60,...) at stat+0x2b syscall(c5fbad38) at syscall+0x27b Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (188, FreeBSD ELF32, stat), eip = 0x830916b, esp = 0xbfbfe98c, ebp = 0xbfbfee98 --- lock order reversal: (sleepable after non-sleepable) 1st 0xc66367c8 vnode interlock (vnode interlock) @ /usr/src/sys/modules/unionfs/../../fs/unionfs/union_subr.c:362 2nd 0xc66367ac unionfs (unionfs) @ /usr/src/sys/modules/unionfs/../../fs/unionfs/union_subr.c:371 KDB: stack backtrace: db_trace_self_wrapper(c0b228e2,c5fba8a8,c07d16c8,4,c0b1e285,...) at db_trace_self_wrapper+0x26 kdb_backtrace(4,c0b1e285,c0d9c620,c60764c0,c5fba900,...) at kdb_backtrace+0x29 _witness_debugger(c0b251ac,c66367ac,c0f2b791,c60764c0,c0f2b688,...) at _witness_debugger+0x1e witness_checkorder(c66367ac,9,c0f2b688,173,c60cad20,...) at witness_checkorder+0x7ad __lockmgr_args(c66367ac,80100,c66367c8,0,0,...) at __lockmgr_args+0x75b unionfs_noderem(c6636754,c60cad20,c5fbaa0c,c0a94966,c5fbaa30,...) at unionfs_noderem+0xa2 unionfs_reclaim(c5fbaa30,c60cad20,0,c6636754,c5fbaa54,...) at unionfs_reclaim+0x1b VOP_RECLAIM_APV(c0f2cd60,c5fbaa30,c60cad20,0,0,...) at VOP_RECLAIM_APV+0x9f vgonel(c66367c8,0,c0b2bf05,98a,c5fbaab4,...) at vgonel+0x1b1 vrecycle(c6636754,c60cad20,c5fbaa9c,c0a94a36,c5fbaab4,...) at vrecycle+0x45 unionfs_inactive(c5fbaab4,c66367c8,c6636754,c66367c8,c5fbaacc,...) at unionfs_inactive+0x28 VOP_INACTIVE_APV(c0f2cd60,c5fbaab4,c0b2bf05,8f1,c0c2f460,...) at VOP_INACTIVE_APV+0xa0 vinactive(c0f2cd60,c5fbaae8,c0b2bf05,87d,c5fbab08,...) at vinactive+0x82 vput(c6636754,ffffffdf,c608cb00,0,c60cad20,...) at vput+0x1c9 kern_statat(c60cad20,0,ffffff9c,834fd7a,0,...) at kern_statat+0xc0 kern_stat(c60cad20,834fd7a,0,c5fbac1c,100ff01,...) at kern_stat+0x36 stat(c60cad20,c5fbacf8,8,c0b25d4c,c0bf5c60,...) at stat+0x2b syscall(c5fbad38) at syscall+0x27b Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (188, FreeBSD ELF32, stat), eip = 0x830916b, esp = 0xbfbfe98c, ebp = 0xbfbfee98 ---
Hi, After some investigation I have isolated the commit that results in the breakage. Please see: http://www.freshbsd.org/2007/10/14/13/46 If I have a unionfs.ko built from before this commit then the system does not hang at chroot but after this commit it does hang. Regards David
Hi, I have found a work-a-round for this problem (see "However if" below): I would also like to clarify the problem (the following steps are the ones I take to prepare my environment, some probably are not required): (Create a compressed image of 'world') # cd /usr/src # make buildworld buildkernel # mkdir /tmp/world && make installworld distribution DESTDIR=/tmp/world # cat << _EOF > /tmp/world/etc/rc.conf hostname="dragonbsd" root_rw_mount="NO" _EOF # cat << _EOF > /tmp/world/etc/fstab tmpfs /tmp tmpfs rw 0 0 _EOF # makefs /tmp/world.ufs /tmp/world # mkuzip -o /tmp/world.uzip /tmp/world.ufs (Prepare a USB UMASS device: da0) # fdisk -BI /dev/da0 # cat << _EOF > /tmp/bsdlabel 8 partitions: a: 256m * 4.2BSD b: * * 4.2BSD c: * * unused _EOF # bsdlabel -BR /dev/da0s1 /tmp/bsdlabel # newfs -L DragonBSDBase /dev/da0s1a # newfs -UL DragonBSD /dev/da0s1b (Install FreeBSD onto USB UMASS device: da0) # mount /dev/da0s1a /mnt # mkdir -p /mnt/base /mnt/dev /mnt/tmp /mnt/usr/lib # cp -rp /tmp/world/boot /mnt/ # cp /tmp/world/usr/lib/kgzldr.o /mnt/usr/lib # cd /usr/src && make installkernel DESTDIR=/mnt # mkdir /mnt/rescue && cd /usr/src/rescue && make install DESTDIR=/mnt # cp /tmp/world.uzip /mnt/base.uzip # cat << _EOF > /mnt/boot/loader.conf geom_uzip_load="YES" unionfs_load="YES" zlib_load="YES" init_script="/chroot" init_chroot="/base" vfs.root.mountfrom="ufs:/dev/ufs/DragonBSDBase" _EOF (The chroot script) # cat << _EOF > /mnt/chroot #!/rescue/sh PATH=/rescue mount -o ro /dev/\$(mdconfig -a -t vnode -o readonly -f /base.uzip).uzip /base mount /dev/ufs/DragonBSD /tmp mount -t unionfs -o noatime /tmp /base echo "Chroot to /base..." _EOF # chmod a+x /mnt/chroot If you boot from da0 then after "Chroot to /base..." is displayed some lock order reversal debug messages will be displayed and then nothing. ^t does work but says something about not a controlling terminal. Ctrl-Alt-Esc works and hitting the power button causes the system to shutdown. However if: # echo mount -t devfs devfs /base/dev >> /mnt/chroot is done then da0 will boot without a problem... Thus it is my suspicion that the system gets into a dead-lock when trying to mount /dev after the chroot. Hope this clarifies the issue and provides some insight? Attached, for your convenience, is a script that will create the bootable umass (assuming it is da0). Regards, David
I will be happy for this bug to be closed if the work-around was documented. Unfortunately I do not speak groff. The document may be as follows: BUGS If one is using unionfs over a directory that is the target for init_chroot then make sure devfs is explicitly mounted otherwise the system will freeze when init(8) attempts to chroot.
State Changed From-To: open->open commit bit has been taken in for safekeeping.
Responsible Changed From-To: daichi->freebsd-fs
Too complicated to verify if this issue persists.