lock order reversal: 1st 0xffffff0003a2d7f8 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:424 2nd 0xfffffffe53e58bc8 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2443 3rd 0xffffff00034fc7f8 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:545 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xc2a ffs_lock() at ffs_lock+0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 ffs_snapshot() at ffs_snapshot+0x1c09 ffs_mount() at ffs_mount+0x666 vfs_donmount() at vfs_donmount+0xe31 nmount() at nmount+0xa6 syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (378, FreeBSD ELF64, nmount), rip = 0x8007a6cec, rsp = 0x7fffffffea48, rbp = 0x800a02530 --- lock order reversal: 1st 0xfffffffe53e58bc8 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2443 2nd 0xffffff0003892330 snaplk (snaplk) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:794 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xc2a ffs_lock() at ffs_lock+0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 ffs_snapshot() at ffs_snapshot+0x1ad6 ffs_mount() at ffs_mount+0x666 vfs_donmount() at vfs_donmount+0xe31 nmount() at nmount+0xa6 syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (378, FreeBSD ELF64, nmount), rip = 0x8007a6cec, rsp = 0x7fffffffea48, rbp = 0x800a02530 --- lock order reversal: 1st 0xffffff0003892330 snaplk (snaplk) @ /usr/src/sys/kern/vfs_vnops.c:293 2nd 0xffffff0003a2d7f8 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:1588 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xc2a ffs_snapremove() at ffs_snapremove+0xe7 softdep_releasefile() at softdep_releasefile+0x139 ufs_inactive() at ufs_inactive+0x1a7 vinactive() at vinactive+0x72 vput() at vput+0x22c vn_close() at vn_close+0xb9 vn_closefile() at vn_closefile+0x7d _fdrop() at _fdrop+0x23 closef() at closef+0x4d kern_close() at kern_close+0x10d syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (6, FreeBSD ELF64, close), rip = 0x80083cf8c, rsp = 0x7fffffffea48, rbp = 0 --- Fix: unknown How-To-Repeat: kernel crashes while loading snd_hda, no crashdump can be written. on reboot ffsck runs (and backroung fsck) which tiggered this LOR.
On Tuesday 17 February 2009 19:59:22 Dirk Meyer wrote: > >Number: 131784 > >Category: kern > >Synopsis: triple LOR after hard reboot ... > lock order reversal: > 1st 0xffffff0003a2d7f8 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:424 > 2nd 0xfffffffe53e58bc8 bufwait (bufwait) @ ... > >How-To-Repeat: > > kernel crashes while loading snd_hda, no crashdump can be written. > on reboot ffsck runs (and backroung fsck) which tiggered this LOR. I'm seeing a similar (but slightly different) LOR when booting after loosing the last bit of power of a laptop battery : lock order reversal: 1st 0xc5f70ad0 ufs (ufs) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:424 2nd 0xd9637810 bufwait (bufwait) @ /mnt/1/src-current/sys/kern/vfs_bio.c:1783 3rd 0xc5b04058 ufs (ufs) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:545 KDB: stack backtrace: db_trace_self_wrapper(c0be832c,e7dbd40c,c0879735,4,c0be37fb,...) at db_trace_self_wrapper+0x26 kdb_backtrace(4,c0be37fb,c5523740,c55264c0,e7dbd468,...) at kdb_backtrace+0x29 _witness_debugger(c0beb0e2,c5b04058,c0bde712,c55264c0,c0c082e2,...) at _witness_debugger+0x25 witness_checkorder(c5b04058,9,c0c082e2,221,0,...) at witness_checkorder+0x839 __lockmgr_args(c5b04058,80100,c5b04074,0,0,...) at __lockmgr_args+0x797 ffs_lock(e7dbd578,c0e7b5a0,c5b010a4,80100,c5b04000,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0cee5a0,e7dbd578,e7dbd598,c0d06ec0,c5b04000,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c5b04000,80100,c0c082e2,221,c5553200,...) at _vn_lock+0x5e ffs_snapshot(c5b05500,c5861980,c0c09c5b,15e,3,...) at ffs_snapshot+0x1527 ffs_mount(c5b05500,c5b01000,c0bf18e3,3d7,c586cb00,...) at ffs_mount+0x146f vfs_donmount(c5b01000,211000,c5d54380,c5d54380,bfbfed04,...) at vfs_donmount+0x130e nmount(c5b01000,e7dbdcf8,c,e7dbdd38,c0ccbb30,...) at nmount+0xbe syscall(e7dbdd38) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280e6c8b, esp = 0xbfbfeb2c, ebp = 0xbfbfee78 --- lock order reversal: 1st 0xd9637810 bufwait (bufwait) @ /mnt/1/src-current/sys/kern/vfs_bio.c:1783 2nd 0xc5fe9c9c snaplk (snaplk) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:2564 3rd 0xd9610360 bufwait (bufwait) @ /mnt/1/src-current/sys/kern/vfs_bio.c:2443 KDB: stack backtrace: db_trace_self_wrapper(c0be832c,e7dbcfe0,c0879735,4,c0be37fb,...) at db_trace_self_wrapper+0x26 kdb_backtrace(4,c0be37fb,c5527360,c5523740,e7dbd03c,...) at kdb_backtrace+0x29 _witness_debugger(c0beb0e2,d9610360,c0bf0833,c5523740,c0befd8e,...) at _witness_debugger+0x25 witness_checkorder(d9610360,9,c0befd8e,98b,0,...) at witness_checkorder+0x839 __lockmgr_args(d9610360,81900,c5f70b20,c0bf0768,50,...) at __lockmgr_args+0x797 getblk(c5f70a78,c1a2cd,0,4000,0,...) at getblk+0x153 breadn(c5f70a78,c1a2cd,0,4000,0,...) at breadn+0x44 bread(c5f70a78,c1a2cd,0,4000,0,...) at bread+0x4c ffs_blkfree(c5a1e200,dd793000,c5f70a78,60d2210,0,...) at ffs_blkfree+0x328 mapacct_ufs2(c5f70a78,c60a1000,c60a5000,dd793000,8e7d80,...) at mapacct_ufs2+0xe2 indiracct_ufs2(60d1658,0,ff7187f4,ffffffff,8e780c,...) at indiracct_ufs2+0x3f1 indiracct_ufs2(3a6bd18,0,ff7ff7f3,ffffffff,8e780c,...) at indiracct_ufs2+0x338 indiracct_ufs2(4d2ace8,0,ffbff7f2,ffffffff,80080c,...) at indiracct_ufs2+0x338 expunge_ufs2(c0a331c0,2,2,29a,0,...) at expunge_ufs2+0x495 ffs_snapshot(c5b05500,c5861980,c0c09c5b,15e,3,...) at ffs_snapshot+0x239d ffs_mount(c5b05500,c5b01000,c0bf18e3,3d7,c586cb00,...) at ffs_mount+0x146f vfs_donmount(c5b01000,211000,c5d54380,c5d54380,bfbfed04,...) at vfs_donmount+0x130e nmount(c5b01000,e7dbdcf8,c,e7dbdd38,c0ccbb30,...) at nmount+0xbe syscall(e7dbdd38) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280e6c8b, esp = 0xbfbfeb2c, ebp = 0xbfbfee78 --- lock order reversal: 1st 0xc5fe9c9c snaplk (snaplk) @ /mnt/1/src-current/sys/kern/vfs_vnops.c:293 2nd 0xc5f70ad0 ufs (ufs) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:1588 KDB: stack backtrace: db_trace_self_wrapper(c0be832c,e7dbd8c4,c0879735,4,c0be37fb,...) at db_trace_self_wrapper+0x26 kdb_backtrace(4,c0be37fb,c5527360,c55264c0,e7dbd920,...) at kdb_backtrace+0x29 _witness_debugger(c0beb0c9,c5f70ad0,c0bde712,c55264c0,c0c082e2,...) at _witness_debugger+0x25 witness_checkorder(c5f70ad0,9,c0c082e2,634,0,...) at witness_checkorder+0x839 __lockmgr_args(c5f70ad0,80000,0,0,0,...) at __lockmgr_args+0x797 ffs_snapremove(c5f70a78,c5b05500,0,c0bf3105,414,...) at ffs_snapremove+0x11f softdep_releasefile(c5b157f8,e7dbdaa8,2,c0e7b570,c0cd03e4,...) at softdep_releasefile+0x3b ufs_inactive(e7dbdae8,c5f70aec,c5f70a78,c5f70aec,e7dbdb00,...) at ufs_inactive+0x1bc VOP_INACTIVE_APV(c0cee5a0,e7dbdae8,c0bf1f1d,92d,c0d06e80,...) at VOP_INACTIVE_APV+0xa5 vinactive(c0cee5a0,e7dbdb1c,c0bf1f1d,8b3,129,...) at vinactive+0x8e vput(c5f70a78,e7dbdb54,c0bf3105,125,c0d06be0,...) at vput+0x1db vn_close(c5f70a78,1,c556c400,c5b01000,e7dbdbe0,...) at vn_close+0xee vn_closefile(c5a895e8,c5b01000,3,0,c5a895e8,...) at vn_closefile+0xe9 _fdrop(c5a895e8,c5b01000,e7dbdc1c,c087957c,0,c5b010a4,c0e7b570,c0cd1bf0,c0be048b,c5d4712c,44f,c0be048b,e7dbdc44,c0841d80,c5d4712c,8,c0be048b,44f) at _fdrop+0x43 closef(c5a895e8,c5b01000,44f,434,c5a895e8,...) at closef+0x290 kern_close(c5b01000,4,e7dbdd2c,c0b3d5f3,c5b01000,...) at kern_close+0x11d close(c5b01000,e7dbdcf8,4,c0bebe9a,c0cc9850,...) at close+0x1a syscall(e7dbdd38) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (6, FreeBSD ELF32, close), eip = 0x28184573, esp = 0xbfbfeb2c, ebp = 0xbfbfee78 --- System: FreeBSD Racebeest.Vitsch.LAN 8.0-CURRENT FreeBSD 8.0-CURRENT #2: Sat Feb 14 01:34:10 CET 2009 root@Racebeest.Vitsch.LAN:/usr/obj/mnt/1/src-current/sys/GENERIC i386 Regards, -- Daan Vreeken VEHosting http://VEHosting.nl tel: +31-(0)40-7113050 / +31-(0)6-46210825 KvK nr: 17174380
On Tue, 17 Feb 2009, Dirk Meyer wrote: See http://sources.zabbadoz.net/freebsd/lor.html > lock order reversal: > 1st 0xffffff0003a2d7f8 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:424 > 2nd 0xfffffffe53e58bc8 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2443 > 3rd 0xffffff00034fc7f8 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:545 LOR #285 (new) > --- syscall (378, FreeBSD ELF64, nmount), rip = 0x8007a6cec, rsp = 0x7fffffffea48, rbp = 0x800a02530 --- > lock order reversal: > 1st 0xfffffffe53e58bc8 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2443 > 2nd 0xffffff0003892330 snaplk (snaplk) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:794 LOR #269 > lock order reversal: > 1st 0xffffff0003892330 snaplk (snaplk) @ /usr/src/sys/kern/vfs_vnops.c:293 > 2nd 0xffffff0003a2d7f8 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:1588 LOR #240 > lock order reversal: > 1st 0xc5f70ad0 ufs (ufs) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:424 > 2nd 0xd9637810 bufwait (bufwait) @ /mnt/1/src-current/sys/kern/vfs_bio.c:1783 > 3rd 0xc5b04058 ufs (ufs) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:545 LOR #285 > lock order reversal: > 1st 0xd9637810 bufwait (bufwait) @ /mnt/1/src-current/sys/kern/vfs_bio.c:1783 > 2nd 0xc5fe9c9c snaplk (snaplk) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:2564 > 3rd 0xd9610360 bufwait (bufwait) @ /mnt/1/src-current/sys/kern/vfs_bio.c:2443 LOR #286 (new) > lock order reversal: > 1st 0xc5fe9c9c snaplk (snaplk) @ /mnt/1/src-current/sys/kern/vfs_vnops.c:293 > 2nd 0xc5f70ad0 ufs (ufs) @ /mnt/1/src-current/sys/ufs/ffs/ffs_snapshot.c:1588 LOR #240 See http://sources.zabbadoz.net/freebsd/lor.html -- Bjoern A. Zeeb The greatest risk is not taking one.
I got the same thing just taking a "dump -L" of a filesystem, running: FreeBSD server.vk2pj.dyndns.org 8.0-STABLE FreeBSD 8.0-STABLE #11: Mon Nov 30 16:36:34 EST 2009 root@server.vk2pj.dyndns.org:/var/obj/usr/src/sys/server amd64 lock order reversal: 1st 0xffffff002db3e9d0 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:424 2nd 0xffffff8080763388 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2559 3rd 0xffffff0028f14098 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:545 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper0x2a _witness_debugger() at _witness_debugger0x2c witness_checkorder() at witness_checkorder0x66f __lockmgr_args() at __lockmgr_args0x475 ffs_lock() at ffs_lock0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV0x52 _vn_lock() at _vn_lock0x47 ffs_snapshot() at ffs_snapshot0x1af8 ffs_mount() at ffs_mount0x652 vfs_donmount() at vfs_donmount0xc80 nmount() at nmount0x51 syscall() at syscall0x1ac Xfast_syscall() at Xfast_syscall0xe1 --- syscall (378, FreeBSD ELF64, nmount), rip = 0x80069bb4c, rsp = 0x7fffffffe518, rbp = 0x7fffffffef1e --- lock order reversal: 1st 0xffffff8080763388 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2559 2nd 0xffffff000be70630 snaplk (snaplk) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:794 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper0x2a _witness_debugger() at _witness_debugger0x2c witness_checkorder() at witness_checkorder0x66f __lockmgr_args() at __lockmgr_args0x475 ffs_lock() at ffs_lock0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV0x52 _vn_lock() at _vn_lock0x47 ffs_snapshot() at ffs_snapshot0x154d ffs_mount() at ffs_mount0x652 vfs_donmount() at vfs_donmount0xc80 nmount() at nmount0x51 syscall() at syscall0x1ac Xfast_syscall() at Xfast_syscall0xe1 --- syscall (378, FreeBSD ELF64, nmount), rip = 0x80069bb4c, rsp = 0x7fffffffe518, rbp = 0x7fffffffef1e --- lock order reversal: 1st 0xffffff000be70630 snaplk (snaplk) @ /usr/src/sys/kern/vfs_vnops.c:296 2nd 0xffffff002db3e9d0 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_snapshot.c:1588 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper0x2a _witness_debugger() at _witness_debugger0x2c witness_checkorder() at witness_checkorder0x66f __lockmgr_args() at __lockmgr_args0x475 ffs_snapremove() at ffs_snapremove0xc6 softdep_releasefile() at softdep_releasefile0x139 ufs_inactive() at ufs_inactive0x1da VOP_INACTIVE_APV() at VOP_INACTIVE_APV0x4a vinactive() at vinactive0x68 vput() at vput0x1ae vn_close() at vn_close0xa1 vn_closefile() at vn_closefile0x5a _fdrop() at _fdrop0x23 closef() at closef0x4a fdfree() at fdfree0x204 exit1() at exit10x24f sys_exit() at sys_exit0xe syscall() at syscall0x1ac Xfast_syscall() at Xfast_syscall0xe1 --- syscall (1, FreeBSD ELF64, sys_exit), rip = 0x8006eea2c, rsp = 0x7fffffffe378, rbp = 0x404ec0 --- If anyone wants to look further, the line number references are relative to: __FBSDID("$FreeBSD: src/sys/kern/vfs_bio.c,v 1.568.2.1 2009/08/03 08:13:06 kensmith Exp $"); SVN rev 196045 on 2009-08-03 08:13:06Z by kensmith __FBSDID("$FreeBSD: src/sys/kern/vfs_vnops.c,v 1.286.2.2 2009/09/08 14:43:42 kib Exp $"); SVN rev 196974 on 2009-09-08 14:43:42Z by kib __FBSDID("$FreeBSD: src/sys/ufs/ffs/ffs_snapshot.c,v 1.150.2.1 2009/08/03 08:13:06 kensmith Exp $"); SVN rev 196045 on 2009-08-03 08:13:06Z by kensmith -- Peter Jeremy
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped
Hello. Just to say I'm seeing #269 on 12.1/amd64 when I run mksnap_ffs on an external HD. This can be systematically reproduced. Filesystem is clean as I previously run "fsck -y".