Bug 192923 - Lock order reversal in vfs_bio.c bufwait
Summary: Lock order reversal in vfs_bio.c bufwait
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-08-22 16:23 UTC by Kate
Modified: 2019-07-30 17:11 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kate 2014-08-22 16:23:00 UTC
I saw some similar bug reports, but they're marked FIXED. I'm filing this just in case it's a different issue.

11.0-CURRENT r270176, every time on boot:

lock order reversal:
 1st 0xe3de20f8 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3089
 2nd 0xc8d53800 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284
KDB: stack backtrace:
db_trace_self_wrapper(c118ebb4,a3438,c1198f69,412,4000,...) at db_trace_self_wrapper+0x2d/frame 0xed2736e0
kdb_backtrace(c1192801,c8d53800,c11c5b76,c85a3a88,c11c57c1,...) at kdb_backtrace+0x30/frame 0xed273744
witness_checkorder(c8d53800,9,c11c57c1,11c,0,...) at witness_checkorder+0xd04/frame 0xed273790
_sx_xlock(c8d53800,0,c11c57c1,11c,c8d41000,...) at _sx_xlock+0x75/frame 0xed2737c0
ufsdirhash_add(ca0a56cc,ed2738b8,3760,ed273840,ed273844,...) at ufsdirhash_add+0x37/frame 0xed2737f0
ufs_direnter(ca0ead50,ca0eab18,ed2738b8,ed273bcc,0,...) at ufs_direnter+0x604/frame 0xed273870
ufs_makeinode(ed273bb8,ed273bcc) at ufs_makeinode+0x534/frame 0xed2739f0
ufs_create(ed273ad8,669,c8d37d30,2,c8d37d74,...) at ufs_create+0x2f/frame 0xed273a04
VOP_CREATE_APV(c1425f7c,ed273ad8,ed273bcc,ed273a68,c0af79e0,...) at VOP_CREATE_APV+0xfe/frame 0xed273a30
vn_open_cred(ed273b70,ed273bfc,1a4,0,c8701b00,c96e2f50) at vn_open_cred+0x2f0/frame 0xed273b00
vn_open(ed273b70,ed273bfc,1a4,c96e2f50,28c5d3ba,...) at vn_open+0x3d/frame 0xed273b28
kern_openat(c8ff9930,ffffff9c,28c5d3ba,0,100202,1a4) at kern_openat+0x310/frame 0xed273c1c
sys_open(c8ff9930,ed273cc8,c1826660,13e6e00,14b,...) at sys_open+0x39/frame 0xed273c40
syscall(ed273d08) at syscall+0x30c/frame 0xed273cfc
Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xed273cfc
--- syscall (5, FreeBSD ELF32, sys_open), eip = 0x28659923, esp = 0xbfbfd81c, ebp = 0xbfbfd830 ---
lock order reversal:
 1st 0xca293150 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2137
 2nd 0xe3e0ce18 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:262
 3rd 0xca16d034 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2137
KDB: stack backtrace:
db_trace_self_wrapper(c118ebb4,ca0f000a,c8ce0800,c8796d00,c17893b8,...) at db_trace_self_wrapper+0x2d/frame 0xed39c728
kdb_backtrace(c119281a,ca16d034,c1179712,c85a3a20,c119be80,...) at kdb_backtrace+0x30/frame 0xed39c78c
witness_checkorder(ca16d034,9,c119be80,859,ca16d054,...) at witness_checkorder+0xd04/frame 0xed39c7d8
__lockmgr_args(ca16d034,80100,ca16d054,0,0,...) at __lockmgr_args+0x8f3/frame 0xed39c8b8
ffs_lock(ed39c938,c0b17463,c859c110,c85a1c10,c859c110,...) at ffs_lock+0x87/frame 0xed39c8f4
VOP_LOCK1_APV(c1425f7c,ed39c938,100000,c1187e7d,c143b768,...) at VOP_LOCK1_APV+0x10a/frame 0xed39c920
_vn_lock(ca16d000,80100,c119be80,859,c119b0bd,...) at _vn_lock+0xa6/frame 0xed39c960
vget(ca16d000,80100,ca0fb930,57,0,...) at vget+0x74/frame 0xed39c998
vfs_hash_get(c8d37d20,3481cb4,80000,ca0fb930,ed39ca98,...) at vfs_hash_get+0xfc/frame 0xed39c9c4
ffs_vgetf(c8d37d20,3481cb4,80000,ed39ca98,1,...) at ffs_vgetf+0x44/frame 0xed39ca20
softdep_sync_buf(ca29311c,e3e0cdc0,1,0,0,...) at softdep_sync_buf+0x35b/frame 0xed39cab0
ffs_syncvnode(ca29311c,1,0,88c,1,...) at ffs_syncvnode+0x2dd/frame 0xed39cb08
softdep_fsync(ca16e000,1,0,ca16e084,c1425f7c,...) at softdep_fsync+0x64f/frame 0xed39cb88
ffs_fsync(ed39cc18,8,c119d2fc,d7d,ca11ad68,...) at ffs_fsync+0x71/frame 0xed39cbac
VOP_FSYNC_APV(c1425f7c,ed39cc18,d7d,0,0,...) at VOP_FSYNC_APV+0xfe/frame 0xed39cbd8
sys_fsync(ca0fb930,ed39ccc8,c1826660,13e6e00,1ae,...) at sys_fsync+0x21a/frame 0xed39cc40
syscall(ed39cd08) at syscall+0x30c/frame 0xed39ccfc
Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xed39ccfc
--- syscall (95, FreeBSD ELF32, sys_fsync), eip = 0x282b68ef, esp = 0xbdfc2ac4, ebp = 0xbdfc2adc ---
Comment 1 Kate 2014-08-22 16:24:47 UTC
Full dmesg up to the LOR:

Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #0 r270176: Tue Aug 19 20:48:51 BST 2014
    root@iona:/usr/obj/usr/src/sys/GENERIC i386
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
WARNING: WITNESS option enabled, expect reduced performance.
CPU: Intel(R) Core(TM)2 Duo CPU     P7550  @ 2.26GHz (2255.39-MHz 686-class CPU)
  Origin="GenuineIntel"  Id=0x1067a  Family=0x6  Model=0x17  Stepping=10
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x408e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE>
  AMD Features=0x20100000<NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 4294967296 (4096 MB)
avail memory = 2834522112 (2703 MB)
kbd0 at kbdmux0
random: <Software, Yarrow> initialized
pcib0 pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
pci0: <memory, RAM> at device 0.1 (no driver attached)
isab0: <PCI-ISA bridge> port 0x2000-0x20ff at device 3.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory, RAM> at device 3.1 (no driver attached)
pci0: <memory, RAM> at device 3.3 (no driver attached)
pci0: <memory, RAM> at device 3.4 (no driver attached)
pci0: <processor> at device 3.5 (no driver attached)
ohci0: <nVidia nForce MCP79 USB Controller> mem 0xd3388000-0xd3388fff irq 11 at device 4.0 on pci0
usbus0 on ohci0
ehci0: <NVIDIA nForce MCP79 USB 2.0 controller> mem 0xd3389200-0xd33892ff irq 10 at device 4.1 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
ohci1: <nVidia nForce MCP79 USB Controller> mem 0xd3387000-0xd3387fff irq 7 at device 6.0 on pci0
usbus2 on ohci1
ehci1: <NVIDIA nForce MCP79 USB 2.0 controller> mem 0xd3389100-0xd33891ff irq 5 at device 6.1 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci1
hdac0: <NVIDIA MCP79 HDA Controller> mem 0xd3380000-0xd3383fff irq 15 at device 8.0 on pci0
pcib1: <PCI-PCI bridge> at device 9.0 on pci0
pci1: <PCI bus> on pcib1
nfe0: <NVIDIA nForce MCP79 Networking Adapter> port 0x21e0-0x21e7 mem 0xd3386000-0xd3386fff,0xd3389000-0xd33890ff,0xd3389300-0xd338930f irq 14 at device 10.0 on pci0
miibus0: <MII bus> on nfe0
rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
nfe0: Ethernet address: 34:15:9e:31:63:da
ahci0: <NVIDIA MCP79 AHCI SATA controller> port 0x21d8-0x21df,0x21ec-0x21ef,0x21d0-0x21d7,0x21e8-0x21eb,0x21c0-0x21cf mem 0xd3384000-0xd3385fff irq 11 at device 11.0 on pci0
ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier supported
ahci0: quirks=0x200<NOAA>
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
pcib2: <PCI-PCI bridge> at device 16.0 on pci0
pci2: <PCI bus> on pcib2
vgapci0: <VGA-compatible display> port 0x1000-0x107f mem 0xd2000000-0xd2ffffff,0xc0000000-0xcfffffff,0xd0000000-0xd1ffffff irq 11 at device 0.0 on pci2
nvidia0: <GeForce 9400M> on vgapci0
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: Boot video device
pcib3: <PCI-PCI bridge> at device 21.0 on pci0
pci3: <PCI bus> on pcib3
pci3: <network> at device 0.0 (no driver attached)
cpu0 on motherboard
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 617482306004823
device_attach: est0 attach returned 6
pmtimer0 on isa0
attimer0: <AT timer> at port 0x40-0x5f irq 0 pnpid PNP0100 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
orm0: <ISA Option ROM> at iomem 0xc0000-0xce7ff pnpid ORM0000 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ata0: <ATA channel> at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
ata1: <ATA channel> at port 0x170-0x177,0x376 irq 15 on isa0
atrtc0: <AT realtime clock> at port 0x70 irq 8 on isa0
Event timer "RTC" frequency 32768 Hz quality 0
ppc0: parallel port not found.
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 617482306004823
device_attach: est0 attach returned 6
Timecounters tick every 1.000 msec
hdacc0: <Cirrus Logic CS4206 HDA CODEC> at cad 0 on hdac0
hdaa0: <Cirrus Logic CS4206 Audio Function Group> at nid 1 on hdacc0
pcm0: <Cirrus Logic CS4206 (Internal Analog 3.1/2.0)> at nid 11,10 and 13 on hdaa0
pcm1: <Cirrus Logic CS4206 (Rear Analog Headphones)> at nid 9 on hdaa0
pcm2: <Cirrus Logic CS4206 (Rear Digital)> at nid 16 on hdaa0
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 480Mbps High Speed USB v2.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ugen0.1: <nVidia> at usbus0
uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen3.1: <nVidia> at usbus3
uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen2.1: <nVidia> at usbus2
uhub2: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen1.1: <nVidia> at usbus1
uhub3: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <Samsung SSD 840 EVO 500GB EXT0BB6Q> ATA-9 SATA 3.x device
ada0: Serial Number S1DHNSBF638894T
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
cd0 at ahcich1 bus 0 scbus1 target 0 lun 0
cd0: <MATSHITA DVD-R   UJ-898 HD09> Removable CD-ROM SCSI-0 device 
cd0: Serial Number UK25 112X5M
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray open
random: unblocking device.
Timecounter "TSC-low" frequency 1127694745 Hz quality 1000
WARNING: WITNESS option enabled, expect reduced performance.
Root mount waiting for: usbus3 usbus2 usbus1 usbus0
uhub2: 5 ports with 5 removable, self powered
uhub0: 7 ports with 7 removable, self powered
Root mount waiting for: usbus3 usbus1
uhub1: 5 ports with 5 removable, self powered
Root mount waiting for: usbus3 usbus1
uhub3: 7 ports with 7 removable, self powered
Root mount waiting for: usbus3 usbus1
ugen2.2: <Apple Inc.> at usbus2
uhub4: <Apple Inc. BRCM2070 Hub, class 9/0, rev 2.00/1.00, addr 2> on usbus2
ugen3.2: <vendor 0x0bda> at usbus3
ugen1.2: <Apple Inc.> at usbus1
uhub4: 3 ports with 0 removable, self powered
Trying to mount root from ufs:/dev/ada0s1a [rw]...
lock order reversal:
 1st 0xc8b317f8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2137
 2nd 0xe3d87fd8 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:262
 3rd 0xc8f9c6dc ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2137
KDB: stack backtrace:
db_trace_self_wrapper(c118ebb4,a,ed2412a0,c0af79e0,c8cf22f8,...) at db_trace_self_wrapper+0x2d/frame 0xed241260
kdb_backtrace(c119281a,c8f9c6dc,c1179712,c85a3a20,c119be80,...) at kdb_backtrace+0x30/frame 0xed2412c4
witness_checkorder(c8f9c6dc,9,c119be80,859,c8f9c6fc,...) at witness_checkorder+0xd04/frame 0xed241310
__lockmgr_args(c8f9c6dc,80100,c8f9c6fc,0,0,...) at __lockmgr_args+0x8f3/frame 0xed2413f0
ffs_lock(ed241470,c1198f69,c859c110,c85a1b40,c859c110,...) at ffs_lock+0x87/frame 0xed24142c
VOP_LOCK1_APV(c1425f7c,ed241470,234,3f7,c143b768,...) at VOP_LOCK1_APV+0x10a/frame 0xed241458
_vn_lock(c8f9c6a8,80100,c119be80,859,c119b0bd,...) at _vn_lock+0xa6/frame 0xed241498
vget(c8f9c6a8,80100,c8d08c40,57,0,...) at vget+0x74/frame 0xed2414d0
vfs_hash_get(c8d38d20,71e3d5,80000,c8d08c40,ed2415d0,...) at vfs_hash_get+0xfc/frame 0xed2414fc
ffs_vgetf(c8d38d20,71e3d5,80000,ed2415d0,1,...) at ffs_vgetf+0x44/frame 0xed241558
softdep_sync_buf(c8b317c4,e3d87f80,1,0,0,...) at softdep_sync_buf+0xac7/frame 0xed2415e8
ffs_syncvnode(c8b317c4,1,0,c1400390,0,...) at ffs_syncvnode+0x2dd/frame 0xed241640
ffs_truncate(c8b317c4,200,0,880,c8702d80,...) at ffs_truncate+0x6eb/frame 0xed2417f0
ufs_direnter(c8b317c4,c8f9c6a8,ed2418b8,ed241bcc,0,...) at ufs_direnter+0x79e/frame 0xed241870
ufs_makeinode(ed241bb8,ed241bcc) at ufs_makeinode+0x534/frame 0xed2419f0
ufs_create(ed241ad8,669,c8d38d30,2,c8d38d74,...) at ufs_create+0x2f/frame 0xed241a04
VOP_CREATE_APV(c1425f7c,ed241ad8,ed241bcc,ed241a68,c0af79e0,...) at VOP_CREATE_APV+0xfe/frame 0xed241a30
vn_open_cred(ed241b70,ed241bfc,1a4,0,c8702d80,c8d5f738) at vn_open_cred+0x2f0/frame 0xed241b00
vn_open(ed241b70,ed241bfc,1a4,c8d5f738,2882a610,...) at vn_open+0x3d/frame 0xed241b28
kern_openat(c8d08c40,ffffff9c,2882a610,0,601,1b6) at kern_openat+0x310/frame 0xed241c1c
sys_open(c8d08c40,ed241cc8,c134fe9a,e8,5,...) at sys_open+0x39/frame 0xed241c40
syscall(ed241d08) at syscall+0x30c/frame 0xed241cfc
Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xed241cfc
--- syscall (5, FreeBSD ELF32, sys_open), eip = 0x28210923, esp = 0xbfbfddf4, ebp = 0xbfbfdea8 ---
Comment 2 dksayers 2015-10-12 22:29:26 UTC
I am also getting this problem on each reboot.
FreeBSD 11.0-CURRENT #1 r289163
------
Trying to mount root from ufs:/dev/gpt/ssdroot [rw]...
lock order reversal:
 1st 0xfffff8001a89c068 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2231
 2nd 0xfffffe07c6843280 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:263
 3rd 0xfffff80018f1fb78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2231
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085e0b8d40
witness_checkorder() at witness_checkorder+0xe79/frame 0xfffffe085e0b8dc0
__lockmgr_args() at __lockmgr_args+0xd3b/frame 0xfffffe085e0b8e70
ffs_lock() at ffs_lock+0xa6/frame 0xfffffe085e0b8ec0
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x100/frame 0xfffffe085e0b8ef0
_vn_lock() at _vn_lock+0x9a/frame 0xfffffe085e0b8f60
vget() at vget+0x63/frame 0xfffffe085e0b8fb0
vfs_hash_get() at vfs_hash_get+0xcc/frame 0xfffffe085e0b9000
ffs_vgetf() at ffs_vgetf+0x40/frame 0xfffffe085e0b9090
softdep_sync_buf() at softdep_sync_buf+0xad1/frame 0xfffffe085e0b9170
ffs_syncvnode() at ffs_syncvnode+0x256/frame 0xfffffe085e0b91f0
ffs_truncate() at ffs_truncate+0x6cd/frame 0xfffffe085e0b93e0
ufs_direnter() at ufs_direnter+0x7bb/frame 0xfffffe085e0b94b0
ufs_makeinode() at ufs_makeinode+0x5f3/frame 0xfffffe085e0b9670
ufs_create() at ufs_create+0x2d/frame 0xfffffe085e0b9690
VOP_CREATE_APV() at VOP_CREATE_APV+0xf1/frame 0xfffffe085e0b96c0
vn_open_cred() at vn_open_cred+0x2f8/frame 0xfffffe085e0b9830
kern_openat() at kern_openat+0x25c/frame 0xfffffe085e0b99a0
amd64_syscall() at amd64_syscall+0x2de/frame 0xfffffe085e0b9ab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085e0b9ab0
--- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800b5cb0a, rsp = 0x7fffffffd538, rbp = 0x7fffffffd620 ---
Comment 3 Marcus von Appen freebsd_committer freebsd_triage 2016-05-28 20:13:07 UTC
And a third one. It often occurs on heavy disk access (e.g. running synth upgrade-system)

FreeBSD ... 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r300409M: Wed May 25 22:39:21 CEST 2016 ...:/usr/obj/usr/src/sys/GENERIC  amd64

lock order reversal:
 1st 0xfffffe03d06bc900 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3512
 2nd 0xfffff800142ac400 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:281
stack backtrace:
#0 0xffffffff80a9b7c0 at witness_debugger+0x70
#1 0xffffffff80a9b6b4 at witness_checkorder+0xe54
#2 0xffffffff80a452c2 at _sx_xlock+0x72
#3 0xffffffff80cfd74e at ufsdirhash_add+0x3e
#4 0xffffffff80d004fa at ufs_direnter+0x4da
#5 0xffffffff80d07b80 at ufs_rename+0xf60
#6 0xffffffff81002420 at VOP_RENAME_APV+0x100
#7 0xffffffff80b08928 at kern_renameat+0x4a8
#8 0xffffffff80ea6a2b at amd64_syscall+0x2db
#9 0xffffffff80e86d9b at Xfast_syscall+0xfb
lock order reversal:
 1st 0xfffff80014d35240 ufs (ufs) @ /usr/src/sys/ufs/ufs/ufs_vnops.c:1157
 2nd 0xfffffe03d06ce790 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:263
 3rd 0xfffff80014f7c9a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2498
stack backtrace:
#0 0xffffffff80a9b7c0 at witness_debugger+0x70
#1 0xffffffff80a9b6b4 at witness_checkorder+0xe54
#2 0xffffffff80a155c6 at __lockmgr_args+0x4d6
#3 0xffffffff80cf8466 at ffs_lock+0xa6
#4 0xffffffff81002f30 at VOP_LOCK1_APV+0x100
#5 0xffffffff80b0b73a at _vn_lock+0x9a
#6 0xffffffff80afbbb3 at vget+0x63
#7 0xffffffff80aee68c at vfs_hash_get+0xcc
#8 0xffffffff80cf4200 at ffs_vgetf+0x40
#9 0xffffffff80cebcea at softdep_sync_buf+0xb3a
#10 0xffffffff80cf9056 at ffs_syncvnode+0x256
#11 0xffffffff80cd05e7 at ffs_truncate+0x8a7
#12 0xffffffff80d08030 at ufs_rename+0x1410
#13 0xffffffff81002420 at VOP_RENAME_APV+0x100
#14 0xffffffff80b08928 at kern_renameat+0x4a8
#15 0xffffffff80ea6a2b at amd64_syscall+0x2db
#16 0xffffffff80e86d9b at Xfast_syscall+0xfb
Comment 4 vidwer+fbsdbugs 2018-02-09 21:37:12 UTC
On 12.0-CURRENT r329060. booting from freebsd-ufs+GPT:

lock order reversal:
 1st 0xfffffe0001651cc0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3566
 2nd 0xfffff800b8b07400 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:289
stack backtrace:
#0 0xffffffff80b2dbf3 at witness_debugger+0x73
#1 0xffffffff80b2da74 at witness_checkorder+0xe34
#2 0xffffffff80ad1ac8 at _sx_xlock+0x68
#3 0xffffffff80dca40d at ufsdirhash_add+0x3d
#4 0xffffffff80dcd2df at ufs_direnter+0x49f
#5 0xffffffff80dd4a4e at ufs_rename+0x111e
#6 0xffffffff810fcf41 at VOP_RENAME_APV+0xe1
#7 0xffffffff80ba5f8a at kern_renameat+0x3ca
#8 0xffffffff822215d9 at filemon_wrapper_rename+0x19
#9 0xffffffff80f7ec2b at amd64_syscall+0x79b
#10 0xffffffff80f59ffd at fast_syscall_common+0x101
Comment 5 Mark Johnston freebsd_committer freebsd_triage 2019-07-30 17:11:13 UTC
These reports are false positives.  As of r350450 we no longer generate the reports listed here.