Bug 224176 - Lock Order Reversals somewhere in VFS/UFS
Summary: Lock Order Reversals somewhere in VFS/UFS
Status: Closed Not A Bug
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 11.1-STABLE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-fs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-08 14:11 UTC by lankfordandrew
Modified: 2017-12-19 18:38 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description lankfordandrew 2017-12-08 14:11:42 UTC
OK, a couple of them.  One during an svnlite update of /usr/src and another while I'm trying to rebuild firefox (see below).  Let me know what else you need.  My kernel config file is at the very end.

$ mount
/dev/ada0s3a on / (ufs, local, journaled soft-updates)
devfs on /dev (devfs, local, multilabel)
/dev/ada0s2 on /D (msdosfs, local)
tmpfs on /var/tmp (tmpfs, local)

Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-STABLE #3 r326622: Wed Dec  6 19:59:18 EST 2017
    root@bogushost2:/usr/obj/usr/src/sys/ARL11MIN amd64
FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn)
WARNING: WITNESS option enabled, expect reduced performance.
VT(vga): resolution 640x480
CPU: Intel(R) Core(TM)2 Duo CPU     T7500  @ 2.20GHz (2194.55-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x6fb  Family=0x6  Model=0xf  Stepping=11
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xe3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  VT-x: (disabled in BIOS) HLT,PAUSE
  TSC: P-state invariant, performance statistics
real memory  = 2147483648 (2048 MB)
avail memory = 2043154432 (1948 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table: <DELL   M08    >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
random: unblocking device.
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.0> irqs 0-23 on motherboard
SMP: AP CPU #1 Launched!
Timecounter "TSC-low" frequency 1097277076 Hz quality 1000
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff8085cc50, 0) error 19
nexus0
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <DELL M08    > on motherboard
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 450
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71,0x72-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 2 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0xeff8-0xefff mem 0xfea00000-0xfeafffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
agp0: <Intel GM965 SVGA controller> on vgapci0
agp0: aperture size is 256M, detected 7676k stolen memory
vgapci0: Boot video device
vgapci1: <VGA-compatible display> mem 0xfeb00000-0xfebfffff at device 2.1 on pci0
uhci0: <Intel 82801H (ICH8) USB controller USB-D> port 0x6f20-0x6f3f irq 20 at device 26.0 on pci0
uhci0: LegSup = 0x2f00
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
uhci1: <Intel 82801H (ICH8) USB controller USB-E> port 0x6f00-0x6f1f irq 21 at device 26.1 on pci0
uhci1: LegSup = 0x2f00
usbus1 on uhci1
usbus1: 12Mbps Full Speed USB v1.0
ehci0: <Intel 82801H (ICH8) USB 2.0 controller USB2-B> mem 0xfed1c400-0xfed1c7ff irq 22 at device 26.7 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci0
usbus2: 480Mbps High Speed USB v2.0
hdac0: <Intel 82801H HDA Controller> mem 0xfe9fc000-0xfe9fffff irq 21 at device 27.0 on pci0
pcib1: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 28.1 on pci0
pci2: <ACPI PCI bus> on pcib2
wpi0: <Intel(R) PRO/Wireless 3945ABG> mem 0xfe8ff000-0xfe8fffff irq 17 at device 0.0 on pci2
pcib3: <ACPI PCI-PCI bridge> at device 28.3 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 28.5 on pci0
pci4: <ACPI PCI bus> on pcib4
bge0: <Broadcom BCM5906 A2, ASIC rev. 0x00c002> mem 0xfe5f0000-0xfe5fffff irq 17 at device 0.0 on pci4
bge0: CHIP ID 0x0000c002; ASIC REV 0x0c; CHIP REV 0xc0; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM5906 10/100baseTX media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow
bge0: Ethernet address: 00:1e:c9:01:be:70
uhci2: <Intel 82801H (ICH8) USB controller USB-A> port 0x6f80-0x6f9f irq 20 at device 29.0 on pci0
uhci2: LegSup = 0x2f00
usbus3 on uhci2
usbus3: 12Mbps Full Speed USB v1.0
uhci3: <Intel 82801H (ICH8) USB controller USB-B> port 0x6f60-0x6f7f irq 21 at device 29.1 on pci0
uhci3: LegSup = 0x2f00
usbus4 on uhci3
usbus4: 12Mbps Full Speed USB v1.0
uhci4: <Intel 82801H (ICH8) USB controller USB-C> port 0x6f40-0x6f5f irq 22 at device 29.2 on pci0
uhci4: LegSup = 0x2f00
usbus5 on uhci4
usbus5: 12Mbps Full Speed USB v1.0
ehci1: <Intel 82801H (ICH8) USB 2.0 controller USB2-A> mem 0xfed1c000-0xfed1c3ff irq 20 at device 29.7 on pci0
usbus6: EHCI version 1.0
usbus6 on ehci1
usbus6: 480Mbps High Speed USB v2.0
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pci5: <serial bus, FireWire> at device 1.0 (no driver attached)
sdhci_pci0: <RICOH R5C822 SD> mem 0xfe4ff400-0xfe4ff4ff irq 18 at device 1.1 on pci5
sdhci_pci0: 1 slot(s) allocated
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <mass storage, ATA> at device 31.1 (no driver attached)
ahci0: <Intel ICH8M AHCI SATA controller> port 0x6eb0-0x6eb7,0x6eb8-0x6ebb,0x6ec0-0x6ec7,0x6ec8-0x6ecb,0x6ee0-0x6eff mem 0xfe9fb800-0xfe9fbfff irq 17 at device 31.2 on pci0
ahci0: AHCI v1.10 with 3 3Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
acpi_lid0: <Control Method Lid Switch> on acpi0
acpi_button0: <Power Button> on acpi0
acpi_button1: <Sleep Button> on acpi0
acpi_acad0: <AC Adapter> on acpi0
battery0: <ACPI Control Method Battery> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64,0x62,0x66 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model GlidePoint, device ID 0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcefff,0xcf000-0xcffff on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
Timecounters tick every 1.000 msec
ugen6.1: <Intel EHCI root HUB> at usbus6
ugen5.1: <Intel UHCI root HUB> at usbus5
ugen4.1: <Intel UHCI root HUB> at usbus4
ugen2.1: <Intel EHCI root HUB> at usbus2
ugen3.1: <Intel UHCI root HUB> at usbus3
ugen0.1: <Intel UHCI root HUB> at usbus0
ugen1.1: <Intel UHCI root HUB> at usbus1
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus6
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
hdacc0: <Sigmatel STAC9228X HDA CODEC> at cad 0 on hdac0
hdaa0: <Sigmatel STAC9228X Audio Function Group> at nid 1ses0 at ahciem0 bus 0 scbus2 target 0 lun 0
 on hdacc0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <FUJITSU MHY2250BH 0085000B> ATA8-ACS SATA 1.x device
ada0: Serial Number K432T8127RSH
ada0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada0: Command Queueing enabled
pcm0: ada0: 238475MB (488397168 512 byte sectors)
<Sigmatel STAC9228X (Analog 2.0+HP/2.0)> at nid 13,10 and 14 on hdaa0
pcm1: <Sigmatel STAC9228X (Analog)> at nid 15 and 19 on hdaa0
pcm2: <Sigmatel STAC9228X (Left Digital)> at nid 33 on hdaa0
hdacc1: <Conexant (0x2c06) HDA CODEC> at cad 1 on hdac0
unknown: <Conexant (0x2c06) HDA CODEC Modem Function Group> at nid 2 on hdacc1 (no driver attached)
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/ada0s3a [rw]...
uhub4: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub6: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
uhub2: 4 ports with 4 removable, self powered
[3] uhub0: 6 ports with 6 removable, self powered
[6] wlan0: Ethernet address: 00:1c:bf:c9:e9:79
[8] wlan0: link state changed to UP
[18] info: [drm] Initialized drm 1.1.0 20060810
[18] drmn0: <Intel i965GM> on vgapci0
[18] intel_iicbb0 on drmn0
[18] iicbus0: <Philips I2C bus> on iicbb0 addr 0xff
[19] iic0: <I2C generic I/O> on iicbus0
[19] iicbus1: <Philips I2C bus> on intel_gmbus0
[19] iic1: <I2C generic I/O> on iicbus1
[19] intel_iicbb1 on drmn0
[19] iicbus2: <Philips I2C bus> on iicbb1 addr 0xff
[19] iic2: <I2C generic I/O> on iicbus2
[19] iicbus3: <Philips I2C bus> on intel_gmbus1
[19] iic3: <I2C generic I/O> on iicbus3
[19] intel_iicbb2 on drmn0
[19] iicbus4: <Philips I2C bus> on iicbb2 addr 0xff
[19] iic4: <I2C generic I/O> on iicbus4
[19] iicbus5: <Philips I2C bus> on intel_gmbus2
[19] iic5: <I2C generic I/O> on iicbus5
[19] intel_iicbb3 on drmn0
[19] iicbus6: <Philips I2C bus> on iicbb3 addr 0xff
[19] iic6: <I2C generic I/O> on iicbus6
[19] iicbus7: <Philips I2C bus> on intel_gmbus3
[19] iic7: <I2C generic I/O> on iicbus7
[19] intel_iicbb4 on drmn0
[19] iicbus8: <Philips I2C bus> on iicbb4 addr 0xff
[19] iic8: <I2C generic I/O> on iicbus8
[19] iicbus9: <Philips I2C bus> on intel_gmbus4
[19] iic9: <I2C generic I/O> on iicbus9
[19] intel_iicbb5 on drmn0
[19] iicbus10: <Philips I2C bus> on iicbb5 addr 0xff
[19] iic10: <I2C generic I/O> on iicbus10
[19] iicbus11: <Philips I2C bus> on intel_gmbus5
[19] iic11: <I2C generic I/O> on iicbus11
[19] info: [drm] MSI enabled 1 message(s)
[19] info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
[19] info: [drm] Driver supports precise vblank timestamp query.
[19] drmn0: taking over the fictitious range 0xe0000000-0xf0000000
[19] info: [drm] initialized overlay support
[19] info: [drm] Connector LVDS-1: get mode from tunables:
[19] info: [drm]   - kern.vt.fb.modes.LVDS-1
[19] info: [drm]   - kern.vt.fb.default_mode
[19] info: [drm] Connector VGA-1: get mode from tunables:
[19] info: [drm]   - kern.vt.fb.modes.VGA-1
[19] info: [drm]   - kern.vt.fb.default_mode
[19] info: [drm] Connector SVIDEO-1: get mode from tunables:
[19] info: [drm]   - kern.vt.fb.modes.SVIDEO-1
[19] info: [drm]   - kern.vt.fb.default_mode
[19] fbd0 on drmn0
[19] VT: Replacing driver "vga" with new "fb".
[19] info: [drm] Initialized i915 1.6.0 20080730 for drmn0 on minor 0
[241] lock order reversal:
[241]  1st 0xfffffe007ba4c0e0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3550
[241]  2nd 0xfffff800438b6c00 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:281
[241] stack backtrace:
[241] #0 0xffffffff80580bf0 at witness_debugger+0x70
[241] #1 0xffffffff80580a72 at witness_checkorder+0xe02
[241] #2 0xffffffff80527908 at _sx_xlock+0x68
[241] #3 0xffffffff807d7ba3 at ufsdirhash_newblk+0x33
[241] #4 0xffffffff807da1dd at ufs_direnter+0x5ad
[241] #5 0xffffffff807e1223 at ufs_rename+0x10d3
[241] #6 0xffffffff808b63c8 at VOP_RENAME_APV+0xe8
[241] #7 0xffffffff805f15da at kern_renameat+0x3ca
[241] #8 0xffffffff8083fd08 at amd64_syscall+0x798
[241] #9 0xffffffff8081ea6b at Xfast_syscall+0xfb
[851] lock order reversal:
[851]  1st 0xfffff8000b72b5f0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2537
[851]  2nd 0xfffffe007ba325a0 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:277
[851]  3rd 0xfffff8000b72b418 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2537
[851] stack backtrace:
[851] #0 0xffffffff80580bf0 at witness_debugger+0x70
[851] #1 0xffffffff80580a72 at witness_checkorder+0xe02
[851] #2 0xffffffff804fcd83 at __lockmgr_args+0x883
[851] #3 0xffffffff807d1ea5 at ffs_lock+0xa5
[851] #4 0xffffffff808b6f30 at VOP_LOCK1_APV+0xe0
[851] #5 0xffffffff805f4546 at _vn_lock+0x66
[851] #6 0xffffffff805e4dc2 at vget+0x82
[851] #7 0xffffffff805d6fb1 at vfs_hash_get+0xd1
[851] #8 0xffffffff807cdb4e at ffs_vgetf+0x3e
[851] #9 0xffffffff807c3da9 at softdep_sync_buf+0x7d9
[851] #10 0xffffffff807d2dc1 at ffs_syncvnode+0x321
[851] #11 0xffffffff807c2dc7 at softdep_fsync+0x4b7
[851] #12 0xffffffff807d1d7c at ffs_fsync+0x7c
[851] #13 0xffffffff808b5f80 at VOP_FSYNC_APV+0xe0
[851] #14 0xffffffff805f112c at kern_fsync+0x1bc
[851] #15 0xffffffff8083fd08 at amd64_syscall+0x798
[851] #16 0xffffffff8081ea6b at Xfast_syscall+0xfb
$ 


$ cat /usr/src/sys/amd64/conf/ARL11MIN 
#
# MINIMAL -- Mostly Minimal kernel configuration file for FreeBSD/amd64
#
# Many definitions of minimal are possible. The one this file follows is
# GENERIC, minus all functionality that can be replaced by loading kernel
# modules.
#
# Exceptions:
# o While UFS is buildable as a module, the current module lacks
#   some features (ACL, GJOURNAL) that GENERIC includes.
# o acpi as a module has been reported flakey and not well tested, so
#   is included in the kernel.
# o random is included due to uncertaty...
# o Many networking things are included
#
# For now, please run changes to these list past imp@freebsd.org
#
# For more information on this file, please read the config(5) manual page,
# and/or the handbook section on Kernel Configuration Files:
#
#    http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html
#
# The handbook is also available locally in /usr/share/doc/handbook
# if you've installed the doc distribution, otherwise always see the
# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the
# latest information.
#
# An exhaustive list of options and more detailed explanations of the
# device lines is also present in the ../../conf/NOTES and NOTES files.
# If you are in doubt as to the purpose or necessity of a line, check first
# in NOTES.
#
# $FreeBSD: stable/11/sys/amd64/conf/MINIMAL 318763 2017-05-24 00:00:55Z jhb $

cpu		HAMMER
ident		ARL11MIN

makeoptions	DEBUG=-g		# Build kernel with gdb(1) debug symbols
makeoptions	WITH_CTF=1		# Run ctfconvert(1) for DTrace support

options 	SCHED_ULE		# ULE scheduler
options 	PREEMPTION		# Enable kernel thread preemption
options 	INET			# InterNETworking
options 	INET6			# IPv6 communications protocols
options 	IPSEC			# IP (v4/v6) security
options 	IPSEC_SUPPORT		# Allow kldload of ipsec and tcpmd5
options 	TCP_OFFLOAD		# TCP offload
options 	SCTP			# Stream Control Transmission Protocol
options 	FFS			# Berkeley Fast Filesystem
options 	SOFTUPDATES		# Enable FFS soft updates support
options 	UFS_ACL			# Support for access control lists
options 	UFS_DIRHASH		# Improve performance on big directories
options 	UFS_GJOURNAL		# Enable gjournal-based UFS journaling
options 	MSDOSFS			# MSDOS Filesystem
options 	PSEUDOFS		# Pseudo-filesystem framework
options 	CD9660			# ISO 9660 Filesystem
options 	PROCFS			# Process filesystem (requires PSEUDOFS)
options 	MD_ROOT			# MD is a potential root device
options 	COMPAT_FREEBSD32	# Compatible with i386 binaries
options 	COMPAT_FREEBSD9		# Compatible with FreeBSD9
options 	COMPAT_FREEBSD10	# Compatible with FreeBSD10
options 	SCSI_DELAY=1000		# Delay (in ms) before probing SCSI
options 	KTRACE			# ktrace(1) support
options 	STACK			# stack(9) support
options 	SYSVSHM			# SYSV-style shared memory
options 	SYSVMSG			# SYSV-style message queues
options 	SYSVSEM			# SYSV-style semaphores
options 	_KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options 	PRINTF_BUFR_SIZE=128	# Prevent printf output being interspersed.
options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
options 	AUDIT			# Security event auditing
options 	CAPABILITY_MODE		# Capsicum capability mode
options 	CAPABILITIES		# Capsicum capabilities
options 	MAC			# TrustedBSD MAC Framework
options 	KDTRACE_FRAME		# Ensure frames are compiled in
options 	KDTRACE_HOOKS		# Kernel DTrace hooks
options 	DDB_CTF			# Kernel ELF linker loads CTF data

# Debugging support.  Always need this:
options 	KDB			# Enable kernel debugger support.
options 	KDB_TRACE		# Print a stack trace for a panic.
# For full debugger support use (turn off in stable branch):
options 	DDB			# Support DDB.
options 	GDB			# Support remote GDB.
options 	DEADLKRES		# Enable the deadlock resolver
options 	INVARIANTS		# Enable calls of extra sanity checking
options 	INVARIANT_SUPPORT	# Extra sanity checks of internal structures, required by INVARIANTS
options 	WITNESS			# Enable checks to detect deadlocks and cycles
options 	WITNESS_SKIPSPIN	# Don't run witness on spinlocks for speed
options 	MALLOC_DEBUG_MAXZONES=8	# Separate malloc(9) zones

# Make an SMP-capable kernel by default
options 	SMP			# Symmetric MultiProcessor Kernel
options 	DEVICE_NUMA		# I/O Device Affinity
options 	EARLY_AP_STARTUP

# CPU frequency control
device		cpufreq

# ATA controllers
device		ahci			# AHCI-compatible SATA controllers

# Bus support.
device		acpi
options 	ACPI_DMAR
device		pci


# ATA/SCSI peripherals
device		scbus			# SCSI bus (required for ATA/SCSI)
device		da			# Direct Access (disks)
device		cd			# CD
device		pass			# Passthrough device (direct ATA/SCSI access)
device		ses			# Enclosure Services (SES and SAF-TE)
#device		ctl			# CAM Target Layer

# atkbdc0 controls both the keyboard and the PS/2 mouse
device		atkbdc			# AT keyboard controller
device		atkbd			# AT keyboard
device		psm			# PS/2 mouse

device		kbdmux			# keyboard multiplexer

device		vga			# VGA video card driver
options 	VESA			# Add support for VESA BIOS Extensions (VBE)

device		splash			# Splash screen and screen saver support

# syscons is the default console driver, resembling an SCO console
device		sc
options 	SC_PIXEL_MODE		# add support for the raster text mode

# vt is the new video console driver
device		vt
device		vt_vga
device		vt_efifb

device		agp			# support several AGP chipsets


# PCCARD (PCMCIA) support
# PCMCIA and cardbus bridge support
device		cbb			# cardbus (yenta) bridge
device		pccard			# PC Card (16-bit) bus
device		cardbus			# CardBus (32-bit) bus

# Serial (COM) ports
device		uart			# Generic UART driver
# USB support
options 	USB_DEBUG		# enable debug msgs
device		uhci			# UHCI PCI->USB interface
device		ohci			# OHCI PCI->USB interface
device		ehci			# EHCI PCI->USB interface (USB 2.0)
device		xhci			# XHCI PCI->USB interface (USB 3.0)
device		usb			# USB Bus (required)
device		ukbd			# Keyboard
device		umass			# Disks/Mass storage - Requires scbus and da

# Sound support
device		sound			# Generic sound driver (required)
device		snd_hda			# Intel High Definition Audio

# MMC/SD
device		mmc			# MMC/SD bus
device		mmcsd			# MMC/SD memory card
device		sdhci			# Generic PCI SD Host Controller

# Pseudo devices.
device		loop			# Network loopback
device		random			# Entropy device
device		rdrand_rng		# Intel Bull Mountain RNG
device		ether			# Ethernet support
device		vlan			# 802.1Q VLAN support
device		tun			# Packet tunnel.
device		gif			# IPv6 and IPv4 tunneling
device		firmware		# firmware assist module

# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device		bpf			# Berkeley packet filter

# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device		miibus			# MII bus support
device		bge			# Broadcom BCM570xx Gigabit Ethernet

# Wireless NIC cards
device		wlan			# 802.11 support
options 	IEEE80211_DEBUG		# enable debug msgs
options 	IEEE80211_AMPDU_AGE	# age frames in AMPDU reorder q's
options 	IEEE80211_SUPPORT_MESH	# enable 802.11s draft support
device		wlan_wep		# 802.11 WEP support
device		wlan_ccmp		# 802.11 CCMP support
device		wlan_tkip		# 802.11 TKIP support
device		wlan_amrr		# AMRR transmit rate control algorithm
device		wpi			# Intel 3945ABG wireless NICs.

# Xen HVM Guest Optimizations
# NOTE: XENHVM depends on xenpci.  They must be added or removed together.
options 	XENHVM			# Xen HVM kernel infrastructure
device		xenpci			# Xen HVM Hypervisor services driver

# Netmap provides direct access to TX/RX rings on supported NICs
device		netmap			# netmap(4) support

# The crypto framework is required by IPSEC
device		crypto			# Required by IPSEC
$
Comment 1 lankfordandrew 2017-12-08 14:32:06 UTC
Just in case you need this:

$ kldstat
Id Refs Address            Size     Name
 1   28 0xffffffff80200000 1070d08  kernel
 2    1 0xffffffff81272000 4d80     coretemp.ko
 3    1 0xffffffff81421000 100df    tmpfs.ko
 4    1 0xffffffff81432000 9fa      blank_saver.ko
 5    1 0xffffffff81433000 991bf    i915kms.ko
 6    1 0xffffffff814cd000 56589    drm2.ko
 7    4 0xffffffff81524000 299f     iicbus.ko
 8    1 0xffffffff81527000 1cd7     iic.ko
 9    1 0xffffffff81529000 1e66     iicbb.ko
$
Comment 2 lankfordandrew 2017-12-08 22:39:50 UTC
Some more after a reboot.  I'm probably getting one or several right at shutdown , but I'm unsure if I can capture them.

[70] lock order reversal:
[70]  1st 0xfffffe007b886f80 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3550
[70]  2nd 0xfffff80002aa9400 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:281
[70] stack backtrace:
[70] #0 0xffffffff80580bf0 at witness_debugger+0x70
[70] #1 0xffffffff80580a72 at witness_checkorder+0xe02
[70] #2 0xffffffff80527908 at _sx_xlock+0x68
[70] #3 0xffffffff807d75b7 at ufsdirhash_remove+0x37
[70] #4 0xffffffff807da779 at ufs_dirremove+0x129
[70] #5 0xffffffff807dfe95 at ufs_remove+0x75
[70] #6 0xffffffff808b60e0 at VOP_REMOVE_APV+0xe0
[70] #7 0xffffffff805ee708 at kern_unlinkat+0x1e8
[70] #8 0xffffffff8083fd08 at amd64_syscall+0x798
[70] #9 0xffffffff8081ea6b at Xfast_syscall+0xfb
[218] lock order reversal:
[218]  1st 0xfffff800638f89a0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2537
[218]  2nd 0xfffffe007b900a60 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:277
[218]  3rd 0xfffff800638f87c8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2537
[218] stack backtrace:
[218] #0 0xffffffff80580bf0 at witness_debugger+0x70
[218] #1 0xffffffff80580a72 at witness_checkorder+0xe02
[218] #2 0xffffffff804fcd83 at __lockmgr_args+0x883
[218] #3 0xffffffff807d1ea5 at ffs_lock+0xa5
[218] #4 0xffffffff808b6f30 at VOP_LOCK1_APV+0xe0
[218] #5 0xffffffff805f4546 at _vn_lock+0x66
[218] #6 0xffffffff805e4dc2 at vget+0x82
[218] #7 0xffffffff805d6fb1 at vfs_hash_get+0xd1
[218] #8 0xffffffff807cdb4e at ffs_vgetf+0x3e
[218] #9 0xffffffff807c3da9 at softdep_sync_buf+0x7d9
[218] #10 0xffffffff807d2dc1 at ffs_syncvnode+0x321
[218] #11 0xffffffff807c2dc7 at softdep_fsync+0x4b7
[218] #12 0xffffffff807d1d7c at ffs_fsync+0x7c
[218] #13 0xffffffff808b5f80 at VOP_FSYNC_APV+0xe0
[218] #14 0xffffffff805f112c at kern_fsync+0x1bc
[218] #15 0xffffffff8083fd08 at amd64_syscall+0x798
[218] #16 0xffffffff8081ea6b at Xfast_syscall+0xfb
Comment 3 Benjamin Kaduk freebsd_committer 2017-12-09 02:06:46 UTC
While the enthusiasm at reporting what appears to be a bug is appreciated, the reality seems to be that all of these LORs are "well-known" and presumed harmless (though, IIRC, not thoroughly analyzed and proven to be harmless).

http://sources.zabbadoz.net/freebsd/lor.html is no longer up-to-date, but is still a good place to get a handle on what has been seen before.

I'll close this bug, as I don't think there is anything actionable here.
Comment 4 lankfordandrew 2017-12-09 02:46:33 UTC
Okidoke.  Seem to be lots and lots of these LORs anyway.  But thanks for the link.
Comment 5 Wolfram Schneider freebsd_committer 2017-12-17 10:16:48 UTC
I must admit that these LOR kernel message looks scary if you see them for the first time.

But if these messages are mostly harmless, why doesn't we say this?

My proposed change request:

instead:

lock order reversal:
 1st 0xfffffe0000cbf2c0 bufwait (bufwait) @ /home/projects/freebsd/sys/kern/vfs_bio.c:3564
 2nd 0xfffff8001c445800 dirhash (dirhash) @ /home/projects/freebsd/sys/ufs/ufs/ufs_dirhash.c:289
stack backtrace:

we display

lock order reversal (mostly harmless, see https://www.freebsd.org/doc/handbook/freebsd-glossary.html#lor-glossary )
 1st 0xfffffe0000cbf2c0 bufwait (bufwait) @ /home/projects/freebsd/sys/kern/vfs_bio.c:3564
 2nd 0xfffff8001c445800 dirhash (dirhash) @ /home/projects/freebsd/sys/ufs/ufs/ufs_dirhash.c:289
stack backtrace:
Comment 6 Conrad Meyer freebsd_committer 2017-12-17 17:23:42 UTC
If we could identify the harmless ones accurately, we may as well not print them at all.
Comment 7 Andriy Gapon freebsd_committer 2017-12-17 21:36:59 UTC
(In reply to Wolfram Schneider from comment #5)
The idea is that those messages can be useful when you get and report a real problem (crash, hang, etc).  Reporting the messages themselves is almost never useful if you do not have a problem.
Comment 8 Wolfram Schneider freebsd_committer 2017-12-19 18:38:31 UTC
(In reply to Andriy Gapon from comment #7)

99.9% of these kernel warnings are false positives. We scare our users. We shouldn't.

I see two options to solve the issue:

1. "If we could identify the harmless ones accurately, we may as well not print them at all." (Conrad)

2. tell the users that they can ignore these warnings as long as the machine runs fine.

I prefer 1), but in the meantime we should do 2)