r253687 introduced a lock order reversal. It's a lock of a sleepable lock while a nonsleepable lock is still held. The problem is that lacp_attach creates some sysctl nodes, but further up the stack lagg_ioctl called LAGG_WLOCK() on the softc. Fix: Two possible solutions come to mind: 1) Move the sysctl node creation out of lacp_attach. Call it from lagg_ioctl after LAGG_WUNLOCK() 2) Turn the sysctl nodes into ioctls instead. That was proposed for a different reason at http://lists.freebsd.org/pipermail/svn-src-head/2013-July/049602.html . How-To-Repeat: ifconfig tap0 create ifconfig tap1 create ifconfig tap2 create ifconfig lagg0 create ifconfig lagg0 up laggproto lacp laggport tap0 laggport tap1 laggport tap2 192.0.0.2/24
Forgot to include the stack trace: lock order reversal: (sleepable after non-sleepable) 1st 0xfffff8000252ca08 if_lagg rmlock (if_lagg rmlock) @/usr/home/alans/freebsd/head/sys/modules/if_lagg/../../net/if_lagg.c:1040 2nd 0xffffffff814ef4e0 sysctl lock (sysctl lock) @/usr/home/alans/freebsd/head/sys/kern/kern_sysctl.c:474 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00977485b0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0097748660 witness_checkorder() at witness_checkorder+0xdc2/frame 0xfffffe00977486f0 _sx_xlock() at _sx_xlock+0x75/frame 0xfffffe0097748730 sysctl_add_oid() at sysctl_add_oid+0x4a/frame 0xfffffe0097748780 lacp_attach() at lacp_attach+0xf7/frame 0xfffffe00977487f0 lagg_lacp_attach() at lagg_lacp_attach+0x88/frame 0xfffffe0097748810 lagg_ioctl() at lagg_ioctl+0x98a/frame 0xfffffe00977488f0 in_control() at in_control+0x38e/frame 0xfffffe0097748970 ifioctl() at ifioctl+0xba2/frame 0xfffffe0097748a30 kern_ioctl() at kern_ioctl+0x22b/frame 0xfffffe0097748a90 sys_ioctl() at sys_ioctl+0x13c/frame 0xfffffe0097748ae0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe0097748bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0097748bf0 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x800fa045a, rsp =0x7fffffffe118, rbp = 0x7fffffffe1a0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive rm if_lagg rmlock (if_lagg rmlock) r = 0 (0xfffff8000252ca08) locked @/usr/home/alans/freebsd/head/sys/modules/if_lagg/../../net/if_lagg.c:1040 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0097748500 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00977485b0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe0097748670 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe00977486e0 malloc() at malloc+0x194/frame 0xfffffe0097748730 sysctl_add_oid() at sysctl_add_oid+0x11f/frame 0xfffffe0097748780 lacp_attach() at lacp_attach+0xf7/frame 0xfffffe00977487f0 lagg_lacp_attach() at lagg_lacp_attach+0x88/frame 0xfffffe0097748810 lagg_ioctl() at lagg_ioctl+0x98a/frame 0xfffffe00977488f0 in_control() at in_control+0x38e/frame 0xfffffe0097748970 ifioctl() at ifioctl+0xba2/frame 0xfffffe0097748a30 kern_ioctl() at kern_ioctl+0x22b/frame 0xfffffe0097748a90 sys_ioctl() at sys_ioctl+0x13c/frame 0xfffffe0097748ae0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe0097748bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0097748bf0 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x800fa045a, rsp = 0x7fffffffe118, rbp = 0x7fffffffe1a0 ---
State Changed From-To: open->analyzed Scott volunteered himself
Responsible Changed From-To: freebsd-bugs->scottl Scott volunteered himself
Fixed by hrs in r272386