Bug 227065

Summary: Lock order reversal for if_addr_lock <-> ifnet_rw and if_addr_lock <-> re0
Product: Base System Reporter: Dom <dom>
Component: kernAssignee: freebsd-net (Nobody) <net>
Status: New ---    
Severity: Affects Some People CC: avos, dom
Priority: ---    
Version: 11.1-RELEASE   
Hardware: Any   
OS: Any   

Description Dom 2018-03-29 14:01:07 UTC 
After booting a GENERIC kernel with the addition of WITNESS+INVARIANTS+INVARIANTS_SUPPORT the following LRO is observed shortly after every boot:

Mar 29 12:27:37 box kernel: lock order reversal:
Mar 29 12:27:37 box kernel: 1st 0xfffff8000474d190 if_addr_lock (if_addr_lock) @ /usr/src/sys/netinet/igmp.c:1708
Mar 29 12:27:37 box kernel: 2nd 0xffffffff81e68d60 ifnet_rw (ifnet_rw) @ /usr/src/sys/net/if.c:258
Mar 29 12:27:37 box kernel: stack backtrace:
Mar 29 12:27:37 box kernel: #0 0xffffffff80aa0010 at witness_debugger+0x70
Mar 29 12:27:37 box kernel: #1 0xffffffff80a9ff03 at witness_checkorder+0xe23
Mar 29 12:27:37 box kernel: #2 0xffffffff80a3df3d at __rw_rlock+0x8d
Mar 29 12:27:37 box kernel: #3 0xffffffff80b28210 at ifnet_byindex+0x20
Mar 29 12:27:37 box kernel: #4 0xffffffff80b99a1d at igmp_intr+0x1d
Mar 29 12:27:37 box kernel: #5 0xffffffff80b48a60 at netisr_dispatch_src+0x80
Mar 29 12:27:37 box kernel: #6 0xffffffff80b99826 at igmp_v1v2_queue_report+0x1c6
Mar 29 12:27:37 box kernel: #7 0xffffffff80b9774d at igmp_fasttimo+0x50d
Mar 29 12:27:37 box kernel: #8 0xffffffff80ac2f94 at pffasttimo+0x54
Mar 29 12:27:37 box kernel: #9 0xffffffff80a59048 at softclock_call_cc+0x158
Mar 29 12:27:37 box kernel: #10 0xffffffff80a5942c at softclock+0x7c
Mar 29 12:27:37 box kernel: #11 0xffffffff80a07e99 at intr_event_execute_handlers+0x99
Mar 29 12:27:37 box kernel: #12 0xffffffff80a08186 at ithread_loop+0xb6
Mar 29 12:27:37 box kernel: #13 0xffffffff80a05584 at fork_exit+0x84
Mar 29 12:27:37 box kernel: #14 0xffffffff80e9052e at fork_trampoline+0xe
Mar 29 12:27:37 box kernel: lock order reversal:
Mar 29 12:27:37 box kernel: 1st 0xfffff8000474d190 if_addr_lock (if_addr_lock) @ /usr/src/sys/netinet/igmp.c:1708
Mar 29 12:27:37 box kernel: 2nd 0xfffffe0000edb218 re0 (network driver) @ /usr/src/sys/dev/re/if_re.c:2912
Mar 29 12:27:37 box kernel: stack backtrace:
Mar 29 12:27:37 box kernel: #0 0xffffffff80aa0010 at witness_debugger+0x70
Mar 29 12:27:37 box kernel: #1 0xffffffff80a9ff03 at witness_checkorder+0xe23
Mar 29 12:27:37 box kernel: #2 0xffffffff80a21add at __mtx_lock_flags+0x9d
Mar 29 12:27:37 box kernel: #3 0xffffffff8078d9e9 at re_start+0x29
Mar 29 12:27:37 box kernel: #4 0xffffffff80b2f72d at if_transmit+0x10d
Mar 29 12:27:37 box kernel: #5 0xffffffff80b317e0 at ether_output+0x750
Mar 29 12:27:37 box kernel: #6 0xffffffff80bac229 at ip_output+0x1439
Mar 29 12:27:37 box kernel: #7 0xffffffff80b99c7d at igmp_intr+0x27d
Mar 29 12:27:37 box kernel: #8 0xffffffff80b48a60 at netisr_dispatch_src+0x80
Mar 29 12:27:37 box kernel: #9 0xffffffff80b99826 at igmp_v1v2_queue_report+0x1c6
Mar 29 12:27:37 box kernel: #10 0xffffffff80b9774d at igmp_fasttimo+0x50d
Mar 29 12:27:37 box kernel: #11 0xffffffff80ac2f94 at pffasttimo+0x54
Mar 29 12:27:37 box kernel: #12 0xffffffff80a59048 at softclock_call_cc+0x158
Mar 29 12:27:37 box kernel: #13 0xffffffff80a5942c at softclock+0x7c
Mar 29 12:27:37 box kernel: #14 0xffffffff80a07e99 at intr_event_execute_handlers+0x99
Mar 29 12:27:37 box kernel: #15 0xffffffff80a08186 at ithread_loop+0xb6
Mar 29 12:27:37 box kernel: #16 0xffffffff80a05584 at fork_exit+0x84
Mar 29 12:27:37 box kernel: #17 0xffffffff80e9052e at fork_trampoline+0xe


There's seemingly no system impact - everything (including the network) works fine.

Tested on FreeBSD 11.1-RELEASE-p8 running on amd64.
Comment 1 Andriy Voskoboinyk freebsd_committer freebsd_triage 2019-01-26 17:52:19 UTC 
Assign to the net@ list