Bug 222680

Summary: bnxt (iflib model): While VLAN TCP Tx is progress, "ifconfig destroy" caused kernel Crash
Product: Base System Reporter: Bhargava Chenna. Marreddy <bhargava.marreddy>
Component: kernAssignee: Matt Joras <mjoras>
Status: Open ---    
Severity: Affects Some People CC: imp, mjoras, shurd
Priority: --- Keywords: crash, needs-qa
Version: CURRENTFlags: koobs: mfc-stable11?
koobs: mfc-stable10?
Hardware: Any   
OS: Any   
Attachments:
Description Flags
core.txt.3 none

Description Bhargava Chenna. Marreddy 2017-09-29 10:14:22 UTC
Repro Steps;- 

1. Boot into FreeBSD rev src_head_322823 
2. Create VLAN interface - 
   ## ifconfig bnxt0.5 create vlan 5 vlandev bnxt0 inet 5.0.0.1/24 
3. Start Tx TCP using Stratus 100G A1 NIC 
4. While Throughput is ~ 90 Gbps, destroy VLAN interface 
   ##ifconfig bnxt0.5 destroy 
5. Server Crashed
Comment 1 Bhargava Chenna. Marreddy 2017-09-29 10:15:36 UTC
Crash dump Analysis:-

Crash dump analysis:-
(kgdb) bt
#0 doadump (textdump=1) at pcpu.h:232
#1 0xffffffff80a6f2f5 in kern_reboot (howto=<value optimized out>) at /usr/src_head_322823/sys/kern/kern_shutdown.c:386
#2 0xffffffff80a6f8e1 in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src_head_322823/sys/kern/kern_shutdown.c:779
#3 0xffffffff80a6f933 in panic (fmt=0x0) at /usr/src_head_322823/sys/kern/kern_shutdown.c:710
#4 0xffffffff80eff9b4 in trap_fatal (frame=0xfffffe00003f05d0, eva=16) at /usr/src_head_322823/sys/amd64/amd64/trap.c:799
#5 0xffffffff80effbcd in trap_pfault (frame=0xfffffe00003f05d0, usermode=0) at /usr/src_head_322823/sys/amd64/amd64/trap.c:656
#6 0xffffffff80eff18c in trap (frame=0xfffffe00003f05d0) at /usr/src_head_322823/sys/amd64/amd64/trap.c:420
#7 0xffffffff80edf041 in calltrap () at /usr/src_head_322823/sys/amd64/amd64/exception.S:237
#8 0xffffffff80b6d9c1 in ether_nh_input (m=0xfffff8046a82d800) at /usr/src_head_322823/sys/net/if_ethersubr.c:517
#9 0xffffffff80b86740 in netisr_dispatch_src (proto=5, source=0, m=0xfffff8046a82d800) at /usr/src_head_322823/sys/net/netisr.c:1120
#10 0xffffffff80b6d172 in ether_input (ifp=<value optimized out>, m=0x0) at /usr/src_head_322823/sys/net/if_ethersubr.c:759
#11 0xffffffff80b75ad9 in vlan_input (ifp=<value optimized out>, m=<value optimized out>) at /usr/src_head_322823/sys/net/if_vlan.c:1387
#12 0xffffffff80b6cc1c in ether_demux (ifp=0xfffff8022db01800, m=0xfffff8046a82d800) at /usr/src_head_322823/sys/net/if_ethersubr.c:803
#13 0xffffffff80b6dc07 in ether_nh_input (m=<value optimized out>) at /usr/src_head_322823/sys/net/if_ethersubr.c:639
#14 0xffffffff80b86740 in netisr_dispatch_src (proto=5, source=0, m=0xfffff8046a82d800) at /usr/src_head_322823/sys/net/netisr.c:1120
#15 0xffffffff80b6d172 in ether_input (ifp=<value optimized out>, m=0x0) at /usr/src_head_322823/sys/net/if_ethersubr.c:759
#16 0xffffffff80b7f2b6 in _task_fn_rx (context=<value optimized out>) at /usr/src_head_322823/sys/net/iflib.c:2568
#17 0xffffffff80ab0d39 in gtaskqueue_run_locked (queue=<value optimized out>) at /usr/src_head_322823/sys/kern/subr_gtaskqueue.c:329
#18 0xffffffff80ab0ac8 in gtaskqueue_thread_loop (arg=<value optimized out>) at /usr/src_head_322823/sys/kern/subr_gtaskqueue.c:504
#19 0xffffffff80a328c4 in fork_exit (callout=0xffffffff80ab0a40 <gtaskqueue_thread_loop>, arg=0xfffffe0000e39230, frame=0xfffffe00003f0ac0) at /usr/src_head_322823/sys/kern/kern_fork.c:1038
#20 0xffffffff80edf62e in fork_trampoline () at /usr/src_head_322823/sys/amd64/amd64/exception.S:651
#21 0x0000000000000000 in ?? ()
Current language: auto; currently minimal
(kgdb) frame 8
#8 0xffffffff80b6d9c1 in ether_nh_input (m=0xfffff8046a82d800) at /usr/src_head_322823/sys/net/if_ethersubr.c:517
517 ETHER_BPF_MTAP(ifp, m);
(kgdb) frame 7
Snip from /usr/src_head_322823/sys/net/ethernet.h
==>
/*
The ETHER_BPF_MTAP macro should be used by drivers which support hardware
offload for VLAN tag processing. It will check the mbuf to see if it has
M_VLANTAG set, and if it does, will pass the packet along to
ether_vlan_mtap. This function will re-insert VLAN tags for the duration
of the tap, so they show up properly for network analyzers.
*/
{{#define ETHER_BPF_MTAP(_ifp, _m) do
Unknown macro: { if (bpf_peers_present((_ifp)->if_bpf)) { \ M_ASSERTVALID(_m); \ if (((_m)->m_flags & M_VLANTAG) != 0) \ ether_vlan_mtap((_ifp)->if_bpf, (_m), NULL, 0); \ else \ bpf_mtap((_ifp)->if_bpf, (_m)); \ } }
while (0)}}
Comment 2 Matt Joras freebsd_committer freebsd_triage 2017-10-04 01:43:38 UTC
Do you still have the dump? I am guessing this crashed on this line:
if (bpf_peers_present((_ifp)->if_bpf)) {
The ifp at this point is the VLAN ifp. Since the the VLAN ifp was destroyed this ends up accessing free'd memory and hence your crash.
Comment 3 Bhargava Chenna. Marreddy 2017-10-04 06:17:02 UTC
Created attachment 186897 [details]
core.txt.3

(In reply to Matt Joras from comment #2)

Thank you Matt.

Yes, I still have them, let me attach to this Bug.
Comment 4 Bhargava Chenna. Marreddy 2017-10-04 06:19:17 UTC
(In reply to Matt Joras from comment #2)
>> The ifp at this point is the VLAN ifp. 
>> Since the the VLAN ifp was destroyed this ends 
>> up accessing free'd memory and hence your crash.
<Chenna> Looks like you are correct. Any thoughts on how to fix this?
Comment 5 Bhargava Chenna. Marreddy 2017-11-06 13:50:28 UTC
Proposed a fix:
https://reviews.freebsd.org/D12973
Comment 6 Warner Losh freebsd_committer freebsd_triage 2023-08-23 00:28:37 UTC
https://reviews.freebsd.org/D41558

Kevin Bowling has come up with this. Can anyone test?