Bug 292993 - panic in ng_eiface_mediastatus when stopping vnet_jail with netgraph
Summary: panic in ng_eiface_mediastatus when stopping vnet_jail with netgraph
Status: In Progress
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 16.0-CURRENT
Hardware: amd64 Any
: --- Affects Some People
Assignee: Zhenlei Huang
URL:
Keywords: crash
: 294186 (view as bug list)
Depends on:
Blocks:
 
Reported: 2026-02-06 07:45 UTC by Martin Lesser
Modified: 2026-05-06 03:31 UTC (History)
5 users (show)

See Also:


Attachments
ng_eiface_lock patch (2.37 KB, patch)
2026-03-09 10:48 UTC, Pouria Mousavizadeh Tehrani
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Lesser 2026-02-06 07:45:45 UTC
Stopping jail[s] with vnet (created with /usr/share/examples/jails/jng) crashes the system:

Reading symbols from kernel.debug...

Unread portion of the kernel message buffer:
<6>ng0_acme: link state changed to DOWN


Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address	= 0x10
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff8214e9bc
stack pointer	        = 0x28:0xfffffe008841da60
frame pointer	        = 0x28:0xfffffe008841da60
code segment		= base rx0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 0 (jail_remove taskq)
rdi: fffff8000fdfe000 rsi: fffffe008841dad0 rdx: fffff80040f4ba08
rcx: 0000000000000000  r8: 0000000000000000  r9: 0000000000000010
rax: fffff80040f4ba00 rbx: fffffe008841dad0 rbp: fffffe008841da60
r10: 0000000000000000 r11: 0000000000000000 r12: 0000000000000001
r13: 0000000000208802 r14: fffff80040f4ba08 r15: fffff8006937002c
trap number		= 12
panic: page fault
cpuid = 1
time = 1770201212
KDB: stack backtrace:
#0 0xffffffff80bbe1ed at kdb_backtrace+0x5d
#1 0xffffffff80b71576 at vpanic+0x136
#2 0xffffffff80b71433 at panic+0x43
#3 0xffffffff81079f69 at trap_pfault+0x3c9
#4 0xffffffff8104ffe8 at calltrap+0x8
#5 0xffffffff80ca6cde at ifmedia_ioctl+0x17e
#6 0xffffffff80dc93fa at dump_iface+0x11a
#7 0xffffffff80dc8ce5 at rtnl_handle_ifnet_event+0x85
#8 0xffffffff80c97818 at if_detach_internal+0x148
#9 0xffffffff80c9e6e5 at vnet_if_return+0x105
#10 0xffffffff80cd7864 at vnet_destroy+0x134
#11 0xffffffff80b311c8 at prison_deref+0x968
#12 0xffffffff80bd4de2 at taskqueue_run_locked+0x182
#13 0xffffffff80bd5fb2 at taskqueue_thread_loop+0xc2
#14 0xffffffff80b2786b at fork_exit+0x7b
#15 0xffffffff8105100e at fork_trampoline+0xe
Uptime: 7m54s

Debugging with kgdb the instruction pointer when system crashes:

(kgdb) list *0xffffffff8214e9bc
0xffffffff8214e9bc is in ng_eiface_mediastatus (/usr/src/sys/netgraph/ng_eiface.c:365).
warning: Source file is more recent than executable.
360	ng_eiface_mediastatus(struct ifnet *ifp, struct ifmediareq *ifmr)
361	{
362		const priv_p priv = (priv_p)ifp->if_softc;
363		struct ifmedia *ifm = &priv->media;
364	
365		if (ifm->ifm_cur->ifm_media == (IFM_ETHER | IFM_AUTO) &&
366		    (priv->link_status & IFM_ACTIVE))
367			ifmr->ifm_active = IFM_ETHER | IFM_1000_T | IFM_FDX;
368		else
369			ifmr->ifm_active = ifm->ifm_cur->ifm_media;

Running jails the "old" way (binding L2/L3 to the main NIC) did not cause any problems the last 2 decades but I want to separate my jails with virtual NICs so netgraph looked promising but until today I did not manage to get it up reliable.

Not sure whether the information above helps so if you need more information please let me know. Also not sure whether I missed something in the conf of the jail but anyway I think that an eventual bug in configuration should not crash the whole system.
Comment 1 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-03-04 09:50:40 UTC
Hi,

Thank you for your report.
Could you please write the steps to reproduce?
Comment 2 Martin Lesser 2026-03-08 07:08:30 UTC
Hi,

following the steps to reproduce. Both host and jail are running 15.0-RELEASE-p4.

If you need further information please let me know. So here we go:

# netgraph related modules loaded - kldstat:
 3    1 0xffffffff82157000     47f0 ng_eiface.ko
 4    4 0xffffffff8215c000    16ca8 netgraph.ko
 5    1 0xffffffff82173000     5b90 ng_ether.ko
 6    1 0xffffffff8217a000     6620 ng_bridge.ko

# net related lines in /etc/rc.conf of host:
ifconfig_em0="inet 192.168.168.3 netmask 255.255.255.0 up"
ifconfig_em0_ipv6="inet6 2a00:6020:50d7:5300::3/64 accept_rtadv"

# ifconfig (on host):
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=4e524bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 68:05:ca:0d:6a:fd
	inet 192.168.168.3 netmask 0xffffff00 broadcast 192.168.168.255
	inet6 fe80::6a05:caff:fe0d:6afd%em0 prefixlen 64 scopeid 0x2
	inet6 2a00:6020:50d7:5300::3 prefixlen 64
	inet6 2a00:6020:50d7:5300:6a05:caff:fe0d:6afd prefixlen 64 autoconf pltime 14400 vltime 86400
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

# /usr/local/etc/jail.conf.d/web.conf:
web { 
  ### STARTUP / LOGGING
  exec.start = "/bin/sh /etc/rc"; 
  exec.stop = "/bin/sh /etc/rc.shutdown"; 
  exec.consolelog = "/var/log/jail_console_${name}.log"; 

  ### PERMISSIONS
  allow.raw_sockets;
  allow.sysvipc;
  exec.clean; 
  mount.devfs; 

  ### HOSTNAME / PATH
  host.hostname = "${name}"; 
  path = "/mnt/jails/nullfs/${name}";
  $rpath = "/mnt/jails/realjails/${name}";

  ### NETWORKS / INTERFACES
  $id      = "210";
  $ip4     = "192.168.168.${id}/24";
  $ip6     = "2a00:6020:50d7:5300::${id}:3";
  $gw6     = "2a00:6020:50d7:5300::1";
  $gw4     = "192.168.168.1";
  $jif     = "em0";
  $if_uplink="${jif}";
  $if_name="ng0_${name}";
  vnet;
  vnet.interface = "${if_name}";
  exec.prestart += "jng bridge $name $if_uplink";
  exec.start    += "ifconfig ${if_name} ${ip4} up";
  exec.start    += "ifconfig ${if_name} inet6 ${ip6}";
  exec.start    += "route add default ${gw4}";
  exec.start    += "route add -inet6 default ${gw6}";
  exec.poststart += "ngctl msg ${if_uplink}: setpromisc 1";
  exec.poststart += "ngctl msg ${if_uplink}: setautosrc 0";
  exec.poststop = "ngctl shutdown $if_name:";
  mount += "${rpath}                     ${path}       nullfs   rw     0 0";
  mount += "/mnt/jails/templates/base15  ${path}/base  nullfs   ro     0 0";

# Starting the jail, jexec into it and ifconfig:
service jail start web
jexec web
ifconfig:

ng0_web: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=28<VLAN_MTU,JUMBO_MTU>
	ether 02:40:b2:0d:6a:fd
	hwaddr 58:9c:fc:10:a0:6e
	inet 192.168.168.210 netmask 0xffffff00 broadcast 192.168.168.255
	inet6 2a00:6020:50d7:5300::210:3 prefixlen 64
	inet6 fe80::40:b2ff:fe0d:6afd%ng0_web prefixlen 64 scopeid 0x6
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

# Leaving jail and stopping it => panic

service jail web stop
Comment 3 Zhenlei Huang freebsd_committer freebsd_triage 2026-03-09 02:43:28 UTC
The fault line appears same with https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244247 but the later has been fixed years ago.
Comment 4 Zhenlei Huang freebsd_committer freebsd_triage 2026-03-09 10:08:16 UTC
A different stack while I was trying to repeat this issue, with 16-CURRENT,


```
Fatal trap 9: general protection fault while in kernel mode
cpuid = 3; apic id = 06
instruction pointer	= 0x20:0xffffffff80d1e382
stack pointer	        = 0x28:0xfffffe00d3071c30
frame pointer	        = 0x28:0xfffffe00d3071c50
code segment		= base rx0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 12 (swi6: task queue)
rdi: fffff801b8776800 rsi: 00000000c0306938 rdx: fffff8001b9db608
rcx: 00000000c0306938  r8: 0000000000000000  r9: 0000000000000010
rax: deadc0dedeadc0de rbx: fffffe00d3071ca0 rbp: fffffe00d3071c50
r10: 0000000000000000 r11: 0000000000000000 r12: fffff802bc20402c
r13: 0000000000000000 r14: 0000000000000016 r15: 0000000000208803
trap number		= 9
panic: general protection fault
cpuid = 3
time = 1773027303
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d30719b0
vpanic() at vpanic+0x136/frame 0xfffffe00d3071ae0
panic() at panic+0x43/frame 0xfffffe00d3071b40
trap_fatal() at trap_fatal+0x68/frame 0xfffffe00d3071b60
calltrap() at calltrap+0x8/frame 0xfffffe00d3071b60
--- trap 0x9, rip = 0xffffffff80d1e382, rsp = 0xfffffe00d3071c30, rbp = 0xfffffe00d3071c50 ---
ifmedia_ioctl() at ifmedia_ioctl+0x122/frame 0xfffffe00d3071c50
dump_iface() at dump_iface+0x126/frame 0xfffffe00d3071d00
rtnl_handle_ifnet_event() at rtnl_handle_ifnet_event+0x88/frame 0xfffffe00d3071d70
do_link_state_change() at do_link_state_change+0x5b/frame 0xfffffe00d3071dc0
taskqueue_run_locked() at taskqueue_run_locked+0x1c2/frame 0xfffffe00d3071e40
taskqueue_run() at taskqueue_run+0x4d/frame 0xfffffe00d3071e60
ithread_loop() at ithread_loop+0x266/frame 0xfffffe00d3071ef0
fork_exit() at fork_exit+0x82/frame 0xfffffe00d3071f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d3071f30
--- trap 0x5099011, rip = 0x5c5049f260b912, rsp = 0x60d536691340d912, rbp = 0x238aabe1b72b86a7 ---
Uptime: 1d20h28m23s
Dumping 5972 out of 16195 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%


__curthread () at /home/zlei/freebsd-src/sys/amd64/include/pcpu_aux.h:57
57		__asm("movq %%gs:%c1,%0" : "=r" (td)
(kgdb) bt
#0  __curthread () at /home/zlei/freebsd-src/sys/amd64/include/pcpu_aux.h:57
#1  doadump (textdump=textdump@entry=1) at /home/zlei/freebsd-src/sys/kern/kern_shutdown.c:399
#2  0xffffffff80bc94f0 in kern_reboot (howto=260) at /home/zlei/freebsd-src/sys/kern/kern_shutdown.c:519
#3  0xffffffff80bc9a27 in vpanic (fmt=0xffffffff812b59a7 "%s", ap=ap@entry=0xfffffe00d3071b20) at /home/zlei/freebsd-src/sys/kern/kern_shutdown.c:974
#4  0xffffffff80bc9853 in panic (fmt=<unavailable>) at /home/zlei/freebsd-src/sys/kern/kern_shutdown.c:887
#5  0xffffffff81128bd8 in trap_fatal (frame=0xfffffe00d3071b70, eva=<optimized out>) at /home/zlei/freebsd-src/sys/amd64/amd64/trap.c:969
#6  <signal handler called>
#7  ifmedia_ioctl (ifp=0xfffff801b8776800, ifr=0xfffffe00d3071ca0, ifm=0xfffff8001b9db608, cmd=3224398136) at /home/zlei/freebsd-src/sys/net/if_media.c:287
#8  0xffffffff80e5dae6 in get_operstate_ether (ifp=0xfffff801b8776800, pstate=<optimized out>) at /home/zlei/freebsd-src/sys/netlink/route/iface.c:126
#9  get_operstate (ifp=0xfffff801b8776800, pstate=<optimized out>) at /home/zlei/freebsd-src/sys/netlink/route/iface.c:183
#10 dump_iface (nw=nw@entry=0xfffffe00d3071d10, ifp=ifp@entry=0xfffff801b8776800, hdr=hdr@entry=0xfffffe00d3071d48, if_flags_mask=if_flags_mask@entry=0, ifname=ifname@entry=0x0)
    at /home/zlei/freebsd-src/sys/netlink/route/iface.c:312
#11 0xffffffff80e5d2e8 in rtnl_handle_ifevent (ifp=0xfffff801b8776800, nlmsg_type=16, if_flags_mask=0, ifname=0x0) at /home/zlei/freebsd-src/sys/netlink/route/iface.c:1424
#12 rtnl_handle_ifnet_event (ifp=0xfffff801b8776800, if_flags_mask=0) at /home/zlei/freebsd-src/sys/netlink/route/iface.c:1460
#13 0xffffffff80d1514b in do_link_state_change (arg=0xfffff801b8776800, pending=1) at /home/zlei/freebsd-src/sys/net/if.c:2030
#14 0xffffffff80c35da2 in taskqueue_run_locked (queue=queue@entry=0xfffff800c9bbb300) at /home/zlei/freebsd-src/sys/kern/subr_taskqueue.c:517
#15 0xffffffff80c35b9d in taskqueue_run (queue=0xfffff800c9bbb300) at /home/zlei/freebsd-src/sys/kern/subr_taskqueue.c:532
#16 0xffffffff80b7cc56 in intr_event_execute_handlers (ie=0xfffff8000445e600, p=<optimized out>) at /home/zlei/freebsd-src/sys/kern/kern_intr.c:1191
#17 ithread_execute_handlers (ie=0xfffff8000445e600, p=<optimized out>) at /home/zlei/freebsd-src/sys/kern/kern_intr.c:1204
#18 ithread_loop (arg=arg@entry=0xfffff80004458020) at /home/zlei/freebsd-src/sys/kern/kern_intr.c:1297
#19 0xffffffff80b78f32 in fork_exit (callout=0xffffffff80b7c9f0 <ithread_loop>, arg=0xfffff80004458020, frame=0xfffffe00d3071f40)
    at /home/zlei/freebsd-src/sys/kern/kern_fork.c:1201
#20 <signal handler called>
#21 0x005c5049f260b912 in ?? ()
Backtrace stopped: Cannot access memory at address 0x60d536691340d912
(kgdb) frame 7
#7  ifmedia_ioctl (ifp=0xfffff801b8776800, ifr=0xfffffe00d3071ca0, ifm=0xfffff8001b9db608, cmd=3224398136) at /home/zlei/freebsd-src/sys/net/if_media.c:287
287				    compat_media(ifm->ifm_cur->ifm_media) : IFM_NONE;
(kgdb) p ifm
$1 = (struct ifmedia *) 0xfffff8001b9db608
(kgdb) p ifm->ifm_cur
$2 = (struct ifmedia_entry *) 0xdeadc0dedeadc0de
(kgdb) p ifp->if_xname 
$3 = "ng0_web\000\000\000\000\000\000\000\000"
(kgdb) frame 7
#7  ifmedia_ioctl (ifp=0xfffff801b8776800, ifr=0xfffffe00d3071ca0, ifm=0xfffff8001b9db608, cmd=3224398136) at /home/zlei/freebsd-src/sys/net/if_media.c:287
287				    compat_media(ifm->ifm_cur->ifm_media) : IFM_NONE;
```

It appears that ng_eiface(4) lacks proper locks to protect driver private data, hence it is possible `ng_eiface_disconnect()  / if_link_state_change()` running in deferred context races with `ng_eiface_rmnode()`.
Comment 5 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-03-09 10:48:10 UTC
Created attachment 268649 [details]
ng_eiface_lock patch

(In reply to Zhenlei Huang from comment #4)
Comment 6 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-03-09 10:49:42 UTC
(In reply to Zhenlei Huang from comment #4)

Hi,

I wrote the patch for proper locking of ng_eiface yesterday.
But I didn't create a revision for it since it revealed another bug.

Stopping jails: webpanic: Assertion ifindex_table[ifp->if_index].ife_ifnet == ifp failed at /usr/src/sys/net/if.c:817
cpuid = 7
time = 1773012076
KDB: stack backtrace:
#0 0xffffffff80bee09d at kdb_backtrace+0x5d
#1 0xffffffff80b9ccce at vpanic+0x16e
#2 0xffffffff80b9cb53 at panic+0x43
#3 0xffffffff80cd8dc0 at if_attach_internal+0x530
#4 0xffffffff80ce04fd at if_vmove+0x6d
#5 0xffffffff80ce042a at vnet_if_return+0xfa
#6 0xffffffff80d1d304 at vnet_destroy+0x144
#7 0xffffffff80b579b5 at prison_deref+0xa65
#8 0xffffffff80c06712 at taskqueue_run_locked+0x1b2
#9 0xffffffff80c07643 at taskqueue_thread_loop+0xd3
#10 0xffffffff80b4d6b2 at fork_exit+0x82
#11 0xffffffff810b73ce at fork_trampoline+0xe
KDB: enter: panic

Yes, it's if_vmove() again.

To understand what happening see this heavily simplified order of events:
vnet tries to vmove the ng_eiface to its home_vnet.
thread 1: IF_WLOCK
thread 1: hit EPOCH_DRAIN
thread 2: ngthread starts working on the request received from `ngctl shutdown`
thread 2: ng_eiface_disconnect (thread 3: taskqueue for link_down)
thread 2: ng_eiface_rmnode
thread 2: ether_detach -> (cause trouble for bpf_vmove in thread 1)
thread 2: if_free -> (thread 4: epoch_call to if_detach_internal)
thread 4: IF_WLOCK (panic)

by delaying the sx_unlock in ng_eiface_rmnode:
thread 1: IF_WLOCK
thread 1: hit EPOCH_DRAIN
thread 2: ngthread starts working on the request received from `ngctl shutdown`
thread 2: ng_eiface_disconnect (thread 3: taskqueue for link_down)
thread 2: ng_eiface_rmnode
thread 1: taskqueue drain
thread 2: new sx_xlock [let's assume this one wins] || thread 3: new sx_slock on ioctl
thread 2: ether_detach
thread 1: taskqueue drain
thread 1: if_down (new thread looking for ng_eiface_ioctl for priv->media just like thread 3 by dump_iface in netlink)
thread 1: hit assertion in if_attach_internal since unlinked from ifindex_table by ether_detach above.

let's add an exception to if_attach_internal before MPASS:
```
/* allow exception for vmove if interface detach by another thread */
if (vmove && ifindex_table[ifp->if_index].ife_ifnet == NULL)
    return;
MPASS(...)
```
.
.
.
thread 1: bpf_vmove after if_attach_internal
remember thread 2 performing bpf_detach in ether_detach before?
another PANIC.
Comment 7 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-03-09 10:56:45 UTC
We can use a reassign function in ng_eiface to block if_vmove after if_detach internal.
However, we can't stop if_vmove and we have to add two more exception in if_attach_internal and bpf_vmove.
Comment 8 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2026-04-02 07:14:16 UTC
*** Bug 294186 has been marked as a duplicate of this bug. ***
Comment 9 Palle Girgensohn freebsd_committer freebsd_triage 2026-04-02 07:56:30 UTC
I've seen that fork_trampoline problems en masse, but did a small change to the order of things happeningm and since it's been smooth sailing. I've long thought to write a bug report about this. 

According to `/usr/share/examples/jails/jail.xxx.conf`, interfaces should be destroyed in poststop: 

exec.poststop += "jng shutdown xxx";	# destroy interface(s)

this setting will very often crash the box when shutting down the jail. Especially if we have two interfaces and with high certainty if one of them has no IP. Either way, it will crash and panic the kernel with the below trace quite often.

Now, changing the order and pull down the interface *first* mitigates this problem completely:

exec.prestop += "jng shutdown $name";  # destroy interface(s)


I believe the problem is somehow that when the jail is pulled down, the interface becomes orphaned.

This is just my observations, and it may or may not be related to your problem, but IO would be surprised if it didin't help.
Comment 10 Zhenlei Huang freebsd_committer freebsd_triage 2026-04-23 19:24:13 UTC
(In reply to Palle Girgensohn from comment #9)
> I believe the problem is somehow that when the jail is pulled down, the interface
> becomes orphaned.

The underlaying cause is that on jail removal, the teardown progress of an interface races with the return of it. To be precisely, that is races between if_detach() and vnet_if_return(). For details, see the patch https://reviews.freebsd.org/D56374 .

There is also another race, that is race between bpfdetach() and bpf_vmove(). That one is only observable on CURRENT/16, given bpf_vmove() was introduced while virtualizing the bpf(4), which is only available on CURRENT/16.

(In reply to Martin Lesser from comment #0)
As a workaround, the ng_eiface(4) interface can be reclaimed / returned prior to remove of the jail, then after the jail been removed, destroy the interface. That is 

```
    exec.prestop = "ifconfig ${if_name} -vnet ${name}";
...
    exec.poststop = "ngctl shutdown $if_name:";
```
Comment 11 Zhenlei Huang freebsd_committer freebsd_triage 2026-04-23 19:30:14 UTC
A minimal jail config to repeat,

```
# cat test.conf
test { 
	path = "/";

	$if_uplink="em0";
	$if_name="ng0_${name}";

	vnet;
	vnet.interface = "${if_name}";
	exec.prestart += "sh /usr/share/examples/jails/jng bridge $name $if_uplink";
	exec.start = "sh -c 'ifconfig $if_name; sleep 1 &'";
	exec.poststop = "ngctl shutdown $if_name:";
}

# jail -f test.conf -c test && jail -f test.conf -r test
... crash ...
```
Comment 12 commit-hook freebsd_committer freebsd_triage 2026-04-25 19:58:45 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=ba7f47d47dc1a177e4d8f115f791ec25f3da0eab

commit ba7f47d47dc1a177e4d8f115f791ec25f3da0eab
Author:     Zhenlei Huang <zlei@FreeBSD.org>
AuthorDate: 2026-04-25 19:56:07 +0000
Commit:     Zhenlei Huang <zlei@FreeBSD.org>
CommitDate: 2026-04-25 19:56:07 +0000

    ifnet: if_detach(): Fix races with vmove operations

    The rationality is that the driver private data holds a strong reference
    to the interface, and the detach operation shall never fail. Given the
    vmove operation, if_vmove_loan(), if_vmove_reclaim() or vnet_if_return()
    is not atomic and spans multiple steps, acquire ifnet_detach_sxlock only
    for if_detach_internal() and if_vmove() is not sufficient. It is possible
    that the thread running if_detach() sees stale vnet, or the vmoving is
    in progress, then if_unlink_ifnet() will fail.

    Fix that by extending coverage of ifnet_detach_sxlock a bit to also
    cover if_unlink_ifnet(), so that the entire detach and vmove operation
    is serialized.

    Given it is an error when the if_unlink_ifnet() fails, and if_detach()
    is a public KPI, prefer panic() over assertion on failure, to indicate
    explicitly that bad thing happens. That shall also prevent potential
    corrupted status of the interface, which is a bit hard to diagnose.

    PR:             292993
    Reviewed by:    glebius
    MFC after:      5 days
    Differential Revision:  https://reviews.freebsd.org/D56374

 sys/net/if.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
Comment 13 commit-hook freebsd_committer freebsd_triage 2026-04-27 19:21:07 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=afa4648b6a30603faab56b649605a3393eac9986

commit afa4648b6a30603faab56b649605a3393eac9986
Author:     Zhenlei Huang <zlei@FreeBSD.org>
AuthorDate: 2026-04-27 19:17:22 +0000
Commit:     Zhenlei Huang <zlei@FreeBSD.org>
CommitDate: 2026-04-27 19:17:22 +0000

    tests: Temporarily skip two testcases

    The changes [1] and [2] made to CURRENT introduce races between ifnet
    detach and vmove operations. That requires extra effort to fix. They
    are not MFCed to stable branches so the latter are not affected.
    Temporarily skip two affected tests on CURRENT right now.

    [1] 0bf42a0a05b9 bpf: virtualize bpf_iflist
    [2] a4d766caf711 bpf: add a crutch to support if_vmove

    PR:             292993
    Discussed with: kp

 tests/sys/net/if_clone_test.sh    | 2 ++
 tests/sys/netgraph/eiface_test.sh | 2 ++
 2 files changed, 4 insertions(+)
Comment 14 commit-hook freebsd_committer freebsd_triage 2026-04-27 19:21:09 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=0988abd52352ae0977cd3e5c10316b7d94e1cac8

commit 0988abd52352ae0977cd3e5c10316b7d94e1cac8
Author:     Zhenlei Huang <zlei@FreeBSD.org>
AuthorDate: 2026-04-27 19:17:22 +0000
Commit:     Zhenlei Huang <zlei@FreeBSD.org>
CommitDate: 2026-04-27 19:17:22 +0000

    tests/netgraph: Add a test for races between if_detach() and vnet_if_return()

    A ng_eiface(4) or physical interface does not involve the cloner hence
    the detaching is a bit different with epair(4). Add more tests to cover
    that.

    PR:             292993
    MFC after:      3 days
    Differential Revision:  https://reviews.freebsd.org/D56609

 tests/sys/netgraph/Makefile             |  2 +
 tests/sys/netgraph/eiface_test.sh (new) | 67 +++++++++++++++++++++++++++++++++
 2 files changed, 69 insertions(+)
Comment 15 commit-hook freebsd_committer freebsd_triage 2026-04-30 09:53:38 UTC
A commit in branch stable/15 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=5c4021ca0abe4e17200f5faa2fd71014ef0a5f09

commit 5c4021ca0abe4e17200f5faa2fd71014ef0a5f09
Author:     Zhenlei Huang <zlei@FreeBSD.org>
AuthorDate: 2026-04-25 19:56:07 +0000
Commit:     Zhenlei Huang <zlei@FreeBSD.org>
CommitDate: 2026-04-30 09:52:02 +0000

    ifnet: if_detach(): Fix races with vmove operations

    The rationality is that the driver private data holds a strong reference
    to the interface, and the detach operation shall never fail. Given the
    vmove operation, if_vmove_loan(), if_vmove_reclaim() or vnet_if_return()
    is not atomic and spans multiple steps, acquire ifnet_detach_sxlock only
    for if_detach_internal() and if_vmove() is not sufficient. It is possible
    that the thread running if_detach() sees stale vnet, or the vmoving is
    in progress, then if_unlink_ifnet() will fail.

    Fix that by extending coverage of ifnet_detach_sxlock a bit to also
    cover if_unlink_ifnet(), so that the entire detach and vmove operation
    is serialized.

    Given it is an error when the if_unlink_ifnet() fails, and if_detach()
    is a public KPI, prefer panic() over assertion on failure, to indicate
    explicitly that bad thing happens. That shall also prevent potential
    corrupted status of the interface, which is a bit hard to diagnose.

    PR:             292993
    Reviewed by:    glebius
    MFC after:      5 days
    Differential Revision:  https://reviews.freebsd.org/D56374

    (cherry picked from commit ba7f47d47dc1a177e4d8f115f791ec25f3da0eab)

 sys/net/if.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
Comment 16 commit-hook freebsd_committer freebsd_triage 2026-04-30 09:53:40 UTC
A commit in branch stable/15 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3a4c4d61ac2c9f369e55d0ab994c9da78aa4cb74

commit 3a4c4d61ac2c9f369e55d0ab994c9da78aa4cb74
Author:     Zhenlei Huang <zlei@FreeBSD.org>
AuthorDate: 2026-04-27 19:17:22 +0000
Commit:     Zhenlei Huang <zlei@FreeBSD.org>
CommitDate: 2026-04-30 09:52:02 +0000

    tests/netgraph: Add a test for races between if_detach() and vnet_if_return()

    A ng_eiface(4) or physical interface does not involve the cloner hence
    the detaching is a bit different with epair(4). Add more tests to cover
    that.

    PR:             292993
    MFC after:      3 days
    Differential Revision:  https://reviews.freebsd.org/D56609

    (cherry picked from commit 0988abd52352ae0977cd3e5c10316b7d94e1cac8)

 tests/sys/netgraph/Makefile             |  2 +
 tests/sys/netgraph/eiface_test.sh (new) | 67 +++++++++++++++++++++++++++++++++
 2 files changed, 69 insertions(+)
Comment 17 commit-hook freebsd_committer freebsd_triage 2026-05-04 16:52:11 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=77b8bc06cf73c66ed9a4ebb4d88d072056059ff4

commit 77b8bc06cf73c66ed9a4ebb4d88d072056059ff4
Author:     Zhenlei Huang <zlei@FreeBSD.org>
AuthorDate: 2026-04-25 19:56:07 +0000
Commit:     Zhenlei Huang <zlei@FreeBSD.org>
CommitDate: 2026-05-04 16:49:43 +0000

    ifnet: if_detach(): Fix races with vmove operations

    The rationality is that the driver private data holds a strong reference
    to the interface, and the detach operation shall never fail. Given the
    vmove operation, if_vmove_loan(), if_vmove_reclaim() or vnet_if_return()
    is not atomic and spans multiple steps, acquire ifnet_detach_sxlock only
    for if_detach_internal() and if_vmove() is not sufficient. It is possible
    that the thread running if_detach() sees stale vnet, or the vmoving is
    in progress, then if_unlink_ifnet() will fail.

    Fix that by extending coverage of ifnet_detach_sxlock a bit to also
    cover if_unlink_ifnet(), so that the entire detach and vmove operation
    is serialized.

    Given it is an error when the if_unlink_ifnet() fails, and if_detach()
    is a public KPI, prefer panic() over assertion on failure, to indicate
    explicitly that bad thing happens. That shall also prevent potential
    corrupted status of the interface, which is a bit hard to diagnose.

    PR:             292993
    Reviewed by:    glebius
    MFC after:      5 days
    Differential Revision:  https://reviews.freebsd.org/D56374

    (cherry picked from commit ba7f47d47dc1a177e4d8f115f791ec25f3da0eab)
    (cherry picked from commit 5c4021ca0abe4e17200f5faa2fd71014ef0a5f09)

 sys/net/if.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
Comment 18 commit-hook freebsd_committer freebsd_triage 2026-05-04 16:52:13 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=33da0a256bfc75f72ea789fd71dce0e07515800c

commit 33da0a256bfc75f72ea789fd71dce0e07515800c
Author:     Zhenlei Huang <zlei@FreeBSD.org>
AuthorDate: 2026-04-27 19:17:22 +0000
Commit:     Zhenlei Huang <zlei@FreeBSD.org>
CommitDate: 2026-05-04 16:49:42 +0000

    tests/netgraph: Add a test for races between if_detach() and vnet_if_return()

    A ng_eiface(4) or physical interface does not involve the cloner hence
    the detaching is a bit different with epair(4). Add more tests to cover
    that.

    PR:             292993
    MFC after:      3 days
    Differential Revision:  https://reviews.freebsd.org/D56609

    (cherry picked from commit 0988abd52352ae0977cd3e5c10316b7d94e1cac8)
    (cherry picked from commit 3a4c4d61ac2c9f369e55d0ab994c9da78aa4cb74)

 tests/sys/netgraph/Makefile             |  2 +
 tests/sys/netgraph/eiface_test.sh (new) | 67 +++++++++++++++++++++++++++++++++
 2 files changed, 69 insertions(+)