Bug 296883 - routing: IPv6 forwarding regressed with bridge after base d05d1f256082
Summary: routing: IPv6 forwarding regressed with bridge after base d05d1f256082
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 16.0-CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Pouria Mousavizadeh Tehrani
URL: https://reviews.freebsd.org/D58347
Keywords: regression
Depends on:
Blocks:
 
Reported: 2026-07-18 16:15 UTC by Evgenii Khramtsov
Modified: 2026-08-05 15:32 UTC (History)
4 users (show)

See Also:


Attachments
output for c3 and c4 (1.37 KB, application/zstd)
2026-07-18 21:08 UTC, Evgenii Khramtsov
no flags Details
output for c9 (808 bytes, application/zstd)
2026-07-20 13:53 UTC, Evgenii Khramtsov
no flags Details
output for c12+ (5.67 KB, application/zstd)
2026-07-21 15:39 UTC, Evgenii Khramtsov
no flags Details
c18 output (14.36 KB, application/zstd)
2026-07-22 13:23 UTC, Evgenii Khramtsov
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Evgenii Khramtsov 2026-07-18 16:15:57 UTC
I run multi-fib IPv6-only desktop/router/firewall/virtualization all-in-one host and I see two regressions after updating post base d05d1f256082. I'm now at base 4d37c2ea3c86 (with cherry-picked base 30d4d3db431a build fix) with base be42ac4dceac and base d05d1f256082 reverted, the latter revert unbreaks bridge IPv6 forwarding for me.

1) Starting a VM with something like:

#!/bin/sh

UEFI=/usr/local/share/uefi-firmware/BHYVE_UEFI.fd
MEM=8G
VM="windoze"
IF="$(ifconfig tap create up)"
MAC="mac=00:15:17:d2:c0:11"

ifconfig bridge0 addm $IF

while true ; do
        bhyve \
                -AHPSw \
                -s 0,hostbridge \
                -s 4,nvme,/usr/vm/$VM/$VM-0.img,sectorsize=4096 \
                -s 10,e1000,$IF,$MAC \
                -s 14,virtio-rnd \
                -s 31,lpc \
                -o cpus="1" \
                -o cores="1" \
                -o threads="1" \
                -o sockets="1" \
                -l com1,stdio \
                -l bootrom,$UEFI,fwcfg=qemu \
                -m $MEM \
                $VM
        RES=$?
        bhyvectl --destroy --vm=$VM
        if [ $RES -eq 1 ] ; then
                ifconfig $IF destroy
                exit 1
        fi
        sleep 5
done

Causes the IPv6 EUI-64 address of guest to become unreachable, I get "destination unreachable no route to host" when pinging guest from host.



2) After I start VM with the script above (tap0 becomes member of bridge0), IPv6-only nodes behind host (bridge0 -> em1 -> dumb wireless AP -> nodes (e.g. Linux/Android/iOS nodes) no longer can route packets to the Internet. I didn't try to ping default gateway on them (host) or ff02::1, but I can reboot into affected environment on request later.

-
I have a bit unusual of a setup to reproduce, but I'll try to include some details:

ifconfig_bridge0_name="brer"
ifconfig_brer="addm em1 up"
ifconfig_brer_ipv6="inet6 2001:db8:b0d7:fed5::1/64"
ifconfig_brer_alias0="inet6 2001:db8:b0d7:fed5:38b8:42d:c42a:da00 prefer_source"
ifconfig_brer_alias1="inet6 2001:db8:b0d7:fed5::123/64"

root@desktop:~ # cat /etc/rtadvd.conf
brer:\
	:raflags="h":\
	:rdnss="2001:4860:4860::6464":rdnssltime#604800:mtu#1492:\
	:pref64="64:ff9b::":

I get IPv6 Internet access with this convoluted DHCPv6-PD setup of net/mpd5 + net/dhcpcd that I didn't have time to figure out if right or wrong, yet:

root@desktop:~ # cat /usr/local/etc/mpd5/ipv6_up.sh:
#!/bin/sh

route add -inet6 default -interface $1
exit 0

root@desktop:~ # cat /usr/local/etc/dhcpcd.conf:
allowinterfaces ng0 brer
duid
ipv6only
noipv6rs
waitip 6
timeout 15

interface ng0
  ia_pd ng0 brer/0


Rrouting table, it is the same with working/regressed base:

root@desktop:~ # netstat -nrF0 -6
Routing tables

Internet6:
Destination                       Gateway                       Flags         Netif Expire
::/96                             link#4                        URS             lo0
default                           link#9                        US              ng0
::1                               link#4                        UHS             lo0
::ffff:0.0.0.0/96                 link#4                        URS             lo0
2001:db8:b0d7:ef92::/64          link#5                        U              brer
2001:db8:b0d7:ef92::1            link#4                        UHS             lo0
2001:db8:b0d7:ef92::123          link#4                        UHS             lo0
2001:db8:b0d7:fed5:38b8:42d:c42a:da00 link#4                   UHS             lo0
fe80::%lo0/10                     link#4                        URS             lo0
fe80::%lo0/64                     link#4                        U               lo0
fe80::1%lo0                       link#4                        UHS             lo0
fe80::%brer/64                    link#5                        U              brer
fe80::6361:2dcc:e3a3:2fed%lo0     link#4                        UHS             lo0
fe80::%ng0/64                     link#9                        U               ng0
fe80::5a9c:fcff:fe10:9a95%lo0     link#4                        UHS             lo0
ff02::/16                         link#4                        URS             lo0

Except that, when I do "ndp -a" with working base:

2001:db8:b0d7:fed5:215:17ff:fed2:c011                           00:15:17:d2:c0:11   brer 23h59m18s S
fe80::215:17ff:fed2:c011%brer        00:15:17:d2:c0:11   brer 23h59m29s S

With regressed base:
fe80::215:17ff:fed2:c011%brer        (incomplete)        brer permanent N

P.S. I might be slow to provide feedback this time, I'm unfortunately very busy with something else now.
Comment 1 Evgenii Khramtsov 2026-07-18 16:18:04 UTC
(In reply to Evgenii Khramtsov from comment #0)

err, I've got lost a bit with bridge0 -> brer rename, I have multiple of bridges.

When I mention "bridge0" assume that it is brer.
Comment 2 Evgenii Khramtsov 2026-07-18 16:20:04 UTC
(In reply to Evgenii Khramtsov from comment #0)

> rdnssltime#604800

That is a workaround to avoid iOS losing RDNSS, there is a patch on Bugzilla already to remove limit for rtadvd RDNSS lifetime for this to work.

Sorry for adding more details in new messages, for some reason I didn't see that before I posted this PR.
Comment 3 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-18 18:38:03 UTC
Hi,

Could you please post the output of these commands?
```
netstat -rn6W
netstat -on6W
netstat -On6W
ifconfig tapX
```

Please replace tapX with your $IF interface.
Comment 4 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-18 18:40:59 UTC
(In reply to Pouria Mousavizadeh Tehrani from comment #3)

I need those output with regressed base.

Also, please post the output of your bridge0 interface too:
ifconfig bridge0
Comment 5 Evgenii Khramtsov 2026-07-18 21:08:50 UTC
Created attachment 272977 [details]
output for c3 and c4

(In reply to Pouria Mousavizadeh Tehrani from comment #4)

Inside:

revert/: base 4d37c2ea3c86 output (with cherry-picked base 30d4d3db431a build fix) with base be42ac4dceac and base d05d1f256082 reverted (works)

regressed/: above but base d05d1f256082 reapplied (forwarding dies after VM starts)

1.txt -> netstat -rn6W
2.txt -> netstat -on6W
3.txt -> netstat -On6W
4.txt -> ifconfig tap0
5.txt -> ifconfig brer ("brer" is bridge0 renamed, that tap0 gets attached to)

I removed interfaces from fibs unrelated and their routing entries from output (they do not touch brer's network in any way / are irrelevant to this PR, and are also banned by firewall in case of local misconfiguration).

I redacted my real GUA /64 prefix with "2001:db8:b0d7:fed5" (prior mention of "ef92" was myopic redacting of GUA prefix).
Comment 6 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-18 21:41:23 UTC
(In reply to Evgenii Khramtsov from comment #5)

Looks normal, you don't have a nexthop group so be42ac4dceac is not the problem.
I will try to reproduce your setup.

Just to retrigger the nhop subscription event:
1. Could you please manually set your guest interfaces (brer and tap0) down and then up? (for example: ifconfig tap0 down && ifconfig tap0 up)
I suspect the link_state capability of tap0 might have some issues.
2. Also, would it be possible to test with the virtio-net driver instead of e1000 for the guest VM?
Comment 7 Evgenii Khramtsov 2026-07-19 10:33:54 UTC
(In reply to Pouria Mousavizadeh Tehrani from comment #6)

I suspected that this is not related to tap0, so instead I tried first:

ifconfig brer deletem em1
ifconfig brer addm em1
-> no forwarding afterwards

ifconfig down em1
ifconfig up em1
-> still no forwarding

ifconfig down brer
ifconfig up brer
-> still no forwarding

I tried the sequence above with tap0 and the result was the same. I didn't bother with trying virtio-net as I doubt anything would change.

Doing the same sequence (all steps) works fine on non-regressed base.

> I will try to reproduce your setup.

em0 faces PPPoE ISP, ng0 gets created by net/mpd5 (set bundle enable ipv6cp + ipv6_up.sh for LL communication with ISP), then net/dhcpcd requests DHCPv6-PD via LL, then delegates the GUA prefix (/64) to the bridge. I only added /64 to bridge in rc.conf because the /64 is static for me and I want to bind sockets (e.g. dns/unbound) before net/dhcpcd gets to delegate GUA to bridge.
Comment 8 Evgenii Khramtsov 2026-07-20 10:23:07 UTC
Do I need to apply D58347 and post new output with regressed base or not?
Comment 9 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-20 12:22:42 UTC
(In reply to Evgenii Khramtsov from comment #8)

I was waiting for my tests to complete.

For the first case, I attempted to replicate your setup using a Windows VM with UEFI and an e1000 tap interface as a bridge member. I didn't encounter any issues.

For the second case, I have an almost identical setup to yours, except I don't use mpd5 + ng_iface. My current implementation uses bridge0 → bge0 → AP → nodes, plus rtadvd on bridge0 and dhcpcd on another interface. This setup worked fine (with that revision applied) long before I commit that.

I suspect the issue might be with the bridge driver (since it lacks LINKSTATE capability BUT reports its link status based on its members) or possibly ng_iface.

Could you please apply the patch (netstat only) and run the command below on the regressed kernel, then share the results?
```
netstat -on6W --libxo json
```

The netstat output with XML/JSON should now emit nhop flags with an "nh_flags_pretty" key. If the regression was caused by d05d1f256082, there must be "invalid" flag somewhere in your nexthop lists.
This should help identify the issue.

Could can filter it with jq too:
```
netstat -on6W --libxo json | jq '.statistics."route-nhop-information"."nhop-table"."rt-family".[]."nh-entry".[] | select(.nh_flags_pretty | index("invalid"))'
```
Comment 10 Evgenii Khramtsov 2026-07-20 13:53:02 UTC
Created attachment 273026 [details]
output for c9

(In reply to Pouria Mousavizadeh Tehrani from comment #9)

> Could you please apply the patch (netstat only) and run the command below on the regressed kernel, then share the results?

Attached output for regressed kernel.

pre: booted and routing is working, no deletem and no addm1 done yet
post-{deletem,addm}: self-describing for em1
Comment 11 Evgenii Khramtsov 2026-07-20 13:56:10 UTC
(In reply to Evgenii Khramtsov from comment #10)

Hmm after now I've got this for filter and post-addm, not sure if it's inside archive:

{
  "index": 2,
  "type_str": "v6/resolve",
  "ifa": "2001:db8:b0d7:fed5::1",
  "gateway": "brer/resolve",
  "flags": "",
  "rt_flags_pretty": [],
  "nh_flags_pretty": [
    "invalid"
  ],
  "use": 0,
  "mtu": 1500,
  "interface-name": "brer",
  "address-interface-name": "",
  "refcount": 2
}
Comment 12 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-20 14:42:41 UTC
(In reply to Evgenii Khramtsov from comment #11)

Interesting, Thank you!

Could you please tell me which routes are using index 13?
It might get change so use point me the routes with the nhop index that are invalid (post addm and post deletem).
```
netstat -rn6W
```
(look for "Nhop#" column")

Also which interface do you mean by addm/deletem? (tapX or emX?)
And after deletem, do you make the bridge empty or is there a member left in that bridge?

The scope of problem is now limited to bridge driver.
But it might be the problem of its member that incorrectly state its linkstate.
So I need to know which member is it?
Comment 13 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-20 14:45:34 UTC
(In reply to Evgenii Khramtsov from comment #11)

No it was not there. include routes with that index too please.
Again, it might change, verify that it's invalid.
Comment 14 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-20 14:48:23 UTC
Also, please include ifconfig result of your lo0 interface too.
Not sure why but it says it's down.
Comment 15 Evgenii Khramtsov 2026-07-20 15:10:42 UTC
(In reply to Pouria Mousavizadeh Tehrani from comment #12)

> Also which interface do you mean by addm/deletem? (tapX or emX?)

em1, as I stopped attaching tap0 since I can reproduce this without bhyve.

> And after deletem, do you make the bridge empty or is there a member left in that bridge?

brer has no members after deletem em1 afterwards.

> Could you please tell me which routes are using index 13?
> include routes with that index too please.
> ifconfig result of your lo0 interface too

I'll be able to reboot next time tomorrow.
Comment 16 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-20 18:33:33 UTC
(In reply to Evgenii Khramtsov from comment #15)

> brer has no members after deletem em1 afterwards.

Ok, so it's normal to have invalid flag on empty bridge. Because it's empty.

The problem is it reports the lo0 and also your bridge with em0 is down.
Comment 17 Evgenii Khramtsov 2026-07-21 15:39:38 UTC
Created attachment 273061 [details]
output for c12+

I simply decided to omit almost all of unrelated interfaces in rc.conf and boot with the minimum that includes the affected brer, so I barely have to redact anything.

I've included the following output for every step:
netstat -nrF0 -6
ndp -a
netstat -rn6W
netstat -on6W
netstat -On6W
ifconfig em1
ifconfig brer
ifconfig em0
ifconfig ng0
ifconfig lo0
netstat -on6W --libxo json
netstat -on6W --libxo json | jq '.statistics."route-nhop-information"."nhop-table"."rt-family".[]."nh-entry".[] | select(.nh_flags_pretty | index("invalid"))'

You can also see "get.sh" inside archive.

The only redaction now is:

find . -type f -exec sed -i '' 's/real-gua-64/2001:db8:b0d7:fed5/g' {} +
+ the same for globally routable IPv4 /32 on ng0


I'll be extra clear:

All the logs inside archive are with regressed kernel boot + Phabricator patch.

after_boot_and_1_min -> I booted, then I was using the Internet on nodes for ~1m, forwarding OK, then I run get.sh
*immediately -> I did the command mentioned in the name of directory and then I run get.sh immediately
*1min -> 1 minute passed after the related *immediate, then I run get.sh again

FWIW, SLAAC still works even when routing on brer is dead.

> your bridge with em0

As previously mentioned:

brer adds em1 in rc.conf on boot,
rtadvd runs on brer in rc.conf.

em0 faces ISP for net/mpd5 to create ng0, then net/dhcpcd delegates /64 to brer,
brer never adds em0.
Comment 18 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-21 19:43:51 UTC
(In reply to Evgenii Khramtsov from comment #17)

AFAICU:

I see your em1 become `status: no carrier` in "04_after_addm_em1_immediately". Its media is also (100baseTX ).

When you move it under a bridge, it resets and reports its link_state as down in:

2308         } else if (!link_check && (sc->link_active == 1)) {
2309                 sc->link_speed = 0;
2310                 sc->link_duplex = 0;
2311                 sc->link_active = 0;
2312                 iflib_link_state_change(ctx, LINK_STATE_DOWN, 0);
2313         }

However, it only reports the link as up when TSO (automasked) is set, and it won't set TSO (automasked) for media under 1G. (This should not be related to TSO, but the code logic does it anyway.)

2300                 /* Only do TSO on gigabit for older chips due to errata */
2301                 if (hw->mac.type < igb_mac_min)
2302                         automasked = em_automask_tso(ctx);
2303
2304                 /* Automasking resets the interface so don't mark it up yet */
2305                 if (!automasked)
2306                         iflib_link_state_change(ctx, LINK_STATE_UP,
2307                             IF_Mbps(sc->link_speed));

So em reports the link state as down, but it won't report it back as up again when it transitions to 100 Mbps media.

The original patch that *might* cause this bug is https://reviews.freebsd.org/D41170.

Could you please try setting this sysctl tunable:

```
sysctl hw.em.unsupported_tso=1
```

and resetting your em1 interface? (might not be enough, but worth trying)
Comment 19 Evgenii Khramtsov 2026-07-21 20:00:37 UTC
(In reply to Pouria Mousavizadeh Tehrani from comment #18)

> Its media is also (100baseTX )

It's expected, the AP is dirt cheap (100baseTX is the max).

> Could you please try setting this sysctl tunable:
> ```
> sysctl hw.em.unsupported_tso=1
> ```
> and resetting your em1 interface? (might not be enough, but worth trying)

Resetting means the down/up em1 from Comment 17 but with hw.em.unsupported_tso=1 set before doing that? If I guessed it right I might be able to do that tomorrow.

I'll also try to do the same with em1 connected to something with a GbE.
Comment 20 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-21 20:16:53 UTC
(In reply to Evgenii Khramtsov from comment #19)

> Resetting means the down/up em1 from Comment 17 but with hw.em.unsupported_tso=1 set before doing that? If I guessed it right I might be able to do that tomorrow.

Looks like, 'hw.em.unsupported_tso' is a read only tunable.
So, As you know, you should set in /boot/loader.conf
Therefore, you have to reboot, that should be enough :)

> I'll also try to do the same with em1 connected to something with a GbE.

Awesome! Thank you for your report and your testing.
Comment 21 Evgenii Khramtsov 2026-07-22 13:23:01 UTC
Created attachment 273079 [details]
c18 output

Inside:

01_em1_GbE/0* - self-describing
02_unsup_tso_GbE/0* - above but booted with hw.em.unsupported_tso="1" in loader.conf
03_unsup_tso_FE/0* - above (including unsup tso) but this time it is 100baseTX
04_no_em1_tap0_virtio/0* - brer has no members after boot, then I start an "open-webui" VM (virtio-net in bhyve start script, attached tap0, Linux guest), and I can't ping the VM from host (UDP destination no route, routing died again), EUI-64 of VM has reverse zone entry so it resolves as "open-webui" in output

FWIW: I could ping LL when routing died.

It is unlikely that I can provide more logs for the rest of this week.
Comment 22 Pouria Mousavizadeh Tehrani freebsd_committer freebsd_triage 2026-07-25 21:21:56 UTC
(In reply to Evgenii Khramtsov from comment #21)

Hi, please try again with this commit: 0bd6a167c1561f01c227b1c428a3d8adf0e38833 (it's in CURRENT)
I understand you were able to reproduce without e1000, but I'm curious if this solves em1 problem or not.

Thank you for your time.
Comment 23 commit-hook freebsd_committer freebsd_triage 2026-07-28 08:19:44 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=5286cbea9118f79b43fb7416d9fc4f081c400949

commit 5286cbea9118f79b43fb7416d9fc4f081c400949
Author:     Pouria Mousavizadeh Tehrani <pouria@FreeBSD.org>
AuthorDate: 2026-07-28 08:16:14 +0000
Commit:     Pouria Mousavizadeh Tehrani <pouria@FreeBSD.org>
CommitDate: 2026-07-28 08:17:32 +0000

    netstat(1): Emit nhop flags for encoding styles with Wflag

    Show nhop flags like invalid nexthop to debug cases like
    the PR below.

    PR:             296883
    Reviewed by:    glebius
    Differential Revision: https://reviews.freebsd.org/D58347

 usr.bin/netstat/nhops.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
Comment 24 Evgenii Khramtsov 2026-07-28 16:06:39 UTC
(In reply to Pouria Mousavizadeh Tehrani from comment #22)

> Hi, please try again with this commit: 0bd6a167c1561f01c227b1c428a3d8adf0e38833
> (it's in CURRENT)
> I understand you were able to reproduce without e1000, but I'm curious if this
> solves em1 problem or not.

No change with 0bd6a167c156 cherry-picked.

How bothersome would it be to have a chicken switch to previous routing behavior? My -CURRENT is getting stale, cherry-picking security fixes would not be sustainable, -RELEASE and -STABLE branches often lack desktop stability fixes and features from -CURRENT. I've never found non-CURRENT to be usable for a semi-modern desktop.

I don't think my PPPoE DHCPv6-PD bridge setup was robust in the first place (e.g. net/dhcpcd receives PD, then needs to timeout before forking to background), and it would be a while to debug this either by you (no reproducer) or by me (free time strangled).
Comment 25 Evgenii Khramtsov 2026-07-28 16:10:56 UTC
(In reply to Evgenii Khramtsov from comment #24)

> How bothersome would it be to have a chicken switch to previous routing behavior?

Don't bother, I think I can figure it out on my own which hunk causes this, later when I would have time.
Comment 26 Gleb Smirnoff freebsd_committer freebsd_triage 2026-07-28 17:13:26 UTC
No need for a chicken switch! The d05d1f256082 doesn't change any behavior intentionally, it just exposed a bug. Your setup is quite complex, so please be patient and provide info to Pouria. The bug is going to be fixed.