tested on: FreeBSD hemlock.eden.le-fay.org 14.0-RELEASE-p4 FreeBSD 14.0-RELEASE-p4 #8 releng/14.0-n265400-4edf3b80733e: Sat Feb 10 07:13:23 GMT 2024 root@hemlock.eden.le-fay.org:/data/src/obj/data/src/releng/14.0/amd64.amd64/sys/HEMLOCK amd64 sample broken configuration: # ifconfig tap0 create # ifconfig tap1 create # ifconfig bridge0 create addm ix1 addm tap0 up # ifconfig ix1.107 create vlan 107 vlandev ix0 # ifconfig bridge1 create addm ix1.107 addm tap1 up although this config appears to work, the bridge1 interface will not pass packets correctly, and a VM attached to tap1 will have no network connectivity. a VM attached to tap0 (on bridge) works fine. this configuration works correctly, when both bridges use tagged interfaces: # ifconfig tap0 create # ifconfig tap1 create # ifconfig ix1.106 create vlan 106 vlandev ix0 # ifconfig bridge0 create addm ix1.106 addm tap0 up # ifconfig ix1.107 create vlan 107 vlandev ix0 # ifconfig bridge1 create addm ix1.107 addm tap1 up i've only tested this with an ixgbe(4), so it may be specific to this hardware, but disabling all VLAN-related hardware acceleration with ifconfig did not fix the problem. ix1: <Intel(R) X520 82599ES (SFI/SFP+)> port 0xe000-0xe01f mem 0xfb680000-0xfb6fffff,0xfbc00000-0xfbc03fff irq 55 at device 0.1 on pci10 # ifconfig ix1 ix1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG> ether 90:e2:ba:1a:e1:15 inet6 fe80::92e2:baff:fe1a:e115%ix1 prefixlen 64 scopeid 0x3 media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>) status: active nd6 options=1<PERFORMNUD> there are several other people who have run into this issue, for example: - https://github.com/churchers/vm-bhyve/issues/390#issuecomment-785537662 - https://genneko.github.io/playing-with-bsd/networking/freebsd-vlan/#do-not-bridge-the-parent-interface-of-the-vlans maybe related to (or a dupe of) bug 240106.
# ifconfig ix1.107 create vlan 107 vlandev ix0 this, and the other references to "ix0" in the original report, should have said ix1 instead: # ifconfig ix1.107 create vlan 107 vlandev ix1 ix0 is not involved in test case, only ix1.
Maybe the same issue as bug 296652. You can try two things to verify this: a) tcpdump on the ix0 interface and on the receiving interface on another host. If you see outgoing ARP Replay packages on ix0 but can't see them on the receiving interface, this is an indication that the ix driver silently drops these frames. b) Replace the ix0 interface with a NIC that uses another driver, for example, bge or bnxt should work.
the system in the original report no longer has an ix(4) in it and this may have been fixed by bridge/vlan changes in the mean time, so i'm going to close this.