On a freshly installed 13.0-RC2 in bhyve: root@test:~ # ifconfig bridge0 create root@test:~ # ifconfig bridge0 addm vtnet0 ifconfig: BRDGADD vtnet0: Operation not supported
I've had a brief look. Adding the interface to the bridge fails because it can't enable promiscuous mode on it. It's not clear to me if setting promiscuous mode is supposed to work on vtnet interfaces without CtrlRxMode. Either way, it looks like the interface flag handling is somewhat buggy as well, because vtnet interfaces which have been brought up cannot be added, but those who have not can be. tl;dr: Needs to be investigated by someone familiar with if_vtnet.
I now tested bringing my vtnet interfaces down, adding them to the bridges, and then bringing them up again. FWIW, this seems to work fine with any traffic being forwarded through the bridge.
As long as this is unfixed, here's the workaround for /etc/rc.conf: 1. make sure you configure your bridges *before* the vtnet devices. 2. assuming you have vtnet0 as a member to bridge0, add /etc/start_if.bridge0: ifconfig vtnet0 down With settings like this, my machine comes up fine and all bridging works as expected.
I think this is regression in if_vtnet(4). 12-STABLE https://github.com/freebsd/freebsd-src/blob/stable/12/sys/dev/virtio/network/if_vtnet.c#L1103 : if ((ifp->if_flags ^ sc->vtnet_if_flags) & (IFF_PROMISC | IFF_ALLMULTI)) { if (sc->vtnet_flags & VTNET_FLAG_CTRL_RX) vtnet_rx_filter(sc); else { ifp->if_flags |= IFF_PROMISC; if ((ifp->if_flags ^ sc->vtnet_if_flags) & IFF_ALLMULTI) error = ENOTSUP; } } RELENG 13.0 https://github.com/freebsd/freebsd-src/blob/releng/13.0/sys/dev/virtio/network/if_vtnet.c#L1297 : if ((ifp->if_flags ^ sc->vtnet_if_flags) & (IFF_PROMISC | IFF_ALLMULTI)) { if ((sc->vtnet_flags & VTNET_FLAG_CTRL_RX) == 0) return (ENOTSUP); vtnet_rx_filter(sc); } Therefore, if the hypervisor does not confirm support for VTNET_FLAG_CTRL_RX, the latest driver version cannot enable PROMISC mode. For example, bhyve doesn't support VTNET_FLAG_CTRL_RX. Technically, if the hypervisor doesn't support the VTNET_FLAG_CTRL_RX, then the interface is always in PROMISC mode. I see no reason to prohibit switching the interface to PROMISC mode if VTNET_FLAG_CTRL_RX is not supported by the hypervisor.
To confirm the issue, simply start FreeBSD-13 in bhyve and call: root:~ # tcpdump -i vtnet0 tcpdump: WARNING: vtnet0: That device doesn't support promiscuous mode (BIOCPROMISC: Operation not supported) tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel
(In reply to Aleksandr Fedorov from comment #4) > Technically, if the hypervisor doesn't support the VTNET_FLAG_CTRL_RX, then > the interface is always in PROMISC mode. This is confirmed by my observation that adding the interface to the bridge while it's down and bringing it up afterwards results in correct operation of the bridge, in bhyve. This clumsy workaround also makes tcpdump work fine.
Sorry for the confusion with the FreeBSD version, I must have misused the bugzilla UI somehow… putting it on 13.0-RELEASE now that this choice is available, as this release is definitely affected.
(In reply to Felix Palmen from comment #2) Same issue. As workaround, we do not have to move the interface in 'down', just force/perform to persist promisc: 'ifconfig vtnet0 promisc'
(In reply to Oleg Ginzburg from comment #8) Well thanks, this is a *simpler* workaround! Still, my expectation would be that I can just add a vtnet interface to a bridge, as it (obviously) works and the adding wasn't a problem in 12 either…
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fc035df8af32d496885e5da26e519ce6a262c9bf commit fc035df8af32d496885e5da26e519ce6a262c9bf Author: Aleksandr Fedorov <afedorov@FreeBSD.org> AuthorDate: 2022-02-05 15:47:46 +0000 Commit: Aleksandr Fedorov <afedorov@FreeBSD.org> CommitDate: 2022-02-05 15:47:46 +0000 if_vtnet(4): Restore the ability to set promisc mode. PR: 254343, 255054 Reviewed by: vmaffione (mentor), donner Approved by: vmaffione (mentor), donner MFC after: 2 weeks Sponsored by: vstack.com Differential Revision: https://reviews.freebsd.org/D30639 sys/dev/virtio/network/if_vtnet.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=009a56b2e416cf32b28be405876ab38bd7544d98 commit 009a56b2e416cf32b28be405876ab38bd7544d98 Author: Aleksandr Fedorov <afedorov@FreeBSD.org> AuthorDate: 2022-02-05 15:47:46 +0000 Commit: Aleksandr Fedorov <afedorov@FreeBSD.org> CommitDate: 2022-02-23 11:08:24 +0000 if_vtnet(4): Restore the ability to set promisc mode. PR: 254343, 255054 Reviewed by: vmaffione (mentor), donner Approved by: vmaffione (mentor), donner MFC after: 2 weeks Sponsored by: vstack.com Differential Revision: https://reviews.freebsd.org/D30639 (cherry picked from commit fc035df8af32d496885e5da26e519ce6a262c9bf) sys/dev/virtio/network/if_vtnet.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
assigning this net@ and marking as in progress, until a new release or at least the reporters confirm this working
(In reply to Mina Galić from comment #12) I think this can be closed, as the fix is landed in current/14 and stable/13, and 13.2 has been released at April 11, 2023 .