Bug 254343 - 13.0-RC2: adding a vtnet (VirtIO network) interface to bridge fails
Summary: 13.0-RC2: adding a vtnet (VirtIO network) interface to bridge fails
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.0-RELEASE
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks: 255054
  Show dependency treegraph
 
Reported: 2021-03-16 19:01 UTC by Felix Palmen
Modified: 2022-04-28 11:52 UTC (History)
12 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Palmen 2021-03-16 19:01:01 UTC
On a freshly installed 13.0-RC2 in bhyve:

root@test:~ # ifconfig bridge0 create
root@test:~ # ifconfig bridge0 addm vtnet0
ifconfig: BRDGADD vtnet0: Operation not supported
Comment 1 Kristof Provost freebsd_committer 2021-03-16 19:49:28 UTC
I've had a brief look.

Adding the interface to the bridge fails because it can't enable promiscuous mode on it.
It's not clear to me if setting promiscuous mode is supposed to work on vtnet interfaces without CtrlRxMode.
Either way, it looks like the interface flag handling is somewhat buggy as well, because vtnet interfaces which have been brought up cannot be added, but those who have not can be.

tl;dr: Needs to be investigated by someone familiar with if_vtnet.
Comment 2 Felix Palmen 2021-03-18 20:57:42 UTC
I now tested bringing my vtnet interfaces down, adding them to the bridges, and then bringing them up again.

FWIW, this seems to work fine with any traffic being forwarded through the bridge.
Comment 3 Felix Palmen 2021-03-20 14:45:39 UTC
As long as this is unfixed, here's the workaround for /etc/rc.conf:

1. make sure you configure your bridges *before* the vtnet devices.

2. assuming you have vtnet0 as a member to bridge0, add /etc/start_if.bridge0:

     ifconfig vtnet0 down


With settings like this, my machine comes up fine and all bridging works as expected.
Comment 4 Aleksandr Fedorov freebsd_committer 2021-04-21 15:21:48 UTC
I think this is regression in if_vtnet(4).

12-STABLE https://github.com/freebsd/freebsd-src/blob/stable/12/sys/dev/virtio/network/if_vtnet.c#L1103 :
			if ((ifp->if_flags ^ sc->vtnet_if_flags) &
			    (IFF_PROMISC | IFF_ALLMULTI)) {
				if (sc->vtnet_flags & VTNET_FLAG_CTRL_RX)
					vtnet_rx_filter(sc);
				else {
					ifp->if_flags |= IFF_PROMISC;
					if ((ifp->if_flags ^ sc->vtnet_if_flags)
					    & IFF_ALLMULTI)
						error = ENOTSUP;
				}
			}


RELENG 13.0 https://github.com/freebsd/freebsd-src/blob/releng/13.0/sys/dev/virtio/network/if_vtnet.c#L1297 :

	if ((ifp->if_flags ^ sc->vtnet_if_flags) &
	    (IFF_PROMISC | IFF_ALLMULTI)) {
		if ((sc->vtnet_flags & VTNET_FLAG_CTRL_RX) == 0)
			return (ENOTSUP);
		vtnet_rx_filter(sc);
	}

Therefore, if the hypervisor does not confirm support for VTNET_FLAG_CTRL_RX, the latest driver version cannot enable PROMISC mode.

For example, bhyve doesn't support VTNET_FLAG_CTRL_RX.

Technically, if the hypervisor doesn't support the VTNET_FLAG_CTRL_RX, then the interface is always in PROMISC mode.
I see no reason to prohibit switching the interface to PROMISC mode if VTNET_FLAG_CTRL_RX is not supported by the hypervisor.
Comment 5 Aleksandr Fedorov freebsd_committer 2021-04-21 15:24:35 UTC
To confirm the issue, simply start FreeBSD-13 in bhyve and call:
root:~ # tcpdump -i vtnet0
tcpdump: WARNING: vtnet0: That device doesn't support promiscuous mode
(BIOCPROMISC: Operation not supported)
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
Comment 6 Felix Palmen 2021-04-22 00:21:04 UTC
(In reply to Aleksandr Fedorov from comment #4)
> Technically, if the hypervisor doesn't support the VTNET_FLAG_CTRL_RX, then
> the interface is always in PROMISC mode.

This is confirmed by my observation that adding the interface to the bridge while it's down and bringing it up afterwards results in correct operation of the bridge, in bhyve. This clumsy workaround also makes tcpdump work fine.
Comment 7 Felix Palmen 2021-05-13 01:07:18 UTC
Sorry for the confusion with the FreeBSD version, I must have misused the bugzilla UI somehow… putting it on 13.0-RELEASE now that this choice is available, as this release is definitely affected.
Comment 8 Oleg Ginzburg 2021-06-04 12:37:29 UTC
(In reply to Felix Palmen from comment #2)
Same issue.

As workaround, we do not have to move the interface in 'down', just force/perform to persist promisc: 'ifconfig vtnet0 promisc'
Comment 9 Felix Palmen 2021-10-05 16:56:09 UTC
(In reply to Oleg Ginzburg from comment #8)
Well thanks, this is a *simpler* workaround!

Still, my expectation would be that I can just add a vtnet interface to a bridge, as it (obviously) works and the adding wasn't a problem in 12 either…
Comment 10 commit-hook freebsd_committer 2022-02-05 15:54:40 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=fc035df8af32d496885e5da26e519ce6a262c9bf

commit fc035df8af32d496885e5da26e519ce6a262c9bf
Author:     Aleksandr Fedorov <afedorov@FreeBSD.org>
AuthorDate: 2022-02-05 15:47:46 +0000
Commit:     Aleksandr Fedorov <afedorov@FreeBSD.org>
CommitDate: 2022-02-05 15:47:46 +0000

    if_vtnet(4): Restore the ability to set promisc mode.

    PR:     254343, 255054
    Reviewed by:    vmaffione (mentor), donner
    Approved by:    vmaffione (mentor), donner
    MFC after:      2 weeks
    Sponsored by:   vstack.com
    Differential Revision:  https://reviews.freebsd.org/D30639

 sys/dev/virtio/network/if_vtnet.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
Comment 11 commit-hook freebsd_committer 2022-02-23 11:10:03 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=009a56b2e416cf32b28be405876ab38bd7544d98

commit 009a56b2e416cf32b28be405876ab38bd7544d98
Author:     Aleksandr Fedorov <afedorov@FreeBSD.org>
AuthorDate: 2022-02-05 15:47:46 +0000
Commit:     Aleksandr Fedorov <afedorov@FreeBSD.org>
CommitDate: 2022-02-23 11:08:24 +0000

    if_vtnet(4): Restore the ability to set promisc mode.

    PR:     254343, 255054
    Reviewed by:    vmaffione (mentor), donner
    Approved by:    vmaffione (mentor), donner
    MFC after:      2 weeks
    Sponsored by:   vstack.com
    Differential Revision:  https://reviews.freebsd.org/D30639

    (cherry picked from commit fc035df8af32d496885e5da26e519ce6a262c9bf)

 sys/dev/virtio/network/if_vtnet.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)