Bug 254343 - 13.0-RC2: adding a vtnet (VirtIO network) interface to bridge fails
Summary: 13.0-RC2: adding a vtnet (VirtIO network) interface to bridge fails
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.0-RELEASE
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks: 255054
  Show dependency treegraph
 
Reported: 2021-03-16 19:01 UTC by Felix Palmen
Modified: 2021-05-13 01:07 UTC (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Palmen 2021-03-16 19:01:01 UTC
On a freshly installed 13.0-RC2 in bhyve:

root@test:~ # ifconfig bridge0 create
root@test:~ # ifconfig bridge0 addm vtnet0
ifconfig: BRDGADD vtnet0: Operation not supported
Comment 1 Kristof Provost freebsd_committer 2021-03-16 19:49:28 UTC
I've had a brief look.

Adding the interface to the bridge fails because it can't enable promiscuous mode on it.
It's not clear to me if setting promiscuous mode is supposed to work on vtnet interfaces without CtrlRxMode.
Either way, it looks like the interface flag handling is somewhat buggy as well, because vtnet interfaces which have been brought up cannot be added, but those who have not can be.

tl;dr: Needs to be investigated by someone familiar with if_vtnet.
Comment 2 Felix Palmen 2021-03-18 20:57:42 UTC
I now tested bringing my vtnet interfaces down, adding them to the bridges, and then bringing them up again.

FWIW, this seems to work fine with any traffic being forwarded through the bridge.
Comment 3 Felix Palmen 2021-03-20 14:45:39 UTC
As long as this is unfixed, here's the workaround for /etc/rc.conf:

1. make sure you configure your bridges *before* the vtnet devices.

2. assuming you have vtnet0 as a member to bridge0, add /etc/start_if.bridge0:

     ifconfig vtnet0 down


With settings like this, my machine comes up fine and all bridging works as expected.
Comment 4 Aleksandr Fedorov freebsd_committer 2021-04-21 15:21:48 UTC
I think this is regression in if_vtnet(4).

12-STABLE https://github.com/freebsd/freebsd-src/blob/stable/12/sys/dev/virtio/network/if_vtnet.c#L1103 :
			if ((ifp->if_flags ^ sc->vtnet_if_flags) &
			    (IFF_PROMISC | IFF_ALLMULTI)) {
				if (sc->vtnet_flags & VTNET_FLAG_CTRL_RX)
					vtnet_rx_filter(sc);
				else {
					ifp->if_flags |= IFF_PROMISC;
					if ((ifp->if_flags ^ sc->vtnet_if_flags)
					    & IFF_ALLMULTI)
						error = ENOTSUP;
				}
			}


RELENG 13.0 https://github.com/freebsd/freebsd-src/blob/releng/13.0/sys/dev/virtio/network/if_vtnet.c#L1297 :

	if ((ifp->if_flags ^ sc->vtnet_if_flags) &
	    (IFF_PROMISC | IFF_ALLMULTI)) {
		if ((sc->vtnet_flags & VTNET_FLAG_CTRL_RX) == 0)
			return (ENOTSUP);
		vtnet_rx_filter(sc);
	}

Therefore, if the hypervisor does not confirm support for VTNET_FLAG_CTRL_RX, the latest driver version cannot enable PROMISC mode.

For example, bhyve doesn't support VTNET_FLAG_CTRL_RX.

Technically, if the hypervisor doesn't support the VTNET_FLAG_CTRL_RX, then the interface is always in PROMISC mode.
I see no reason to prohibit switching the interface to PROMISC mode if VTNET_FLAG_CTRL_RX is not supported by the hypervisor.
Comment 5 Aleksandr Fedorov freebsd_committer 2021-04-21 15:24:35 UTC
To confirm the issue, simply start FreeBSD-13 in bhyve and call:
root:~ # tcpdump -i vtnet0
tcpdump: WARNING: vtnet0: That device doesn't support promiscuous mode
(BIOCPROMISC: Operation not supported)
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
Comment 6 Felix Palmen 2021-04-22 00:21:04 UTC
(In reply to Aleksandr Fedorov from comment #4)
> Technically, if the hypervisor doesn't support the VTNET_FLAG_CTRL_RX, then
> the interface is always in PROMISC mode.

This is confirmed by my observation that adding the interface to the bridge while it's down and bringing it up afterwards results in correct operation of the bridge, in bhyve. This clumsy workaround also makes tcpdump work fine.
Comment 7 Felix Palmen 2021-05-13 01:07:18 UTC
Sorry for the confusion with the FreeBSD version, I must have misused the bugzilla UI somehow… putting it on 13.0-RELEASE now that this choice is available, as this release is definitely affected.