Bug 249427 - Creating bHyve virtual machines cause network flaps intel
Summary: Creating bHyve virtual machines cause network flaps intel
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bhyve (show other bugs)
Version: 12.1-STABLE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-09-18 09:04 UTC by unspam
Modified: 2020-09-20 10:00 UTC (History)
1 user (show)

See Also:


Attachments
dmesg output (20.89 KB, text/plain)
2020-09-18 09:04 UTC, unspam
no flags Details
ifconfig output (2.46 KB, text/plain)
2020-09-18 09:10 UTC, unspam
no flags Details
bootloader output (193 bytes, text/plain)
2020-09-18 09:10 UTC, unspam
no flags Details
sysctl output (631 bytes, text/plain)
2020-09-18 09:11 UTC, unspam
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description unspam 2020-09-18 09:04:01 UTC
Created attachment 218042 [details]
dmesg output

Version: FreeBSD 12.2-BETA1

Server 1: Intel(R) PRO/1000 PCI-Express Network Driver
Server 2: Broadcom NetXtreme Gigabit Ethernet

I have two servers, one with intel (igb) and one with broadcom (bge).

I have also setup a bridge for bHyve but whenever I start or stop a bHyve Virtual Machine on the server with the Intel Network Adapter, I get a brief network flap

Server 1 - Intel

    igb3: link state changed to UP
    tap0: link state changed to DOWN
    igb3: link state changed to DOWN
    igb3: link state changed to UP

Server 2 - Broadcom

    tap0: link state changed to DOWN
    tap0: Ethernet address:
    tap0: promiscuous mode enabled
    nd6_dad_timer: cancel DAD on tap0 because of ND6_IFF_IFDISABLED.
    tap0: link state changed to UP
Comment 1 unspam 2020-09-18 09:10:01 UTC
Created attachment 218044 [details]
ifconfig output
Comment 2 unspam 2020-09-18 09:10:26 UTC
Created attachment 218045 [details]
bootloader output
Comment 3 unspam 2020-09-18 09:11:03 UTC
Created attachment 218046 [details]
sysctl output
Comment 4 Aleksandr Fedorov freebsd_committer 2020-09-20 10:00:22 UTC
This is not the problem from bhyve side. The main problem is adding if_tuntap(4) to if_bridge(4) with igb(4) interface. When you add an interface without support TSO/CSUM e.t.c to if_bridge(4) which already has igb(4) with TSO/CSUM enabled, than if_bridge(4) resets offload flags on all interfaces. This is leads to link flap on igb(4).

So, it's work as expected, and described in if_bridge(4) manual page:

     The TOE, TSO, TXCSUM and TXCSUM6 capabilities on all interfaces added to
     the bridge are disabled if any of the interfaces doesn't support/enable
     them.  The LRO capability is always disabled.  All the capabilities are
     restored when the interface is removed from the bridge.  Changing
     capabilities at run-time may cause NIC reinit and a link flap.

As a workaround, you can turn off the offload flags on the igb(4) interface at boot time in /etc/rc.conf