Bug 251208 - bridge(4) combines multiple packets together when traffic load is high
Summary: bridge(4) combines multiple packets together when traffic load is high
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.1-RELEASE
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-net (Nobody)
Depends on:
Reported: 2020-11-17 08:20 UTC by Brian Buhrow
Modified: 2020-11-19 17:55 UTC (History)
1 user (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Brian Buhrow 2020-11-17 08:20:03 UTC
When running a bridge between em(4) and Xen PV virtual hosts, the
virtual hosts see a lot of oversized packets when inbound traffic is  heavy
from the physical interface (em0) to the virtual interface of the bridge
(xnb0.domid)  Here are some of the messages from the Xen VM that illustrate
the problem:

Nov 16 23:53:45 test-new /netbsd: xennet0: discarding oversize frame (len= 1518)

this happens over and over again, with two different frame sizes: 1518 and

The MTU on all interfaces is 1500.

Output of ifconfig bridge0 is as follows:

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 02:05:35:15:6f:00
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: xnb34.0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 6 priority 128 path cost 2000000
	member: xnb33.0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 5 priority 128 path cost 2000000
	        ifmaxaddr 0 port 1 priority 128 path cost 2000000
	groups: bridge

	All the packets the vm is complaining about originate on a host on the
other side of a Cisco switch which reports no giant packets on any of its
interfaces.  In addition, a tcpdump session on the bridge itself shows no
giant packets coming into it from any of its members.

	What I think is happening is that packets are coming in at such a
rate, approximately 7500 pps, that the output side of the bridge is
grabbing 2 packets when it processes the output queue and sendning them
out as 1.

	Additional notes:
This doesn't seem to affect hvm guests with Xen.  I don't know if that's
because the emulated ethernet chips in Qemu silently discard the oversized
frames or if it's because the output routines used from the bridge(4)
driver are different.  I'd guess the former, but I'm not familiar enough
with the bridge(4) code in FreeBSD to say for sure.

	to reproduce, set up a Xen PV guest on a bridge paired with an em(4)
interface and send it as much traffic as you can.  I'm driving the inbound
interface at about 100 mbits/sec, the maximum available on this particular
switch port.
Also, in case it isn't obvious, the FreeBSD installation in question, is
runing as the dom0 for this Xen installation.  It's also running
xen-4.14.0, with the same version of the xentools instaled.

Comment 1 Brian Buhrow 2020-11-19 17:55:45 UTC
Hello.   I forgot to note in the original bug report that there is a vlan
attached to  em0 in this configuration, but the vlan does not participate in
the bridge.  And, in checking with tcpdump, it appears that the oversized
packets do not coincide with the generation of vlan traffic through the em0
physical interface.  This leads me to believe this can be reproduced
without a vlan attached to the em0 interface, but in case that is not true,
I'm noting the presence of a vlan here as a way of more accurately
producing the conditions I'm seeing.