Bug 243554 - multicast packets not seen on PHY bridge member
Summary: multicast packets not seen on PHY bridge member
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.1-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-net (Nobody)
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2020-01-24 02:01 UTC by James Blachly
Modified: 2020-10-22 04:00 UTC (History)
3 users (show)

See Also:


Attachments
diff against releng/12.1 (328 bytes, patch)
2020-01-24 03:12 UTC, Kyle Evans
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description James Blachly 2020-01-24 02:01:02 UTC
Summary: if_bridge(4) purports in the man page[1] to forward multicast traffic to all members of the bridge. However, this does not appear to be the case.

Extended summary: a bridge with members tap0, tap1, ... comprising bhyve virtual machines, as well as igb1 (the host's primary interface) forwards multicast traffic (mDNS specifically) among the taps, and OUT the PHY interface (igb1), however, the PHY interface does not receive inbound multicast traffic (on the FreeBSD side). Unicast traffic operates fine.

Details:
I use FreeBSD 12.1 as a VM host and ran into a problem with multicast packets over a bridge not being seen by programs [on the host] listening on the bridge's physical interface constituent (igb1), which I discovered when running avahi-daemon.

Briefly, my setup is as follows:
FreeBSD 11.2 host, bare metal, eth PHY igb1
    bridge0 with members igb1, tap0, tap1
VM linux guest virtio-net to tap0 to bridge on FreeBSD
VM freebsd guest virtio-net to tap1 to bridge on FreeBSD
Mac, ethernet to same switch as FreeBSD

mDNS query/response operates properly between the Mac and any of the
others (both physical and virtual), and all work in the converse
direction with the Mac.  The guests, all of which are constituents of
the bridge, are able to communicate via mDNS with one another.  However,
the guests are _unable_ to communicate with the host via mDNS.

tcpdump shows the query packets appearing on igb1, but truss on avahi-daemon
shows they are not received.

This means multicast packets are forwarded OUT all members of the
bridge, but not IN (at least, to physical interfaces -- they do
go both directions on the taps)

If I add an IP address to the bridge, avahi-daemon on the host binds to
the bridge interface directly and then receives incoming packets,
responding with the IP of the bridge. All then operates correctly,
except that the host now has two IPs on the same subnet of course.


Given that if_bridge(4) is described as a virtual switch [1] and

Given that unicast packets originating on one of the bridge's
taps are received by host programs bound to igb1, it seems to me that
anything bound to igb1 should also be receiving the multicast packets.


Is the discrepancy between handling of unicast and multicast packets

* an error specifically related to multicast and bridging, or
* an accident that unicast connections work? [unlikely]
* (or none of the above)

Kind regards and thanks in advance.



[1]      A bridge works like a switch, forwarding traffic from one interface to
     another.  Multicast and broadcast packets are always forwarded to all
     interfaces that are part of the bridge.  For unicast traffic, the bridge
     learns which MAC addresses are associated with which interfaces and will
     forward the traffic selectively.
Comment 1 James Blachly 2020-01-24 02:06:11 UTC
(In reply to James Blachly from comment #0)

I have to correct the below, I meant to type FreeBSD 12.1-RELEASE as my current version. The problem first manifest for me in version 11.2 and the text I pasted from an unanswered 2018 email to free-net ML. Sorry for any confusion.
Comment 2 Kyle Evans freebsd_committer 2020-01-24 02:08:48 UTC
CC'ing kp, since he's done some work on bridge, too
Comment 3 Kyle Evans freebsd_committer 2020-01-24 03:12:34 UTC
Created attachment 211002 [details]
diff against releng/12.1

It looks similar to some of the other observability problems I've fixed in the past. While the conventional setup is that the bridge alone would get the IP and not igb1, I think being able to observe the packets in question on igb1 is still important for debugging purposes.

There's also an incorrect looking comment in if_bridge.c that I'll dig into a little later; in bridge_forward(), we claim that tapping multicast/broadcast traffic isn't important because it will be reinjected into ether_input. I can't see how this is true. AFAICT these packets will travel bridge_broadcast() -> bridge_enqueue() -> if_transmit OR just bridge_enqueue() -> if_transmit, which will typically not involve ether_input.
Comment 4 James Blachly 2020-01-29 04:02:42 UTC
@Kyle Evans:

Patch did not do the trick. I can try to tcpdump this weekend (perhaps) to prove it, but behavior is same as before.

Also I note that the patch adds the call to ETHER_BPF_MTAP at a different place than in HEAD -- in the patch it adds right before bridge_enque, but in HEAD (but not in 12.1) the call is higher, outside the if{} block:

https://github.com/freebsd/freebsd/blob/0f0a35a04846fc4f4bdb6caa2852336d7de9447d/sys/net/if_bridge.c#L2058-L2060

I am sure this is intended but just thought I'd mention here for others.

Anyway, thanks for looking at this. If you can think of other diagnostics I can do that would be helpful, let me know.
Comment 5 Kyle Evans freebsd_committer 2020-01-29 04:16:29 UTC
(In reply to James Blachly from comment #4)

Interesting- re-reading your original post... so the proper packets are appearing via tcpdump, but they're not being tapped off to avahi?

That may be indicative of an architectural problem that we can't actually solve, and would instead require something like switch(4) [that we don't have yet] with a cleaner separation between the interface receiving local packets and the physical interface.