Bug 209491
| Summary: | Broadcast storm with ipfw+natd+gateway | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | Rudolf Čejka <cejkar> | ||||||||
| Component: | kern | Assignee: | Andrey V. Elsukov <ae> | ||||||||
| Status: | Closed FIXED | ||||||||||
| Severity: | Affects Some People | CC: | ae, dipdill, eugen, gnn, marcinkk, pi, ports | ||||||||
| Priority: | --- | ||||||||||
| Version: | CURRENT | ||||||||||
| Hardware: | Any | ||||||||||
| OS: | Any | ||||||||||
| Attachments: |
|
||||||||||
As quick workaround, insert following rule to your ipfw ruleset: ipfw add 51 deny ip from any to any out recv em0 xmit em0 diverted The same problem here. Observed after upgrading from 10.3 to 11.0.
When I have 1 or 2 machines with natd in the network everything seems ok. Wireshark shows incereased number of broadcasts, but the network still works. After runnig a third and next machines the whole local network goes down because of broadcast storm :(
The workaround works fine. I've addedd to /etc/rc.firewall on all my machines after "divert natd" rule this one:
${fwcmd} add 51 deny ip from any to any out recv ${natd_interface} xmit ${natd_interface} diverted
*** Bug 215256 has been marked as a duplicate of this bug. *** Created attachment 182485 [details]
Proposed patch (untested)
Can you test this patch?
The problem appeared after transition to ip_tryforward(). Due to the lack of M_BCAST flag on the diverted mbuf, it does not considered as broadcast packet and handled by tryforward code, that sends such packets to the wire, instead of handling in ip_input().
Created attachment 182486 [details]
Proposed patch (untested)
Simplify the patch.
Created attachment 182488 [details]
teach divert(4) to restore M_[BM]CAST flags
My version of patch does nearly same thing but it additionally includes incoming multicast packets having same problem.
Another note: one should reboot system after rebuilding sources even if ipdivert.ko modules is used. It is not safe currently to re-load ipdivert.ko without reboot, so do not force it.
A commit references this bug: Author: ae Date: Wed May 17 09:04:09 UTC 2017 New revision: 318399 URL: https://svnweb.freebsd.org/changeset/base/318399 Log: Set M_BCAST and M_MCAST flags on mbuf sent via divert socket. r290383 has changed how mbufs sent by divert socket are handled. Previously they are always handled by slow path processing in ip_input(). Now ip_tryforward() is invoked from ip_input() before in_broadcast() check. Since diverted packet lost all mbuf flags, it passes the broadcast check in ip_tryforward() due to missing M_BCAST flag. In the result the broadcast packet is forwarded to the wire instead of be consumed by network stack. Add in_broadcast() check to the div_output() function. And restore the M_BCAST flag if destination address is broadcast for the given network interface. PR: 209491 MFC after: 1 week Changes: head/sys/netinet/ip_divert.c A commit references this bug: Author: ae Date: Wed May 24 09:03:46 UTC 2017 New revision: 318778 URL: https://svnweb.freebsd.org/changeset/base/318778 Log: MFC r318399: Set M_BCAST and M_MCAST flags on mbuf sent via divert socket. r290383 has changed how mbufs sent by divert socket are handled. Previously they are always handled by slow path processing in ip_input(). Now ip_tryforward() is invoked from ip_input() before in_broadcast() check. Since diverted packet lost all mbuf flags, it passes the broadcast check in ip_tryforward() due to missing M_BCAST flag. In the result the broadcast packet is forwarded to the wire instead of be consumed by network stack. Add in_broadcast() check to the div_output() function. And restore the M_BCAST flag if destination address is broadcast for the given network interface. PR: 209491 Changes: _U stable/11/ stable/11/sys/netinet/ip_divert.c |
After commit 290383 (replace fastforward path with tryforward), there is atleast one possible system configuration, which can be source of broadcast storm. Simply put in your /etc/rc.conf: firewall_enable="YES" firewall_type="OPEN" natd_enable="YES" natd_interface="em0" gateway_enable="YES" After reboot, you have a divert rule in ipfw with runnig natd: 00050 divert 8668 ip4 from any to any via em0 Then, simply send a broadcast to this system, and it responds with 2 * (TTL - 1) broadcasts sent back. Have atleast two of these systems on one local subnet with configured samba server, which sends one broadcast per minute, and this is practically sufficient to shut down your site with broadcast storm. Pre-290383 system just receives the broadcast: 08:55:25.167489 IP 10.0.2.4.21680 > 10.0.2.255.netbios-ns: [|SMB] After-290383 system with IP 10.0.2.15 receives the broadcast, and then responds with 126 broadcasts with reowned(translated) source address: 09:02:33.939027 IP 10.0.2.4.21490 > 10.0.2.255.netbios-ns: [|SMB] 09:02:33.939255 IP 10.0.2.15.21490 > 10.0.2.255.netbios-ns: [|SMB] 09:02:33.939303 IP 10.0.2.15.21490 > 10.0.2.255.netbios-ns: [|SMB] 09:02:33.939472 IP 10.0.2.15.44294 > 10.0.2.255.netbios-ns: [|SMB] 09:02:33.939524 IP 10.0.2.15.44294 > 10.0.2.255.netbios-ns: [|SMB] 09:02:33.939630 IP 10.0.2.15.40288 > 10.0.2.255.netbios-ns: [|SMB] 09:02:33.939661 IP 10.0.2.15.40288 > 10.0.2.255.netbios-ns: [|SMB] ... (I thought that it was just one or two replied broadcasts, but in my testing environment in VirtualBox with two systems, one sending broadcast and one storming, it really showed this output and I could not find any other explanation of this. However, example above with one samba server and three storming systems was really real...)