Bug 215041

Summary: [pf] Handshake to certain (fixed) hosts is dropped
Product: Base System Reporter: bsd
Component: kernAssignee: freebsd-pf (Nobody) <pf>
Status: New ---    
Severity: Affects Only Me CC: kp, maximos
Priority: ---    
Version: 11.0-RELEASE   
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
Captures from internal interfaces, external interface, and PF none

Description bsd 2016-12-04 12:37:47 UTC
Created attachment 177653 [details]
Captures from internal interfaces, external interface, and PF

(The same behavior was observed on 10.3-RELEASE, but remained unchanged after upgrading to 11-RELEASE)

I am running a bridge configured as follows:

cloned_interfaces="bridge0"
ifconfig_bridge0="addm em0 addm re0 SYNCDHCP"
ifconfig_em0="up -tso" # Internal interface
ifconfig_re0="up -tso" # External interface, connecting to NAT router

And this extremely minimal firewall config:

pass log all

The issue is that while PF is running, a host connected to the internal interface attempting to connect to 185.60.115.40:443 (something related to the login of Blizzard's battle.net service), will not receive a response to the initial SYN packet, see em0.pcap in the attached zip. However, on the external interface (see re0.pcap) the SYN/ACKs do plainly show up, both for the initial SYN an the retries. The logs of PF itself align with the view of the internal interface, the SYN/ACKs do not show up at all:

00:00:00.000000 rule 0..16777216/0(match): pass in on re0: 192.168.0.186.56465 > 185.60.115.40.443: Flags [S], seq 1914506337, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
00:00:00.000058 rule 0..16777216/0(match): pass out on bridge0: 192.168.0.186.56465 > 185.60.115.40.443: Flags [S], seq 1914506337, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
00:00:00.250999 rule 0..16777216/0(match): pass in on re0: 192.168.0.186.56467 > 185.60.115.40.443: Flags [S], seq 2119186033, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
00:00:00.000059 rule 0..16777216/0(match): pass out on bridge0: 192.168.0.186.56467 > 185.60.115.40.443: Flags [S], seq 2119186033, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0#

Disabling PF via "pfctl -d" instantly makes the problem disappear, "pfctl -e" makes it reappear just as reliably, so the issue definitely seems to be linked to PF and not a general networking or hardware/driver problem.
Comment 1 Kristof Provost freebsd_committer 2016-12-11 21:21:07 UTC
Just to confirm, your host (running pf) is bridging, not routing, right?
Comment 2 bsd 2016-12-11 21:45:24 UTC
(In reply to Kristof Provost from comment #1)
Correct, it's just working as a MAC-level bridge.
Comment 3 bsd 2016-12-15 09:03:05 UTC
Update: The problem seems to center on the line "pass [log] all". When I comment out the line and do "pfctl -F all -f configfile", the handshake to 185.60.115.40:443 works. Comment it in again, flush/reload, and the handshakes disappear again.

Same story with a slightly bigger config:

int_if="em0"
ext_if="re0"

rdr on $int_if inet proto tcp from any to any port www -> 127.0.0.1 port 3128

pass in quick on $int_if route-to lo0 inet proto tcp from any to 127.0.0.1 port 3128 keep state
pass all

-> Handshakes get dropped. Remove the "pass all", handshakes work.


Is this some intricacy of the rule syntax I'm missing or a legit bug?


PS: Sorry for not testing this earlier, a "pass all" ruleset seemed too minimal to have any effect...
Comment 4 Kristof Provost freebsd_committer 2016-12-30 14:18:06 UTC
That would suggest that the state tracking is involved in the problem.

Unfortunately I've had no luck reproducing this. Do you have any unusual sysctl's set?
Comment 5 bsd 2017-01-21 13:23:03 UTC
(In reply to Kristof Provost from comment #4)
Sorry, nothing related to networking there
Comment 6 Max 2017-01-23 05:45:53 UTC
(In reply to bsd from comment #3)

int_if="em0"
ext_if="re0"

rdr on $int_if inet proto tcp from any to any port www -> 127.0.0.1 port 3128

pass in quick on $int_if route-to lo0 inet proto tcp from any to 127.0.0.1 port 3128 keep state
pass all


Why "route-to"?..
It's just 
 rdr pass on $int_if inet proto tcp from any to any port www -> 127.0.0.1 port 3128
or
 rdr on $int_if inet proto tcp from any to any port www -> 127.0.0.1 port 3128
 pass in quick on $int_if inet proto tcp from any to 127.0.0.1 port 3128
or
 rdr on $int_if inet proto tcp from any to any port www tag PROXY -> 127.0.0.1 port 3128
 pass in quick on $int_if tagged PROXY


And there is something weird in your pflog-filtered.pcap...

Can you show full network and PF configs? ...and bridge0 dump?