Bug 186385 - pf don't work as expected in 10.0 with same configuration used on 9.1
Summary: pf don't work as expected in 10.0 with same configuration used on 9.1
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: Unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: Gleb Smirnoff
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-02-02 19:30 UTC by Nicolas
Modified: 2014-06-05 08:11 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nicolas 2014-02-02 19:30:00 UTC
It seem there is a bug or an important change requiring a different configuration in FreeBSD 10.0-RELEASE (and also FreeBSD 10.0-STABLE r261402 (20140202)) as pf don't work as expected in 10.0 with same configuration used on 9.1:

Network topology

[LAN vlan1: 192.168.0.1/24 - FreeBSD router 1 (pf) - WAN gre3] <-GRE tunnel-> [WAN gre3 FreeBSD router 2 (no firewall) - LAN vlan1 10.0.0.1/24] <-> Server 10.0.0.4/24

Rules

@0 pass quick inet proto tcp from 10.0.0.4 to 192.168.0.1 port = ssh flags S/SA keep state
  [ Evaluations: 1061      Packets: 0         Bytes: 0           States: 0     ]
  [ Inserted: uid 0 pid 10062 State Creations: 0     ]
@1 block drop in log all
  [ Evaluations: 1061      Packets: 62        Bytes: 6680        States: 0     ]
  [ Inserted: uid 0 pid 10062 State Creations: 0     ]
@2 block drop out log all
  [ Evaluations: 1061      Packets: 420       Bytes: 54339       States: 0     ]
  [ Inserted: uid 0 pid 10062 State Creations: 0     ]


FreeBSD router 1 with FreeBSD 9.1-RELEASE and the same pf rules and same network configuration allow SSH connection from 10.0.0.4 to 192.168.0.1.

FreeBSD router 1 with FreeBSD 10.0-RELEASE and the same pf rules and same network configuration block SSH connection from 10.0.0.4 to 192.168.0.1.


Logs pf on FreeBSD router 1 with FreeBSD 9.1-RELEASE:

FreeBSD 9.1-RELEASE work fine, there is no logs.


Logs pf on FreeBSD router 1 with FreeBSD 10.0-RELEASE:

pf: 2014-02-02 18:43:33.835486 rule 1..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57295: Flags [S.], seq 3441651510, ack 4193145984, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2222369418 ecr 1412851403], length 0
pf: 2014-02-02 18:43:37.840940 rule 1..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57295: Flags [S.], seq 3441651510, ack 4193145984, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 40982819 ecr 1412852404], length 0
pf: 2014-02-02 18:43:45.854816 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57295: Flags [S.], seq 3963336982, ack 4193145984, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 3613658962 ecr 1412854408], length 0
pf: 2014-02-02 18:44:01.886653 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57295: Flags [S.], seq 4098644495, ack 4193145984, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 779398636 ecr 1412858416], length 0
pf: 2014-02-02 18:45:19.989679 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57386: Flags [S.], seq 1915509128, ack 892855858, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 3712426345 ecr 1412877941], length 0
pf: 2014-02-02 18:45:20.984554 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57386: Flags [S.], seq 1915509128, ack 892855858, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2907582533 ecr 1412878191], length 0
pf: 2014-02-02 18:45:23.021138 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57386: Flags [S.], seq 1915509128, ack 892855858, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 770182855 ecr 1412878699], length 0
pf: 2014-02-02 18:45:27.023275 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57386: Flags [S.], seq 3184076358, ack 892855858, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 3100252351 ecr 1412879700], length 0
pf: 2014-02-02 18:45:35.037416 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57386: Flags [S.], seq 3184076358, ack 892855858, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 800877918 ecr 1412881704], length 0
pf: 2014-02-02 18:45:51.068999 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57386: Flags [S.], seq 2425515114, ack 892855858, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1412798070 ecr 1412885712], length 0
pf: 2014-02-02 18:48:04.553610 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57528: Flags [S.], seq 693546962, ack 165319204, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 4205042630 ecr 1412919083], length 0
pf: 2014-02-02 18:48:05.549581 rule 2..16777216/0(match): block out on gre3: 192.168.0.1.22 > 10.0.0.4.57528: Flags [S.], seq 693546962, ack 165319204, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1380441280 ecr 1412919333], length 0


On FreeBSD router 1 with FreeBSD 10.0-RELEASE, I have also done the following test:
"pass quick on gre3 all keep state" added = SSH connection from 10.0.0.4 to 192.168.0.1 blocked
"pass quick on gre3 all no state" added = SSH connection from 10.0.0.4 to 192.168.0.1 work
=> So if state are disabled it works.

Please note that between FreeBSD 9.1-RELEASE and FreeBSD 10.0-RELEASE I use the same /etc for make sure to have exactly the same rc.conf and pf configuration.

It's no related to TCP as UDP have the same problem (can't do SNMP connection from 10.0.0.4 to 192.168.0.1).

Fix: 

Note: pf have been rewritten between 9.1 and 10.0
How-To-Repeat: See full description.
Comment 1 Nicolas 2014-02-23 13:36:01 UTC
Related to:
kern/185876: ipfw not matching incoming packets decapsulating ipsec.
example l2tp/ipsec
kern/186755: ipsec tunnels don't work with pf or ipfw

After very long testing, i have discovered the route cause.

The revision 254519 break the firewall with IPsec.
http://svnweb.freebsd.org/base?view=revision&revision=254519

"Move the global M_SKIP_FIREWALL mbuf flags to a protocol layer specific
flag instead.  The flag is only used within the IP and IPv6 layer 3
protocols.

Because some firewall packages treat IPv4 and IPv6 packets the same the
flag should have the same value for both."

It seem that some code doesn't have been updated for allow firewall to
work with IPsec.

-- 
Nicolas DEFFAYET
Comment 2 Gleb Smirnoff freebsd_committer freebsd_triage 2014-03-12 14:33:36 UTC
State Changed
From-To: open->patched

Fixed in head in r263091. 


Comment 3 Gleb Smirnoff freebsd_committer freebsd_triage 2014-03-12 14:33:36 UTC
Responsible Changed
From-To: freebsd-bugs->glebius

Fixed in head in r263091.