Hi, I have setup site to site IPSEc between FreeBSD 14.0 Current and Palo Alto includes NAT with IPsec Phase 2 Networks NAT allows remote host to directly contact local host using their equivalent NAT addresses, provided that IPsec rules allow the traffic to pass. My Configuration: VPN GW : FreeBSD 14 CURRENT IP Adress: 65.65.65.2 Auth Method: PSK DH Group:DH14 (2048) Encryption Algorithm: AES-256 Hash Algorithm: SHA-256 Mode: Main SA Life Time: 86400 sec Encapsulation: ESP <Phase2> Encryption Algorithm: AES-256 Hash Algorithm: SHA-256 PFS: Enable, DH14 (2048) SA Life Time: 3600 sec Encrypted Domain: 85.85.85.86 NAT IP: 192.168.1.100 strongswan conf: local_ts = 85.85.85.86/32|192.168.1.100/32 remote_ts = 75.75.75.76/32 ############################# Remeote Side Configuration: VPN GW : FreeBSD 14 CURRENT IP Adress: 55.55.55.38 Auth Method: PSK DH Group:DH14 (2048) Encryption Algorithm: AES-256 Hash Algorithm: SHA-256 Mode: Main SA Life Time: 86400 sec Encapsulation: ESP <Phase2> Encryption Algorithm: AES-256 Hash Algorithm: SHA-256 PFS: Enable, DH14 (2048) SA Life Time: 3600 sec Encrypted Domain: 75.75.75.76 NAT IP: 192.168.250.100 Our local encryption domains behind nat, so if the remote peer wants to send data to my server (192.168.1.100) first the data arrives at 85.85.85.86 and then is redirected to 192.168.1.100: # IPSec virtual interface to LAN Redirect rule rdr log(all) on { enc0 } proto { tcp } from { 0.0.0.0/0 } to { 85.85.85.86/32 } port { 80 } -> 192.168.1.100 port 80 The problem is: When remote peer's server (192.168.250.100) tries to send data to me, PF recognizes it as coming from WAN (em0) you can see the pflog0 logs shown below But remote peer's traffic must come from enc0 root@test~ # tcpdump -ttt -n -e -i pflog0 action block tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytes 00:00:00.000000 rule 2048/0(match) [ridentifier 100000001]: block in on em0: 192.168.250.100.40900 > 85.85.85.86.80: Flags [S], seq 2765053457, win 1024, length 0 If i write a em0 pass rule for Remote Peer Server IP, connection continues as expected But why traffic comes from WAN(em0)? # PF pass in log(all) quick on { em0 } inet from { 192.168.250.100/32 } to { 192.168.1.100/32 } flags S/SA keep state ridentifier 37 #DEFAULT NAT RULES nat log(all) on em0 inet from { !em0 } to any port 500 -> (em0:0) static-port #static-port -> for_ipsec nat log(all) on em0 inet from { !em0 } to any -> (em0:0) port 1024:65535 #IPSec Allow Rule pass out log (all) quick route-to ( em0 65.65.65.1 ) proto udp from any to 55.55.55.38 port = 500 keep state label "ipsec_vpn_1_500" pass in log (all) quick on em0 reply-to ( em0 65.65.65.1 ) proto udp from 55.55.55.38 to any port = 500 keep state label "ipsec_vpn_1_500" pass out log (all) quick route-to ( em0 65.65.65.1 ) proto udp from any to 55.55.55.38 port = 4500 keep state label "ipsec_vpn_1_4500" pass in log (all) quick on em0 reply-to ( em0 65.65.65.1 ) proto udp from 55.55.55.38 to any port = 4500 keep state label "ipsec_vpn_1_4500" pass out log (all) quick route-to ( em0 65.65.65.1 ) from any to 55.55.55.38 keep state label "ipsec_vpn_80_esp" pass in log (all) quick on em0 reply-to ( em0 65.65.65.1 ) from 55.55.55.38 to any keep state label "ipsec_vpn_80_esp" # enc0 pass all pass in log(all) quick on { enc0 } inet from { any } to { any } flags S/SA keep state ridentifier 5 # setkey -DP 192.168.1.100[any] 75.75.75.76[any] any out ipsec esp/tunnel/65.65.65.2-55.55.55.38/unique:1 created: Aug 18 05:52:51 2023 lastused: Aug 18 09:33:50 2023 lifetime: 9223372036854775807(s) validtime: 0(s) spid=13254 seq=36 pid=2553 scope=global refcnt=1 75.75.75.76[any] 85.85.85.86[any] any in ipsec esp/tunnel/55.55.55.38-65.65.65.2/unique:1 created: Aug 18 05:52:51 2023 lastused: Aug 18 05:52:51 2023 lifetime: 9223372036854775807(s) validtime: 0(s) spid=13253 seq=941 pid=2553 scope=global refcnt=1
Hello, this error persists in FreeBSD-14 RELEASE. I tested it today. The response from Ipsec still seems to be coming from the wan interface. # Since it seems to be coming from the wan, it is blocked by entering my wrong rule. block drop in log quick on pppoe_igc1 from any to any tag "wan" ridentifier 100000001 # pflog 100000001]: block in on pppoe_igc1: 32.32.32.32.443 > 192.168.1.233.54146: Flags [S.], seg 1260103609, ack 142834308, win 65535,options [mss 1460, nop, wscale 8, nop, nop, sackoK], length o # my nat rule nat log on enc0 inet from { 192.168.1.0/24 } to { 32.32.32.32/32 } -> 10.200.100.1/32 # swanctl --list-sas ipsec2000: #18, ESTABLISHED, IKEv1, 006cc2d48e260de2_i 768af4a1fdc970bf_r* local '95.95.95.95' @ 95.95.95.8[4500] remote '212.212.212.212' @ 212.212.212.212[4500] AES_CBC-256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048 established 20485s ago, reauth in 56685s ipsec2001: #23, reqid 1, INSTALLED, TUNNEL-in-UDP, ESP:AES_CBC-256/HMAC_SHA2_256_128 installed 2757s ago, rekeying in 135s, expires in 843s in c2ad555f, 716504 bytes, 535 packets, 14249s ago out c89f82d4, 70100 bytes, 523 packets, 1143s ago local 10.200.100.1/32|192.168.1.0/24 remote 32.32.32.32/32|/0
(In reply to cArleone from comment #1) I've applied the initial analysis of the case. I've managed to test it using jails and vnet. From my testing both if_enc and pf work as expected, i.e. I can catch ESP or the payload on enc0. If you want you may run the same test on your system to verify that basic behavior work fine: - fetch the test file as /usr/tests/sys/netpfil/pf/ipsec - # echo "atf_test_program{name="ipsec", is_exclusive=true}" >> /usr/tests/sys/netpfil/pf/Kyuafile - # kyua test -k /usr/tests/sys/netpfil/pf/Kyuafile ipsec I believe the test does not cover your case completely. If more details and sequence of actions/manipulations over the IPsec traffic are provided, then probably I will have higher chances to reproduce the issue. CURRENT 5dbf886104b45fea255987ee2ae4828b8d002ffe was used for the testing.
Created attachment 251676 [details] /usr/tests/sys/netpfil/pf/ipsec
(In reply to Alfa from comment #0) > When remote peer's server (192.168.250.100) tries to send data to me, > PF recognizes it as coming from WAN (em0) As an idea, "sysctl net.inet.ipsec.filtertunnel=1" could be the reason. It reveals the payload on the same interface ESP came on. For example, a single "pass log all no state" pf rule provides the following sequence of matches for a single ping roundtrip: Case 1: sysctl net.inet.ipsec.filtertunnel=0 # off sysctl net.enc.in.ipsec_filter_mask=2 # after IPsec sysctl net.enc.out.ipsec_filter_mask=1 # before IPsec pflog: pass in on wan: ESP pass in on enc0: ICMP echo request pass out on enc0: ICMP echo reply pass out on wan: ESP Case 2: sysctl net.inet.ipsec.filtertunnel=1 # on sysctl net.enc.in.ipsec_filter_mask=2 # after IPsec sysctl net.enc.out.ipsec_filter_mask=1 # before IPsec pflog: pass in on wan: ESP pass in on enc0: ICMP echo request pass in on wan: ICMP echo request # this is the diff pass out on enc0: ICMP echo reply pass out on wan: ESP
I've got port forwarding working with "rdr on enc0" pf rule using the "pf_if_enc.diff" patch and the following settings: sysctl net.inet.ipsec.filtertunnel=0 sysctl net.enc.in.ipsec_filter_mask=2 sysctl net.enc.out.ipsec_filter_mask=1 If you still have interest in the mentioned setup then it will be appreciated to hear results of your testing. FYI: the patch was committed to 15-CURRENT.
Created attachment 251931 [details] pf_if_enc.diff