I'm trying to nat packet incoming from enc0 to an other machine. The RDR statement in pf works for the incoming packet, but the reply from the other machine is forwarded back to the issuer without encryption. It might be because of the state matching on the reply, witch send back the reply bypassing the SPD rules. This is working flawlessly with encryption. I’m in a gateway setup, is there any chances to get the RDR working with ipsec in transport mode ? Computer1 em0 | ----->IPSEC-----> | em0 Computer2 em1 | -----> | em0 Computer3 | 10.11.1.3 172.31.0.1 10.56.1.10 10.56.1.224 Here is my setup : ifconfig em0 : 172.31.0.1/24 em1 : 10.56.1.10/24 pf.conf : rdr on enc0 inet from 10.11.0.0/16 to 172.31.0.1 tag "balance-1" -> 10.56.1.224 pass all setkey.conf : add -4 10.11.1.3 172.31.0.1 esp 0x100 -m transport -E rijndael-cbc "This is secret AES 256 bits key!" -A hmac-sha2-256 "This is secret HMAC 256 bits key"; add -4 172.31.0.1 10.11.1.3 esp 0x101 -m transport -E rijndael-cbc "This is secret AES 256 bits key!" -A hmac-sha2-256 "This is secret HMAC 256 bits key"; spdadd 10.11.1.3 172.31.0.1 any -P in ipsec esp/transport//require; spdadd 172.31.0.1 10.11.1.3 any -P out ipsec esp/transport//require; Results: ping -W1 -c1 -S 10.11.1.3 172.31.0.1 PING 172.31.0.1 (172.31.0.1) from 10.11.1.3: 56 data bytes tcpdump -ni em0 11:43:31.276852 IP 10.11.1.3 > 172.31.0.1: ESP(spi=0x00000100,seq=0x16), length 120 11:43:31.277594 IP 172.31.0.1 > 10.11.1.3: ICMP echo reply, id 49496, seq 0, length 64 Thank you for your help.
(In reply to Jérôme-Charles LALLEMAND from comment #0) > I'm trying to nat packet incoming from enc0 to an other machine. > > The RDR statement in pf works for the incoming packet, but the reply from > the other machine is forwarded back to the issuer without encryption. > > It might be because of the state matching on the reply, witch send back the > reply bypassing the SPD rules. No, this doesn't work, because PF consumes packet when it does "rdr". PF does output directly, thus a packet does not coming back to the IPsec. > This is working flawlessly with encryption. > > I’m in a gateway setup, is there any chances to get the RDR working with > ipsec in transport mode ? 11:43:31.277594 IP 172.31.0.1 > 10.11.1.3: ICMP echo reply, id 49496, seq 0, length 64 I guess you have another rdr rule to make this work? I'm not quite familiar with PF, but probably you can use route via loopback interface and make rdr via it. You need a route to the 10.11.1.3, so when encrypted packet will be routed via loopback interface, you can forward it to needed address..
(In reply to Andrey V. Elsukov from comment #1) > (In reply to Jérôme-Charles LALLEMAND from comment #0) > > I'm trying to nat packet incoming from enc0 to an other machine. Ah, I missed "nat" keyword here. So, replies just translated back by NAT? Can you show route table (netstat -rnf inet) from Computer2?
Thank you for your answers. Yes, it seems that the answers, go directly using the nat state. I got a route to 10.11.1.3, via em0. 172.31.0.1 is a loopback, advertised through em0 The routing table of Computer 2 follow: netstat -rnf inet Routing tables Internet: Destination Gateway Flags Netif Expire 10.56.0.0/16 10.56.1.1 UGS em1 10.56.1.0/24 link#2 U em1 10.56.1.10 link#2 UHS lo0 10.11.1.0/24 10.112.1.9 UGS em0 10.11.1.8/29 link#1 U em0 10.11.1.11 link#1 UHS lo0 127.0.0.1 link#6 UH lo0 172.31.0.1 link#9 UH lo0 192.168.0.0/16 10.56.1.1 UGS em1
I've run some other tests, using a default gateway toward lo0 and route-to rules to force ipsec to apply on lo0 with the correct src/dst. I don't understand where exactly the SPD apply : I've tested à setup with 2 SPD rules f 0.0.0.0/0 0.0.0.0/0 any -P in ipsec esp/transport//require; 0.0.0.0/0 0.0.0.0/0 any -P out ipsec esp/transport//require; And still, some packets are leaving m'y box unencrypted, despite the require flag
(In reply to Jérôme-Charles LALLEMAND from comment #5) > I've run some other tests, using a default gateway toward lo0 and route-to > rules to force ipsec to apply on lo0 with the correct src/dst. > > I don't understand where exactly the SPD apply : > > I've tested à setup with 2 SPD rules f > 0.0.0.0/0 0.0.0.0/0 any -P in ipsec esp/transport//require; > 0.0.0.0/0 0.0.0.0/0 any -P out ipsec esp/transport//require; > > And still, some packets are leaving m'y box unencrypted, despite the require > flag This how PF works. For some type of actions (route-to, nat, probably some others) it takes mbuf from the FreeBSD's network stack and then don't return it back. It does direct if_output() into network interface. SPD applied in the ip_output(). ip_output() ip_ipsec_output() IPsec processing begins: call to if_enc() PF handles packet and does direct if_output IPsec thinks that packet was dropped by PF and didn't finish its work ---------- You may try to use ipfw(4), it reuses ip_output() for NAT'ed and forwarded packets. For ipfw this will work like: ip_output() ip_ipsec_output() IPsec processing begins: call to if_enc() ipfw handles packet, do NAT then return to IPSec IPsec do encryption IPsec call ip_output() with encrypted packet ip_ipsec_output() decides that no IPsec required ipfw checks ESP packet if_output() using interface determined by route lookup
AFAIK, Franco Fichtner <franco at lastsummer dot de> is working on the solution, where PF will work like ipfw does.
I've tried IPFW and I get better results but I got still a problem with the reply packet. On my gateway the SPD for the reply match correctly, because when I disable it, the reply is forwarded encrypted. But when I enable the SPD in rule, the packet disappear, I don't know where. The SA corresponding seems OK, and I got no error log (ipsec.debug=1) Here is my setup : 10.160.64.3 10.112.1.3 172.31.0.1 LAN\ /WAN0 em0\ \ / \ |Client| ---->clear----> |Gateway| ---->ESP----> |Router| ---->ESP----> |Server| \ FreeBSD / \ \em0 eth0/ \eth1 10.160.64.100 10.112.1.1 172.31.0.254 The gateway is NATing source and destination. The setup works without encryption. But with encryption I can't get unencrypted reply to the client. With encryption on, the rule 211 get no more match. routing: ======== netstat -rnf inet Routing tables Internet: Destination Gateway Flags Netif Expire default 10.56.1.1 UGS em1 10.56.1.0/24 link#2 U em1 10.56.1.46 link#2 UHS lo0 10.112.1.0/29 link#1 U WAN0 10.112.1.3 link#1 UHS lo0 10.160.64.0/24 link#3 U LAN 10.160.64.3 link#3 UHS lo0 127.0.0.1 link#5 UH lo0 172.31.0.1 10.112.1.1 UGHS WAN0 sysctl values: ============== sysctl net.inet.ip.forwarding=1 sysctl net.inet.ip.fw.one_pass=0 sysctl net.enc.out.ipsec_bpf_mask=1 sysctl net.enc.out.ipsec_filter_mask=1 sysctl net.enc.in.ipsec_bpf_mask=2 sysctl net.enc.in.ipsec_filter_mask=2 ipfw rules: =========== ipfw -q -f flush ipfw -q add 100 allow ip from any to any via lo0 # Admin iface ipfw -q add 101 allow ip from any to any via em1 ipfw nat 10 config if LAN redirect_addr 172.31.0.1 10.160.64.3 ipfw nat 11 config ip 10.112.1.3 ipfw add 200 nat 10 tag 10 ip4 from any to 10.160.64.3 in recv LAN ipfw add 201 nat 11 tagged 10 out xmit WAN0 ipfw add 210 nat 11 tag 11 ip4 from any to 10.112.1.3 in recv WAN0 ipfw add 211 nat 10 out xmit LAN ipfw add 10000 allow ip4 from any to any setkey.conf: ============ flush; spdflush; add -4 10.160.64.100 172.31.0.1 esp 0x100 -m transport -E rijndael-cbc "This is secret AES 256 bits key0"; add -4 172.31.0.1 10.160.64.100 esp 0x101 -m transport -E rijndael-cbc "This is secret AES 256 bits key1"; spdadd 10.160.64.100 172.31.0.1 any -P out ipsec esp/transport//require; spdadd 172.31.0.1 10.160.64.100 any -P in ipsec esp/transport//require;
(In reply to Jérôme-Charles LALLEMAND from comment #8) > I've tried IPFW and I get better results but I got still a problem with the > reply packet. If I understand correctly, do you want to see on the WAN ESP packets with following addresses 10.112.1.3 <-> 172.31.0.1? > On my gateway the SPD for the reply match correctly, because when I disable > it, the reply is forwarded encrypted. But when I enable the SPD in rule, the > packet disappear, I don't know where. > > The SA corresponding seems OK, and I got no error log (ipsec.debug=1) > > Here is my setup : > > > 10.160.64.3 10.112.1.3 172.31.0.1 > LAN\ /WAN0 em0\ > \ / \ > |Client| ---->clear----> |Gateway| ---->ESP----> |Router| ---->ESP----> > |Server| > \ FreeBSD / \ > \em0 eth0/ \eth1 > 10.160.64.100 10.112.1.1 172.31.0.254 > > > The gateway is NATing source and destination. The setup works without > encryption. But with encryption I can't get unencrypted reply to the client. > With encryption on, the rule 211 get no more match. > > routing: > ======== > netstat -rnf inet > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > default 10.56.1.1 UGS em1 > 10.56.1.0/24 link#2 U em1 > 10.56.1.46 link#2 UHS lo0 > 10.112.1.0/29 link#1 U WAN0 > 10.112.1.3 link#1 UHS lo0 > 10.160.64.0/24 link#3 U LAN > 10.160.64.3 link#3 UHS lo0 > 127.0.0.1 link#5 UH lo0 > 172.31.0.1 10.112.1.1 UGHS WAN0 > > sysctl values: > ============== > > sysctl net.inet.ip.forwarding=1 > sysctl net.inet.ip.fw.one_pass=0 > sysctl net.enc.out.ipsec_bpf_mask=1 > sysctl net.enc.out.ipsec_filter_mask=1 > sysctl net.enc.in.ipsec_bpf_mask=2 > sysctl net.enc.in.ipsec_filter_mask=2 > > > ipfw rules: > =========== > > ipfw -q -f flush > ipfw -q add 100 allow ip from any to any via lo0 > # Admin iface > ipfw -q add 101 allow ip from any to any via em1 > > ipfw nat 10 config if LAN redirect_addr 172.31.0.1 10.160.64.3 > ipfw nat 11 config ip 10.112.1.3 > > ipfw add 200 nat 10 tag 10 ip4 from any to 10.160.64.3 in recv LAN > ipfw add 201 nat 11 tagged 10 out xmit WAN0 > ipfw add 210 nat 11 tag 11 ip4 from any to 10.112.1.3 in recv WAN0 > ipfw add 211 nat 10 out xmit LAN > > ipfw add 10000 allow ip4 from any to any > > > setkey.conf: > ============ > > flush; > spdflush; > > add -4 10.160.64.100 172.31.0.1 esp 0x100 -m transport -E rijndael-cbc "This > is secret AES 256 bits key0"; > add -4 172.31.0.1 10.160.64.100 esp 0x101 -m transport -E rijndael-cbc "This > is secret AES 256 bits key1"; > > spdadd 10.160.64.100 172.31.0.1 any -P out ipsec esp/transport//require; > spdadd 172.31.0.1 10.160.64.100 any -P in ipsec esp/transport//require; Inbound traffic is handled using corresponding SA. If ESP packet was not destined to your IP, it will not be decrypted. After decryption it will be checked to match with security policy. So, If I understand your needs correctly, you need something like: spdadd 10.160.64.100 172.31.0.1 any -P out ipsec esp/transport//require; spdadd 172.31.0.1 10.112.1.3 any -P in ipsec esp/transport//require; add -4 10.160.64.100 172.31.0.1 esp 0x100 -m transport -E rijndael-cbc "This is secret AES 256 bits key0"; add -4 172.31.0.1 10.112.1.3 esp 0x101 -m transport -E rijndael-cbc "This is secret AES 256 bits key1"; Then configure NAT: ipfw nat 11 config ip 10.112.1.3 ipfw add nat 11 ip from any to any via enc0 Configure if_enc: net.enc.out.ipsec_filter_mask=1 net.enc.in.ipsec_filter_mask=1 How it should work: ping -S 10.160.64.100 172.31.0.1 Gateway receives packet on the LAN interface. It routes packet via the WAN0 interface. Then it founds security policy and does IPsec transform. It does lookup for corresponding SA, this is SA with SPI 0x100. Then it calls to ipfw, where packet will be translated to ICMP 10.112.1.3 -> 172.31.0.1, then we should continue IPsec processing and encrypt packet. Then it just will be routed to 172.31.0.1. When we receive the reply, it should be ESP packet from 172.31.0.1 to 10.112.1.3 with SPI 0x101. Since the packet is destined to us, ESP protocol handler will try to decrypt it. It will do SA lookup for ESP proto with SPI 0x101 and destination 10.112.1.3. And it should find our SA. Then packet will be decrypted and after that it will be passed to ipfw. ipfw nat should translate it back to the ICMP 172.31.0.1->10.160.64.100 and return to IPsec processing. Since there no additional IPsec processing required, packet will be routed to 10.160.64.100. Something similar should be done on the 172.31.0.1 side.
> Inbound traffic is handled using corresponding SA. If ESP packet was > not destined to your IP, it will not be decrypted. After decryption it > will be checked to match with security policy. Thank you for your useful explanations. I understand way better now how it works. > If I understand correctly, do you want to see on the WAN ESP packets > with following addresses 10.112.1.3 <-> 172.31.0.1? Exactly my needs. I want to do RDR and NAT on the gateway. My test command on the client is : ping -S 10.160.64.100 10.160.64.3 I'll adapt your solution to do: 1/ RDR on LAN 2/ ipsec encryption 3/ NAT on enc0