When setting a policy with the src port like this, spdadd -n 2620:10e:6024:a::2[2049] 2620:10e:6026:b::/64[any] tcp -P out ipsec esp/transport//require; the outbound traffic from 2049/tcp (response to NFSv4 client request) was not encrypted. If I use [any] then it will be.
To which part of the system does this PR apply?
IPsec?
The IPsec stack. The line defines an IPsec security policy requiring all TCP traffic from 2620:10e:6024:a::2 with a source port of 2049 to 2620:10e:6026:b::/64 be protected by IPsec in transport mode. I suspect this isn't a technical problem but a different between what the user expects and IPsec does. IPsec requires one policy for each direction unless the policy is general enough to match both directions. Successful session establishment also requires a correctly configured IKE daemon on both sides.
Successful session establish only needs SAD entry, which can be configured with setkey, e.g. add -n 2620:10e:6024:a::2 2620:10e:6026:b:b414:48e2:de5d:d145 esp 2000 -E aes-gcm-16 0x9ea908964f5fdd2d8a8d249b8e2313c5e7a5b425; It doesn’t actually require IKE. But that’s not the problem. Change the policy to src port 2049, it doesn’t work. Change the policy to src port any, and it works. Even though the traffic is originating from 2049 because it’s the response to NFSv4 client requests.
Just to be clear, there is a policy for the other direction, spdadd -n 2620:10e:6026:b::/64[any] 2620:10e:6024:a::2[2049] tcp -P in ipsec esp/transport//require; that works just fine for the inbound traffic to 2049/tcp (NFSv4 server).
It looks like NFS doesn't use user level sockets. IPsec doesn't use SPD port matching without corresponding PCBs.
I think the problem is in ipsec_setspidx_inpcb() function. It should be fixed in the similar way: https://svnweb.freebsd.org/base?view=revision&revision=309358
Created attachment 178602 [details] Proposed patch (untested) Can you test this patch?
Comment on attachment 178602 [details] Proposed patch (untested) This worked for me. root@safety:/usr/src/sys/netipsec # patch < ~/ipsec.c.diff Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |Index: sys/netipsec/ipsec.c |=================================================================== |--- sys/netipsec/ipsec.c (revision 311647) |+++ sys/netipsec/ipsec.c (working copy) -------------------------- Patching file ipsec.c using Plan A... Hunk #1 succeeded at 241. Hunk #2 succeeded at 344 (offset 1 line). Hunk #3 succeeded at 501 (offset 1 line). Hunk #4 succeeded at 511 (offset 1 line). done root@safety:~ # setkey -DP fe80::%em0/64[any] fe80::a00:27ff:fefc:de09%em0[2049] tcp in ipsec esp/transport//require spid=1 seq=1 pid=806 refcnt=1 fe80::a00:27ff:fefc:de09%em0[2049] fe80::%em0/64[any] tcp out ipsec esp/transport//require spid=2 seq=0 pid=806 refcnt=1 I was able to NFSv4 mount a filesystem, and tcpdump is showing me that everything is ESP.
A commit references this bug: Author: ae Date: Sun Jan 8 12:40:08 UTC 2017 New revision: 311679 URL: https://svnweb.freebsd.org/changeset/base/311679 Log: Add direction argument to ipsec_setspidx_inpcb() function. This function is used only by ipsec_getpolicybysock() to fill security policy index selector for locally generated packets (that have INPCB). The function incorrectly assumes that spidx is the same for both directions. Fix this by using new direction argument to specify correct INPCB security policy - sp_in or sp_out. There is no need to fill both policy indeces, because they are overwritten for each packet. This fixes security policy matching for outbound packets when user has specified TCP/UDP ports in the security policy upperspec. PR: 213869 MFC after: 1 week Changes: head/sys/netipsec/ipsec.c
A commit references this bug: Author: ae Date: Sun Jan 15 15:43:19 UTC 2017 New revision: 312233 URL: https://svnweb.freebsd.org/changeset/base/312233 Log: MFC r311679: Add direction argument to ipsec_setspidx_inpcb() function. This function is used only by ipsec_getpolicybysock() to fill security policy index selector for locally generated packets (that have INPCB). The function incorrectly assumes that spidx is the same for both directions. Fix this by using new direction argument to specify correct INPCB security policy - sp_in or sp_out. There is no need to fill both policy indeces, because they are overwritten for each packet. This fixes security policy matching for outbound packets when user has specified TCP/UDP ports in the security policy upperspec. PR: 213869 Changes: _U stable/11/ stable/11/sys/netipsec/ipsec.c
There might still be a bug even after the patch. I'm using the patch on an NFSv4 server, and encountered a new NFSv4 mount that did not work. So I reverted to [any] on the source in the policy and got the mount to succeed. 19:23:21.702627 IP6 coconut > safe: ESP(spi=0xb04a3109,seq=0x7), length 76 19:23:21.702666 IP6 safe.nfsd > coconut.849: Flags [S.], seq 3009240135, ack 1156380226, win 65535, options [mss 1440,nop,wscale 9,sackOK,TS val 182932015 ecr 615648], length 0
(In reply to Jason Mader from comment #12) > There might still be a bug even after the patch. I'm using the patch on an > NFSv4 server, and encountered a new NFSv4 mount that did not work. So I > reverted to [any] on the source in the policy and got the mount to succeed. > > 19:23:21.702627 IP6 coconut > safe: ESP(spi=0xb04a3109,seq=0x7), length 76 > 19:23:21.702666 IP6 safe.nfsd > coconut.849: Flags [S.], seq 3009240135, ack > 1156380226, win 65535, options [mss 1440,nop,wscale 9,sackOK,TS val > 182932015 ecr 615648], length 0 At least I need the policies list at time when it doesn't work. Do you use some IKEd? It would be good if you try to reproduce this problem with projects/ipsec. You can apply the patch to stable/11 with 'svn patch': https://lists.freebsd.org/pipermail/freebsd-net/2017-January/046888.html