Bug 146190 - [ipsec][patch] NAT traversal does not work in transport mode
Summary: [ipsec][patch] NAT traversal does not work in transport mode
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 8.0-STABLE
Hardware: Any Any
: Normal Affects Only Me
Assignee: Andrey V. Elsukov
Depends on:
Reported: 2010-04-30 17:40 UTC by Denis Antrushin
Modified: 2017-09-08 04:29 UTC (History)
6 users (show)

See Also:

file.diff (9.83 KB, patch)
2010-04-30 17:40 UTC, Denis Antrushin
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Denis Antrushin 2010-04-30 17:40:01 UTC
IPSEC NAT-Traversal does not work in transport mode.
It requires fixing of TCP/UDP checksums of packets protected by ESP
(or ignoring checksum mismatches), which is not implemented in kernel.

Fix: There are two ways to handle this issue: recalculate TCP/UDP checksums
using NAT-OA information from IKE exchange or just ignore checksums of
packets protected by ESP.
The former case requires support from IKED daemon.

Attached prototype patch implements both cases.
I didn't tried isakmpd, but racoon currently does not send NAT-OAi/NAT-OAr
info to the kernel, so without racoon patching, only ignoring of checksum
mismatch is available (put under sysctl control in the patch)

Patch attached with submission follows:
How-To-Repeat: Try to setup IPSEC transport mode connection between two hosts with NAT box
in the middle. Observe incoming packets dropped by kernel due to checksum
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2010-04-30 20:13:57 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-net

Over to maintainer(s).
Comment 2 VANHULLEBUS Yvan freebsd_committer 2010-05-03 08:57:47 UTC
Responsible Changed
From-To: freebsd-net->vanhu

Taking it, I'll also handle userland (racoon) part.
Comment 3 Denis.Antrushin 2010-05-06 20:56:36 UTC
There is one problem with this patch on SMP system: packet's checksum is 
fixed properly, but somehow source and destination ports in security 
policy index created from mbuf (ipsec4_get_ulp()) gets corrupted: they
become 0x4000 and 0x0000 (always the same) instead of actual port 
numbers. As a result, incoming packet is rejected by kernel
(my config has few ports configured for IPSEC and rest is denied).
If I remove 'deny all' rule, IPSEC connection works OK.
Also, I get couple of debug messages garbled in the log near just before
bad SP index appears:

kernel: DP key_freesav SA:0xffffff002d5eec00 (SPI 2540512i5p9s)e frcom 
46/_usirn/_srrecj/escyts:/ nmebtuif =p 
sec/xf0oxrfmf_fefsfp.fc:00603f3d7e500;,  irnepcfbc n= t now0 

On uniprocessor system everything works OK, so this looks like race
condition, which I don't understand: how the same mbuf could be 
processed in parallel by two threads? So far I've been unable to
figure out what's happening here...
Comment 4 Denis.Antrushin 2010-05-14 17:34:18 UTC
Please ignore my previous comment.
The same issue exists on uniprocessor system as well.
That bogus ports numbers are, in fact, not from TCP header, but are
pieces of IP header.
The problem is that tcp_input() (partially) zeroes out IP header when
computing TCP checksum and then call IPSEC stuff. But IPSEC uses
IP header length field to get TCP/UDP port numbers from mbuf
(ipsec4_get_ulp()). With zero ip_hl field, it access IP header instead
of TCP or UDP ones.

I don't know how to add new patch to existing PR, so I've put it here:

Also, a bit improved userland (racoon) patch is here:
Comment 5 Nat Howard 2015-01-22 18:33:11 UTC
I wasn't able to get the updated patch that DAntrushin mentions in comment #4 (http://den.homeunix.org/public_html/freebsd/ipsec_natt.v4.diff) -- that page is offline, and the wayback machine doesn't have a copy of it.  Does anyone have a copy?   I did try installing it, and it seems like exactly what I need, but I'm no kernel expert these days, and I had to guess at one or two updated ways of doing things to make it work.

Ideally, a working version would apply cleanly against the 10.1 kernel, and fix my IPSEC Transport mode problems!

Also, I think this bug would affect anyone trying to do L2TP/mpd5/ipsec with windows, so perhaps its importance could be updated?

Thanks for any help you can lend!
Comment 6 Nat Howard 2015-01-27 17:26:25 UTC
I've found an answer that works for me -- butchering the code so that all UDP checksums ae turned off.   I've posted this as a comment here:


Here's the text of it:

I found that the existing patches, the ones that let you set a sysctl called "net.inet.esp.esp_ignore_natt_cksum" to 1. These work fine for IOS and MacOS, both from behind NAT boxes and on fully-routed address, and succeed on windows on a fully-routed address... and then fail on windows when the windows client is behind a NAT wall.

The problem there seems to be that FreeBSD and windows have different notions of what the udp checksum should be, and the net.inet.esp.esp_ignore_natt_cksum fix isn't sufficient to fix this entirely. You can try this yourself, and see netstat -s report a couple of "with bad checksum" packets each time you try to connect with windows.

I therefore performed the following butchery on a 10.1 system, which turns off all UDP checksum-checking:


diff --git a/sys/netinet/udp_usrreq.c b/sys/netinet/udp_usrreq.c
index 6c7d341..b1c1e37 100644
--- a/sys/netinet/udp_usrreq.c
+++ b/sys/netinet/udp_usrreq.c
@@ -461,7 +461,9 @@ udp_input(struct mbuf *m, int off)
    * Checksum extended UDP header and data.
-  if (uh->uh_sum) {
+  /* NRH %%% DISABLE UDP CHECKSUMMING! Incoming stuff will show up as "no checksum", I think */
+/*  if (uh->uh_sum) { */
+  if (FALSE) {
     u_short uh_sum;
     if ((m->m_pkthdr.csum_flags & CSUM_DATA_VALID) &&

and find that I can now connect via L2TP/IPSEC from a windows box behind a NAT wall.

I'm sure someone can figure out the discrepancy in checksum computation, and I'm sure someone can make this an "ignore all UDP checksums" sysctl (I may do this last part myself, eventually), but for now, this works for me.
Comment 7 andywhite 2015-09-21 11:56:09 UTC
Can we get some progress on fixing checksums in transport mode, and the use of Client NAT and Server NAT ?

The patches here break mobile ipsec, specifically these two lines

memcpy(&((struct sockaddr_in *)&spidx.dst)->sin_addr,&((struct sockaddr_in *)iph2->src)->sin_addr, sizeof(in_addr_t)); 


memcpy(&((struct sockaddr_in *)&spidx.src)->sin_addr,&((struct sockaddr_in *)iph2->dst)->sin_addr, sizeof(in_addr_t)); 

in mobile ipsec uses (transport mode), the phase2 automatically generated policy is incorrect as it is formed between the NATd addresses and not the Iaddr and Raddr

Also, see bug report in NetBSD http://gnats.netbsd.org/47894
Comment 8 Luoqi Chen freebsd_committer 2016-10-24 00:02:13 UTC
Is anyone still actively working on this bug?
Comment 9 Andrey V. Elsukov freebsd_committer 2016-12-11 22:53:01 UTC
Yvan, I'm taking this PR, since I'm currently working with this code.
I already have submitted the call for testers request, so if someone is able to test, please try.
Comment 10 MyDevil.net Hosting 2017-04-14 00:35:59 UTC

any info about merge projects/ipsec into head?
Comment 11 Andrey V. Elsukov freebsd_committer 2017-04-14 08:48:35 UTC
(In reply to MyDevil.net Hosting from comment #10)
> Hello,
> any info about merge projects/ipsec into head?

It is already merged even in stable/11. You can try and report about the result.
Comment 12 Victor Volpe 2017-09-08 04:29:36 UTC
(In reply to Andrey V. Elsukov from comment #11)

There will no be merge fix for 10.3?

# uname -a
FreeBSD xxx 10.3-RELEASE-p21 FreeBSD 10.3-RELEASE-p21 #3: Fri Sep  8 01:02:53 BRT 2017     victor@xxx:/usr/obj/usr/src/sys/CUSTOM  amd64