Bug 252990

Summary: net/wireguard: WG don't use CARP IP as source
Product: Ports & Packages Reporter: Michael Muenz <m.muenz>
Component: Individual Port(s)Assignee: Bernhard Froehlich <decke>
Status: Closed FIXED    
Severity: Affects Only Me CC: franco, grehan, jason, kp, m.muenz, vvd
Priority: --- Flags: bugzilla: maintainer-feedback? (decke)
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Untested patch none

Description Michael Muenz 2021-01-25 10:29:40 UTC 
Hi,

I asked Jason regarding CARP HA with FreeBSD twice since when using CARP IP as the destination, the reply packet will be sent as the system IP and therefore doesn't match.

https://lists.zx2c4.com/pipermail/wireguard/2020-September/005840.html

Now I thought I can maybe do some tricks via pf and NAT.

My first test was outbound NAT with source as CARP and source port wireguard:

nat on igb0 inet proto udp from (self) port 51820 to any -> 82.34.74.60 static-port # Outbound NAT fuer WireGuard HA

But for reply packets, so when other side connects first, this doesn't match:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:24:19.887082 IP 80.151.56.127.19335 > 82.34.74.60.51820: UDP, length 148
11:24:19.887422 IP 82.34.74.61.51820 > 80.151.56.127.19335: UDP, length 92
11:24:25.037698 IP 80.151.56.127.19335 > 82.34.74.60.51820: UDP, length 148
11:24:25.038026 IP 82.34.74.61.51820 > 80.151.56.127.19335: UDP, length 92

82.34.74.60 = CARP IP
82.34.74.61 = IP of Firewall1

Then I tried a portforward when connecting to CARP IP redirecting to localhost, but it has the same result and I don't see the packets on interface lo0:

rdr log on igb0 inet proto udp from {any} to {82.34.74.60} port {51820} -> 127.0.0.1 port 51820 # Portforward auf localhost fuer WireGuard HA

I would guess it's blocked when I don't see the packet on lo0, but I still see the reply in the tcpdump going out.

Also, I flip between LTE and Wifi so it's nothing like pf state.

Any idea how to dig deeper into it? Maybe is there an option for outgoing NAT to be state-less like with usual pf rules?

I add kprovost@ like discussed via Twitter.
Comment 1 Kristof Provost freebsd_committer freebsd_triage 2021-01-25 13:20:05 UTC 
Created attachment 221902 [details]
Untested patch

From a very brief look at the WireGuard code (in kernel and ifconfig) I think that what's needed is to configure the local endpoint address (e_local in wg_endpoint in the kernel).
Ifconfig doesn't support that (yet), so that needs to be added too.

It doesn't actually look very hard to do. Here's a totally untested patch with what I think is needed. (Seriously, I've not even tried to compile this. Use at your own risk, etc, etc, rivers of fire, etc, etc)
Comment 2 Michael Muenz 2021-01-25 15:28:35 UTC 
Wow, thank you Kristof, really appreciated.
Never expected a patch already.

I'll test it and use as a base for discussion with wg-devs :)
Comment 3 Michael Muenz 2021-02-25 08:36:51 UTC 
Dear Kristov,

Sorry for coming back so late. On the same day I had a quick chat with Jason about the patch, but he was "not enthusiastic about knobs" :)

I saw that pfsense guys added a patch for the kernel implementation of wireguard:

https://redmine.pfsense.org/issues/11354

Maybe they will prepare the patch upstream (if not already).

Again, thanks for your (super-quick) work!

Best,
Michael
Comment 4 Peter Grehan freebsd_committer freebsd_triage 2021-03-02 11:58:30 UTC 
This should be fixed as part of the pfSense wg import (D28962, 95331c228a39)

Let us know how it goes.
Comment 5 Jason A. Donenfeld 2021-04-17 23:43:51 UTC 
I suspect this bug can now be closed as resolved?
Comment 6 Michael Muenz 2021-05-03 19:51:00 UTC 
Hi Jason,

sorry for the delay. I tested it with an alias on the latest -kmod version.
Packets from client to server main IP are replied correctly, packets initiated to client are sent via main IP. Same to alias IP.

Great job Jason! :)