Bug 254520 - net/wireguard-kmod: pf not working
Summary: net/wireguard-kmod: pf not working
Status: Closed Not A Bug
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: Bernhard Froehlich
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-03-24 07:23 UTC by Matthias Pfaller
Modified: 2021-04-23 06:57 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (decke)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Pfaller 2021-03-24 07:23:13 UTC
I have the following rules in my pf.conf:

rdr on wg0 inet6 proto tcp from xxx::/48 to xxx::1 port = 443 -> xxx::1 port 344
rdr on wg0 inet  proto tcp from 10.33.68.0/23 to 10.33.66.130 port = 443 -> 10.33.66.130 port 344

With wireguard-go this is working as expected (by me). With wireguard-kmod this seems to get ignored. Connections coming in over wg0 will connect to port 443 and not to port 344.

regards, Matthias
Comment 1 Bernhard Froehlich freebsd_committer freebsd_triage 2021-03-24 07:26:43 UTC
Could you please update to the latest wireguard-kmod to check if it helps? There were some fixes that might fix this issue as well.
Comment 2 Matthias Pfaller 2021-03-24 08:07:49 UTC
I am at:
wireguard-kmod-0.0.20210323
Name           : wireguard-kmod
Version        : 0.0.20210323

I think this is the latest version.
Comment 3 Jason A. Donenfeld 2021-03-24 16:19:10 UTC
Can you supply a short self-contained script to fully reproduce the issue?
Comment 4 Jason A. Donenfeld 2021-03-24 16:29:55 UTC
I'm unable to reproduce this issue:

[root@devil ~]# cat /etc/pf.conf 
rdr on server2 inet proto tcp from 192.168.194.1 to 192.168.194.2 port 1 -> 192.168.194.2 port 2
[root@devil ~]# ifconfig server2
server2: flags=c1<UP,RUNNING,NOARP> metric 0 mtu 1420
        options=80000<LINKSTATE>
        inet 192.168.194.2 netmask 0xffffffff
        inet6 abcd::2 prefixlen 128
        groups: wg
        nd6 options=101<PERFORMNUD,NO_DAD>
[root@devil ~]# wg show server2
interface: server2
  public key: KTfdg1TsKmlT+jf4zzPIOImOCKs4JtKizPvW7Ci/tQ8=
  private key: (hidden)
  listening port: 25150

peer: mxG79uOKJwExqbEhl7zf+Lf00hvk+STestQBE1NWRAc=
  endpoint: 192.168.1.2:28413
  allowed ips: abcd::1/128, 192.168.194.1/32
  latest handshake: 1 minute, 10 seconds ago
  transfer: 62.91 KiB received, 60.92 KiB sent

Are you sure you're running the latest wireguard-kmod? You may need to kldunload the old one or reboot.
Comment 5 Matthias Pfaller 2021-03-24 21:06:13 UTC
Yes, I did a kldunload and I switched several times between wireguard-go and wireguard-kmod.

As an experiment I now redirected 443 to 25. When using wireguard-kmod I will see  my webserver. With wireguard-go I will get protocol errors (as expected).

I seem to have trouble with ipv6 as well. With wireguard-go ipv6 traffic is working, with wireguard-kmod it isn't.

ipv4 traffic is working. It just takes a while for my stubborn phone to switch to ipv4 :-)
Comment 6 Matthias Pfaller 2021-03-24 21:11:11 UTC
(In reply to Jason A. Donenfeld from comment #4)

Try redirecting to e.g. port 21 to 22. Now you should be able to do a "ssh -p 21" it should work.
Comment 7 Jason A. Donenfeld 2021-03-24 21:40:10 UTC
I put this in my /etc/pf.conf

rdr on server2 inet proto tcp from 192.168.194.1 to 192.168.194.2 port 21 -> 192.168.194.2 port 22
rdr on server2 inet6 proto tcp from abcd::1 to abcd::2 port 21 -> abcd::2 port 22

And then I did:

    pfctl -e
    pfctl -f /etc/pf.conf

Afterwards, I was able to use:

    ssh -p 21 root@192.168.194.2
and
    ssh -p 21 root@abcd::2

And that worked just fine.

So, I'm in fact unable to reproduce your issue. Please supply a standalone shell script that demonstrates it. Otherwise I won't be able to fix it.
Comment 8 Jason A. Donenfeld 2021-03-25 02:38:24 UTC
I just tried this on 12.2 too and it seems to be working.

I wonder if there's another complication in your setup that might be relevant here?

Probably trying to reduce this down to a simple shell script reproducer would go a long way.
Comment 9 Matthias Pfaller 2021-03-25 09:20:54 UTC
I'll try to do a simpler (and better testable) setup. It might take a while. I'll use wireguard-go for the moment.
Comment 10 Victor Volpe 2021-03-26 20:22:11 UTC
(In reply to Matthias Pfaller from comment #0)

Another pf problem over here. Rules based on wg0 is not recognized on boot. It's like the interface doesn't exists. I need to start pf manually every time I reboot my server. Nice!
Comment 11 Jason A. Donenfeld 2021-03-27 02:19:21 UTC
(In reply to Victor Volpe from comment #10)

I'm unable to reproduce this unfortunately. I ran `pfctl -f /etc/pf.conf && pfctl -e`, and then only afterwards did I enable and load wireguard, and my rule worked just fine.

Can you create a simple reproducer script, just as I've asked from the original reporter?
Comment 12 Bernhard Froehlich freebsd_committer freebsd_triage 2021-03-27 21:09:31 UTC
(In reply to Victor Volpe from comment #10)

To make it clear this was a different issue caused by a pf nat rule for a wg0 interface that did not exist when pf was starting. This is expected behaviour and one needs to write (wg0) in the rule to tell pf about it.
Comment 13 Jason A. Donenfeld 2021-04-13 17:56:11 UTC
Any additional reproduction info for this bug?
Comment 14 Matthias Pfaller 2021-04-14 05:36:05 UTC
(In reply to Jason A. Donenfeld from comment #13)
Sorry, I haven't had the time to setup a test case.
Comment 15 Matthias Pfaller 2021-04-14 10:28:34 UTC
(In reply to Matthias Pfaller from comment #14)
I just retried with wireguard-kmod-0.0.20210412. No change in behaviour :-( I'm reverting to wireguard-go again.
Comment 16 Jason A. Donenfeld 2021-04-14 13:30:14 UTC
This is not going to get fixed without additional information on how to reproduce the problem.
Comment 17 Matthias Pfaller 2021-04-23 06:57:44 UTC
This was a configuration error on my side.