255227 – route(8) ignoring ifp parameter after upgrade from 12.2-RELEASE to 13.0-RELEASE

Bug 255227 - route(8) ignoring ifp parameter after upgrade from 12.2-RELEASE to 13.0-RELEASE

Summary: route(8) ignoring ifp parameter after upgrade from 12.2-RELEASE to 13.0-RELEASE

Status:	New

Alias:	None

Product:	Base System
Classification:	Unclassified
Component:	bin (show other bugs)
Version:	13.0-RELEASE
Hardware:	Any Any

Importance:	--- Affects Only Me
Assignee:	Alexander V. Chernikov

URL:
Keywords:

Depends on:
Blocks:

Reported:	2021-04-19 15:39 UTC by net
Modified:	2021-05-17 13:14 UTC (History)
CC List:	1 user (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description net 2021-04-19 15:39:32 UTC

Hi,

maybe first how to re-produce things. A 12.2-RELEASE system running Strongswan and using gif interfaces for route-based VPN's between several systems.
In rc.conf:

cloned_interfaces="gif0"
ifconfig_gif0="inet 10.0.1.1 10.0.1.2 netmask 255.255.255.255 inet6 tunnel 2a01:...  2a02:..."

Then using Strongswan or ipsec-tools to create (successfully) an ipsec tunnel from 2a01:... to 2a02:...

In 12.2-RELEASE and earlier, it was possible setting additional routes via:

route add -net 172.16.0.0/24 10.0.1.2 -ifp gif0

To reach hosts in 172.16.0.0/24 without any problem. netstat -rn reported:

172.16.0/24    10.0.1.2         UGS         gif0

Creating the same route in 13.0-RELEASE, the route command seems to ignore the ifp parameter and instead creates:

172.16.0/24    10.0.1.2         UGS         lo0

Which of course causes problems. Doing a

ping -S 10.0.1.1 172.16.0.1

works.

Comment 1 Alexander V. Chernikov freebsd_committer

2021-04-22 08:29:53 UTC

Hi!

Do I understand correctly that creating p2p gif0 interface & trying to use the remote end as a gateway should trigger the problem?

I'm trying the following with 13-S (don't have 13-R handy, will test later today):

8:25 [0] m@devel0 s ifconfig gif0
gif0: flags=8011<UP,POINTOPOINT,MULTICAST> metric 0 mtu 1280
	options=80000<LINKSTATE>
	inet 10.0.1.1 --> 10.0.1.2 netmask 0xffffffff
	groups: gif
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
8:25 [0] m@devel0 netstat -4rnW | grep 10.0.1
10.0.1.1           link#4             UHS       152  16384        lo0
10.0.1.2           link#4             UH        149   1280       gif0
8:25 [0] m@devel0 s route add -net 172.16.0.0/24 10.0.1.2
add net 172.16.0.0: gateway 10.0.1.2 fib 0
8:25 [0] m@devel0 netstat -4rnW | grep 172.16.0.0
172.16.0.0/24      10.0.1.2           UGS       153   1280       gif0
8:25 [0] m@devel0 s route delete 172.16.0.0/24
delete net 172.16.0.0 fib 0
8:25 [0] m@devel0 s route add -net 172.16.0.0/24 10.0.1.2 -ifp gif0
add net 172.16.0.0: gateway 10.0.1.2 fib 0
8:25 [0] m@devel0 netstat -4rnW | grep 172.16.0.0
172.16.0.0/24      10.0.1.2           UGS       153   1280       gif0


Meanwhile, could you consider sharing `netstat -4rnW` output (at least for 10.0 routes) after gif0 creation?

Also: could you please clarify the reason of adding `-ifp gif0`?

Comment 2 Alexander V. Chernikov freebsd_committer

2021-04-22 20:12:14 UTC

(In reply to Alexander V. Chernikov from comment #1)
Tried with 13.0-R, the behaviour looks the same as in 13-S.

Could you by any chance clarify your setup a bit more?
The `netstat -4rnW` output requested above, custom sysctls, any other relevant network-related customisations?

Comment 3 net 2021-05-17 13:14:28 UTC

Hi,

sorry for the delay, here's the requested netstat output:

Routing tables

Internet:
Destination        Gateway            Flags   Nhop#    Mtu      Netif Expire
default            178.63.103.193     UGS        13   1500        re0
10.0.1.1           link#12            UHS        19  16384        lo0
10.0.1.2           link#12            UH         15   1280       gif0
10.0.1.3           link#13            UHS        24  16384        lo0
10.0.1.4           link#13            UH         23   1280       gif1
10.0.1.5           link#14            UHS        27  16384        lo0
10.0.1.6           link#14            UH         26   1280       gif2
10.0.1.101         link#15            UHS        21  16384        lo0
10.0.2.1           link#17            UH         14   1500       tun0
10.0.2.2           link#17            UHS        16  16384        lo0
127.0.0.1          link#2             UH          1  16384        lo0
127.0.10.1         link#3             UH          4  16384        lo1
127.0.20.1         link#4             UH          5  16384        lo2
127.0.40.1         link#5             UH          6  16384        lo3
127.0.50.1         link#6             UH          7  16384        lo4
127.0.70.1         link#7             UH          8  16384        lo5
127.0.80.1         link#8             UH          9  16384        lo6
127.0.90.1         link#9             UH         10  16384        lo7
127.0.100.1        link#10            UH         11  16384        lo8
127.0.110.1        link#11            UH         12  16384        lo9
999.999.999.999/26 link#1             U           2   1500        re0
999.999.999.999    link#1             UHS         3  16384        lo0
192.168.10.0/28    10.0.1.101         UGS        22   1280        lo0
192.168.11.0/27    10.0.1.101         UGS        22   1280        lo0
192.168.11.6       link#15            UH         20   1280        lo0
192.168.13.0/27    10.0.1.101         UGS        22   1280        lo0
192.168.14.0/28    10.0.1.101         UGS        22   1280        lo9

999.999.999.999 is my public facing IP and the 192.168.x networks are the ones the gif tunnels should address.

How to reproduce (manually):

/sbin/ifconfig gif3 destroy
/sbin/ifconfig gif3 create
/sbin/ifconfig gif3 inet 10.0.1.101 192.168.11.6 netmask 255.255.255.255
/sbin/ifconfig gif3 inet6 tunnel 2bcd::1 2xyz::2

While 2bcd::1 is the external IPv6 of "my" host, 2xyz::2 the other side. Then start Strongswan as example to successfully establish an ipsec tunnel:

ipsec status

Security Associations (4 up, 0 connecting):
     tunnel[166]: ESTABLISHED 33 minutes ago, 2bcd::1 [C=WS, ST=State, L=Town, O=Local, OU=Local Ipsec, CN=host.something.any]...2xyz::2[C=WS, ST=State, L= Town, O=Local, OU=Local Ipsec, CN=host2.something.any]
     tunnel{1094}:  INSTALLED, TUNNEL, reqid 4, ESP SPIs: ced63352_i 8ba22adf_o
     tunnel{1094}:   10.0.1.101/32 === 192.168.10.0/28
     tunnel{1095}:  INSTALLED, TUNNEL, reqid 5, ESP SPIs: c86c116f_i df56c110_o
     tunnel{1095}:   10.0.1.101/32 === 192.168.11.0/27
     tunnel{1096}:  INSTALLED, TUNNEL, reqid 6, ESP SPIs: cb01bb22_i 64178bdb_o
     tunnel{1096}:   10.0.1.101/32 === 192.168.13.0/27
     tunnel{1097}:  INSTALLED, TUNNEL, reqid 7, ESP SPIs: c7e69894_i 56292473_o
     tunnel{1097}:   10.0.1.101/32 === 192.168.14.0/28

Now running "ping -S" and it's no problem to ping any host behind 192.168.... though setting a route with:

route add -net 192.168.14.0/28 10.0.1.101 -ifp gif3

Will result in the netstat result above. It works if the route command is set via:

route add -net 192.168.14.0/28 192.168.11.6

Then "gif3" will be chosen as interface.

Related to sysctl's the folllowing have been set:

kern.coredump=0
kern.msgbuf_show_timestamp=1

security.jail.chflags_allowed=0
security.jail.allow_raw_sockets=0
security.jail.enforce_statfs=2
security.jail.set_hostname_allowed=0
security.jail.param.persist=1
security.jail.param.securelevel=3

security.bsd.hardlink_check_uid=1
security.bsd.hardlink_check_gid=1
security.bsd.see_jail_proc=0
security.bsd.see_other_gids=0
security.bsd.see_other_uids=0
security.bsd.stack_guard_page=1
security.bsd.unprivileged_proc_debug=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.see_jail_proc=0

net.inet.icmp.drop_redirect=1
net.inet.ip.check_interface=0
net.inet.ip.portrange.randomized=1
net.inet.ip.process_options=0
net.inet.ip.random_id=1
net.inet.ip.redirect=0
net.inet.tcp.drop_synfin=1
net.inet.tcp.icmp_may_rst=0

hw.usb.disable_enumeration=1

vfs.zfs.txg.timeout=5
vfs.zfs.no_scrub_prefetch=1

Running the route command as stated above like:

route add -net 192.168.14.0/28 192.168.11.6

will work. If I specify it like:

route add -net 192.168.14.0/28 10.0.1.101 -ifp gif3

it doesn't. The second made some sense, choose "10.0.1.101" as "gateway" for reaching subnets through the "gif3" ipsec tunnel.