Hi, maybe first how to re-produce things. A 12.2-RELEASE system running Strongswan and using gif interfaces for route-based VPN's between several systems. In rc.conf: cloned_interfaces="gif0" ifconfig_gif0="inet 10.0.1.1 10.0.1.2 netmask 255.255.255.255 inet6 tunnel 2a01:... 2a02:..." Then using Strongswan or ipsec-tools to create (successfully) an ipsec tunnel from 2a01:... to 2a02:... In 12.2-RELEASE and earlier, it was possible setting additional routes via: route add -net 172.16.0.0/24 10.0.1.2 -ifp gif0 To reach hosts in 172.16.0.0/24 without any problem. netstat -rn reported: 172.16.0/24 10.0.1.2 UGS gif0 Creating the same route in 13.0-RELEASE, the route command seems to ignore the ifp parameter and instead creates: 172.16.0/24 10.0.1.2 UGS lo0 Which of course causes problems. Doing a ping -S 10.0.1.1 172.16.0.1 works.
Hi! Do I understand correctly that creating p2p gif0 interface & trying to use the remote end as a gateway should trigger the problem? I'm trying the following with 13-S (don't have 13-R handy, will test later today): 8:25 [0] m@devel0 s ifconfig gif0 gif0: flags=8011<UP,POINTOPOINT,MULTICAST> metric 0 mtu 1280 options=80000<LINKSTATE> inet 10.0.1.1 --> 10.0.1.2 netmask 0xffffffff groups: gif nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> 8:25 [0] m@devel0 netstat -4rnW | grep 10.0.1 10.0.1.1 link#4 UHS 152 16384 lo0 10.0.1.2 link#4 UH 149 1280 gif0 8:25 [0] m@devel0 s route add -net 172.16.0.0/24 10.0.1.2 add net 172.16.0.0: gateway 10.0.1.2 fib 0 8:25 [0] m@devel0 netstat -4rnW | grep 172.16.0.0 172.16.0.0/24 10.0.1.2 UGS 153 1280 gif0 8:25 [0] m@devel0 s route delete 172.16.0.0/24 delete net 172.16.0.0 fib 0 8:25 [0] m@devel0 s route add -net 172.16.0.0/24 10.0.1.2 -ifp gif0 add net 172.16.0.0: gateway 10.0.1.2 fib 0 8:25 [0] m@devel0 netstat -4rnW | grep 172.16.0.0 172.16.0.0/24 10.0.1.2 UGS 153 1280 gif0 Meanwhile, could you consider sharing `netstat -4rnW` output (at least for 10.0 routes) after gif0 creation? Also: could you please clarify the reason of adding `-ifp gif0`?
(In reply to Alexander V. Chernikov from comment #1) Tried with 13.0-R, the behaviour looks the same as in 13-S. Could you by any chance clarify your setup a bit more? The `netstat -4rnW` output requested above, custom sysctls, any other relevant network-related customisations?
Hi, sorry for the delay, here's the requested netstat output: Routing tables Internet: Destination Gateway Flags Nhop# Mtu Netif Expire default 178.63.103.193 UGS 13 1500 re0 10.0.1.1 link#12 UHS 19 16384 lo0 10.0.1.2 link#12 UH 15 1280 gif0 10.0.1.3 link#13 UHS 24 16384 lo0 10.0.1.4 link#13 UH 23 1280 gif1 10.0.1.5 link#14 UHS 27 16384 lo0 10.0.1.6 link#14 UH 26 1280 gif2 10.0.1.101 link#15 UHS 21 16384 lo0 10.0.2.1 link#17 UH 14 1500 tun0 10.0.2.2 link#17 UHS 16 16384 lo0 127.0.0.1 link#2 UH 1 16384 lo0 127.0.10.1 link#3 UH 4 16384 lo1 127.0.20.1 link#4 UH 5 16384 lo2 127.0.40.1 link#5 UH 6 16384 lo3 127.0.50.1 link#6 UH 7 16384 lo4 127.0.70.1 link#7 UH 8 16384 lo5 127.0.80.1 link#8 UH 9 16384 lo6 127.0.90.1 link#9 UH 10 16384 lo7 127.0.100.1 link#10 UH 11 16384 lo8 127.0.110.1 link#11 UH 12 16384 lo9 999.999.999.999/26 link#1 U 2 1500 re0 999.999.999.999 link#1 UHS 3 16384 lo0 192.168.10.0/28 10.0.1.101 UGS 22 1280 lo0 192.168.11.0/27 10.0.1.101 UGS 22 1280 lo0 192.168.11.6 link#15 UH 20 1280 lo0 192.168.13.0/27 10.0.1.101 UGS 22 1280 lo0 192.168.14.0/28 10.0.1.101 UGS 22 1280 lo9 999.999.999.999 is my public facing IP and the 192.168.x networks are the ones the gif tunnels should address. How to reproduce (manually): /sbin/ifconfig gif3 destroy /sbin/ifconfig gif3 create /sbin/ifconfig gif3 inet 10.0.1.101 192.168.11.6 netmask 255.255.255.255 /sbin/ifconfig gif3 inet6 tunnel 2bcd::1 2xyz::2 While 2bcd::1 is the external IPv6 of "my" host, 2xyz::2 the other side. Then start Strongswan as example to successfully establish an ipsec tunnel: ipsec status Security Associations (4 up, 0 connecting): tunnel[166]: ESTABLISHED 33 minutes ago, 2bcd::1 [C=WS, ST=State, L=Town, O=Local, OU=Local Ipsec, CN=host.something.any]...2xyz::2[C=WS, ST=State, L= Town, O=Local, OU=Local Ipsec, CN=host2.something.any] tunnel{1094}: INSTALLED, TUNNEL, reqid 4, ESP SPIs: ced63352_i 8ba22adf_o tunnel{1094}: 10.0.1.101/32 === 192.168.10.0/28 tunnel{1095}: INSTALLED, TUNNEL, reqid 5, ESP SPIs: c86c116f_i df56c110_o tunnel{1095}: 10.0.1.101/32 === 192.168.11.0/27 tunnel{1096}: INSTALLED, TUNNEL, reqid 6, ESP SPIs: cb01bb22_i 64178bdb_o tunnel{1096}: 10.0.1.101/32 === 192.168.13.0/27 tunnel{1097}: INSTALLED, TUNNEL, reqid 7, ESP SPIs: c7e69894_i 56292473_o tunnel{1097}: 10.0.1.101/32 === 192.168.14.0/28 Now running "ping -S" and it's no problem to ping any host behind 192.168.... though setting a route with: route add -net 192.168.14.0/28 10.0.1.101 -ifp gif3 Will result in the netstat result above. It works if the route command is set via: route add -net 192.168.14.0/28 192.168.11.6 Then "gif3" will be chosen as interface. Related to sysctl's the folllowing have been set: kern.coredump=0 kern.msgbuf_show_timestamp=1 security.jail.chflags_allowed=0 security.jail.allow_raw_sockets=0 security.jail.enforce_statfs=2 security.jail.set_hostname_allowed=0 security.jail.param.persist=1 security.jail.param.securelevel=3 security.bsd.hardlink_check_uid=1 security.bsd.hardlink_check_gid=1 security.bsd.see_jail_proc=0 security.bsd.see_other_gids=0 security.bsd.see_other_uids=0 security.bsd.stack_guard_page=1 security.bsd.unprivileged_proc_debug=0 security.bsd.unprivileged_read_msgbuf=0 security.bsd.see_jail_proc=0 net.inet.icmp.drop_redirect=1 net.inet.ip.check_interface=0 net.inet.ip.portrange.randomized=1 net.inet.ip.process_options=0 net.inet.ip.random_id=1 net.inet.ip.redirect=0 net.inet.tcp.drop_synfin=1 net.inet.tcp.icmp_may_rst=0 hw.usb.disable_enumeration=1 vfs.zfs.txg.timeout=5 vfs.zfs.no_scrub_prefetch=1 Running the route command as stated above like: route add -net 192.168.14.0/28 192.168.11.6 will work. If I specify it like: route add -net 192.168.14.0/28 10.0.1.101 -ifp gif3 it doesn't. The second made some sense, choose "10.0.1.101" as "gateway" for reaching subnets through the "gif3" ipsec tunnel.
I confirm this bug. Example: ifconfig epair1 create ifconfig epair1a 192.0.2.1/32 up ifconfig epair1b 192.0.2.2/32 up route -4q add -net 192.0.2.32/30 192.0.2.2 -ifp epair1a netstat -nrW4 | egrep '192.0.2.32/30' route -4q change -net 192.0.2.32/30 192.0.2.2 -ifp lo0 netstat -nrW4 | egrep '192.0.2.32/30' ifconfig epair1a destroy On 11.4-RELEASE-p9: ... 192.0.2.32/30 192.0.2.2 UGS 0 1500 epair1a 192.0.2.32/30 192.0.2.2 UGS 0 1500 lo0 On 14.1-RELEASE-p5: ... 192.0.2.32/30 192.0.2.2 UGS 7 1500 epair1b 192.0.2.32/30 192.0.2.2 UGS 7 1500 epair1b