Bug 257499 - IPv6 link routes are not removed from routing table when interface goes down
Summary: IPv6 link routes are not removed from routing table when interface goes down
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.0-STABLE
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-07-30 08:45 UTC by Alexis Savin
Modified: 2021-08-04 14:31 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexis Savin 2021-07-30 08:45:01 UTC
Hello FreeBSD community,

I'm facing some kind of inconsistent behavior with the kernel's IPv6 implementation.

Directly Connected routes (link routes) are consistently created at startup or when an network interface is created.

However, these routes remain in the routing table when associated network interface goes down.
If then deleted (manually or by a specific process such as quagga), these routes are not restored in the routing table when the interface goes back up, leading to unexpected behavior.

Tested on 11.4-STABLE and 13.0-STABLE.

Step to Reproduce:
1. Configure an additional network interface on a FreeBSD device with IPv6 and get it up and running (connected + up)
```
ifconfig_em1="up  media autoselect -txcsum"
ifconfig_em1_alias0="inet 10.10.10.24 netmask 255.255.255.0"
ifconfig_em1_ipv6="inet6 2607:5300:203:2e61::2 prefixlen 64"

```

2. Observe that interface is up and that link routes are pushed into the routing table (both IPv4 and IPv6)
```
# ifconfig em1
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
	ether 08:00:27:57:85:9f
	hwaddr 08:00:27:57:85:9f
	inet6 fe80::a00:27ff:fe57:859f%em1 prefixlen 64 scopeid 0x2
	inet6 2607:5300:203:2e61::2 prefixlen 64
	inet 10.10.10.24 netmask 0xffffff00 broadcast 10.10.10.255
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active

# netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.254      UGS         em0
10.10.10.0/24      link#2             U           em1
10.10.10.24        link#2             UHS         lo0
127.0.0.1          link#3             UH          lo0
192.168.1.0/24     link#1             U           em0
192.168.1.18       link#1             UHS         lo0

# netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#3                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2607:5300:203:2e61::/64           link#2                        U           em1
2607:5300:203:2e61::2             link#2                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em1/64                     link#2                        U           em1
fe80::a00:27ff:fe57:859f%em1      link#2                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0

```

3. Take down the interface em1 and observe that IPv4 link routes (10.10.10.0/24) are removed from the routing table while IPv6 link routes (2607:5300:203:2e61::/64, 2607:5300:203:2e61::2 and fe80::%em1/64) remain.
```
# ifconfig em1 down
# ifconfig em1 
em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
	ether 08:00:27:57:85:9f
	hwaddr 08:00:27:57:85:9f
	inet6 fe80::a00:27ff:fe57:859f%em1 prefixlen 64 tentative scopeid 0x2
	inet6 2607:5300:203:2e61::2 prefixlen 64 tentative
	inet 10.10.10.24 netmask 0xffffff00 broadcast 10.10.10.255
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
# netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.254      UGS         em0
10.10.10.24        link#2             UHS         lo0
127.0.0.1          link#3             UH          lo0
192.168.1.0/24     link#1             U           em0
192.168.1.18       link#1             UHS         lo0
# netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#3                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2607:5300:203:2e61::/64           link#2                        U           em1
2607:5300:203:2e61::2             link#2                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em1/64                     link#2                        U           em1
fe80::a00:27ff:fe57:859f%em1      link#2                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
```

4. Delete remaining routes
```
route delete -inet6 fe80::%em1/64
route delete -inet6 2607:5300:203:2e61::2
route delete -inet6 2607:5300:203:2e61::/64

# netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#3                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::a00:27ff:fe57:859f%em1      link#2                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
```

# Bring the interface back up, observer that link routes are not restored
```
# ifconfig em1 up
# ifconfig em1 
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
	ether 08:00:27:57:85:9f
	hwaddr 08:00:27:57:85:9f
	inet6 fe80::a00:27ff:fe57:859f%em1 prefixlen 64 scopeid 0x2
	inet6 2607:5300:203:2e61::2 prefixlen 64
	inet 10.10.10.24 netmask 0xffffff00 broadcast 10.10.10.255
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
# netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#3                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::a00:27ff:fe57:859f%em1      link#2                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0

```

From my understanding, this is definitely not the expected behavior.
It's not consistent regarding FreeBSD IPv4 implementation, nor consistant regarding other kernel implementation.
On Linux for instance, these link routes are taken down when network interface goes down and brough back once back up.
So unless I'm missing some specific configuration setting, I suspect an issue with the IPv6 network stack.

I guess this doesn't affect everybody, we figured it out while using dynamic routing (Quagga/FRR +BGP).

Thank in advance for your feedback

Kind regards