Bug 196361

Summary: Constrain IPv6 routes to each FIB (Consistent with IPv4 route behaviour)
Product: Base System Reporter: Jason Healy <jhealy>
Component: kernAssignee: Alan Somers <asomers>
Status: Closed FIXED    
Severity: Affects Only Me CC: asomers, bz, hrs, jhealy, jhujhiti, melifaro, philipp
Priority: --- Keywords: needs-patch, needs-qa
Version: 10.1-RELEASEFlags: asomers: mfc-stable10-
asomers: mfc-stable11+
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
Respect net.add_addr_allfibs=0 for inet6 (revision 1)
none
inet6 test cases none

Description Jason Healy 2014-12-29 18:06:00 UTC
When configuring multiple FIBs, IPv4 routes are properly constrained to each FIB, but IPv6 routes leak between FIBs.

Steps to reproduce:

loader.conf: set net.fibs=3

Turn off default of adding new interfaces routes to all FIBs:

# sysctl net.add_addr_allfibs=0

Create two new interfaces, with IPv4 and IPv6 addresses on each, with each assigned to its own FIB:

# ifconfig em1 inet 192.0.2.1/24 fib 1
# ifconfig em1 inet6 2001:db8:dead:beef::1/64 fib 1
# ifconfig em2 inet 203.0.113.1/24 fib 2
# ifconfig em2 inet6 2001:db8:cafe:babe::1/64 fib 2

Consult the routing table for each FIB:

# setfib -F 1 netstat -rn

Routing tables (fib: 1)

Internet:
Destination        Gateway            Flags      Netif Expire
192.0.2.0/24       link#2             U           em1
192.0.2.1          link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
2001:db8:cafe:babe::/64           link#3                        U           em2
2001:db8:dead:beef::/64           link#2                        U           em1
2001:db8:dead:beef::1             link#2                        UHS         lo0
fe80::%em1/64                     link#2                        U           em1
fe80::a00:27ff:fef6:162a%em1      link#2                        UHS         lo0
fe80::%em2/64                     link#3                        U           em2
fe80::%lo0/64                     link#5                        U           lo0


# setfib -F 2 netstat -rn

Routing tables (fib: 2)

Internet:
Destination        Gateway            Flags      Netif Expire
203.0.113.0/24     link#3             U           em2
203.0.113.1        link#3             UHS         lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
2001:db8:cafe:babe::/64           link#3                        U           em2
2001:db8:cafe:babe::1             link#3                        UHS         lo0
2001:db8:dead:beef::/64           link#2                        U           em1
fe80::%em1/64                     link#2                        U           em1
fe80::%em2/64                     link#3                        U           em2
fe80::a00:27ff:fe62:d267%em2      link#3                        UHS         lo0
fe80::%lo0/64                     link#5                        U           lo0


Note that as expected, the IPv4 routes are constrained to their FIB (192.0.2.0 to FIB 1 and 203.0.113.0 to FIB 2). However, the IPv6 routes (deadbeef and cafebabe) leak between the FIBs; both prefixes that I add are listed in both FIBs (as well as the link-local stuff).

Posted to freebsd-net and bz@ said this is a regression and to file this PR.

Thanks,

Jason
Comment 1 Erick Turnquist 2015-10-26 00:23:39 UTC
From my perspective as an amateur kernel hacker, this appears to be intentional behavior. We have comments in the "FIB-aware" code like this: "/* Announce arrival of local address to all FIBs. */".

I do not understand the logic here.
Comment 2 Jason Healy 2015-10-27 00:28:10 UTC
Sorry... don't understand the logic of the code in the kernel, or of the bug as reported?  Participants on freebsd-net concurred that this was a regression, so I believe the bug to be valid.  As to the kernel sources, I can't say.

The FIB code is working correctly for IPv4, so it's only the v6 that seems to leak routes.

Thanks,

Jason
Comment 3 Erick Turnquist 2015-10-27 01:28:49 UTC
Sorry - I should have been more verbose. I agree with your assessment. What doesn't make sense to me is the logic in the code that clearly intends to add these routes to all FIBs. The logic existed in the merge at r231852, and I don't see the code ever changing, so I'm not sure how it's a regression (I did see your mailing list post - in fact bz@, the committer of r231852, was the one who said it's a regression).

There is plenty of other good multi-FIB code in IPv6 areas of the kernel, so I must be missing a good reason for this behavior to exist.
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2015-10-27 03:02:31 UTC
Adjust summary to reflect desired behaviour, since existing behaviour appears intended (documented), thus this is more a  'change request' than a 'bug report'.
 CC original committer
Comment 5 Alexander V. Chernikov freebsd_committer 2015-10-30 17:02:01 UTC
As far as I understand, initially it was done to make in[6]_localip and similar functions just work.
alanjude@ put a lot of stuff in making IPv6 code fib-aware.
(So one day net.add_addr_allfibs=0 can become the default), but things are far from here currently.

The actual logic resides in nd6_prefix_onlink_rtrequest() and it (currently) ignores allfibs tunable.

We should either make it work like rtinit() or even merge it into rtinit() (with proper testing).
Comment 6 Erick Turnquist 2015-10-30 23:48:36 UTC
It's a bit more complex than that. I think we also need to install link-local multicast routes in the correct FIB for each interface. I'm working on a patch against 10.2, but this is my first time looking at FreeBSD kernel code...
Comment 7 Alexander V. Chernikov freebsd_committer 2015-10-31 10:49:03 UTC
(In reply to jhujhiti from comment #6)
FreeBSD development model assumes all new features goes to HEAD first and, after some period of testing, are merged to stable branches if possible.
So, it would be great if this patch can be applied/tested on HEAD - otherwise it would wouldn't be possible to commit it to FreeBSD.

Note also that HEAD is pretty different in netinet6/ from 10 (and especially 9).
For example, link/interface multicast routes does not exists at all. (removed from HEAD in r273742 and r274346)
Comment 8 Erick Turnquist 2015-10-31 15:14:10 UTC
(In reply to Alexander V. Chernikov from comment #7)
I chose 10.2 since it's of immediate utility for me (this prevents me from deploying IPv6 at home, and I don't run HEAD on these machines). Once I identify and fix all of the issues, I'll definitely forward-port to HEAD before attaching a patch.
Comment 9 Erick Turnquist 2016-12-22 02:53:01 UTC
Created attachment 178192 [details]
Respect net.add_addr_allfibs=0 for inet6 (revision 1)

I didn't forget about this! I implemented against 10.3 about a year ago and finally found the time to port it to HEAD. This patch essentially makes IPv6 respect net.add_addr_allfibs the same way IPv4 does. This is my first patch against base - any feedback is welcome.

The changes here are mostly straightforward: where we have an ifp, we can use its FIB, and where we've previously assumed the default FIB, we should consider that local routes can exist outside of it now. A couple changes are more noteworthy:

* Default router selection (defrouter_ functions) can select multiple routers, up to one per FIB. defrouter_select() now takes a FIB argument to simplify the logic inside the function. It is up to the caller to determine if we should re-select routers for all FIBs, by making multiple calls, or not.
* In icmp6_reflect(), there may be an edge case where source address selection fails to use the correct FIB if in6ifa_ifwithaddr() returns NULL. I don't fully understand the situations in which this can happen (or if it's possible at all).
* rtinit() didn't use the interface's FIB for both AF_INET as well as AF_INET6 and I don't understand why. For all uses of the function in AF_INET context, using the interface FIB seems correct to me, but previous in_addprefix() and rip_ctlinput() seem a little strange.

Here's what this looks like when net.add_addr_allfibs is 0. em0 and epair0b here are bridged together and there is a router advertising fd00::/64.

em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
	ether e0:cb:4e:00:5c:99
	inet6 fe80::e2cb:4eff:fe00:5c99%em0 prefixlen 64 scopeid 0x1 
	inet6 fd00::e2cb:4eff:fe00:5c99 prefixlen 64 autoconf 
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 04:ef:30:02:88:af
	inet6 fe80::6ef:30ff:fe02:88af%epair0b prefixlen 64 scopeid 0x6 
	inet6 fd00::6ef:30ff:fe02:88af prefixlen 64 autoconf 
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	fib: 1
	groups: epair 

% ndp -na
Neighbor                             Linklayer Address  Netif Expire    S Flags
fe80::ff:30ff:fe02:80d%epair0b       02:ff:30:02:08:0d epair0b 23h45m16s S R
fd00::6ef:30ff:fe02:88af             04:ef:30:02:88:af epair0b permanent R 
fe80::6ef:30ff:fe02:88af%epair0b     04:ef:30:02:88:af epair0b permanent R 
fe80::ff:30ff:fe02:80d%em0           02:ff:30:02:08:0d    em0 23h43m46s S R
fd00::e2cb:4eff:fe00:5c99            e0:cb:4e:00:5c:99    em0 permanent R 
fe80::e2cb:4eff:fe00:5c99%em0        e0:cb:4e:00:5c:99    em0 permanent R

% ndp -np 
fd00::/64 if=epair0b
flags=LAO vltime=600, pltime=300, expire=8m8s, ref=1
  advertised by
    fe80::ff:30ff:fe02:80d%epair0b (reachable)
fe80::%epair0b/64 if=epair0b
flags=LAO vltime=infinity, pltime=infinity, expire=Never, ref=0
  No advertising router
fd00::/64 if=em0
flags=LAO vltime=600, pltime=300, expire=8m8s, ref=1
  advertised by
    fe80::ff:30ff:fe02:80d%em0 (reachable)
fe80::%em0/64 if=em0
flags=LAO vltime=infinity, pltime=infinity, expire=Never, ref=0
  No advertising router
fe80::%lo0/64 if=lo0
flags=LAO vltime=infinity, pltime=infinity, expire=Never, ref=0
  No advertising router

% netstat -rnf inet6 -F0
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::ff:30ff:fe02:80d%em0    UG          em0
::1                               link#3                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fd00::/64                         link#1                        U           em0
fd00::e2cb:4eff:fe00:5c99         link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em0/64                     link#1                        U           em0
fe80::e2cb:4eff:fe00:5c99%em0     link#1                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0

% netstat -rnf inet6 -F1
Routing tables (fib: 1)

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::ff:30ff:fe02:80d%epair0b UG     epair0b
::1                               lo0                           UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fd00::/64                         link#6                        U       epair0b
fd00::6ef:30ff:fe02:88af          link#6                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%epair0b/64                 link#6                        U       epair0b
fe80::6ef:30ff:fe02:88af%epair0b  link#6                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0

% ifmcstat -f inet6
em0:
	inet6 fe80::e2cb:4eff:fe00:5c99%em0 scopeid 0x1
	mldv2 flags=2<USEALLOW> rv 2 qi 125 qri 10 uri 3
		group ff01::1%em0 scopeid 0x1 mode exclude
			mcast-macaddr 33:33:00:00:00:01
		group ff02::2:c26b:b618%em0 scopeid 0x1 mode exclude
			mcast-macaddr 33:33:c2:6b:b6:18
		group ff02::2:ffc2:6bb6%em0 scopeid 0x1 mode exclude
			mcast-macaddr 33:33:ff:c2:6b:b6
		group ff02::1%em0 scopeid 0x1 mode exclude
			mcast-macaddr 33:33:00:00:00:01
		group ff02::1:ff00:5c99%em0 scopeid 0x1 mode exclude
			mcast-macaddr 33:33:ff:00:5c:99
lo0:
	inet6 fe80::1%lo0 scopeid 0x3
	mldv2 flags=2<USEALLOW> rv 2 qi 125 qri 10 uri 3
		group ff01::1%lo0 scopeid 0x3 mode exclude
		group ff02::2:c26b:b618%lo0 scopeid 0x3 mode exclude
		group ff02::2:ffc2:6bb6%lo0 scopeid 0x3 mode exclude
		group ff02::1%lo0 scopeid 0x3 mode exclude
		group ff02::1:ff00:1%lo0 scopeid 0x3 mode exclude
epair0b:
	inet6 fe80::6ef:30ff:fe02:88af%epair0b scopeid 0x6
	mldv2 flags=2<USEALLOW> rv 2 qi 125 qri 10 uri 3
		group ff01::1%epair0b scopeid 0x6 mode exclude
			mcast-macaddr 33:33:00:00:00:01
		group ff02::2:c26b:b618%epair0b scopeid 0x6 mode exclude
			mcast-macaddr 33:33:c2:6b:b6:18
		group ff02::2:ffc2:6bb6%epair0b scopeid 0x6 mode exclude
			mcast-macaddr 33:33:ff:c2:6b:b6
		group ff02::1%epair0b scopeid 0x6 mode exclude
			mcast-macaddr 33:33:00:00:00:01
		group ff02::1:ff02:88af%epair0b scopeid 0x6 mode exclude
			mcast-macaddr 33:33:ff:02:88:af
Comment 10 Alan Somers freebsd_committer 2016-12-22 03:22:10 UTC
jhujhiti it looks good so far.  Do you think you could also add regression tests to tests/sys/netinet/fibs_test.sh?  You can probably just mirror the logic in the existing loopback_and_network_routes_on_nondefault_fib, default_route_with_multiple_fibs_on_same_subnet, same_ip_multiple_ifaces_fib0, and subnet_route_with_multiple_fibs_on_same_subnet tests.
Comment 11 Erick Turnquist 2016-12-29 02:33:54 UTC
Created attachment 178370 [details]
inet6 test cases

(In reply to Alan Somers from comment #10)

Hi Alan,

Here are the test cases. I've created inet6 variants of loopback_and_network_routes_on_nondefault_fib, default_route_with_multiple_fibs_on_same_subnet, and subnet_route_with_multiple_fibs_on_same_subnet.

Regarding same_ip_multiple_ifaces_fib0: should this even work for IPv4? To me, it seems invalid to allow conflicting local addresses in the same FIB. In any case, neither current HEAD nor my patch allow this situation for inet6. The second address addition will fail with EEXIST. However, assigning the same address to interfaces in different FIBs does make sense and work, so I've created an inet6 variant of same_ip_multiple_ifaces instead.
Comment 12 Alan Somers freebsd_committer 2016-12-29 21:40:23 UTC
(In reply to jhujhiti from comment #11)
Regarding same_ip_multiple_ifaces_fib0, the configuration isn't valid.  But that's not the point of the test.  As the comment and the PR say, the point of the test is just to make sure you don't panic if you try to configure a system that way.

For that matter, assigning the same IP to different interfaces on different fibs (as same_ip_multiple_ifaces) tests doesn't really make sense either, but I added a test case for it because it worked on older versions of FreeBSD and some people were actually using that feature.  But the situation is much more interesting with IPv6 because of the concept of scopes.  With IPv6, it makes sense to assign the same address to multiple interfaces, as long as their scopes are not overlapping.  They can even use the same FIB.  For example, the same link-local address can be assigned to two different interfaces, as long as they're on separate networks.

Since these patches are starting to get fairly complicated, could you please create an account at https://reviews.freebsd.org/ and upload the patches there?  It's far easier to review patches there than on Bugzilla.
Comment 13 Alan Somers freebsd_committer 2017-01-31 20:08:08 UTC
I've fully reviewed jhujhiti's testcases.  Apart from using the wrong syntax to delete an IPv6 route in same_ip_multiple_ifaces_inet6, it looks good.  I'll commit it with minor changes.
Comment 14 Erick Turnquist 2017-01-31 20:10:06 UTC
(In reply to Alan Somers from comment #13)

Hi Alan,

Sorry for the radio silence - I can fix the test case and submit for formal code review in the next couple of days if you'd like.
Comment 15 commit-hook freebsd_committer 2017-01-31 20:14:34 UTC
A commit references this bug:

Author: asomers
Date: Tue Jan 31 20:13:50 UTC 2017
New revision: 313025
URL: https://svnweb.freebsd.org/changeset/base/313025

Log:
  Add tests for multi-fib IPv6 routing

  PR:		196361
  Submitted by:	jhujhiti@adjectivism.org
  Reported by:	Jason Healy <jhealy@logn.net>
  MFC after:	4 weeks
  Sponsored by:	Spectra Logic Corp

Changes:
  head/tests/sys/netinet/fibs_test.sh
Comment 16 Erick Turnquist 2017-02-05 16:41:58 UTC
I've created a diff in Phabricator: https://reviews.freebsd.org/D9451
Comment 17 commit-hook freebsd_committer 2017-02-07 17:41:28 UTC
A commit references this bug:

Author: asomers
Date: Tue Feb  7 17:40:59 UTC 2017
New revision: 313395
URL: https://svnweb.freebsd.org/changeset/base/313395

Log:
  Add fibs_test:udp_dontroute6, another IPv6 multi-FIB test

  PR:		196361
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

Changes:
  head/tests/sys/netinet/fibs_test.sh
  head/tests/sys/netinet/udp_dontroute.c
Comment 18 commit-hook freebsd_committer 2017-03-01 00:28:48 UTC
A commit references this bug:

Author: asomers
Date: Wed Mar  1 00:28:04 UTC 2017
New revision: 314442
URL: https://svnweb.freebsd.org/changeset/base/314442

Log:
  Add an ATF test for IPv6 SLAAC with multiple fibs

  Tests that an interface can get a SLAAC address and that it inserts its
  routes into the correct fib. Does not test anything to do with NDP.

  PR:		196361
  Reviewed by:	Erick Turnquist <jhujhiti@adjectivism.org>
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9776

Changes:
  head/tests/sys/netinet/fibs_test.sh
Comment 19 commit-hook freebsd_committer 2017-03-17 16:51:43 UTC
A commit references this bug:

Author: asomers
Date: Fri Mar 17 16:50:38 UTC 2017
New revision: 315458
URL: https://svnweb.freebsd.org/changeset/base/315458

Log:
  Constrain IPv6 routes to single FIBs when net.add_addr_allfibs=0

  sys/netinet6/icmp6.c
  	Use the interface's FIB for source address selection in ICMPv6 error
  	responses.

  sys/netinet6/in6.c
  	In in6_newaddrmsg, announce arrival of local addresses on the
  	interface's FIB only.  In in6_lltable_rtcheck, use a per-fib ND6
  	cache instead of a single cache.

  sys/netinet6/in6_src.c
  	In in6_selectsrc, use the caller's fib instead of the default fib.
  	In in6_selectsrc_socket, remove a superfluous check.

  sys/netinet6/nd6.c
  	In nd6_lle_event, use the interface's fib for routing socket
  	messages.  In nd6_is_new_addr_neighbor, check all FIBs when trying
  	to determine whether an address is a neighbor.  Also, simplify the
  	code for point to point interfaces.

  sys/netinet6/nd6.h
  sys/netinet6/nd6.c
  sys/netinet6/nd6_rtr.c
  	Make defrouter_select fib-aware, and make all of its callers pass in
  	the interface fib.

  sys/netinet6/nd6_nbr.c
  	When inputting a Neighbor Solicitation packet, consider the
  	interface fib instead of the default fib for DAD.  Output NS and
  	Neighbor Advertisement packets on the correct fib.

  sys/netinet6/nd6_rtr.c
  	Allow installing the same host route on different interfaces in
  	different FIBs.  If rt_add_addr_allfibs=0, only install or delete
  	the prefix route on the interface fib.

  tests/sys/netinet/fibs_test.sh
  	Clear some expected failures, but add a skip for the newly revealed
  	BUG217871.

  PR:		196361
  Submitted by:	Erick Turnquist <jhujhiti@adjectivism.org>
  Reported by:	Jason Healy <jhealy@logn.net>
  Reviewed by:	asomers
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9451

Changes:
  head/sys/netinet6/icmp6.c
  head/sys/netinet6/in6.c
  head/sys/netinet6/in6_src.c
  head/sys/netinet6/nd6.c
  head/sys/netinet6/nd6.h
  head/sys/netinet6/nd6_nbr.c
  head/sys/netinet6/nd6_rtr.c
  head/tests/sys/netinet/fibs_test.sh
Comment 20 commit-hook freebsd_committer 2017-04-17 20:13:59 UTC
A commit references this bug:

Author: asomers
Date: Mon Apr 17 20:13:21 UTC 2017
New revision: 317067
URL: https://svnweb.freebsd.org/changeset/base/317067

Log:
  MFC r313025, r313395, r314113, r314442, r315458, r315656

  r313025:
  Add tests for multi-fib IPv6 routing

  PR:		196361
  Submitted by:	jhujhiti@adjectivism.org
  Reported by:	Jason Healy <jhealy@logn.net>
  MFC after:	4 weeks
  Sponsored by:	Spectra Logic Corp

  r313395:
  Add fibs_test:udp_dontroute6, another IPv6 multi-FIB test

  PR:		196361
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314113:
  Remove tests/sys/netinet/fibs_tests's dependency on net/socat

  Instead of bridging two tap interfaces with socat, just use an epair pair.

  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314442:
  Add an ATF test for IPv6 SLAAC with multiple fibs

  Tests that an interface can get a SLAAC address and that it inserts its
  routes into the correct fib. Does not test anything to do with NDP.

  PR:		196361
  Reviewed by:	Erick Turnquist <jhujhiti@adjectivism.org>
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9776

  r315458:
  Constrain IPv6 routes to single FIBs when net.add_addr_allfibs=0

  sys/netinet6/icmp6.c
  	Use the interface's FIB for source address selection in ICMPv6 error
  	responses.

  sys/netinet6/in6.c
  	In in6_newaddrmsg, announce arrival of local addresses on the
  	interface's FIB only.  In in6_lltable_rtcheck, use a per-fib ND6
  	cache instead of a single cache.

  sys/netinet6/in6_src.c
  	In in6_selectsrc, use the caller's fib instead of the default fib.
  	In in6_selectsrc_socket, remove a superfluous check.

  sys/netinet6/nd6.c
  	In nd6_lle_event, use the interface's fib for routing socket
  	messages.  In nd6_is_new_addr_neighbor, check all FIBs when trying
  	to determine whether an address is a neighbor.  Also, simplify the
  	code for point to point interfaces.

  sys/netinet6/nd6.h
  sys/netinet6/nd6.c
  sys/netinet6/nd6_rtr.c
  	Make defrouter_select fib-aware, and make all of its callers pass in
  	the interface fib.

  sys/netinet6/nd6_nbr.c
  	When inputting a Neighbor Solicitation packet, consider the
  	interface fib instead of the default fib for DAD.  Output NS and
  	Neighbor Advertisement packets on the correct fib.

  sys/netinet6/nd6_rtr.c
  	Allow installing the same host route on different interfaces in
  	different FIBs.  If rt_add_addr_allfibs=0, only install or delete
  	the prefix route on the interface fib.

  tests/sys/netinet/fibs_test.sh
  	Clear some expected failures, but add a skip for the newly revealed
  	BUG217871.

  PR:		196361
  Submitted by:	Erick Turnquist <jhujhiti@adjectivism.org>
  Reported by:	Jason Healy <jhealy@logn.net>
  Reviewed by:	asomers
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9451

  r315656:
  Fix back-to-back runs of sys/netinet/fibs_test;slaac_on_nondefault_fib6

  This test was failing if run twice because rtadvd takes too long to die.
  The rtadvd process from the first run was still running when the
  second run created its interfaces.  The solution is to use SIGKILL during
  the cleanup instead of SIGTERM so rtadvd will die faster.

  While I'm here, randomize the addresses used for the test, which makes bugs
  like this easier to spot, and fix the cleanup order to be the opposite of
  the setup order

  PR:		217871
  MFC after:	18 days
  X-MFC-With:	315458
  Sponsored by:	Spectra Logic Corp

Changes:
_U  stable/11/
  stable/11/sys/netinet6/icmp6.c
  stable/11/sys/netinet6/in6.c
  stable/11/sys/netinet6/in6_src.c
  stable/11/sys/netinet6/nd6.c
  stable/11/sys/netinet6/nd6.h
  stable/11/sys/netinet6/nd6_nbr.c
  stable/11/sys/netinet6/nd6_rtr.c
  stable/11/tests/sys/netinet/fibs_test.sh
  stable/11/tests/sys/netinet/udp_dontroute.c
Comment 21 commit-hook freebsd_committer 2017-04-17 20:14:02 UTC
A commit references this bug:

Author: asomers
Date: Mon Apr 17 20:13:21 UTC 2017
New revision: 317067
URL: https://svnweb.freebsd.org/changeset/base/317067

Log:
  MFC r313025, r313395, r314113, r314442, r315458, r315656

  r313025:
  Add tests for multi-fib IPv6 routing

  PR:		196361
  Submitted by:	jhujhiti@adjectivism.org
  Reported by:	Jason Healy <jhealy@logn.net>
  MFC after:	4 weeks
  Sponsored by:	Spectra Logic Corp

  r313395:
  Add fibs_test:udp_dontroute6, another IPv6 multi-FIB test

  PR:		196361
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314113:
  Remove tests/sys/netinet/fibs_tests's dependency on net/socat

  Instead of bridging two tap interfaces with socat, just use an epair pair.

  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314442:
  Add an ATF test for IPv6 SLAAC with multiple fibs

  Tests that an interface can get a SLAAC address and that it inserts its
  routes into the correct fib. Does not test anything to do with NDP.

  PR:		196361
  Reviewed by:	Erick Turnquist <jhujhiti@adjectivism.org>
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9776

  r315458:
  Constrain IPv6 routes to single FIBs when net.add_addr_allfibs=0

  sys/netinet6/icmp6.c
  	Use the interface's FIB for source address selection in ICMPv6 error
  	responses.

  sys/netinet6/in6.c
  	In in6_newaddrmsg, announce arrival of local addresses on the
  	interface's FIB only.  In in6_lltable_rtcheck, use a per-fib ND6
  	cache instead of a single cache.

  sys/netinet6/in6_src.c
  	In in6_selectsrc, use the caller's fib instead of the default fib.
  	In in6_selectsrc_socket, remove a superfluous check.

  sys/netinet6/nd6.c
  	In nd6_lle_event, use the interface's fib for routing socket
  	messages.  In nd6_is_new_addr_neighbor, check all FIBs when trying
  	to determine whether an address is a neighbor.  Also, simplify the
  	code for point to point interfaces.

  sys/netinet6/nd6.h
  sys/netinet6/nd6.c
  sys/netinet6/nd6_rtr.c
  	Make defrouter_select fib-aware, and make all of its callers pass in
  	the interface fib.

  sys/netinet6/nd6_nbr.c
  	When inputting a Neighbor Solicitation packet, consider the
  	interface fib instead of the default fib for DAD.  Output NS and
  	Neighbor Advertisement packets on the correct fib.

  sys/netinet6/nd6_rtr.c
  	Allow installing the same host route on different interfaces in
  	different FIBs.  If rt_add_addr_allfibs=0, only install or delete
  	the prefix route on the interface fib.

  tests/sys/netinet/fibs_test.sh
  	Clear some expected failures, but add a skip for the newly revealed
  	BUG217871.

  PR:		196361
  Submitted by:	Erick Turnquist <jhujhiti@adjectivism.org>
  Reported by:	Jason Healy <jhealy@logn.net>
  Reviewed by:	asomers
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9451

  r315656:
  Fix back-to-back runs of sys/netinet/fibs_test;slaac_on_nondefault_fib6

  This test was failing if run twice because rtadvd takes too long to die.
  The rtadvd process from the first run was still running when the
  second run created its interfaces.  The solution is to use SIGKILL during
  the cleanup instead of SIGTERM so rtadvd will die faster.

  While I'm here, randomize the addresses used for the test, which makes bugs
  like this easier to spot, and fix the cleanup order to be the opposite of
  the setup order

  PR:		217871
  MFC after:	18 days
  X-MFC-With:	315458
  Sponsored by:	Spectra Logic Corp

Changes:
_U  stable/11/
  stable/11/sys/netinet6/icmp6.c
  stable/11/sys/netinet6/in6.c
  stable/11/sys/netinet6/in6_src.c
  stable/11/sys/netinet6/nd6.c
  stable/11/sys/netinet6/nd6.h
  stable/11/sys/netinet6/nd6_nbr.c
  stable/11/sys/netinet6/nd6_rtr.c
  stable/11/tests/sys/netinet/fibs_test.sh
  stable/11/tests/sys/netinet/udp_dontroute.c
Comment 22 commit-hook freebsd_committer 2017-04-17 20:14:05 UTC
A commit references this bug:

Author: asomers
Date: Mon Apr 17 20:13:22 UTC 2017
New revision: 317067
URL: https://svnweb.freebsd.org/changeset/base/317067

Log:
  MFC r313025, r313395, r314113, r314442, r315458, r315656

  r313025:
  Add tests for multi-fib IPv6 routing

  PR:		196361
  Submitted by:	jhujhiti@adjectivism.org
  Reported by:	Jason Healy <jhealy@logn.net>
  MFC after:	4 weeks
  Sponsored by:	Spectra Logic Corp

  r313395:
  Add fibs_test:udp_dontroute6, another IPv6 multi-FIB test

  PR:		196361
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314113:
  Remove tests/sys/netinet/fibs_tests's dependency on net/socat

  Instead of bridging two tap interfaces with socat, just use an epair pair.

  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314442:
  Add an ATF test for IPv6 SLAAC with multiple fibs

  Tests that an interface can get a SLAAC address and that it inserts its
  routes into the correct fib. Does not test anything to do with NDP.

  PR:		196361
  Reviewed by:	Erick Turnquist <jhujhiti@adjectivism.org>
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9776

  r315458:
  Constrain IPv6 routes to single FIBs when net.add_addr_allfibs=0

  sys/netinet6/icmp6.c
  	Use the interface's FIB for source address selection in ICMPv6 error
  	responses.

  sys/netinet6/in6.c
  	In in6_newaddrmsg, announce arrival of local addresses on the
  	interface's FIB only.  In in6_lltable_rtcheck, use a per-fib ND6
  	cache instead of a single cache.

  sys/netinet6/in6_src.c
  	In in6_selectsrc, use the caller's fib instead of the default fib.
  	In in6_selectsrc_socket, remove a superfluous check.

  sys/netinet6/nd6.c
  	In nd6_lle_event, use the interface's fib for routing socket
  	messages.  In nd6_is_new_addr_neighbor, check all FIBs when trying
  	to determine whether an address is a neighbor.  Also, simplify the
  	code for point to point interfaces.

  sys/netinet6/nd6.h
  sys/netinet6/nd6.c
  sys/netinet6/nd6_rtr.c
  	Make defrouter_select fib-aware, and make all of its callers pass in
  	the interface fib.

  sys/netinet6/nd6_nbr.c
  	When inputting a Neighbor Solicitation packet, consider the
  	interface fib instead of the default fib for DAD.  Output NS and
  	Neighbor Advertisement packets on the correct fib.

  sys/netinet6/nd6_rtr.c
  	Allow installing the same host route on different interfaces in
  	different FIBs.  If rt_add_addr_allfibs=0, only install or delete
  	the prefix route on the interface fib.

  tests/sys/netinet/fibs_test.sh
  	Clear some expected failures, but add a skip for the newly revealed
  	BUG217871.

  PR:		196361
  Submitted by:	Erick Turnquist <jhujhiti@adjectivism.org>
  Reported by:	Jason Healy <jhealy@logn.net>
  Reviewed by:	asomers
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9451

  r315656:
  Fix back-to-back runs of sys/netinet/fibs_test;slaac_on_nondefault_fib6

  This test was failing if run twice because rtadvd takes too long to die.
  The rtadvd process from the first run was still running when the
  second run created its interfaces.  The solution is to use SIGKILL during
  the cleanup instead of SIGTERM so rtadvd will die faster.

  While I'm here, randomize the addresses used for the test, which makes bugs
  like this easier to spot, and fix the cleanup order to be the opposite of
  the setup order

  PR:		217871
  MFC after:	18 days
  X-MFC-With:	315458
  Sponsored by:	Spectra Logic Corp

Changes:
_U  stable/11/
  stable/11/sys/netinet6/icmp6.c
  stable/11/sys/netinet6/in6.c
  stable/11/sys/netinet6/in6_src.c
  stable/11/sys/netinet6/nd6.c
  stable/11/sys/netinet6/nd6.h
  stable/11/sys/netinet6/nd6_nbr.c
  stable/11/sys/netinet6/nd6_rtr.c
  stable/11/tests/sys/netinet/fibs_test.sh
  stable/11/tests/sys/netinet/udp_dontroute.c
Comment 23 commit-hook freebsd_committer 2017-04-17 20:14:09 UTC
A commit references this bug:

Author: asomers
Date: Mon Apr 17 20:13:22 UTC 2017
New revision: 317067
URL: https://svnweb.freebsd.org/changeset/base/317067

Log:
  MFC r313025, r313395, r314113, r314442, r315458, r315656

  r313025:
  Add tests for multi-fib IPv6 routing

  PR:		196361
  Submitted by:	jhujhiti@adjectivism.org
  Reported by:	Jason Healy <jhealy@logn.net>
  MFC after:	4 weeks
  Sponsored by:	Spectra Logic Corp

  r313395:
  Add fibs_test:udp_dontroute6, another IPv6 multi-FIB test

  PR:		196361
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314113:
  Remove tests/sys/netinet/fibs_tests's dependency on net/socat

  Instead of bridging two tap interfaces with socat, just use an epair pair.

  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp

  r314442:
  Add an ATF test for IPv6 SLAAC with multiple fibs

  Tests that an interface can get a SLAAC address and that it inserts its
  routes into the correct fib. Does not test anything to do with NDP.

  PR:		196361
  Reviewed by:	Erick Turnquist <jhujhiti@adjectivism.org>
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9776

  r315458:
  Constrain IPv6 routes to single FIBs when net.add_addr_allfibs=0

  sys/netinet6/icmp6.c
  	Use the interface's FIB for source address selection in ICMPv6 error
  	responses.

  sys/netinet6/in6.c
  	In in6_newaddrmsg, announce arrival of local addresses on the
  	interface's FIB only.  In in6_lltable_rtcheck, use a per-fib ND6
  	cache instead of a single cache.

  sys/netinet6/in6_src.c
  	In in6_selectsrc, use the caller's fib instead of the default fib.
  	In in6_selectsrc_socket, remove a superfluous check.

  sys/netinet6/nd6.c
  	In nd6_lle_event, use the interface's fib for routing socket
  	messages.  In nd6_is_new_addr_neighbor, check all FIBs when trying
  	to determine whether an address is a neighbor.  Also, simplify the
  	code for point to point interfaces.

  sys/netinet6/nd6.h
  sys/netinet6/nd6.c
  sys/netinet6/nd6_rtr.c
  	Make defrouter_select fib-aware, and make all of its callers pass in
  	the interface fib.

  sys/netinet6/nd6_nbr.c
  	When inputting a Neighbor Solicitation packet, consider the
  	interface fib instead of the default fib for DAD.  Output NS and
  	Neighbor Advertisement packets on the correct fib.

  sys/netinet6/nd6_rtr.c
  	Allow installing the same host route on different interfaces in
  	different FIBs.  If rt_add_addr_allfibs=0, only install or delete
  	the prefix route on the interface fib.

  tests/sys/netinet/fibs_test.sh
  	Clear some expected failures, but add a skip for the newly revealed
  	BUG217871.

  PR:		196361
  Submitted by:	Erick Turnquist <jhujhiti@adjectivism.org>
  Reported by:	Jason Healy <jhealy@logn.net>
  Reviewed by:	asomers
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D9451

  r315656:
  Fix back-to-back runs of sys/netinet/fibs_test;slaac_on_nondefault_fib6

  This test was failing if run twice because rtadvd takes too long to die.
  The rtadvd process from the first run was still running when the
  second run created its interfaces.  The solution is to use SIGKILL during
  the cleanup instead of SIGTERM so rtadvd will die faster.

  While I'm here, randomize the addresses used for the test, which makes bugs
  like this easier to spot, and fix the cleanup order to be the opposite of
  the setup order

  PR:		217871
  MFC after:	18 days
  X-MFC-With:	315458
  Sponsored by:	Spectra Logic Corp

Changes:
_U  stable/11/
  stable/11/sys/netinet6/icmp6.c
  stable/11/sys/netinet6/in6.c
  stable/11/sys/netinet6/in6_src.c
  stable/11/sys/netinet6/nd6.c
  stable/11/sys/netinet6/nd6.h
  stable/11/sys/netinet6/nd6_nbr.c
  stable/11/sys/netinet6/nd6_rtr.c
  stable/11/tests/sys/netinet/fibs_test.sh
  stable/11/tests/sys/netinet/udp_dontroute.c
Comment 24 Alan Somers freebsd_committer 2017-05-22 15:19:36 UTC
Declining to MFC to stable/10.  I don't think this bug is severe enough that people who haven't upgraded to 11 need it.