Bug 189088

Summary: Assigning same IP to multiple interfaces in different FIBs only creates a host route for the first
Product: Base System Reporter: Alan Somers <asomers>
Component: kernAssignee: Alexander V. Chernikov <melifaro>
Status: Closed FIXED    
Severity: Affects Some People CC: allanjude, gonzo, jamie, jhujhiti, melifaro, net, pat, rgrimes, zlei
Priority: --- Keywords: needs-patch, needs-qa
Version: UnspecifiedFlags: koobs: mfc-stable12?
koobs: mfc-stable11?
Hardware: Any   
OS: Any   

Description Alan Somers freebsd_committer freebsd_triage 2014-04-28 23:30:00 UTC
Assigning the same IP to multiple interfaces simultaneously sounds insane, but some people do it.  However, if you assign them to separate FIBs as well as separate interfaces, bad stuff happens.  For one thing, only the first FIB used will get a host route.  For another, if you delete the addresses in FIFO order the host route will never get deleted (though it will if you delete them in LIFO order).

How-To-Repeat: On a system with net.add_addr_allfibs=0 and net.fibs=4, Run the following commands:

# ifconfig tap0 create
# ifconfig tap1 create
# ifconfig tap0 192.0.2.2/32 fib 2
# ifconfig tap1 192.0.2.2/32 fib 3
# setfib 2 netstat -rn -f inet
Routing tables (fib: 2)

Internet:
Destination        Gateway            Flags    Netif Expire
192.0.2.2          link#3             UHS       lo0
192.0.2.2/32       link#3             U        tap0
# setfib 3 netstat -rn -f inet
Routing tables (fib: 3)

Internet:
Destination        Gateway            Flags    Netif Expire
192.0.2.2/32       link#4             U        tap1


Notice that FIB 3 does not get a 192.0.2.2 host route.  Now try deleting the addresses in FIFO order:


# ifconfig tap0 -alias 192.0.2.2
# ifconfig tap1 -alias 192.0.2.2
# setfib 2 netstat -rn -f inet
Routing tables (fib: 2)

Internet:
Destination        Gateway            Flags    Netif Expire
192.0.2.2          link#4             UHS       lo0
Comment 1 Alan Somers freebsd_committer freebsd_triage 2014-04-29 16:08:24 UTC
Responsible Changed
From-To: freebsd-bugs->asomers

I'll take it.
Comment 2 dfilter service freebsd_committer freebsd_triage 2014-04-29 16:12:27 UTC
Author: asomers
Date: Tue Apr 29 15:12:23 2014
New Revision: 265094
URL: http://svnweb.freebsd.org/changeset/base/265094

Log:
  Add regression test for PR kern/189088.
  
  PR:		kern/189088
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic

Modified:
  head/tests/sys/netinet/fibs_test.sh

Modified: head/tests/sys/netinet/fibs_test.sh
==============================================================================
--- head/tests/sys/netinet/fibs_test.sh	Tue Apr 29 14:52:39 2014	(r265093)
+++ head/tests/sys/netinet/fibs_test.sh	Tue Apr 29 15:12:23 2014	(r265094)
@@ -252,6 +252,59 @@ same_ip_multiple_ifaces_fib0_cleanup()
 	cleanup_tap
 }
 
+# Regression test for PR kern/189088
+# Test that removing an IP address works even if the same IP is assigned to a
+# different interface, on a different FIB.  Tests the same code that whose
+# panic was regressed by same_ip_multiple_ifaces_fib0.  
+# Create two tap interfaces and assign them both the same IP address but with
+# different netmasks, and on different FIBs.  Then remove one's IP
+# address.  Hopefully the machine won't panic.  Also, the IP's hostroute should
+# dissappear from the correct fib.
+atf_test_case same_ip_multiple_ifaces cleanup
+same_ip_multiple_ifaces_head()
+{
+	atf_set "descr" "Can remove an IP alias from an interface when the same IP is also assigned to another interface, on non-default FIBs."
+	atf_set "require.user" "root"
+	atf_set "require.config" "fibs"
+}
+same_ip_multiple_ifaces_body()
+{
+	atf_expect_fail "kern/189088 Assigning the same IP to multiple interfaces in different FIBs creates a host route for only one"
+	ADDR="192.0.2.2"
+	MASK0="24"
+	MASK1="32"
+
+	# Unlike most of the tests in this file, this is applicable regardless
+	# of net.add_addr_allfibs
+	get_fibs 2
+
+	# Setup the interfaces, then remove one alias.  It should not panic.
+	setup_tap ${FIB0} ${ADDR} ${MASK0}
+	TAP0=${TAP}
+	setup_tap ${FIB1} ${ADDR} ${MASK1}
+	TAP1=${TAP}
+	ifconfig ${TAP1} -alias ${ADDR}
+	atf_check -o not-match:"^${ADDR}[[:space:]]" \
+		setfib ${FIB1} netstat -rn -f inet
+
+	# Do it again, in the opposite order.  It should not panic.
+	setup_tap ${FIB0} ${ADDR} ${MASK0}
+	TAP0=${TAP}
+	setup_tap ${FIB1} ${ADDR} ${MASK1}
+	TAP1=${TAP}
+	ifconfig ${TAP0} -alias ${ADDR}
+	atf_check -o not-match:"^${ADDR}[[:space:]]" \
+		setfib ${FIB0} netstat -rn -f inet
+}
+same_ip_multiple_ifaces_cleanup()
+{
+	# Due to PR kern/189088, we must destroy the interfaces in LIFO order
+	# in order for the routes to be correctly cleaned up.
+	for TAPD in `tail -r "tap_devices_to_cleanup"`; do
+		ifconfig ${TAPD} destroy
+	done
+}
+
 # Regression test for kern/187550
 atf_test_case subnet_route_with_multiple_fibs_on_same_subnet cleanup
 subnet_route_with_multiple_fibs_on_same_subnet_head()
@@ -349,6 +402,7 @@ atf_init_test_cases()
 	atf_add_test_case loopback_and_network_routes_on_nondefault_fib
 	atf_add_test_case default_route_with_multiple_fibs_on_same_subnet
 	atf_add_test_case same_ip_multiple_ifaces_fib0
+	atf_add_test_case same_ip_multiple_ifaces
 	atf_add_test_case subnet_route_with_multiple_fibs_on_same_subnet
 	atf_add_test_case udp_dontroute
 }
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 3 Rodney W. Grimes freebsd_committer freebsd_triage 2018-01-17 23:59:24 UTC
asomers:  "Assigning the same IP to multiple interfaces simultaneously sounds insane, but some people do it."

Actually this was very common, and should "just work" as long as the interfaces are Point to Point.   However I do not see this as valid on tapX, as those are NOT Point to Point interfaces.

The other thing going on here is the brain dead concept that the kernel should be creating a route to the local IP address of an interface via lo0, that just does not belong in the kernel AT ALL.  This is a policy decision that should be made by the administrator via proper application of route(8) or a routing daemon, the kernel can never be taught to get this correct in all situations, and this just demonstrates one more place the current code fails.
Comment 4 Oleksandr Tymoshenko freebsd_committer freebsd_triage 2019-01-21 09:39:21 UTC
Alan, can this PR be closed or is some work still pending?

Thanks
Comment 5 Alan Somers freebsd_committer freebsd_triage 2019-02-07 18:56:11 UTC
I'm not actively working on it ATM.  I never personally had a use case for it; I just opened the bug because a user complained about it.  I'll throw it back to the pool.
Comment 6 Mark Linimon freebsd_committer freebsd_triage 2020-07-11 06:59:13 UTC
Not currently being worked on.
Comment 7 Eugene Grosbein freebsd_committer freebsd_triage 2020-07-12 05:51:50 UTC
It's no use keeping this open for another 6 years without anyone working on it and without use case even. Closing this.
Comment 8 Rodney W. Grimes freebsd_committer freebsd_triage 2020-07-12 15:42:02 UTC
It is a bug, but being ignored, re-classed as feedback timeout, root cause is in kernel attempt to add/remove routes via lo0 which it can never fully understand and hence should not be doing.
Comment 9 Alexander V. Chernikov freebsd_committer freebsd_triage 2020-07-12 16:03:50 UTC
This should just work w/ multiple fibs.
Take.
Comment 10 Rodney W. Grimes freebsd_committer freebsd_triage 2020-07-12 16:13:16 UTC
Your going to find that difficult.  You shall need to teach ifa_maintain_loopback_route about fibs, and how to walk them to add and remove its attempt at loopback route maintainance.  Or simply kill the ifa_maintain_loopback_route code and put that back in the hands of userland routing protocols where it belongs.
Comment 11 Allan Jude freebsd_committer freebsd_triage 2020-07-18 15:06:48 UTC
Thanks for looking at this
Comment 12 Alexander V. Chernikov freebsd_committer freebsd_triage 2021-02-14 23:39:05 UTC
https://reviews.freebsd.org/D28673
Comment 13 commit-hook freebsd_committer freebsd_triage 2021-02-16 20:05:00 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=9fdbf7eef5c006002769add15b1ebb8fa8d9e220

commit 9fdbf7eef5c006002769add15b1ebb8fa8d9e220
Author:     Alexander V. Chernikov <melifaro@FreeBSD.org>
AuthorDate: 2021-02-16 20:00:46 +0000
Commit:     Alexander V. Chernikov <melifaro@FreeBSD.org>
CommitDate: 2021-02-16 20:00:46 +0000

    Make in_localip_more() fib-aware.

    It fixes loopback route installation for the interfaces
     in the different fibs using the same prefix.

    Reviewed By:    donner
    PR:             189088
    Differential Revision: https://reviews.freebsd.org/D28673
    MFC after:      1 week

 sys/netinet/in.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
Comment 14 commit-hook freebsd_committer freebsd_triage 2021-03-10 22:28:55 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f67641675958cb566b0ae50dc6942017d42393fe

commit f67641675958cb566b0ae50dc6942017d42393fe
Author:     Alexander V. Chernikov <melifaro@FreeBSD.org>
AuthorDate: 2021-02-16 20:00:46 +0000
Commit:     Alexander V. Chernikov <melifaro@FreeBSD.org>
CommitDate: 2021-03-10 21:47:39 +0000

    Make in_localip_more() fib-aware.

    It fixes loopback route installation for the interfaces
     in the different fibs using the same prefix.

    Reviewed By:    donner
    PR:             189088
    Differential Revision: https://reviews.freebsd.org/D28673

    (cherry picked from commit 9fdbf7eef5c006002769add15b1ebb8fa8d9e220)

 sys/netinet/in.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
Comment 15 commit-hook freebsd_committer freebsd_triage 2021-03-11 08:26:28 UTC
A commit in branch releng/13.0 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=dba80ca2cfb0b0c1447269e8a6a3545f428c51e6

commit dba80ca2cfb0b0c1447269e8a6a3545f428c51e6
Author:     Alexander V. Chernikov <melifaro@FreeBSD.org>
AuthorDate: 2021-02-16 20:00:46 +0000
Commit:     Alexander V. Chernikov <melifaro@FreeBSD.org>
CommitDate: 2021-03-11 08:23:07 +0000

    Make in_localip_more() fib-aware.

    It fixes loopback route installation for the interfaces
     in the different fibs using the same prefix.

    Reviewed By:    donner
    PR:             189088
    Approved by:    re(gjb)
    Differential Revision: https://reviews.freebsd.org/D28673

    (cherry picked from commit f67641675958cb566b0ae50dc6942017d42393fe)

 sys/netinet/in.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
Comment 16 Alexander V. Chernikov freebsd_committer freebsd_triage 2021-04-04 08:59:48 UTC
The desired changes landed in 13-R.
Due to the differences in the routing stack, I'm not going to merge them to 12/11.

I'm going to close this one on April 11, unless there are any objections.
Comment 17 Pat Maddox 2022-04-10 18:25:23 UTC
(In reply to Alexander V. Chernikov from comment #16)

Did this change make it impossible to explicitly add an lo0 route for a host that's already in the routing table? bug #263202