Bug 187549

Summary: Host and network routes for a new interface appear in the wrong FIB
Product: Base System Reporter: Alan Somers <asomers>
Component: kernAssignee: Alan Somers <asomers>
Status: Closed FIXED    
Severity: Affects Some People CC: vmiszczak
Priority: Normal    
Version: CURRENT   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
file.diff
none
187549_nosetfib.patch none

Description Alan Somers freebsd_committer freebsd_triage 2014-03-13 17:40:00 UTC
When UPing a new interface, the kernel automatically adds a host route (referred to as a loopback route in some places) and a network route.  These routes should be added to the interface's FIB.  However, they get added to the default FIB instead.

Fix: Patch attached with submission follows:
How-To-Repeat: ifconfig tap0 create
setfib 2 ifconfig tap0 192.0.2.2 netmask 255.255.255.0 fib 2
setfib 2 netstat -rn -f inet

# This is what you ought to see
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
192.0.2.0/24       link#9             UP          0        0   tap0
192.0.2.2          link#9             UHS         0        0    lo0

# This is what you actually see
Routing tables (fib: 2)

Internet:
Destination        Gateway            Flags    Netif Expire
192.0.2.0/24       link#3             U        tap0

Notice that the host route is missing.
Comment 1 Alan Somers freebsd_committer freebsd_triage 2014-03-13 17:43:30 UTC
Responsible Changed
From-To: freebsd-bugs->asomers

I'll take it.  I already have a patch
Comment 2 dfilter service freebsd_committer freebsd_triage 2014-03-20 20:39:48 UTC
Author: asomers
Date: Thu Mar 20 20:39:41 2014
New Revision: 263445
URL: http://svnweb.freebsd.org/changeset/base/263445

Log:
  Add several ATF tests that deal with multiple fibs.  They're described in
  several different PRs, but the tests share some common code, so I'm
  committing them together.
  
  sbin/ifconfig/tests
  sbin/ifconfig/tests/fibs_test.sh
  sbin/ifconfig/tests/Makefile
  sbin/ifconfig/Makefile
  	Add fibs_test.sh, which regresses bin/187551
  
  tests/sys/netinet
  tests/sys/netinet/fibs_test.sh
  tests/sys/netinet/udp_dontroute.c
  tests/sys/netinet/Makefile
  tests/sys/Makefile
  	Add fibs_test.sh, which regresses kern/167947, kern/187552
  	kern/187549, kern/187550, and kern/187553
  
  etc/mtree/BSD.tests.dist
  	Add newly created directories
  
  PR:		bin/187551
  PR:		kern/167947
  PR:		kern/187552
  PR:		kern/187549
  PR:		kern/187550
  PR:		kern/187553
  Discussed with:	melifaro
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corporation

Added:
  head/sbin/ifconfig/tests/
  head/sbin/ifconfig/tests/Makefile   (contents, props changed)
  head/sbin/ifconfig/tests/fibs_test.sh   (contents, props changed)
  head/tests/sys/netinet/
  head/tests/sys/netinet/Makefile   (contents, props changed)
  head/tests/sys/netinet/fibs_test.sh   (contents, props changed)
  head/tests/sys/netinet/udp_dontroute.c   (contents, props changed)
Modified:
  head/etc/mtree/BSD.tests.dist
  head/sbin/ifconfig/Makefile
  head/tests/sys/Makefile

Modified: head/etc/mtree/BSD.tests.dist
==============================================================================
--- head/etc/mtree/BSD.tests.dist	Thu Mar 20 20:33:18 2014	(r263444)
+++ head/etc/mtree/BSD.tests.dist	Thu Mar 20 20:39:41 2014	(r263445)
@@ -75,6 +75,8 @@
             ..
             growfs
             ..
+            ifconfig
+            ..
             mdconfig
             ..
         ..
@@ -91,6 +93,8 @@
         sys
             kern
             ..
+            netinet
+            ..
         ..
         usr.bin
             apply

Modified: head/sbin/ifconfig/Makefile
==============================================================================
--- head/sbin/ifconfig/Makefile	Thu Mar 20 20:33:18 2014	(r263444)
+++ head/sbin/ifconfig/Makefile	Thu Mar 20 20:39:41 2014	(r263445)
@@ -63,4 +63,8 @@ MAN=	ifconfig.8
 CFLAGS+= -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wnested-externs
 WARNS?=	2
 
+.if ${MK_TESTS} != "no"
+SUBDIR+=    tests
+.endif
+
 .include <bsd.prog.mk>

Added: head/sbin/ifconfig/tests/Makefile
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ head/sbin/ifconfig/tests/Makefile	Thu Mar 20 20:39:41 2014	(r263445)
@@ -0,0 +1,9 @@
+# $FreeBSD$
+
+.include <bsd.own.mk>
+
+TESTSDIR=	${TESTSBASE}/sbin/ifconfig
+
+ATF_TESTS_SH=	fibs_test
+
+.include <bsd.test.mk>

Added: head/sbin/ifconfig/tests/fibs_test.sh
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ head/sbin/ifconfig/tests/fibs_test.sh	Thu Mar 20 20:39:41 2014	(r263445)
@@ -0,0 +1,126 @@
+#
+#  Copyright (c) 2014 Spectra Logic Corporation
+#  All rights reserved.
+# 
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  1. Redistributions of source code must retain the above copyright
+#     notice, this list of conditions, and the following disclaimer,
+#     without modification.
+#  2. Redistributions in binary form must reproduce at minimum a disclaimer
+#     substantially similar to the "NO WARRANTY" disclaimer below
+#     ("Disclaimer") and any redistribution must be conditioned upon
+#     including a substantially similar Disclaimer requirement for further
+#     binary redistribution.
+# 
+#  NO WARRANTY
+#  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+#  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#  HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+#  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+#  OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+#  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+#  STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+#  IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+#  POSSIBILITY OF SUCH DAMAGES.
+# 
+#  Authors: Alan Somers         (Spectra Logic Corporation)
+#
+# $FreeBSD$
+
+
+# Regression test for bin/187551
+atf_test_case process_fib cleanup
+process_fib_head()
+{
+	atf_set "descr" "ifconfig will set its process fib whenever configuring an interface with nondefault fib"
+	atf_set "require.user" "root"
+	atf_set "require.config" "fibs"
+}
+process_fib_body()
+{
+	atf_expect_fail "bin/187551 ifconfig should change its process fib when configuring an interface with nondefault fib"
+	# Configure the TAP interface to use an RFC5737 nonrouteable address
+	# and a non-default fib
+	ADDR="192.0.2.2"
+	SUBNET="192.0.2.0"
+	MASK="24"
+
+	# Check system configuration
+	if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then
+		atf_skip "This test requires net.add_addr_allfibs=0"
+	fi
+	get_fibs 1
+
+	# Configure a TAP interface
+	get_tap
+	ktrace ifconfig $TAP ${ADDR}/${MASK} fib $FIB0
+	if kdump -s | egrep -q 'CALL[[:space:]]+setfib'; then
+		atf_pass
+	else
+		atf_fail "ifconfig never called setfib(2)"
+	fi
+}
+
+process_fib_cleanup()
+{
+	cleanup_tap
+}
+
+atf_init_test_cases()
+{
+	atf_add_test_case process_fib
+}
+
+
+# parameter numfibs	The number of fibs to lookup
+get_fibs()
+{
+	NUMFIBS=$1
+	net_fibs=`sysctl -n net.fibs`
+	i=0
+	while [ $i -lt "$NUMFIBS" ]; do
+		fib=`atf_config_get "fibs" | \
+			awk -v i=$(( i + 1 )) '{print $i}'`
+		echo "fib is ${fib}"
+		eval FIB${i}=${fib}
+		if [ "$fib" -ge "$net_fibs" ]; then
+			atf_skip "The ${i}th configured fib is ${fib}, which is not less than net.fibs, which is ${net_fibs}"
+		fi
+		i=$(( $i + 1 ))
+	done
+}
+
+
+
+# Creates a new tap(4) interface, registers it for cleanup, and returns the
+# name via the environment variable TAP
+get_tap()
+{
+	local TAPN=0
+	while ! ifconfig tap${TAPN} create > /dev/null 2>&1; do
+		if [ "$TAPN" -ge 8 ]; then
+			atf_skip "Could not create a tap(4) interface"
+		else
+			TAPN=$(($TAPN + 1))
+		fi
+	done
+	local TAPD=tap${TAPN}
+	# Record the TAP device so we can clean it up later
+	echo ${TAPD} >> "tap_devices_to_cleanup"
+	TAP=${TAPD}
+}
+
+
+
+
+cleanup_tap()
+{
+	for TAPD in `cat "tap_devices_to_cleanup"`; do
+		ifconfig ${TAPD} destroy
+	done
+}
+

Modified: head/tests/sys/Makefile
==============================================================================
--- head/tests/sys/Makefile	Thu Mar 20 20:33:18 2014	(r263444)
+++ head/tests/sys/Makefile	Thu Mar 20 20:39:41 2014	(r263445)
@@ -5,6 +5,7 @@
 .PATH: ${.CURDIR}/..
 
 TESTS_SUBDIRS+=		kern
+TESTS_SUBDIRS+=		netinet
 TESTSDIR= ${TESTSBASE}/sys
 
 KYUAFILE= yes

Added: head/tests/sys/netinet/Makefile
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ head/tests/sys/netinet/Makefile	Thu Mar 20 20:39:41 2014	(r263445)
@@ -0,0 +1,12 @@
+# $FreeBSD$
+
+TESTSDIR=	${TESTSBASE}/sys/netinet
+BINDIR=		${TESTSDIR}
+
+ATF_TESTS_SH+=	fibs_test
+PROG=	udp_dontroute
+SRCS=	udp_dontroute.c
+NO_MAN=
+WARNS?=	6
+
+.include <bsd.test.mk>

Added: head/tests/sys/netinet/fibs_test.sh
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ head/tests/sys/netinet/fibs_test.sh	Thu Mar 20 20:39:41 2014	(r263445)
@@ -0,0 +1,371 @@
+#
+#  Copyright (c) 2014 Spectra Logic Corporation
+#  All rights reserved.
+# 
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  1. Redistributions of source code must retain the above copyright
+#     notice, this list of conditions, and the following disclaimer,
+#     without modification.
+#  2. Redistributions in binary form must reproduce at minimum a disclaimer
+#     substantially similar to the "NO WARRANTY" disclaimer below
+#     ("Disclaimer") and any redistribution must be conditioned upon
+#     including a substantially similar Disclaimer requirement for further
+#     binary redistribution.
+# 
+#  NO WARRANTY
+#  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+#  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#  HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+#  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+#  OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+#  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+#  STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+#  IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+#  POSSIBILITY OF SUCH DAMAGES.
+# 
+#  Authors: Alan Somers         (Spectra Logic Corporation)
+#
+# $FreeBSD$
+
+# All of the tests in this file requires the test-suite config variable "fibs"
+# to be defined to a space-delimited list of FIBs that may be used for testing.
+
+# arpresolve should check the interface fib for routes to a target when
+# creating an ARP table entry.  This is a regression for kern/167947, where
+# arpresolve only checked the default route.
+#
+# Outline:
+# Create two tap(4) interfaces
+# Simulate a crossover cable between them by using net/socat
+# Use nping (from security/nmap) to send an ICMP echo request from one
+# interface to the other, spoofing the source IP.  The source IP must be
+# spoofed, or else it will already have an entry in the arp table. 
+# Check whether an arp entry exists for the spoofed IP
+atf_test_case arpresolve_checks_interface_fib cleanup
+arpresolve_checks_interface_fib_head()
+{
+	atf_set "descr" "arpresolve should check the interface fib, not the default fib, for routes"
+	atf_set "require.user" "root"
+	atf_set "require.config" "fibs"
+	atf_set "require.progs" "socat nping"
+}
+arpresolve_checks_interface_fib_body()
+{
+	atf_expect_fail "kern/167947 arpresolve checks only the default FIB for the interface route"
+	# Configure the TAP interfaces to use a RFC5737 nonrouteable addresses
+	# and a non-default fib
+	ADDR0="192.0.2.2"
+	ADDR1="192.0.2.3"
+	SUBNET="192.0.2.0"
+	# Due to bug TBD (regressed by multiple_fibs_on_same_subnet) we need
+	# diffferent subnet masks, or FIB1 won't have a subnet route.
+	MASK0="24"
+	MASK1="25"
+	# Spoof a MAC that is reserved per RFC7042
+	SPOOF_ADDR="192.0.2.4"
+	SPOOF_MAC="00:00:5E:00:53:00"
+
+	# Check system configuration
+	if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then
+		atf_skip "This test requires net.add_addr_allfibs=0"
+	fi
+	get_fibs 2
+
+	# Configure TAP interfaces
+	setup_tap "$FIB0" ${ADDR0} ${MASK0}
+	TAP0=$TAP
+	setup_tap "$FIB1" ${ADDR1} ${MASK1}
+	TAP1=$TAP
+
+	# Simulate a crossover cable
+	socat /dev/${TAP0} /dev/${TAP1} &
+	SOCAT_PID=$!
+	echo ${SOCAT_PID} >> "processes_to_kill"
+	
+	# Send an ICMP echo request with a spoofed source IP
+	setfib 2 nping -c 1 -e ${TAP0} -S ${SPOOF_ADDR} \
+		--source-mac ${SPOOF_MAC} --icmp --icmp-type "echo-request" \
+		--icmp-code 0 --icmp-id 0xdead --icmp-seq 1 --data 0xbeef \
+		${ADDR1}
+	# For informational and debugging purposes only, look for the
+	# characteristic error message
+	dmesg | grep "llinfo.*${SPOOF_ADDR}"
+	# Check that the ARP entry exists
+	atf_check -o match:"${SPOOF_ADDR}.*expires" setfib 3 arp ${SPOOF_ADDR}
+}
+arpresolve_checks_interface_fib_cleanup()
+{
+	for PID in `cat "processes_to_kill"`; do
+		kill $PID
+	done
+	cleanup_tap
+}
+
+
+# Regression test for kern/187549
+atf_test_case loopback_and_network_routes_on_nondefault_fib cleanup
+loopback_and_network_routes_on_nondefault_fib_head()
+{
+	atf_set "descr" "When creating and deleting loopback routes, use the interface's fib"
+	atf_set "require.user" "root"
+	atf_set "require.config" "fibs"
+}
+
+loopback_and_network_routes_on_nondefault_fib_body()
+{
+	atf_expect_fail "kern/187549 Host and network routes for a new interface appear in the wrong FIB"
+	# Configure the TAP interface to use an RFC5737 nonrouteable address
+	# and a non-default fib
+	ADDR="192.0.2.2"
+	SUBNET="192.0.2.0"
+	MASK="24"
+
+	# Check system configuration
+	if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then
+		atf_skip "This test requires net.add_addr_allfibs=0"
+	fi
+	get_fibs 1
+
+	# Configure a TAP interface
+	setup_tap ${FIB0} ${ADDR} ${MASK}
+
+	# Check whether the host route exists in only the correct FIB
+	setfib ${FIB0} netstat -rn -f inet | grep -q "^${ADDR}.*UHS.*lo0"
+	if [ 0 -ne $? ]; then
+		setfib ${FIB0} netstat -rn -f inet
+		atf_fail "Host route did not appear in the correct FIB"
+	fi
+	setfib 0 netstat -rn -f inet | grep -q "^${ADDR}.*UHS.*lo0"
+	if [ 0 -eq $? ]; then
+		setfib 0 netstat -rn -f inet
+		atf_fail "Host route appeared in the wrong FIB"
+	fi
+
+	# Check whether the network route exists in only the correct FIB
+	setfib ${FIB0} netstat -rn -f inet | \
+		grep -q "^${SUBNET}/${MASK}.*${TAPD}"
+	if [ 0 -ne $? ]; then
+		setfib ${FIB0} netstat -rn -f inet
+		atf_fail "Network route did not appear in the correct FIB"
+	fi
+	setfib 0 netstat -rn -f inet | \
+		grep -q "^${SUBNET}/${MASK}.*${TAPD}"
+	if [ 0 -eq $? ]; then
+		setfib ${FIB0} netstat -rn -f inet
+		atf_fail "Network route appeared in the wrong FIB"
+	fi
+}
+
+loopback_and_network_routes_on_nondefault_fib_cleanup()
+{
+	cleanup_tap
+}
+
+
+# Regression test for kern/187552
+atf_test_case default_route_with_multiple_fibs_on_same_subnet cleanup
+default_route_with_multiple_fibs_on_same_subnet_head()
+{
+	atf_set "descr" "Multiple interfaces on the same subnet but with different fibs can both have default routes"
+	atf_set "require.user" "root"
+	atf_set "require.config" "fibs"
+}
+
+default_route_with_multiple_fibs_on_same_subnet_body()
+{
+	atf_expect_fail "kern/187552 default route uses the wrong interface when multiple interfaces have the same subnet but different fibs"
+	# Configure the TAP interfaces to use a RFC5737 nonrouteable addresses
+	# and a non-default fib
+	ADDR0="192.0.2.2"
+	ADDR1="192.0.2.3"
+	GATEWAY="192.0.2.1"
+	SUBNET="192.0.2.0"
+	MASK="24"
+
+	# Check system configuration
+	if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then
+		atf_skip "This test requires net.add_addr_allfibs=0"
+	fi
+	get_fibs 2
+
+	# Configure TAP interfaces
+	setup_tap "$FIB0" ${ADDR0} ${MASK}
+	TAP0=$TAP
+	setup_tap "$FIB1" ${ADDR1} ${MASK}
+	TAP1=$TAP
+
+	# Attempt to add default routes
+	setfib ${FIB0} route add default ${GATEWAY}
+	setfib ${FIB1} route add default ${GATEWAY}
+
+	# Verify that the default route exists for both fibs, with their
+	# respective interfaces.
+	atf_check -o match:"^default.*${TAP0}$" \
+		setfib ${FIB0} netstat -rn -f inet
+	atf_check -o match:"^default.*${TAP1}$" \
+		setfib ${FIB1} netstat -rn -f inet
+}
+
+default_route_with_multiple_fibs_on_same_subnet_cleanup()
+{
+	cleanup_tap
+}
+
+
+# Regression test for kern/187550
+atf_test_case subnet_route_with_multiple_fibs_on_same_subnet cleanup
+subnet_route_with_multiple_fibs_on_same_subnet_head()
+{
+	atf_set "descr" "Multiple FIBs can have subnet routes for the same subnet"
+	atf_set "require.user" "root"
+	atf_set "require.config" "fibs"
+}
+
+subnet_route_with_multiple_fibs_on_same_subnet_body()
+{
+	atf_expect_fail "kern/187550 Multiple interfaces on different FIBs but the same subnet don't all have a subnet route"
+	# Configure the TAP interfaces to use a RFC5737 nonrouteable addresses
+	# and a non-default fib
+	ADDR0="192.0.2.2"
+	ADDR1="192.0.2.3"
+	SUBNET="192.0.2.0"
+	MASK="24"
+
+	# Check system configuration
+	if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then
+		atf_skip "This test requires net.add_addr_allfibs=0"
+	fi
+	get_fibs 2
+
+	# Configure TAP interfaces
+	setup_tap "$FIB0" ${ADDR0} ${MASK}
+	setup_tap "$FIB1" ${ADDR1} ${MASK}
+
+	# Check that a subnet route exists on both fibs
+	atf_check -o ignore setfib "$FIB0" route get $ADDR1
+	atf_check -o ignore setfib "$FIB1" route get $ADDR0
+}
+
+subnet_route_with_multiple_fibs_on_same_subnet_cleanup()
+{
+	cleanup_tap
+}
+
+# Test that source address selection works correctly for UDP packets with
+# SO_DONTROUTE set that are sent on non-default FIBs.
+# This bug was discovered with "setfib 1 netperf -t UDP_STREAM -H some_host"
+# Regression test for kern/187553
+atf_test_case udp_dontroute cleanup
+udp_dontroute_head()
+{
+	atf_set "descr" "Source address selection for UDP packets with SO_DONTROUTE on non-default FIBs works"
+	atf_set "require.user" "root"
+	atf_set "require.config" "fibs"
+}
+
+udp_dontroute_body()
+{
+	atf_expect_fail "kern/187553 Source address selection for UDP packets with SO_DONTROUTE uses the default FIB"
+	# Configure the TAP interface to use an RFC5737 nonrouteable address
+	# and a non-default fib
+	ADDR="192.0.2.2"
+	SUBNET="192.0.2.0"
+	MASK="24"
+	# Use a different IP on the same subnet as the target
+	TARGET="192.0.2.100"
+
+	# Check system configuration
+	if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then
+		atf_skip "This test requires net.add_addr_allfibs=0"
+	fi
+	get_fibs 1
+
+	# Configure a TAP interface
+	setup_tap ${FIB0} ${ADDR} ${MASK}
+
+	# Send a UDP packet with SO_DONTROUTE.  In the failure case, it will
+	# return ENETUNREACH
+	SRCDIR=`atf_get_srcdir`
+	atf_check -o ignore setfib ${FIB0} ${SRCDIR}/udp_dontroute ${TARGET}
+}
+
+udp_dontroute_cleanup()
+{
+	cleanup_tap
+}
+
+
+atf_init_test_cases()
+{
+	atf_add_test_case arpresolve_checks_interface_fib
+	atf_add_test_case loopback_and_network_routes_on_nondefault_fib 
+	atf_add_test_case default_route_with_multiple_fibs_on_same_subnet 
+	atf_add_test_case subnet_route_with_multiple_fibs_on_same_subnet 
+	atf_add_test_case udp_dontroute
+}
+
+# Looks up one or more fibs from the configuration data and validates them.
+# Returns the results in the env varilables FIB0, FIB1, etc.
+
+# parameter numfibs	The number of fibs to lookup
+get_fibs()
+{
+	NUMFIBS=$1
+	net_fibs=`sysctl -n net.fibs`
+	i=0
+	while [ $i -lt "$NUMFIBS" ]; do
+		fib=`atf_config_get "fibs" | \
+			awk -v i=$(( i + 1 )) '{print $i}'`
+		echo "fib is ${fib}"
+		eval FIB${i}=${fib}
+		if [ "$fib" -ge "$net_fibs" ]; then
+			atf_skip "The ${i}th configured fib is ${fib}, which is not less than net.fibs, which is ${net_fibs}"
+		fi
+		i=$(( $i + 1 ))
+	done
+}
+
+# Creates a new tap(4) interface, registers it for cleanup, and returns the
+# name via the environment variable TAP
+get_tap()
+{
+	local TAPN=0
+	while ! ifconfig tap${TAPN} create > /dev/null 2>&1; do
+		if [ "$TAPN" -ge 8 ]; then
+			atf_skip "Could not create a tap(4) interface"
+		else
+			TAPN=$(($TAPN + 1))
+		fi
+	done
+	local TAPD=tap${TAPN}
+	# Record the TAP device so we can clean it up later
+	echo ${TAPD} >> "tap_devices_to_cleanup"
+	TAP=${TAPD}
+}
+
+# Create a tap(4) interface, configure it, and register it for cleanup.
+# parameters:
+# fib
+# IP address
+# Netmask in number of bits (eg 24 or 8)
+# Return: the tap interface name as the env variable TAP
+setup_tap()
+{
+	local FIB=$1
+	local ADDR=$2
+	local MASK=$3
+	get_tap
+	echo setfib ${FIB} ifconfig $TAP ${ADDR}/${MASK} fib $FIB
+	setfib ${FIB} ifconfig $TAP ${ADDR}/${MASK} fib $FIB
+}
+
+cleanup_tap()
+{
+	for TAPD in `cat "tap_devices_to_cleanup"`; do
+		ifconfig ${TAPD} destroy
+	done
+}

Added: head/tests/sys/netinet/udp_dontroute.c
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ head/tests/sys/netinet/udp_dontroute.c	Thu Mar 20 20:39:41 2014	(r263445)
@@ -0,0 +1,85 @@
+/*
+ *  Copyright (c) 2014 Spectra Logic Corporation
+ *  All rights reserved.
+ * 
+ *  Redistribution and use in source and binary forms, with or without
+ *  modification, are permitted provided that the following conditions
+ *  are met:
+ *  1. Redistributions of source code must retain the above copyright
+ *     notice, this list of conditions, and the following disclaimer,
+ *     without modification.
+ *  2. Redistributions in binary form must reproduce at minimum a disclaimer
+ *     substantially similar to the "NO WARRANTY" disclaimer below
+ *     ("Disclaimer") and any redistribution must be conditioned upon
+ *     including a substantially similar Disclaimer requirement for further
+ *     binary redistribution.
+ * 
+ *  NO WARRANTY
+ *  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+ *  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *  HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ *  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ *  OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ *  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ *  STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ *  IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ *  POSSIBILITY OF SUCH DAMAGES.
+ * 
+ *  Authors: Alan Somers         (Spectra Logic Corporation)
+ *
+ * $FreeBSD$
+ */
+
+#include <arpa/inet.h>
+#include <netinet/in.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+
+#include <err.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+/* 
+ * Sends a single UDP packet to the provided address, with SO_DONTROUTE set
+ * I couldn't find a way to do this with builtin utilities like nc(1)
+ */
+int main(int argc, char **argv)
+{
+	struct sockaddr_in dst;
+	int s;
+	int opt;
+	int ret;
+	const char* buf = "Hello, World!";
+
+	if (argc != 2) {
+		fprintf(stderr, "Usage: %s ip_address\n", argv[0]);
+		exit(2);
+	}
+	s = socket(PF_INET, SOCK_DGRAM, 0);
+	if (s < 0)
+		err(errno, "socket");
+	opt = 1;
+
+	ret = setsockopt(s, SOL_SOCKET, SO_DONTROUTE, &opt, sizeof(opt));
+	if (ret == -1)
+		err(errno, "setsockopt(SO_DONTROUTE)");
+
+	dst.sin_len = sizeof(dst);
+	dst.sin_family = AF_INET;
+	dst.sin_port = htons(46120);
+	dst.sin_addr.s_addr = inet_addr(argv[1]);
+	if (dst.sin_addr.s_addr == htonl(INADDR_NONE)) {
+		fprintf(stderr, "Invalid address: %s\n", argv[1]);
+		exit(2);
+	}
+	ret = sendto(s, buf, strlen(buf), 0, (struct sockaddr*)&dst,
+	    dst.sin_len);
+	if (ret == -1)
+		err(errno, "sendto");
+	
+	return (0);
+}
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 3 Mark Linimon freebsd_committer freebsd_triage 2014-04-20 22:59:54 UTC
State Changed
From-To: open->patched

Over to committer as possible MFC reminder.
Comment 4 Alan Somers freebsd_committer freebsd_triage 2014-04-21 17:16:06 UTC
State Changed
From-To: patched->open

Actually, linimon, this is still open.  Commit 263445 merely added 
regression tests; it did not fix the bug.
Comment 5 asomers 2014-04-21 19:58:07 UTC
At Melifaro's request, I tried another patch that does not involve
invoking setfib(2) from ifconfig.  My original reason for using the
process fib instead of the interface fib in rtinit1_fib was based on
the impossibility of deleting routes on a nondefault fib with the
route(8) command.  However, that is no longer an issue since r243185
by hrs.  Here is my patch for review.
Comment 6 dfilter service freebsd_committer freebsd_triage 2014-04-24 18:23:20 UTC
Author: asomers
Date: Thu Apr 24 17:23:16 2014
New Revision: 264887
URL: http://svnweb.freebsd.org/changeset/base/264887

Log:
  Fix host and network routes for new interfaces when net.add_addr_allfibs=0
  
  sys/net/route.c
  	In rtinit1, use the interface fib instead of the process fib.  The
  	latter wasn't very useful because ifconfig(8) is usually invoked
  	with the default process fib.  Changing ifconfig(8) to use setfib(2)
  	would be redundant, because it already sets the interface fib.
  
  tests/sys/netinet/fibs_test.sh
  	Clear the expected ATF failure
  
  sys/net/if.c
  	Pass the interface fib in calls to rtrequest1_fib and rtalloc1_fib
  
  sys/netinet/in.c
  sys/net/if_var.h
  	Add a fibnum argument to ifa_switch_loopback_route, a subroutine of
  	in_scrubprefix.  Pass it the interface fib.
  
  PR:		kern/187549
  Reviewed by:	melifaro
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corporation

Modified:
  head/sys/net/if.c
  head/sys/net/if_var.h
  head/sys/net/route.c
  head/sys/netinet/in.c
  head/tests/sys/netinet/fibs_test.sh

Modified: head/sys/net/if.c
==============================================================================
--- head/sys/net/if.c	Thu Apr 24 16:19:49 2014	(r264886)
+++ head/sys/net/if.c	Thu Apr 24 17:23:16 2014	(r264887)
@@ -1485,7 +1485,7 @@ ifa_add_loopback_route(struct ifaddr *if
 	info.rti_flags = ifa->ifa_flags | RTF_HOST | RTF_STATIC;
 	info.rti_info[RTAX_DST] = ia;
 	info.rti_info[RTAX_GATEWAY] = (struct sockaddr *)&null_sdl;
-	error = rtrequest1_fib(RTM_ADD, &info, &rt, 0);
+	error = rtrequest1_fib(RTM_ADD, &info, &rt, ifa->ifa_ifp->if_fib);
 
 	if (error == 0 && rt != NULL) {
 		RT_LOCK(rt);
@@ -1517,7 +1517,7 @@ ifa_del_loopback_route(struct ifaddr *if
 	info.rti_flags = ifa->ifa_flags | RTF_HOST | RTF_STATIC;
 	info.rti_info[RTAX_DST] = ia;
 	info.rti_info[RTAX_GATEWAY] = (struct sockaddr *)&null_sdl;
-	error = rtrequest1_fib(RTM_DELETE, &info, NULL, 0);
+	error = rtrequest1_fib(RTM_DELETE, &info, NULL, ifa->ifa_ifp->if_fib);
 
 	if (error != 0)
 		log(LOG_DEBUG, "%s: deletion failed: %u\n", __func__, error);
@@ -1526,11 +1526,11 @@ ifa_del_loopback_route(struct ifaddr *if
 }
 
 int
-ifa_switch_loopback_route(struct ifaddr *ifa, struct sockaddr *sa)
+ifa_switch_loopback_route(struct ifaddr *ifa, struct sockaddr *sa, int fib)
 {
 	struct rtentry *rt;
 
-	rt = rtalloc1_fib(sa, 0, 0, 0);
+	rt = rtalloc1_fib(sa, 0, 0, fib);
 	if (rt == NULL) {
 		log(LOG_DEBUG, "%s: fail", __func__);
 		return (EHOSTUNREACH);

Modified: head/sys/net/if_var.h
==============================================================================
--- head/sys/net/if_var.h	Thu Apr 24 16:19:49 2014	(r264886)
+++ head/sys/net/if_var.h	Thu Apr 24 17:23:16 2014	(r264887)
@@ -490,7 +490,7 @@ struct	ifnet *ifunit_ref(const char *);
 
 int	ifa_add_loopback_route(struct ifaddr *, struct sockaddr *);
 int	ifa_del_loopback_route(struct ifaddr *, struct sockaddr *);
-int	ifa_switch_loopback_route(struct ifaddr *, struct sockaddr *);
+int	ifa_switch_loopback_route(struct ifaddr *, struct sockaddr *, int fib);
 
 struct	ifaddr *ifa_ifwithaddr(struct sockaddr *);
 int		ifa_ifwithaddr_check(struct sockaddr *);

Modified: head/sys/net/route.c
==============================================================================
--- head/sys/net/route.c	Thu Apr 24 16:19:49 2014	(r264886)
+++ head/sys/net/route.c	Thu Apr 24 17:23:16 2014	(r264887)
@@ -1536,7 +1536,7 @@ rtinit1(struct ifaddr *ifa, int cmd, int
 	}
 	if (fibnum == RT_ALL_FIBS) {
 		if (rt_add_addr_allfibs == 0 && cmd == (int)RTM_ADD) {
-			startfib = endfib = curthread->td_proc->p_fibnum;
+			startfib = endfib = ifa->ifa_ifp->if_fib;
 		} else {
 			startfib = 0;
 			endfib = rt_numfibs - 1;

Modified: head/sys/netinet/in.c
==============================================================================
--- head/sys/netinet/in.c	Thu Apr 24 16:19:49 2014	(r264886)
+++ head/sys/netinet/in.c	Thu Apr 24 17:23:16 2014	(r264887)
@@ -708,7 +708,7 @@ in_scrubprefix(struct in_ifaddr *target,
 
 		if (eia != NULL) {
 			error = ifa_switch_loopback_route((struct ifaddr *)eia,
-			    (struct sockaddr *)&target->ia_addr);
+			    (struct sockaddr *)&target->ia_addr, fibnum);
 			ifa_free(&eia->ia_ifa);
 		} else {
 			error = ifa_del_loopback_route((struct ifaddr *)target,

Modified: head/tests/sys/netinet/fibs_test.sh
==============================================================================
--- head/tests/sys/netinet/fibs_test.sh	Thu Apr 24 16:19:49 2014	(r264886)
+++ head/tests/sys/netinet/fibs_test.sh	Thu Apr 24 17:23:16 2014	(r264887)
@@ -116,7 +116,6 @@ loopback_and_network_routes_on_nondefaul
 
 loopback_and_network_routes_on_nondefault_fib_body()
 {
-	atf_expect_fail "kern/187549 Host and network routes for a new interface appear in the wrong FIB"
 	# Configure the TAP interface to use an RFC5737 nonrouteable address
 	# and a non-default fib
 	ADDR="192.0.2.2"
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 7 Alan Somers freebsd_committer freebsd_triage 2014-04-24 19:45:17 UTC
State Changed
From-To: open->patched

patched by change 264887
Comment 8 commit-hook freebsd_committer freebsd_triage 2014-09-18 20:54:06 UTC
A commit references this bug:

Author: asomers
Date: Thu Sep 18 20:53:37 UTC 2014
New revision: 271825
URL: http://svnweb.freebsd.org/changeset/base/271825

Log:
  MFC r264887, except for the ATF test change.  Also,
  ifa_switch_loopback_route doesn't exist in stable/9, so the relevant change
  to that function went into in_scrubprefix instead.

  Fix host and network routes for new interfaces when net.add_addr_allfibs=0

  sys/net/route.c
          In rtinit1, use the interface fib instead of the process fib.  The
          latter wasn't very useful because ifconfig(8) is usually invoked
          with the default process fib.  Changing ifconfig(8) to use setfib(2)
          would be redundant, because it already sets the interface fib.

  tests/sys/netinet/fibs_test.sh
          Clear the expected ATF failure

  sys/net/if.c
          Pass the interface fib in calls to rtrequest1_fib and rtalloc1_fib

  sys/netinet/in.c
  sys/net/if_var.h
          Add a fibnum argument to ifa_switch_loopback_route, a subroutine of
          in_scrubprefix.  Pass it the interface fib.

  PR:             kern/187549

Changes:
_U  stable/9/
_U  stable/9/sys/
_U  stable/9/sys/net/
  stable/9/sys/net/if.c
  stable/9/sys/net/route.c
  stable/9/sys/netinet/in.c
Comment 9 Alan Somers freebsd_committer freebsd_triage 2014-09-18 21:08:53 UTC
MFCed to stable/9 by 271825 and to stable/10 by 267193.
Comment 10 Alan Somers freebsd_committer freebsd_triage 2014-10-21 14:40:27 UTC
*** Bug 155757 has been marked as a duplicate of this bug. ***