arpresolve() checks if a connected route exists before adding ARP entries to the arp table. The problem is that it only checks the default FIB 0. When one does have interface in non default FIB with it's directly attached routes removed from the default FIB, arpresolve fails and no ARP entries can be added : arpresolve: can't allocate llinfo for x.x.x.x Fix: Apply the included patch. Patch attached with submission follows: How-To-Repeat: put these in /boot/loader.conf : net.add_addr_allfibs="0" net.fibs="8" Then : /usr/sbin/setfib 1 /sbin/ifconfig em1 10.180.0.2 netmask 255.255.255.0 fib 1 /usr/sbin/setfib 1 /sbin/route add default 10.180.0.254 Now then I try to ping the host from its gw 10.180.0.254, the host generates "arpresolve: can't allocate llinfo for 10.180.0.254" messages.
Responsible Changed From-To: freebsd-bugs->freebsd-net Fix synopsis and assign.
Hi, I had the problem too, and could confirm the fix. FreeBSD ********.arcor.net 8.3-RELEASE-p2 FreeBSD 8.3-RELEASE-p2 #1: Tue Jun 12 09:06:31 CEST 2012 =20 root@********.arcor.net:/usr/src/sys/amd64/compile/GENERICfib amd64 However, the fix ONLY works if using ifconfig ... fib N to set the fib for an interface, and not the FreeBSD 7.x method to assign the FIB via ipfw rules. Just thought I should mention this - in our case, the ifconfig way is the better choice anyway. Best regards, Rudolf Polzer=
Responsible Changed From-To: freebsd-net->melifaro Take.
Melifaro, I'm working in this area at the moment. I even think that I can write an ATF regression test for this. Do you mind if I take the bug myself? -Alan
> Melifaro, > I'm working in this area at the moment. I even think that I can > write an ATF regression test for this. Do you mind if I take the bug > myself? Well, probably we should discuss what exactly you're planning to do first. Proper fix is quite tricky. (1) Current fib code has an awful hack for interface routes which are added to all fibs (net.add_addr_allfibs). We need to remove that first which involves some route/rtsock code cleanup first (which I stared to do some time ago). We also need to handle interface fib changes properly and so on. (2) Then, we have to do per-interface LLINFO for IPv4/IPv6 (since it is currently global). (3) After that, we can think of changing arpresolve to honor fib, but we should handle cases like 'route leaking' or something similar which involves GW mac from GRT (fib 0) being proxied somehow to fib X
On Tue, Mar 11, 2014 at 1:37 PM, Alexander V. Chernikov <melifaro@freebsd.org> wrote: >> Melifaro, >> I'm working in this area at the moment. I even think that I can >> write an ATF regression test for this. Do you mind if I take the bug >> myself? > Well, probably we should discuss what exactly you're planning to do first. > Proper fix is quite tricky. > > (1) Current fib code has an awful hack for interface routes which are > added to all fibs (net.add_addr_allfibs). We need to remove that first > which involves some route/rtsock code cleanup first (which I stared to > do some time ago). We also need to handle interface fib changes properly > and so on. I'm currently doing all my work with net.add_addr_allfibs=0. To get it working, I had to apply Nikolay's patch for this bug, and fix several other bugs too. I haven't opened GNATS issues for all of them yet, but I intend to. > > (2) Then, we have to do per-interface LLINFO for IPv4/IPv6 (since it is > currently global). Why do you want separate tables per interface? Are you concerned that physically separate networks, connected to different interfaces of the same host, might each contain hosts with the same IP address but different MACs? I don't think that's an issue for IPv4. IPv4 doesn't have any concept of scoping, so I don't think there's any sane way to construct such a network. IPv6 might be a different story. But I would consider that a separate issue from kern/167947. > > (3) After that, we can think of changing arpresolve to honor fib, but we > should handle cases like 'route leaking' or something similar which > involves GW mac from GRT (fib 0) being proxied somehow to fib X I don't understand. Are you talking about a situation where both FIBs would have direct access to the same subnet? I've tested that situation extensively (and had to fix another bug there). With my patches, I can have two interfaces, each with separate FIBs, attached to the same subnet and it works fine. -Alan > >
On 12.03.2014 00:27, Alan Somers wrote: >> (2) Then, we have to do per-interface LLINFO for IPv4/IPv6 (since it is >> currently global). > Why do you want separate tables per interface? Are you concerned that Sorry, actually I mean 'per-fib', but this doesn't differ much. > physically separate networks, connected to different interfaces of the > same host, might each contain hosts with the same IP address but > different MACs? I don't think that's an issue for IPv4. IPv4 doesn't Well, it is. RFC1918 was always popular. And it is even more popular nowadays :) > have any concept of scoping, so I don't think there's any sane way to > construct such a network. IPv6 might be a different story. But I > would consider that a separate issue from kern/167947. > >> (3) After that, we can think of changing arpresolve to honor fib, but we >> should handle cases like 'route leaking' or something similar which >> involves GW mac from GRT (fib 0) being proxied somehow to fib X > I don't understand. Are you talking about a situation where both FIBs > would have direct access to the same subnet? I've tested that Sort of. > situation extensively (and had to fix another bug there). With my > patches, I can have two interfaces, each with separate FIBs, attached > to the same subnet and it works fine. Can you send me these patches? > > -Alan > >>
On Wed, Mar 12, 2014 at 1:36 AM, Alexander V. Chernikov <melifaro@freebsd.org> wrote: > On 12.03.2014 00:27, Alan Somers wrote: >>> >>> (2) Then, we have to do per-interface LLINFO for IPv4/IPv6 (since it is >>> currently global). >> >> Why do you want separate tables per interface? Are you concerned that > > Sorry, actually I mean 'per-fib', but this doesn't differ much. > >> physically separate networks, connected to different interfaces of the >> same host, might each contain hosts with the same IP address but >> different MACs? I don't think that's an issue for IPv4. IPv4 doesn't > > Well, it is. RFC1918 was always popular. And it is even more popular > nowadays :) Sure, private networks are popular. But it's impossible for traffic to pass from one to another if they reuse addresses. Given that, under what circumstances would a FreeBSD box ever need to be able to talk to multiple such networks? I can think of a bad reason or two, but no good reasons. > >> have any concept of scoping, so I don't think there's any sane way to >> construct such a network. IPv6 might be a different story. But I >> would consider that a separate issue from kern/167947. >> >>> (3) After that, we can think of changing arpresolve to honor fib, but we >>> should handle cases like 'route leaking' or something similar which >>> involves GW mac from GRT (fib 0) being proxied somehow to fib X >> >> I don't understand. Are you talking about a situation where both FIBs >> would have direct access to the same subnet? I've tested that > > Sort of. > >> situation extensively (and had to fix another bug there). With my >> patches, I can have two interfaces, each with separate FIBs, attached >> to the same subnet and it works fine. > > Can you send me these patches? I'll open new bugs for each one. First, I need to make ATF test cases for them. I've got about another day's worth of work there. -Alan
Author: asomers Date: Thu Mar 20 20:39:41 2014 New Revision: 263445 URL: http://svnweb.freebsd.org/changeset/base/263445 Log: Add several ATF tests that deal with multiple fibs. They're described in several different PRs, but the tests share some common code, so I'm committing them together. sbin/ifconfig/tests sbin/ifconfig/tests/fibs_test.sh sbin/ifconfig/tests/Makefile sbin/ifconfig/Makefile Add fibs_test.sh, which regresses bin/187551 tests/sys/netinet tests/sys/netinet/fibs_test.sh tests/sys/netinet/udp_dontroute.c tests/sys/netinet/Makefile tests/sys/Makefile Add fibs_test.sh, which regresses kern/167947, kern/187552 kern/187549, kern/187550, and kern/187553 etc/mtree/BSD.tests.dist Add newly created directories PR: bin/187551 PR: kern/167947 PR: kern/187552 PR: kern/187549 PR: kern/187550 PR: kern/187553 Discussed with: melifaro MFC after: 3 weeks Sponsored by: Spectra Logic Corporation Added: head/sbin/ifconfig/tests/ head/sbin/ifconfig/tests/Makefile (contents, props changed) head/sbin/ifconfig/tests/fibs_test.sh (contents, props changed) head/tests/sys/netinet/ head/tests/sys/netinet/Makefile (contents, props changed) head/tests/sys/netinet/fibs_test.sh (contents, props changed) head/tests/sys/netinet/udp_dontroute.c (contents, props changed) Modified: head/etc/mtree/BSD.tests.dist head/sbin/ifconfig/Makefile head/tests/sys/Makefile Modified: head/etc/mtree/BSD.tests.dist ============================================================================== --- head/etc/mtree/BSD.tests.dist Thu Mar 20 20:33:18 2014 (r263444) +++ head/etc/mtree/BSD.tests.dist Thu Mar 20 20:39:41 2014 (r263445) @@ -75,6 +75,8 @@ .. growfs .. + ifconfig + .. mdconfig .. .. @@ -91,6 +93,8 @@ sys kern .. + netinet + .. .. usr.bin apply Modified: head/sbin/ifconfig/Makefile ============================================================================== --- head/sbin/ifconfig/Makefile Thu Mar 20 20:33:18 2014 (r263444) +++ head/sbin/ifconfig/Makefile Thu Mar 20 20:39:41 2014 (r263445) @@ -63,4 +63,8 @@ MAN= ifconfig.8 CFLAGS+= -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wnested-externs WARNS?= 2 +.if ${MK_TESTS} != "no" +SUBDIR+= tests +.endif + .include <bsd.prog.mk> Added: head/sbin/ifconfig/tests/Makefile ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sbin/ifconfig/tests/Makefile Thu Mar 20 20:39:41 2014 (r263445) @@ -0,0 +1,9 @@ +# $FreeBSD$ + +.include <bsd.own.mk> + +TESTSDIR= ${TESTSBASE}/sbin/ifconfig + +ATF_TESTS_SH= fibs_test + +.include <bsd.test.mk> Added: head/sbin/ifconfig/tests/fibs_test.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sbin/ifconfig/tests/fibs_test.sh Thu Mar 20 20:39:41 2014 (r263445) @@ -0,0 +1,126 @@ +# +# Copyright (c) 2014 Spectra Logic Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions, and the following disclaimer, +# without modification. +# 2. Redistributions in binary form must reproduce at minimum a disclaimer +# substantially similar to the "NO WARRANTY" disclaimer below +# ("Disclaimer") and any redistribution must be conditioned upon +# including a substantially similar Disclaimer requirement for further +# binary redistribution. +# +# NO WARRANTY +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING +# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGES. +# +# Authors: Alan Somers (Spectra Logic Corporation) +# +# $FreeBSD$ + + +# Regression test for bin/187551 +atf_test_case process_fib cleanup +process_fib_head() +{ + atf_set "descr" "ifconfig will set its process fib whenever configuring an interface with nondefault fib" + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} +process_fib_body() +{ + atf_expect_fail "bin/187551 ifconfig should change its process fib when configuring an interface with nondefault fib" + # Configure the TAP interface to use an RFC5737 nonrouteable address + # and a non-default fib + ADDR="192.0.2.2" + SUBNET="192.0.2.0" + MASK="24" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 1 + + # Configure a TAP interface + get_tap + ktrace ifconfig $TAP ${ADDR}/${MASK} fib $FIB0 + if kdump -s | egrep -q 'CALL[[:space:]]+setfib'; then + atf_pass + else + atf_fail "ifconfig never called setfib(2)" + fi +} + +process_fib_cleanup() +{ + cleanup_tap +} + +atf_init_test_cases() +{ + atf_add_test_case process_fib +} + + +# parameter numfibs The number of fibs to lookup +get_fibs() +{ + NUMFIBS=$1 + net_fibs=`sysctl -n net.fibs` + i=0 + while [ $i -lt "$NUMFIBS" ]; do + fib=`atf_config_get "fibs" | \ + awk -v i=$(( i + 1 )) '{print $i}'` + echo "fib is ${fib}" + eval FIB${i}=${fib} + if [ "$fib" -ge "$net_fibs" ]; then + atf_skip "The ${i}th configured fib is ${fib}, which is not less than net.fibs, which is ${net_fibs}" + fi + i=$(( $i + 1 )) + done +} + + + +# Creates a new tap(4) interface, registers it for cleanup, and returns the +# name via the environment variable TAP +get_tap() +{ + local TAPN=0 + while ! ifconfig tap${TAPN} create > /dev/null 2>&1; do + if [ "$TAPN" -ge 8 ]; then + atf_skip "Could not create a tap(4) interface" + else + TAPN=$(($TAPN + 1)) + fi + done + local TAPD=tap${TAPN} + # Record the TAP device so we can clean it up later + echo ${TAPD} >> "tap_devices_to_cleanup" + TAP=${TAPD} +} + + + + +cleanup_tap() +{ + for TAPD in `cat "tap_devices_to_cleanup"`; do + ifconfig ${TAPD} destroy + done +} + Modified: head/tests/sys/Makefile ============================================================================== --- head/tests/sys/Makefile Thu Mar 20 20:33:18 2014 (r263444) +++ head/tests/sys/Makefile Thu Mar 20 20:39:41 2014 (r263445) @@ -5,6 +5,7 @@ .PATH: ${.CURDIR}/.. TESTS_SUBDIRS+= kern +TESTS_SUBDIRS+= netinet TESTSDIR= ${TESTSBASE}/sys KYUAFILE= yes Added: head/tests/sys/netinet/Makefile ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/tests/sys/netinet/Makefile Thu Mar 20 20:39:41 2014 (r263445) @@ -0,0 +1,12 @@ +# $FreeBSD$ + +TESTSDIR= ${TESTSBASE}/sys/netinet +BINDIR= ${TESTSDIR} + +ATF_TESTS_SH+= fibs_test +PROG= udp_dontroute +SRCS= udp_dontroute.c +NO_MAN= +WARNS?= 6 + +.include <bsd.test.mk> Added: head/tests/sys/netinet/fibs_test.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/tests/sys/netinet/fibs_test.sh Thu Mar 20 20:39:41 2014 (r263445) @@ -0,0 +1,371 @@ +# +# Copyright (c) 2014 Spectra Logic Corporation +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions, and the following disclaimer, +# without modification. +# 2. Redistributions in binary form must reproduce at minimum a disclaimer +# substantially similar to the "NO WARRANTY" disclaimer below +# ("Disclaimer") and any redistribution must be conditioned upon +# including a substantially similar Disclaimer requirement for further +# binary redistribution. +# +# NO WARRANTY +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING +# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGES. +# +# Authors: Alan Somers (Spectra Logic Corporation) +# +# $FreeBSD$ + +# All of the tests in this file requires the test-suite config variable "fibs" +# to be defined to a space-delimited list of FIBs that may be used for testing. + +# arpresolve should check the interface fib for routes to a target when +# creating an ARP table entry. This is a regression for kern/167947, where +# arpresolve only checked the default route. +# +# Outline: +# Create two tap(4) interfaces +# Simulate a crossover cable between them by using net/socat +# Use nping (from security/nmap) to send an ICMP echo request from one +# interface to the other, spoofing the source IP. The source IP must be +# spoofed, or else it will already have an entry in the arp table. +# Check whether an arp entry exists for the spoofed IP +atf_test_case arpresolve_checks_interface_fib cleanup +arpresolve_checks_interface_fib_head() +{ + atf_set "descr" "arpresolve should check the interface fib, not the default fib, for routes" + atf_set "require.user" "root" + atf_set "require.config" "fibs" + atf_set "require.progs" "socat nping" +} +arpresolve_checks_interface_fib_body() +{ + atf_expect_fail "kern/167947 arpresolve checks only the default FIB for the interface route" + # Configure the TAP interfaces to use a RFC5737 nonrouteable addresses + # and a non-default fib + ADDR0="192.0.2.2" + ADDR1="192.0.2.3" + SUBNET="192.0.2.0" + # Due to bug TBD (regressed by multiple_fibs_on_same_subnet) we need + # diffferent subnet masks, or FIB1 won't have a subnet route. + MASK0="24" + MASK1="25" + # Spoof a MAC that is reserved per RFC7042 + SPOOF_ADDR="192.0.2.4" + SPOOF_MAC="00:00:5E:00:53:00" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 2 + + # Configure TAP interfaces + setup_tap "$FIB0" ${ADDR0} ${MASK0} + TAP0=$TAP + setup_tap "$FIB1" ${ADDR1} ${MASK1} + TAP1=$TAP + + # Simulate a crossover cable + socat /dev/${TAP0} /dev/${TAP1} & + SOCAT_PID=$! + echo ${SOCAT_PID} >> "processes_to_kill" + + # Send an ICMP echo request with a spoofed source IP + setfib 2 nping -c 1 -e ${TAP0} -S ${SPOOF_ADDR} \ + --source-mac ${SPOOF_MAC} --icmp --icmp-type "echo-request" \ + --icmp-code 0 --icmp-id 0xdead --icmp-seq 1 --data 0xbeef \ + ${ADDR1} + # For informational and debugging purposes only, look for the + # characteristic error message + dmesg | grep "llinfo.*${SPOOF_ADDR}" + # Check that the ARP entry exists + atf_check -o match:"${SPOOF_ADDR}.*expires" setfib 3 arp ${SPOOF_ADDR} +} +arpresolve_checks_interface_fib_cleanup() +{ + for PID in `cat "processes_to_kill"`; do + kill $PID + done + cleanup_tap +} + + +# Regression test for kern/187549 +atf_test_case loopback_and_network_routes_on_nondefault_fib cleanup +loopback_and_network_routes_on_nondefault_fib_head() +{ + atf_set "descr" "When creating and deleting loopback routes, use the interface's fib" + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} + +loopback_and_network_routes_on_nondefault_fib_body() +{ + atf_expect_fail "kern/187549 Host and network routes for a new interface appear in the wrong FIB" + # Configure the TAP interface to use an RFC5737 nonrouteable address + # and a non-default fib + ADDR="192.0.2.2" + SUBNET="192.0.2.0" + MASK="24" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 1 + + # Configure a TAP interface + setup_tap ${FIB0} ${ADDR} ${MASK} + + # Check whether the host route exists in only the correct FIB + setfib ${FIB0} netstat -rn -f inet | grep -q "^${ADDR}.*UHS.*lo0" + if [ 0 -ne $? ]; then + setfib ${FIB0} netstat -rn -f inet + atf_fail "Host route did not appear in the correct FIB" + fi + setfib 0 netstat -rn -f inet | grep -q "^${ADDR}.*UHS.*lo0" + if [ 0 -eq $? ]; then + setfib 0 netstat -rn -f inet + atf_fail "Host route appeared in the wrong FIB" + fi + + # Check whether the network route exists in only the correct FIB + setfib ${FIB0} netstat -rn -f inet | \ + grep -q "^${SUBNET}/${MASK}.*${TAPD}" + if [ 0 -ne $? ]; then + setfib ${FIB0} netstat -rn -f inet + atf_fail "Network route did not appear in the correct FIB" + fi + setfib 0 netstat -rn -f inet | \ + grep -q "^${SUBNET}/${MASK}.*${TAPD}" + if [ 0 -eq $? ]; then + setfib ${FIB0} netstat -rn -f inet + atf_fail "Network route appeared in the wrong FIB" + fi +} + +loopback_and_network_routes_on_nondefault_fib_cleanup() +{ + cleanup_tap +} + + +# Regression test for kern/187552 +atf_test_case default_route_with_multiple_fibs_on_same_subnet cleanup +default_route_with_multiple_fibs_on_same_subnet_head() +{ + atf_set "descr" "Multiple interfaces on the same subnet but with different fibs can both have default routes" + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} + +default_route_with_multiple_fibs_on_same_subnet_body() +{ + atf_expect_fail "kern/187552 default route uses the wrong interface when multiple interfaces have the same subnet but different fibs" + # Configure the TAP interfaces to use a RFC5737 nonrouteable addresses + # and a non-default fib + ADDR0="192.0.2.2" + ADDR1="192.0.2.3" + GATEWAY="192.0.2.1" + SUBNET="192.0.2.0" + MASK="24" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 2 + + # Configure TAP interfaces + setup_tap "$FIB0" ${ADDR0} ${MASK} + TAP0=$TAP + setup_tap "$FIB1" ${ADDR1} ${MASK} + TAP1=$TAP + + # Attempt to add default routes + setfib ${FIB0} route add default ${GATEWAY} + setfib ${FIB1} route add default ${GATEWAY} + + # Verify that the default route exists for both fibs, with their + # respective interfaces. + atf_check -o match:"^default.*${TAP0}$" \ + setfib ${FIB0} netstat -rn -f inet + atf_check -o match:"^default.*${TAP1}$" \ + setfib ${FIB1} netstat -rn -f inet +} + +default_route_with_multiple_fibs_on_same_subnet_cleanup() +{ + cleanup_tap +} + + +# Regression test for kern/187550 +atf_test_case subnet_route_with_multiple_fibs_on_same_subnet cleanup +subnet_route_with_multiple_fibs_on_same_subnet_head() +{ + atf_set "descr" "Multiple FIBs can have subnet routes for the same subnet" + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} + +subnet_route_with_multiple_fibs_on_same_subnet_body() +{ + atf_expect_fail "kern/187550 Multiple interfaces on different FIBs but the same subnet don't all have a subnet route" + # Configure the TAP interfaces to use a RFC5737 nonrouteable addresses + # and a non-default fib + ADDR0="192.0.2.2" + ADDR1="192.0.2.3" + SUBNET="192.0.2.0" + MASK="24" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 2 + + # Configure TAP interfaces + setup_tap "$FIB0" ${ADDR0} ${MASK} + setup_tap "$FIB1" ${ADDR1} ${MASK} + + # Check that a subnet route exists on both fibs + atf_check -o ignore setfib "$FIB0" route get $ADDR1 + atf_check -o ignore setfib "$FIB1" route get $ADDR0 +} + +subnet_route_with_multiple_fibs_on_same_subnet_cleanup() +{ + cleanup_tap +} + +# Test that source address selection works correctly for UDP packets with +# SO_DONTROUTE set that are sent on non-default FIBs. +# This bug was discovered with "setfib 1 netperf -t UDP_STREAM -H some_host" +# Regression test for kern/187553 +atf_test_case udp_dontroute cleanup +udp_dontroute_head() +{ + atf_set "descr" "Source address selection for UDP packets with SO_DONTROUTE on non-default FIBs works" + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} + +udp_dontroute_body() +{ + atf_expect_fail "kern/187553 Source address selection for UDP packets with SO_DONTROUTE uses the default FIB" + # Configure the TAP interface to use an RFC5737 nonrouteable address + # and a non-default fib + ADDR="192.0.2.2" + SUBNET="192.0.2.0" + MASK="24" + # Use a different IP on the same subnet as the target + TARGET="192.0.2.100" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 1 + + # Configure a TAP interface + setup_tap ${FIB0} ${ADDR} ${MASK} + + # Send a UDP packet with SO_DONTROUTE. In the failure case, it will + # return ENETUNREACH + SRCDIR=`atf_get_srcdir` + atf_check -o ignore setfib ${FIB0} ${SRCDIR}/udp_dontroute ${TARGET} +} + +udp_dontroute_cleanup() +{ + cleanup_tap +} + + +atf_init_test_cases() +{ + atf_add_test_case arpresolve_checks_interface_fib + atf_add_test_case loopback_and_network_routes_on_nondefault_fib + atf_add_test_case default_route_with_multiple_fibs_on_same_subnet + atf_add_test_case subnet_route_with_multiple_fibs_on_same_subnet + atf_add_test_case udp_dontroute +} + +# Looks up one or more fibs from the configuration data and validates them. +# Returns the results in the env varilables FIB0, FIB1, etc. + +# parameter numfibs The number of fibs to lookup +get_fibs() +{ + NUMFIBS=$1 + net_fibs=`sysctl -n net.fibs` + i=0 + while [ $i -lt "$NUMFIBS" ]; do + fib=`atf_config_get "fibs" | \ + awk -v i=$(( i + 1 )) '{print $i}'` + echo "fib is ${fib}" + eval FIB${i}=${fib} + if [ "$fib" -ge "$net_fibs" ]; then + atf_skip "The ${i}th configured fib is ${fib}, which is not less than net.fibs, which is ${net_fibs}" + fi + i=$(( $i + 1 )) + done +} + +# Creates a new tap(4) interface, registers it for cleanup, and returns the +# name via the environment variable TAP +get_tap() +{ + local TAPN=0 + while ! ifconfig tap${TAPN} create > /dev/null 2>&1; do + if [ "$TAPN" -ge 8 ]; then + atf_skip "Could not create a tap(4) interface" + else + TAPN=$(($TAPN + 1)) + fi + done + local TAPD=tap${TAPN} + # Record the TAP device so we can clean it up later + echo ${TAPD} >> "tap_devices_to_cleanup" + TAP=${TAPD} +} + +# Create a tap(4) interface, configure it, and register it for cleanup. +# parameters: +# fib +# IP address +# Netmask in number of bits (eg 24 or 8) +# Return: the tap interface name as the env variable TAP +setup_tap() +{ + local FIB=$1 + local ADDR=$2 + local MASK=$3 + get_tap + echo setfib ${FIB} ifconfig $TAP ${ADDR}/${MASK} fib $FIB + setfib ${FIB} ifconfig $TAP ${ADDR}/${MASK} fib $FIB +} + +cleanup_tap() +{ + for TAPD in `cat "tap_devices_to_cleanup"`; do + ifconfig ${TAPD} destroy + done +} Added: head/tests/sys/netinet/udp_dontroute.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/tests/sys/netinet/udp_dontroute.c Thu Mar 20 20:39:41 2014 (r263445) @@ -0,0 +1,85 @@ +/* + * Copyright (c) 2014 Spectra Logic Corporation + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions, and the following disclaimer, + * without modification. + * 2. Redistributions in binary form must reproduce at minimum a disclaimer + * substantially similar to the "NO WARRANTY" disclaimer below + * ("Disclaimer") and any redistribution must be conditioned upon + * including a substantially similar Disclaimer requirement for further + * binary redistribution. + * + * NO WARRANTY + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGES. + * + * Authors: Alan Somers (Spectra Logic Corporation) + * + * $FreeBSD$ + */ + +#include <arpa/inet.h> +#include <netinet/in.h> +#include <sys/types.h> +#include <sys/socket.h> + +#include <err.h> +#include <errno.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +/* + * Sends a single UDP packet to the provided address, with SO_DONTROUTE set + * I couldn't find a way to do this with builtin utilities like nc(1) + */ +int main(int argc, char **argv) +{ + struct sockaddr_in dst; + int s; + int opt; + int ret; + const char* buf = "Hello, World!"; + + if (argc != 2) { + fprintf(stderr, "Usage: %s ip_address\n", argv[0]); + exit(2); + } + s = socket(PF_INET, SOCK_DGRAM, 0); + if (s < 0) + err(errno, "socket"); + opt = 1; + + ret = setsockopt(s, SOL_SOCKET, SO_DONTROUTE, &opt, sizeof(opt)); + if (ret == -1) + err(errno, "setsockopt(SO_DONTROUTE)"); + + dst.sin_len = sizeof(dst); + dst.sin_family = AF_INET; + dst.sin_port = htons(46120); + dst.sin_addr.s_addr = inet_addr(argv[1]); + if (dst.sin_addr.s_addr == htonl(INADDR_NONE)) { + fprintf(stderr, "Invalid address: %s\n", argv[1]); + exit(2); + } + ret = sendto(s, buf, strlen(buf), 0, (struct sockaddr*)&dst, + dst.sin_len); + if (ret == -1) + err(errno, "sendto"); + + return (0); +} _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Here is an updated patch for this bug. The test case is sys/netinet/fibs_test:arpresolve_checks_interface_fib . BTW, the other FIB bugs I've opened are kern/187549, kern/187550, bin/187551, kern/187552, kern/187553, and bin/187699.
Author: asomers Date: Wed Mar 26 22:46:03 2014 New Revision: 263779 URL: http://svnweb.freebsd.org/changeset/base/263779 Log: Correct ARP update handling when the routes for network interfaces are restricted to a single FIB in a multifib system. Restricting an interface's routes to the FIB to which it is assigned (by setting net.add_addr_allfibs=0) causes ARP updates to fail with "arpresolve: can't allocate llinfo for x.x.x.x". This is due to the ARP update code hard coding it's lookup for existing routing entries to FIB 0. sys/netinet/in.c: When dealing with RTM_ADD (add route) requests for an interface, use the interface's assigned FIB instead of the default (FIB 0). sys/netinet/if_ether.c: In arpresolve(), enhance error message generated when an lla_lookup() fails so that the interface causing the error is visible in logs. tests/sys/netinet/fibs_test.sh Clear ATF expected error. PR: kern/167947 Submitted by: Nikolay Denev <ndenev@gmail.com> (previous version) Reviewed by: melifaro MFC after: 3 weeks Sponsored by: Spectra Logic Corporation Modified: head/sys/netinet/if_ether.c head/sys/netinet/in.c head/tests/sys/netinet/fibs_test.sh Modified: head/sys/netinet/if_ether.c ============================================================================== --- head/sys/netinet/if_ether.c Wed Mar 26 22:30:38 2014 (r263778) +++ head/sys/netinet/if_ether.c Wed Mar 26 22:46:03 2014 (r263779) @@ -334,8 +334,8 @@ retry: if (la == NULL) { if (flags & LLE_CREATE) log(LOG_DEBUG, - "arpresolve: can't allocate llinfo for %s\n", - inet_ntoa(SIN(dst)->sin_addr)); + "arpresolve: can't allocate llinfo for %s on %s\n", + inet_ntoa(SIN(dst)->sin_addr), ifp->if_xname); m_freem(m); return (EINVAL); } Modified: head/sys/netinet/in.c ============================================================================== --- head/sys/netinet/in.c Wed Mar 26 22:30:38 2014 (r263778) +++ head/sys/netinet/in.c Wed Mar 26 22:46:03 2014 (r263779) @@ -994,8 +994,9 @@ in_lltable_rtcheck(struct ifnet *ifp, u_ KASSERT(l3addr->sa_family == AF_INET, ("sin_family %d", l3addr->sa_family)); - /* XXX rtalloc1 should take a const param */ - rt = rtalloc1(__DECONST(struct sockaddr *, l3addr), 0, 0); + /* XXX rtalloc1_fib should take a const param */ + rt = rtalloc1_fib(__DECONST(struct sockaddr *, l3addr), 0, 0, + ifp->if_fib); if (rt == NULL) return (EINVAL); Modified: head/tests/sys/netinet/fibs_test.sh ============================================================================== --- head/tests/sys/netinet/fibs_test.sh Wed Mar 26 22:30:38 2014 (r263778) +++ head/tests/sys/netinet/fibs_test.sh Wed Mar 26 22:46:03 2014 (r263779) @@ -55,7 +55,6 @@ arpresolve_checks_interface_fib_head() } arpresolve_checks_interface_fib_body() { - atf_expect_fail "kern/167947 arpresolve checks only the default FIB for the interface route" # Configure the TAP interfaces to use a RFC5737 nonrouteable addresses # and a non-default fib ADDR0="192.0.2.2" _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
State Changed From-To: open->closed see kern/187549.
Responsible Changed From-To: melifaro->asomers
State Changed From-To: closed->patched This is merely patched, not closed, because I haven't MFCed the fix
A commit references this bug: Author: asomers Date: Thu Sep 18 20:17:42 UTC 2014 New revision: 271809 URL: http://svnweb.freebsd.org/changeset/base/271809 Log: MFC 263779, except for the ATF test change. Correct ARP update handling when the routes for network interfaces are restricted to a single FIB in a multifib system. Restricting an interface's routes to the FIB to which it is assigned (by setting net.add_addr_allfibs=0) causes ARP updates to fail with "arpresolve: can't allocate llinfo for x.x.x.x". This is due to the ARP update code hard coding it's lookup for existing routing entries to FIB 0. sys/netinet/in.c: When dealing with RTM_ADD (add route) requests for an interface, use the interface's assigned FIB instead of the default (FIB 0). sys/netinet/if_ether.c: In arpresolve(), enhance error message generated when an lla_lookup() fails so that the interface causing the error is visible in logs. PR: kern/167947 Changes: _U stable/9/ _U stable/9/sys/ stable/9/sys/netinet/if_ether.c stable/9/sys/netinet/in.c
MFCed to stable/9 by change 271809 and to stable/10 by change 267175.