Bug 71474 - [route] route lookup does not skip interfaces marked down
Summary: [route] route lookup does not skip interfaces marked down
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 5.2-CURRENT
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-09-07 19:50 UTC by Thomas Quinot
Modified: 2018-05-28 19:50 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Quinot 2004-09-07 19:50:22 UTC
	When a route is set, the interface corresponding to the destination
	hop is looked up among all existing interfaces, not only those that
	are marked up, causing the wrong interface to be selected in some cases.

Fix: 

Fix not determined yet. This problem can be worked around by
	changing the addresses on the down interface so it won't clash
	with those of the up one:

	# ifconfig gif0 192.168.99.1 192.168.99.2
	# ifconfig gif0 down
	# route delete 192.168.1.0 192.168.0.1
	# route add 192.168.1.0 192.168.0.1
	# netstat -rn | grep 192.168
	192.168.0.1        192.168.0.253      UH          1        0   gif1
	192.168.1          192.168.0.1        UGS         0        0   gif1
How-To-Repeat: 	This issue can be demonstrated using gif interfaces:

	# ifconfig gif0 create
	# ifconfig gif0 192.168.0.254 192.168.0.1
	# ifconfig gif0 down
	# ifconfig gif1 create
	# ifconfig gif1 192.168.0.253 192.168.0.1

	Note at this point that 192.168.0.1 is reachable through interface
	gif1.

	# route add 192.168.1.0 192.168.0.1
	# netstat -rn | grep 192.168
	192.168.0.1        192.168.0.253      UH          1        0   gif1
	192.168.1          192.168.0.1        UGS         0        0   gif0

	Note then that, even though 192.168.0.1 is marked as reachable through
	gif1, for the purpose of routing to 192.168.1.0 the selected interface
	is gif0 (which is down).
Comment 1 ru freebsd_committer 2004-09-08 07:24:51 UTC
Just tell it the interface you really want:

route add -net 192.168.1 -iface gif1


Cheers,
-- 
Ruslan Ermilov
ru@FreeBSD.org
FreeBSD committer
Comment 2 ru freebsd_committer 2004-09-08 12:46:54 UTC
On Wed, Sep 08, 2004 at 12:09:39PM +0200, Thomas Quinot wrote:
> * Ruslan Ermilov, 2004-09-08 :
> 
> > This is *not* how I spelled it.  Try the above as shown.
> 
> I stand corrected. The command as proposed does work. Unfortunately this
> applies only to point-to-point interfaces, not in the case where a
> specific next-hop address is required.
> 
But your test case only applies to P2P interfaces, when both ends
of two P2P interfaces share the same IP address.  I'd call this a
misconfiguration.  I do not see how this could apply to broadcast
interfaces.


Cheers,
-- 
Ruslan Ermilov
ru@FreeBSD.org
FreeBSD committer
Comment 3 ru freebsd_committer 2004-09-08 13:49:26 UTC
On Wed, Sep 08, 2004 at 02:26:29PM +0200, Thomas Quinot wrote:
> * Ruslan Ermilov, 2004-09-08 :
> 
> >  But your test case only applies to P2P interfaces, when both ends
> >  of two P2P interfaces share the same IP address.
> 
> No, this is not the case. Only the remote ends of the two interfaces
> share the same IP address. I described that test case because using gif
> interfaces makes it easier to reproduce, but I initially observed this
> symptom with one bge and one Intel wireless (ndis) interface.
> 
That's what I meant (two remote ends of both interfaces).

> >  I do not see how this could apply to broadcast interfaces.
> 
> As follows. Consider a machine with two Ethernet interfaces I1 and I2.
> 
> ifconfig I1 192.168.0.1 netmask 255.255.255.0
> ifconfig I1 down
> ifconfig I2 192.168.0.2 netmask 255.255.255.0
> route add default 192.168.0.254
> 
> The default route will be set to 192.168.0.254 on I1, not on I2 as
> expected.
> 
It's still a misconfiguration -- if you do not bring the I1 interface
down, the "interface" route for I2 will not be installed, and the
ifconfig(8) command will be aborted.  Anyway...

I reproduced this with two ng_eiface(4) nodes.  There's still a useful
option in the route(8) utility -- you can supply an explicit reference
to the interface, like this:

: route add default 192.168.0.254 -ifp I1

This is also vaguely documented in the route(8) manpage, FWIW:

: In a change or add command where the destination and gateway are not suf-
: ficient to specify the route ..., the -ifp or -ifa modifiers may be used
: to determine the interface or interface address.


Cheers,
-- 
Ruslan Ermilov
ru@FreeBSD.org
FreeBSD committer
Comment 4 ru freebsd_committer 2004-09-08 14:49:27 UTC
On Wed, Sep 08, 2004 at 03:06:48PM +0200, Thomas Quinot wrote:
> * Ruslan Ermilov, 2004-09-08 :
> 
> >  > ifconfig I1 192.168.0.1 netmask 255.255.255.0
> >  > ifconfig I1 down
> >  > ifconfig I2 192.168.0.2 netmask 255.255.255.0
> >  > route add default 192.168.0.254
> >  > 
> >  > The default route will be set to 192.168.0.254 on I1, not on I2 as
> >  > expected.
> >  > 
> >  It's still a misconfiguration -- if you do not bring the I1 interface
> >  down, the "interface" route for I2 will not be installed, and the
> >  ifconfig(8) command will be aborted.  Anyway...
> 
> I am not sure I get what you mean. The point of this PR is that I *do*
> bring I1 down, that the kernel has an unambiguous indication of how to
> reach the specified gateway ('ping 192.168.0.254' works in the example
> described above), and in spite of that the wrong interface is selected
> when creating a route.
> 
This is a misconfiguration because you cannot expect two broadcast
interfaces configured with the same IP network to work.  The fact
that it works at all, in a scenario you shown, is probably a bug.

> >  I reproduced this with two ng_eiface(4) nodes.  There's still a useful
> >  option in the route(8) utility -- you can supply an explicit reference
> >  to the interface, like this:
> >  : route add default 192.168.0.254 -ifp I1
> 
> Right, I was not aware of that option.
> 
> >  This is also vaguely documented in the route(8) manpage, FWIW:
> 
> *Vaguely* is the right term I think. :-)
> 
> Having browsed through the code, I think the interface lookup for route
> additions should use a modified version of ifa_ifwithaddr ignoring
> non-up interfaces.
> 
If we change ifa_ifwith*() to ignore !IFF_UP interfaces, I suspect this
will break too many things.  I think the correct change here would be
to set the interface of the route to the interface of its gateway (if
route is through the gateway (RTF_GATEWAY)).  This should be easy to
fix, but I don't have a time for this now.


Cheers,
-- 
Ruslan Ermilov
ru@FreeBSD.org
FreeBSD committer
Comment 5 Bruce M Simpson freebsd_committer 2006-09-25 20:15:12 UTC
Responsible Changed
From-To: freebsd-bugs->bms

I'll take this
Comment 6 Bruce M Simpson 2006-09-27 13:30:42 UTC
Hi,

I've been looking at this PR. A few points, based on code inspection and 
reproducing the test case.

 *ifa_ifwithroute() already checks for RTF_GATEWAY, and unconditionally 
calls ifa_ifwithdstaddr() to obtain the ifa pointer.
 * The rtmsg passed in from userland is well-formed (RTM_ADD).
 * ifa_ifwithroute() in this case is called from rt_getifa() during RTM_ADD.
 * ifa_ifwithdstaddr() is called from many other parts of the stack, so 
being conservative about the change seems sensible.
 * This isn't really a multipath problem so much as a next-hop 
resolution problem.

Perhaps if we change ifa_ifwithdstaddr() to ifa_ifwithdstaddrflags() and 
pass IFF_UP as a necessary flag for the interface match that would be an 
acceptable fix, and change other consumers to pass 0 or use a macro to 
emulate the old function.

We have to call ifa_ifwithdstaddr() to look up the correct ifaddr for 
the rt->rt_ifa anyway; the rt->rt_ifp can in turn be looked up from that.

Just my 2c,
BMS
Comment 7 Bruce M Simpson 2006-09-27 13:59:20 UTC
For a related patch, see also:
http://lists.freebsd.org/pipermail/freebsd-net/2006-February/009884.html

(Provided by glebius)
Comment 8 Bruce M Simpson freebsd_committer 2006-09-27 14:25:46 UTC
State Changed
From-To: open->analyzed

We can reproduce the problem, we've analyzed it, there are possible 
fixes, we need to decide what to do.
Comment 9 Bruce M Simpson 2007-03-19 00:19:13 UTC
This looks like an architectural problem stemming from how next-hop 
resolution is performed in FreeBSD's forwarding plane. Fear not -- XORP 
has similar problems; recursively resolving next-hops when they change 
is not a trivial issue.

To my mind this needs to be fixed with a rewrite. Looking at the code, 
we'd have to do another tree walk on PRC_IFDOWN. We already do this, but 
we ignore static routes.

The reason why the default route doesn't get its ifa/ifp updated when 
the interface it refers to goes down, is because it's static.
Comment 10 Bruce M Simpson 2007-07-18 00:01:56 UTC
This sounds like a job for a 'floating static' forwarding entry.

A 'floating static' entry is one whose next-hop may be re-resolved 
although the entry itself is static and should be treated as such by the 
forwarding code.

A 'floating' flag would allow the PRC_IFDOWN path to tell the 
difference, and update the ifp in those cases where the next-hop would 
change, without clobbering the state of the next-hop for FIB entries 
marked as static.

See IOS or JunOS documentation for more information on 'floating static 
routes'.
Comment 11 Bruce M Simpson freebsd_committer 2010-06-06 16:07:11 UTC
Responsible Changed
From-To: bms->none

I'm afraid I have no free time for the foreseeable future which 
I can commit directly to FreeBSD work, due to a demanding new role. 

I am happy to review the occasional patch, but regrettably cannot 
be directly involved in engineering work beyond this. 

Thank you for your understanding.
Comment 12 Mark Linimon freebsd_committer freebsd_triage 2010-06-07 18:55:17 UTC
Responsible Changed
From-To: none->freebsd-bugs

Canonicalize assignment.
Comment 13 Mark Linimon freebsd_committer freebsd_triage 2010-06-07 18:55:37 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-net

On second thought, it probably belongs to -net.
Comment 14 Eitan Adler freebsd_committer freebsd_triage 2018-05-28 19:50:11 UTC
batch change:

For bugs that match the following
-  Status Is In progress 
AND
- Untouched since 2018-01-01.
AND
- Affects Base System OR Documentation

DO:

Reset to open status.


Note:
I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.