Bug 241191 - route flush panic with RADIX_MPATH
Summary: route flush panic with RADIX_MPATH
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-STABLE
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-net mailing list
URL:
Keywords: crash, needs-qa
Depends on:
Blocks: 240700
  Show dependency treegraph
 
Reported: 2019-10-11 07:25 UTC by Andrey Linkevich
Modified: 2019-10-11 08:38 UTC (History)
4 users (show)

See Also:
koobs: maintainer-feedback? (melifaro)
koobs: maintainer-feedback? (glebius)
koobs: mfc-stable11?
koobs: mfc-stable12?


Attachments
Panic screenshot, 1 (11.52 KB, image/png)
2019-10-11 07:26 UTC, Andrey Linkevich
no flags Details
Panic screenshot, 2 (15.83 KB, image/png)
2019-10-11 07:26 UTC, Andrey Linkevich
no flags Details
Patch (1.45 KB, patch)
2019-10-11 07:33 UTC, Andrey Linkevich
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Linkevich 2019-10-11 07:25:02 UTC
Hello, Colleagues.

We are use RADIX_MPATH .
Have a lot of IPv4 route multipath.

Our FreeBSD 12.0-STABLE is panic when execute `route flush`. (See details and screenshots)

The problem is caused by incorrect processing of return rt_unlinkrte() from sys/net/route.c .

With RADIX_MPATH return may be _NULL_ with *perror = 0.

Than rtrequest1_fib() from sys/net/route.c is panic;
Also route_output() from sys/net/rtsock.c .

I prepared the patch.
Please check it out. If he does not contradict anything, then accept.

Perhaps patch creates some other problems. However, patch solves the original problem successfully.

Thanks.


Add details.

root@noc-srr01:~ # sysctl kern | grep kern.os
kern.ostype: FreeBSD
kern.osrelease: 12.0-STABLE
kern.osrevision: 199506
kern.osreldate: 1200503


Example 1: WITHOUT kernel options RADIX_MPATH

# A lot of routes
root@noc-srr01:/boot/kernel # netstat -rnW | wc -l
   16132

# 
root@noc-srr01:/boot/kernel # netstat -rnW | more
Routing tables

Internet:
Destination        Gateway            Flags       Use    Mtu      Netif Expire
default            10.169.211.1       UGS        3945   1500        xn0
1.1.2.0/30         10.169.213.234     UG1           0   1500        xn1
1.2.2.0/30         10.169.213.234     UG1           0   1500        xn1
1.2.3.0/30         10.169.213.234     UG1           0   1500        xn1
...

# Work flush correctly
root@noc-srr01:/boot/kernel # netstat -rn | wc -l ; route -qn flush ; netstat -rn | wc -l
   16132
     23


Example 2: WITH kernel options RADIX_MPATH

# A lot of routes (16000 * 4 ifs and routers)
root@noc-srr01:~ # netstat -rn | wc -l
   63595

# 
root@noc-srr01:~ # netstat -rn | more
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.169.211.1       UGS         xn0
1.1.2.0/30         10.169.213.234     UG1         xn1
1.1.2.0/30         10.169.213.242     UG1         xn2
1.1.2.0/30         10.169.213.233     UG1         xn1
1.1.2.0/30         10.169.213.241     UG1         xn2
1.2.2.0/30         10.169.213.234     UG1         xn1
1.2.2.0/30         10.169.213.242     UG1         xn2
1.2.2.0/30         10.169.213.233     UG1         xn1
1.2.2.0/30         10.169.213.241     UG1         xn2
1.2.3.0/30         10.169.213.234     UG1         xn1
1.2.3.0/30         10.169.213.242     UG1         xn2
1.2.3.0/30         10.169.213.233     UG1         xn1
1.2.3.0/30         10.169.213.241     UG1         xn2

# flush panic ... see ScreenShots in attach
root@noc-srr01:~ # netstat -rn | wc -l ; route -qn flush ; netstat -rn | wc -l
   63599
Comment 1 Andrey Linkevich 2019-10-11 07:26:13 UTC
Created attachment 208238 [details]
Panic screenshot, 1
Comment 2 Andrey Linkevich 2019-10-11 07:26:43 UTC
Created attachment 208239 [details]
Panic screenshot, 2
Comment 3 Andrey Linkevich 2019-10-11 07:33:02 UTC
Created attachment 208240 [details]
Patch
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2019-10-11 08:38:54 UTC
Crash report with patch on stable/12, potential 12.1-R candidate

CC recent committers around that section of code