Bug 146792 - [flowtable] flowcleaner 100% cpu's core load
Summary: [flowtable] flowcleaner 100% cpu's core load
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: Unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-05-21 11:40 UTC by Nikolay Nevzorov
Modified: 2018-05-28 21:01 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nikolay Nevzorov 2010-05-21 11:40:02 UTC
On high-load network router since some time (1-5 days) process flowcleaner get's 100% load on a one CPU's core. 
ssh works, remote reboot can't be done, only hard reboot.


Bug was seen at many routers:
http://forum.nag.ru/forum/index.php?showtopic=55485&st=0

Fix: 

Without "flowtable" option in kernel's config router works fine.
How-To-Repeat: Bug can be seen only at high-load (more than 30 megabits) routers.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2010-05-21 21:52:17 UTC
Responsible Changed
From-To: freebsd-amd64->freebsd-net

Probably not amd64-specific.
Comment 2 Kurt Jaeger 2010-05-23 15:19:25 UTC
Hi!

I observe a similar behaviour on a 8.0-RELEASE-p2 i386 GENERIC
kernel.

System receives 2 BGP4 fullfeeds (approx. 310K routes each).

The system is still running, a few processes are unkillable or
die only after a long amount (1-2h) of time.

Here's the list of unkillable processes:

80871  ??  R      0:00.00 /bin/sh /etc/periodic/daily/470.status-named
76499  ??  Rs     0:00.01 sshd: [accepted] (sshd)
76922  ??  Rs     0:00.01 sshd: [accepted] (sshd)

flowcleaner looks pretty busy (for an uptime of approx. 40h):

   22  ??  RL   1209:50.98 [flowcleaner]

 4:17PM  up 1 day, 22:22, 2 users, load averages: 7.20, 6.53, 5.81

quagga is running on the system, bgpd mgmt cli is no longer reachable:

# telnet 0 2605
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.

^]
telnet> close
Connection closed.
# 

What can I do to help to debug this ?
No console access available right now, but can probably made available.

This is a production host, but not yet super-critical, so...

-- 
pi@opsec.eu            +49 171 3101372                        10 years to go !
Comment 3 Yar Tikhiy freebsd_committer freebsd_triage 2010-09-21 22:00:47 UTC
FWIW, this flowtable problem still can be triggered in 8.1-RELEASE.
Yesterday it was seen on a machine with approx. 300 routes from OSPF
-- no BGP full views involved.  The machine was unable to reboot
cleanly, was stuck forever waiting for vnlru to exit.  Can it be a
locking issue?

Thanks,
Yar
Comment 4 nti 2010-10-01 22:19:17 UTC
Today I recognize also 100 % CPU load by flowcleaner.

Machine is a fresh installed 8.1 stable with *no* router functionality but with opennms - a netmanagement application - installed. 

Nicola=
Comment 5 mark.boolootian 2011-02-02 19:37:16 UTC
Hi folks,

I hit this problem on a pair of anycast name servers.  What
I'm running:

FreeBSD ns1.example.com 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19
02:36:49
UTC 2010     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

Here's a peak at ps:

ns1b# ps auxwww | head
USER     PID %CPU %MEM   VSZ   RSS  TT  STAT STARTED      TIME COMMAND
root      11 100.0  0.0     0    32  ??  RL   11Jan11 59960:15.43 [idle]
root      21 100.0  0.0     0    16  ??  RL   11Jan11 1112:01.24
[flowcleaner]
root       0  0.0  0.0     0    96  ??  DLs  11Jan11   0:02.94 [kernel]
root       1  0.0  0.0  3204   556  ??  ILs  11Jan11   0:00.01 /sbin/init --
root       2  0.0  0.0     0    16  ??  DL   11Jan11   0:52.87 [g_event]
root       3  0.0  0.0     0    16  ??  DL   11Jan11   0:10.10 [g_up]
root       4  0.0  0.0     0    16  ??  DL   11Jan11   0:15.18 [g_down]
root       5  0.0  0.0     0    16  ??  DL   11Jan11   0:00.00
[mpt_recovery0]

The box is running Quagga with a single OSPF adjacency.  It has about 500
routes.
Both anycast instances of ns1 hit this problem, but neither instance of ns2,
which are
configured identically, saw the trouble.  The ns1 name servers are much
busier than ns2.

It appears that one instance of ns1 died almost a week ago, which went
unnoticed :-(  This
morning, the second instance died.  At that point, it was hard not to notice
:-)

Traffic on the mailing list suggests that 'sysctl
net.inet.flowtable.enable=0' is a work-around.
We'll pursue that path and hope for a bug fix in the not-too-distant future.

thanks,
mark
Comment 6 Bjoern A. Zeeb freebsd_committer freebsd_triage 2011-02-02 20:36:04 UTC
Hi,

things could be improved in HEAD and stable/8 (as of r217168 [1]).

Please test and report back.

/bz

[1] http://svn.freebsd.org/viewvc/base?view=revision&revision=217168

-- 
Bjoern A. Zeeb                                 You have to have visions!
         <ks> Going to jail sucks -- <bz> All my daemons like it!
   http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html
Comment 7 Bjoern A. Zeeb freebsd_committer freebsd_triage 2011-03-19 15:26:11 UTC
State Changed
From-To: open->feedback

Submitter(s) were asked for feedback on a more up-to-date 
stable/8 or HEAD back in February.
Comment 8 dfilter service freebsd_committer freebsd_triage 2011-03-19 15:50:50 UTC
Author: bz
Date: Sat Mar 19 15:50:34 2011
New Revision: 219775
URL: http://svn.freebsd.org/changeset/base/219775

Log:
  For now remove options FLOWTABLE from the remaining GENERIC kernel
  configurations and make it opt-in for those who want it.  LINT will
  still build it.
  
  While it may be a perfect win in some scenarios, it still troubles users
  (see PRs) in general cases.  In addition we are still allocating resources
  even if disabled by sysctl and still leak arp/nd6 entries in case of
  interface destruction.
  
  Discussed with:	qingli (2010-11-24, just never executed)
  Discussed with: juli (OCTEON1)
  PR:		kern/148018, kern/155604, kern/144917, kern/146792
  MFC after:	2 weeks

Modified:
  head/sys/amd64/conf/GENERIC
  head/sys/i386/conf/GENERIC
  head/sys/ia64/conf/GENERIC
  head/sys/mips/conf/OCTEON1

Modified: head/sys/amd64/conf/GENERIC
==============================================================================
--- head/sys/amd64/conf/GENERIC	Sat Mar 19 14:44:24 2011	(r219774)
+++ head/sys/amd64/conf/GENERIC	Sat Mar 19 15:50:34 2011	(r219775)
@@ -61,7 +61,6 @@ options 	KBD_INSTALL_CDEV	# install a CD
 options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
 options 	AUDIT			# Security event auditing
 options 	MAC			# TrustedBSD MAC Framework
-options 	FLOWTABLE		# per-cpu routing cache
 #options 	KDTRACE_FRAME		# Ensure frames are compiled in
 #options 	KDTRACE_HOOKS		# Kernel DTrace hooks
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel

Modified: head/sys/i386/conf/GENERIC
==============================================================================
--- head/sys/i386/conf/GENERIC	Sat Mar 19 14:44:24 2011	(r219774)
+++ head/sys/i386/conf/GENERIC	Sat Mar 19 15:50:34 2011	(r219775)
@@ -62,7 +62,6 @@ options 	KBD_INSTALL_CDEV	# install a CD
 options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
 options 	AUDIT			# Security event auditing
 options 	MAC			# TrustedBSD MAC Framework
-options 	FLOWTABLE		# per-cpu routing cache
 #options 	KDTRACE_HOOKS		# Kernel DTrace hooks
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel
 

Modified: head/sys/ia64/conf/GENERIC
==============================================================================
--- head/sys/ia64/conf/GENERIC	Sat Mar 19 14:44:24 2011	(r219774)
+++ head/sys/ia64/conf/GENERIC	Sat Mar 19 15:50:34 2011	(r219775)
@@ -31,8 +31,6 @@ options 	COMPAT_FREEBSD7	# Compatible wi
 options 	DDB		# Support DDB
 options 	DEADLKRES	# Enable the deadlock resolver
 options 	FFS		# Berkeley Fast Filesystem
-#options 	FLOWTABLE	# per-cpu routing cache (removed due to
-				# misaligned access -- see kern/148018)
 options 	GDB		# Support remote GDB
 options 	GEOM_LABEL	# Provides labelization
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel

Modified: head/sys/mips/conf/OCTEON1
==============================================================================
--- head/sys/mips/conf/OCTEON1	Sat Mar 19 14:44:24 2011	(r219774)
+++ head/sys/mips/conf/OCTEON1	Sat Mar 19 15:50:34 2011	(r219775)
@@ -73,7 +73,6 @@ options 	PRINTF_BUFR_SIZE=128	# Prevent 
 options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
 options 	AUDIT			# Security event auditing
 options 	MAC			# TrustedBSD MAC Framework
-options 	FLOWTABLE		# per-cpu routing cache
 #options 	KDTRACE_FRAME		# Ensure frames are compiled in
 #options 	KDTRACE_HOOKS		# Kernel DTrace hooks
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 9 dfilter service freebsd_committer freebsd_triage 2011-04-09 13:04:46 UTC
Author: bz
Date: Sat Apr  9 12:04:35 2011
New Revision: 220486
URL: http://svn.freebsd.org/changeset/base/220486

Log:
  MFC r219775:
  
    For now remove options FLOWTABLE from the remaining GENERIC kernel
    configurations and make it opt-in for those who want it.  LINT will
    still build it.
  
    While it may be a perfect win in some scenarios, it still troubles users
    (see PRs) in general cases.  In addition we are still allocating resources
    even if disabled by sysctl and still leak arp/nd6 entries in case of
    interface destruction.
  
    Discussed with:	qingli (2010-11-24, just never executed)
    Discussed with:	juli (OCTEON1)
  PR:			kern/148018, kern/155604, kern/144917, kern/146792

Modified:
  stable/8/sys/amd64/conf/GENERIC
  stable/8/sys/i386/conf/GENERIC
  stable/8/sys/ia64/conf/GENERIC
  stable/8/sys/mips/conf/OCTEON1
Directory Properties:
  stable/8/sys/   (props changed)
  stable/8/sys/amd64/include/xen/   (props changed)
  stable/8/sys/cddl/contrib/opensolaris/   (props changed)
  stable/8/sys/contrib/dev/acpica/   (props changed)
  stable/8/sys/contrib/pf/   (props changed)

Modified: stable/8/sys/amd64/conf/GENERIC
==============================================================================
--- stable/8/sys/amd64/conf/GENERIC	Sat Apr  9 11:03:04 2011	(r220485)
+++ stable/8/sys/amd64/conf/GENERIC	Sat Apr  9 12:04:35 2011	(r220486)
@@ -72,7 +72,6 @@ options 	KBD_INSTALL_CDEV	# install a CD
 options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
 options 	AUDIT			# Security event auditing
 options 	MAC			# TrustedBSD MAC Framework
-options 	FLOWTABLE		# per-cpu routing cache
 #options 	KDTRACE_FRAME		# Ensure frames are compiled in
 #options 	KDTRACE_HOOKS		# Kernel DTrace hooks
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel

Modified: stable/8/sys/i386/conf/GENERIC
==============================================================================
--- stable/8/sys/i386/conf/GENERIC	Sat Apr  9 11:03:04 2011	(r220485)
+++ stable/8/sys/i386/conf/GENERIC	Sat Apr  9 12:04:35 2011	(r220486)
@@ -73,7 +73,6 @@ options 	KBD_INSTALL_CDEV	# install a CD
 options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
 options 	AUDIT			# Security event auditing
 options 	MAC			# TrustedBSD MAC Framework
-options 	FLOWTABLE		# per-cpu routing cache
 #options 	KDTRACE_HOOKS		# Kernel DTrace hooks
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel
 

Modified: stable/8/sys/ia64/conf/GENERIC
==============================================================================
--- stable/8/sys/ia64/conf/GENERIC	Sat Apr  9 11:03:04 2011	(r220485)
+++ stable/8/sys/ia64/conf/GENERIC	Sat Apr  9 12:04:35 2011	(r220486)
@@ -31,7 +31,6 @@ options 	COMPAT_43TTY	# BSD 4.3 TTY comp
 options 	COMPAT_FREEBSD6	# Compatible with FreeBSD6
 options 	COMPAT_FREEBSD7	# Compatible with FreeBSD7
 options 	FFS		# Berkeley Fast Filesystem
-options 	FLOWTABLE	# per-cpu routing cache
 options 	GDB		# Support remote GDB
 options 	GEOM_LABEL	# Provides labelization
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel

Modified: stable/8/sys/mips/conf/OCTEON1
==============================================================================
--- stable/8/sys/mips/conf/OCTEON1	Sat Apr  9 11:03:04 2011	(r220485)
+++ stable/8/sys/mips/conf/OCTEON1	Sat Apr  9 12:04:35 2011	(r220486)
@@ -76,7 +76,6 @@ options 	PRINTF_BUFR_SIZE=128	# Prevent 
 options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
 options 	AUDIT			# Security event auditing
 options 	MAC			# TrustedBSD MAC Framework
-options 	FLOWTABLE		# per-cpu routing cache
 #options 	KDTRACE_FRAME		# Ensure frames are compiled in
 #options 	KDTRACE_HOOKS		# Kernel DTrace hooks
 options 	INCLUDE_CONFIG_FILE     # Include this file in kernel
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 10 K. Macy freebsd_committer freebsd_triage 2012-01-23 20:48:09 UTC
Have you tested this workload with 9.0 with the flowtable enabled? Is
the problem as severe?

Thanks
Comment 11 Vladimir V Romadin 2015-07-21 08:49:15 UTC
9.3 same issue
Comment 12 Eitan Adler freebsd_committer freebsd_triage 2018-05-28 19:45:38 UTC
batch change:

For bugs that match the following
-  Status Is In progress 
AND
- Untouched since 2018-01-01.
AND
- Affects Base System OR Documentation

DO:

Reset to open status.


Note:
I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.
Comment 13 Eugene Grosbein freebsd_committer freebsd_triage 2018-05-28 21:01:04 UTC
FLOWTABLE deorbited long time ago.