Bug 176438 - net-mgmt/nrpe consuming cpu when handling new connections
Summary: net-mgmt/nrpe consuming cpu when handling new connections
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Bryan Drewery
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-02-26 11:30 UTC by Kajetan Staszkiewicz
Modified: 2014-09-27 19:38 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kajetan Staszkiewicz 2013-02-26 11:30:00 UTC
When nrpe2 built with SSL support accepts a new connection, it enters a loop waiting for SSL handshake. If no handshake occurs but the tcp connection is established, the loop consumes all available cpu time.

1004             /* keep attempting the request if needed */
1005                         while(((rc=SSL_accept(ssl))!=1) && (SSL_get_error(ssl,rc)==SSL_ERROR_WANT_READ));

I've observed this behavior on FreeBSD 9.0 and 9.1, I was told it happens also on 6.x.

This does not happen on nrpe2 running on Debian. Running strace/gdb on the process shows that in Linux it is waiting on read(), while on FreeBSD it iterates the loop over and over very fast.

Fix: 

A dirty hack is to add usleep(100000) in the aforementioned loop.
How-To-Repeat: `telnet localhost 5666` is enough to observe 100% cpu usage in top.
Comment 1 Edwin Groothuis freebsd_committer freebsd_triage 2013-02-27 00:11:54 UTC
Maintainer of net-mgmt/nrpe2,

Please note that PR ports/176438 has just been submitted.

If it contains a patch for an upgrade, an enhancement or a bug fix
you agree on, reply to this email stating that you approve the patch
and a committer will take care of it.

The full text of the PR can be found at:
    http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/176438

-- 
Edwin Groothuis via the GNATS Auto Assign Tool
edwin@FreeBSD.org
Comment 2 Edwin Groothuis freebsd_committer freebsd_triage 2013-02-27 00:11:55 UTC
State Changed
From-To: open->feedback

Awaiting maintainers feedback (via the GNATS Auto Assign Tool)
Comment 3 Olli Hauer freebsd_committer freebsd_triage 2013-10-06 10:14:18 UTC
Responsible Changed
From-To: freebsd-ports-bugs->ohauer

I'll take it. 

Kajetan thanks for the PR, I can verify the high CPU load. 

Please test the following patch: 
http://people.freebsd.org/~ohauer/diffs/nrpe_PR_176438.diff 

The patch resets the SSL connection after ~60s in case  
the SSL handshake is not finished. 

Additional I included a polling patch from OpenBSD's nrpe
Comment 4 Niclas Zeising freebsd_committer freebsd_triage 2013-10-07 23:10:10 UTC
State Changed
From-To: feedback->closed

Feedback timeout.
Comment 5 Florian Smeets freebsd_committer freebsd_triage 2013-11-13 19:27:04 UTC
State Changed
From-To: closed->open

This is still a problem! Please commit the suggested patch. 


Comment 6 Florian Smeets freebsd_committer freebsd_triage 2013-11-13 19:27:04 UTC
Responsible Changed
From-To: ohauer->zeising

This is still a problem! Please commit the suggested patch.
Comment 7 Florian Smeets freebsd_committer freebsd_triage 2014-09-07 21:36:10 UTC
Can we please move forward with this patch? I just had a host nearly go down because of this again.
Comment 8 Bryan Drewery freebsd_committer freebsd_triage 2014-09-16 02:55:41 UTC
I run into this very often too. The patch looks somewhat hacky, but does look like it would help. I am applying it in my tree.

Please address this.
Comment 9 commit-hook freebsd_committer freebsd_triage 2014-09-27 19:37:21 UTC
A commit references this bug:

Author: bdrewery
Date: Sat Sep 27 19:36:36 UTC 2014
New revision: 369416
URL: https://svnweb.freebsd.org/changeset/ports/369416

Log:
  - Apply patch to fix timed out SSL connections from spinning CPU

  Tested by:	bdrewery
  Submitted by:	kajetan.staszkiewicz@innogames.de
  Submitted by:	ohauer
  PR:		176438
  Approved by:	maintainer timeout

Changes:
  head/net-mgmt/nrpe/Makefile
  head/net-mgmt/nrpe/files/patch-src-nrpe.c
Comment 10 commit-hook freebsd_committer freebsd_triage 2014-09-27 19:38:23 UTC
A commit references this bug:

Author: bdrewery
Date: Sat Sep 27 19:37:56 UTC 2014
New revision: 369417
URL: https://svnweb.freebsd.org/changeset/ports/369417

Log:
  MFH: r369416

  - Apply patch to fix timed out SSL connections from spinning CPU

  Tested by:	bdrewery
  Submitted by:	kajetan.staszkiewicz@innogames.de
  Submitted by:	ohauer
  PR:		176438
  Approved by:	maintainer timeout

Changes:
_U  branches/2014Q3/
  branches/2014Q3/net-mgmt/nrpe/Makefile
  branches/2014Q3/net-mgmt/nrpe/files/patch-src-nrpe.c