Bug 76736 - syslogd(8) pipelines losing messages
Summary: syslogd(8) pipelines losing messages
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 13.0-STABLE
Hardware: Any Any
: Normal Affects Some People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-01-27 02:30 UTC by Felix Hernandez Campos
Modified: 2021-04-07 16:38 UTC (History)
1 user (show)

See Also:


Attachments
file.diff (1.09 KB, patch)
2005-01-27 02:30 UTC, Felix Hernandez Campos
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Hernandez Campos 2005-01-27 02:30:18 UTC
Syslogd supports logging subprocesses that communicate with the daemon using 
pipelines, e.g., in /etc/syslog.conf

local0.*        | exec grep -v DUPLEX >> /root/log

If the arrival rate of syslog messages is high enough (above 100 per second), syslogd may 
try to write to the pipeline too fast. The symptom of this problem is a line like

Jan 22 16:48:57 maryann syslogd:  exec grep -v DUPLEX >> /root/log: Resource 
temporarily unavailable

in /var/log/messages. The cause of the problem is the use of non-blocking writes by 
syslogd, which will fail when the pipe is full. In this case, the file descriptor of the pipe is 
closed, and the syslog message quietly discarded. See

http://lists.freebsd.org/pipermail/freebsd-hackers/2005-January/009921.html

for further details.

Fix: The solution is to check whether the pipeline is full, and if so give the
logging subprocess a chance to empty the pipeline. This has to be implemented
in a way that still protects syslogd against hung logging subprocesses. The
following patch worked well in my environment. I also tried it against CURRENT
(syslogd version 1.140) and the patch worked fine.
How-To-Repeat: 
Configure a syslog pipeline in one of the local facilities (see above) and write a program 
that submits numbered syslog messages to the same facility. You should see resource 
temporarily unavailable in /var/log/messages, and gaps in the sequence of logged 
messages. In my environment, syslog messages come from a large number of hosts, so it 
may be that network buffering aggravates (or even creates) this problem.
Comment 1 dwmalone freebsd_committer 2005-01-27 08:19:24 UTC
Responsible Changed
From-To: freebsd-bugs->dwmalone

I'll have a loog at this.
Comment 2 Eitan Adler freebsd_committer freebsd_triage 2012-07-10 04:41:53 UTC
Responsible Changed
From-To: dwmalone->freebsd-bugs

over to the pool (approved by bugmeister)
Comment 3 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:09 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped