Bug 32420
| Summary: | syslogd accumulates "repeated" messages when piping to programs when the program may want to take immediate action | ||
|---|---|---|---|
| Product: | Base System | Reporter: | brett <brett> |
| Component: | bin | Assignee: | freebsd-bugs (Nobody) <bugs> |
| Status: | Closed FIXED | ||
| Severity: | Affects Only Me | ||
| Priority: | Normal | ||
| Version: | 4.4-RELEASE | ||
| Hardware: | Any | ||
| OS: | Any | ||
Here's a candidate patch. -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com Index: usr.sbin/syslogd/syslogd.8 =================================================================== RCS file: /home/ncvs/src/usr.sbin/syslogd/syslogd.8,v retrieving revision 1.43 diff -u -r1.43 syslogd.8 --- syslogd.8 8 Dec 2001 16:17:49 -0000 1.43 +++ syslogd.8 5 Jan 2002 07:29:17 -0000 @@ -156,6 +156,12 @@ Specify one specific IP address or hostname to bind to. If a hostname is specified, the IPv4 or IPv6 address which corresponds to it is used. +.It Fl c +Disable the compression of repeated instances of the same line +into a single line of the form +.Dq last message repeated N times +when the output is a pipe to another program. +If specified twice, disable this compression in all cases. .It Fl d Put .Nm Index: usr.sbin/syslogd/syslogd.c =================================================================== RCS file: /home/ncvs/src/usr.sbin/syslogd/syslogd.c,v retrieving revision 1.93 diff -u -r1.93 syslogd.c --- syslogd.c 27 Nov 2001 20:02:18 -0000 1.93 +++ syslogd.c 5 Jan 2002 07:29:18 -0000 @@ -274,6 +274,7 @@ #endif int send_to_all = 0; /* send message to all IPv4/IPv6 addresses */ int use_bootfile = 0; /* log entire bootfile for every kern msg */ +int no_compress = 0; /* don't compress messages (1=pipes, 2=all) */ char bootfile[MAXLINE+1]; /* booted kernel file */ @@ -334,7 +335,7 @@ socklen_t len; bindhostname = NULL; - while ((ch = getopt(argc, argv, "46Aa:b:df:kl:m:nop:P:suv")) != -1) + while ((ch = getopt(argc, argv, "46Aa:b:cdf:kl:m:nop:P:suv")) != -1) switch (ch) { case '4': family = PF_INET; @@ -354,6 +355,9 @@ case 'b': bindhostname = optarg; break; + case 'c': + no_compress++; + break; case 'd': /* debug */ Debug++; break; @@ -865,7 +869,8 @@ /* * suppress duplicate lines to this file */ - if ((flags & MARK) == 0 && msglen == f->f_prevlen && + if (no_compress - (f->f_type != F_PIPE) < 1 && + (flags & MARK) == 0 && msglen == f->f_prevlen && !strcmp(msg, f->f_prevline) && !strcasecmp(from, f->f_prevhost)) { (void)strlcpy(f->f_lasttime, timestamp, 16); Archie: It's more complex than I would have made it, but it will certainly work. --Brett At 12:31 AM 1/5/2002, Archie Cobbs wrote: >Here's a candidate patch. > >-Archie > >__________________________________________________________________________ >Archie Cobbs * Packet Design * http://www.packetdesign.com > >Index: usr.sbin/syslogd/syslogd.8 >=================================================================== >RCS file: /home/ncvs/src/usr.sbin/syslogd/syslogd.8,v >retrieving revision 1.43 >diff -u -r1.43 syslogd.8 >--- syslogd.8 8 Dec 2001 16:17:49 -0000 1.43 >+++ syslogd.8 5 Jan 2002 07:29:17 -0000 >@@ -156,6 +156,12 @@ > Specify one specific IP address or hostname to bind to. > If a hostname is specified, > the IPv4 or IPv6 address which corresponds to it is used. >+.It Fl c >+Disable the compression of repeated instances of the same line >+into a single line of the form >+.Dq last message repeated N times >+when the output is a pipe to another program. >+If specified twice, disable this compression in all cases. > .It Fl d > Put > .Nm >Index: usr.sbin/syslogd/syslogd.c >=================================================================== >RCS file: /home/ncvs/src/usr.sbin/syslogd/syslogd.c,v >retrieving revision 1.93 >diff -u -r1.93 syslogd.c >--- syslogd.c 27 Nov 2001 20:02:18 -0000 1.93 >+++ syslogd.c 5 Jan 2002 07:29:18 -0000 >@@ -274,6 +274,7 @@ > #endif > int send_to_all = 0; /* send message to all IPv4/IPv6 addresses */ > int use_bootfile = 0; /* log entire bootfile for every kern msg */ >+int no_compress = 0; /* don't compress messages (1=pipes, 2=all) */ > > char bootfile[MAXLINE+1]; /* booted kernel file */ > >@@ -334,7 +335,7 @@ > socklen_t len; > > bindhostname = NULL; >- while ((ch = getopt(argc, argv, "46Aa:b:df:kl:m:nop:P:suv")) != -1) >+ while ((ch = getopt(argc, argv, "46Aa:b:cdf:kl:m:nop:P:suv")) != -1) > switch (ch) { > case '4': > family = PF_INET; >@@ -354,6 +355,9 @@ > case 'b': > bindhostname = optarg; > break; >+ case 'c': >+ no_compress++; >+ break; > case 'd': /* debug */ > Debug++; > break; >@@ -865,7 +869,8 @@ > /* > * suppress duplicate lines to this file > */ >- if ((flags & MARK) == 0 && msglen == f->f_prevlen && >+ if (no_compress - (f->f_type != F_PIPE) < 1 && >+ (flags & MARK) == 0 && msglen == f->f_prevlen && > !strcmp(msg, f->f_prevline) && > !strcasecmp(from, f->f_prevhost)) { > (void)strlcpy(f->f_lasttime, timestamp, 16); State Changed From-To: open->closed Implemented via the '-c' flag in -current and -stable. On Fri, 04 Jan 2002 23:50:01 PST, Archie Cobbs wrote:
> Here's a candidate patch.
Hi Archie,
Sorry to come in late like this, but you moved pretty quickly on this
one. :-)
Your patch provides an all or nothing solution. What about systems that
have syslogd pipe recipients, some of which are programmed to deal with
aggregated messages and some of which are not?
Wouldn't it be better to extend the ssylog.conf language so that the
operator can choose _which_ pipe recipients should receive unaggregated
messages?
Ciao,
Sheldon.
Sheldon Hearn writes: > > Here's a candidate patch. > > Hi Archie, > > Sorry to come in late like this, but you moved pretty quickly on this > one. :-) > > Your patch provides an all or nothing solution. What about systems that > have syslogd pipe recipients, some of which are programmed to deal with > aggregated messages and some of which are not? > > Wouldn't it be better to extend the ssylog.conf language so that the > operator can choose _which_ pipe recipients should receive unaggregated > messages? Yes :-) __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com On Tue, 29 Jan 2002 10:50:02 PST, Archie Cobbs wrote:
> > Wouldn't it be better to extend the ssylog.conf language so that the
> > operator can choose _which_ pipe recipients should receive unaggregated
> > messages?
>
> Yes :-)
Okay, please bug me before MFC'ing this, in case I find time to work on
a syslog.conf language extension to replace the new option.
Ciao,
Sheldon.
On Wed, 30 Jan 2002 06:10:02 PST, Sheldon Hearn wrote:
> Okay, please bug me before MFC'ing this, in case I find time to work on
> a syslog.conf language extension to replace the new option.
For the sake of the audit trail, Archie informs me that this has already
been MFC'd. It's a real pretty the normal protocol of delaying MFC's
wasn't followed, because this isn't the best solution, but the best
solution would duplicate part of the new functionality, and warrant
deprecation of the new feature.
Ciao,
Sheldon.
|
FreeBSD's syslogd has a "repeat counting" feature, implemented by Eric Allman, which is designed to save file space and prevent flooding of the console with messages. Unfortunately, because it also works when messages are piped to a log monitoring program, it prevents a log monitor from seeing the stream of messages and taking appropriate action (e.g. blackholing an attacker). While it's a good idea to do repeat counting when the destination of log messages is a file, a user, or the console, it should be disabled when the output is piped. Fix: Here's the original code: /* * suppress duplicate lines to this file */ if ((flags & MARK) == 0 && msglen == f->f_prevlen && !strcmp(msg, f->f_prevline) && !strcasecmp(from, f->f_prevhost)) { (void)strncpy(f->f_lasttime, timestamp, 15); f->f_prevcount++; Add f->f_type != F_PIPE to the ANDed conditions in the initial if, preferably before the string length comparison. P.S. -- What are all those comments in the source about a "hollow laugh?"