Bug 239370

Summary: bug sysutils/rsyslog8 in branches/2019Q3
Product: Ports & Packages Reporter: Tomohiro Hosaka <bokutin>
Component: Individual Port(s)Assignee: Matthew Seaman <matthew>
Status: Closed Not Accepted    
Severity: Affects Only Me Flags: matthew: maintainer-feedback-
Priority: ---    
Version: Latest   
Hardware: Any   
OS: Any   

Description Tomohiro Hosaka 2019-07-21 23:24:10 UTC
sysutils/rsyslog8 in branches/2019Q3 does not work well with 12.0-RELEASE-p7.

/var/log/message:
(boot)
Jul 15 23:03:12 host1 kernel: da0: 40.000MB/s transfers
Jul 15 23:03:12 host1 kernel: da0: Attempt to query device size failed: NOT READY, Medium not present
Jul 15 23:03:12 host1 kernel: da0: quirks=0x3<NO_SYNC_CACHE,NO_6_BYTE>
Jul 15 23:03:12 host1 kernel: lo0: link state changed to UP
Jul 15 23:03:12 host1 kernel: em0: link state changed to UP
Jul 15 23:03:12 host1 kernel: em1: link state changed to UP
Jul 15 23:03:13 host1 1 2019-07-15T23:03:13.039949+09:00 host1 ntpd 1054 - - ntpd 4.2.8p12-a (1): Starting
Jul 15 23:03:13 host1 1 2019-07-15T23:03:13.089587+09:00 host1 ntpd 1055 - - leapsecond file ('/var/db/ntpd.leap-seconds.list'): good hash signature
Jul 15 23:03:13 host1 1 2019-07-15T23:03:13.089719+09:00 host1 ntpd 1055 - - leapsecond file ('/var/db/ntpd.leap-seconds.list'): loaded, expire=2019-12-28T00:00
(snip)

The reason is this.
https://github.com/rsyslog/rsyslog/issues/3316

Resolved by merging 8.1907.0 into 2019Q3.
https://www.freshports.org/sysutils/rsyslog8

I am not familiar with the quarterly branch policy, but it may be better to merge.
Comment 1 Matthew Seaman freebsd_committer freebsd_triage 2019-07-22 07:11:43 UTC
The code in 2019Q3 around timestamp formats is not functionally different from what is in HEAD.  The differences are that the patches were merged upstream and that upstream chose a slightly different mechanism for enabling or disabling RFC5424 format handling based on FreeBSD version.

So, I'm not convinced that your proposed fix will actually work.

Let me see if I can reproduce your problem and investigate a bit more.
Comment 2 Tomohiro Hosaka 2019-07-22 16:19:25 UTC
Thank you for your research.

I checked the problem again.

See below.

Is there anything else I can do?

# uname -a
FreeBSD host1 12.0-RELEASE-p7 FreeBSD 12.0-RELEASE-p7 GENERIC  amd64

# pkg install /var/cache/pkg/rsyslog-8.1905.0.txz

# pkg info rsyslog
rsyslog-8.1905.0
Name           : rsyslog
Version        : 8.1905.0
Installed on   : Tue Jul 23 00:46:59 2019 JST
Origin         : sysutils/rsyslog8
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local
Categories     : sysutils
Licenses       : GPLv3, APACHE20, LGPL3
Maintainer     : matthew@FreeBSD.org
WWW            : https://www.rsyslog.com/
Comment        : Syslogd supporting SQL, TCP, and TLS
Options        :
	DBI            : off
	DOCS           : on
	ELASTIC        : off
	GCRYPT         : on
	GNUTLS         : off
	GSSAPI         : off
	HTTP           : off
	JSONPARSE      : on
	KAFKA          : off
	MYSQL          : off
	NORMALIZE      : on
	OPENSSL        : off
	PGSQL          : on
	RABBITMQ       : off
	RELP           : on
	SNMP           : off
Shared Libs required:
	librelp.so.0
	liblognorm.so.5
	liblogging-rfc3195.so.0
	libgpg-error.so.0
	libfastjson.so.4
	libpq.so.5
	libestr.so.0
	libuuid.so.1
	libgcrypt.so.20
	libcurl.so.4
Annotations    :
	FreeBSD_version: 1200086
Flat size      : 1.44MiB
Description    :
RSYSLOG is the rocket-fast system for log processing.

It offers high-performance, great security features and a modular
design. While it started as a regular syslogd, rsyslog has evolved
into a kind of swiss army knife of logging, being able to accept
inputs from a wide variety of sources, transform them, and output to
the results to diverse destinations.

RSYSLOG can deliver over one million messages per second to local
destinations when limited processing is applied (based on v7, December
2013). Even with remote destinations and more elaborate processing the
performance is usually considered "stunning".

WWW: https://www.rsyslog.com/

# /usr/local/etc/rc.d/rsyslogd stop
Stopping rsyslogd.
Waiting for PIDS: 54418.

# grep rsyslog /etc/rc.conf
rsyslogd_enable="YES"

# cat /usr/local/etc/rsyslog.conf
module( load="imuxsock" )
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err   /var/log/messages

# logger message1

# tail /var/log/messages            
Jul 23 00:46:59 host1 pkg[75417] rsyslog-8.1905.0 installed
2019-07-23T01:01:49.355043+09:00 host1 1 2019-07-23T01:01:49.354984+09:00 host1 rsyslogd 83319 - - environment variable TZ is not set, auto correcting this to TZ=/etc/localtime [v8.1905.0 try https://www.rsyslog.com/e/2442 ]
2019-07-23T01:02:00.616531+09:00 host1 1 2019-07-23T01:02:00.616485+09:00 host1 bokutin 83409 - - message1

# pkg install /var/cache/pkg/rsyslog-8.1907.0.txz

# pkg info rsyslog
rsyslog-8.1907.0
Name           : rsyslog
Version        : 8.1907.0
Installed on   : Tue Jul 23 01:04:15 2019 JST
Origin         : sysutils/rsyslog8
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local
Categories     : sysutils
Licenses       : GPLv3, APACHE20, LGPL3
Maintainer     : matthew@FreeBSD.org
WWW            : https://www.rsyslog.com/
Comment        : Syslogd supporting SQL, TCP, and TLS
Options        :
	DBI            : off
	DOCS           : on
	ELASTIC        : off
	GCRYPT         : on
	GNUTLS         : off
	GSSAPI         : off
	HTTP           : off
	JSONPARSE      : on
	KAFKA          : off
	MYSQL          : off
	NORMALIZE      : on
	OPENSSL        : off
	PGSQL          : on
	RABBITMQ       : off
	RELP           : on
	SNMP           : off
Shared Libs required:
	librelp.so.0
	liblognorm.so.5
	liblogging-rfc3195.so.0
	libgpg-error.so.0
	libfastjson.so.4
	libpq.so.5
	libestr.so.0
	libuuid.so.1
	libgcrypt.so.20
	libcurl.so.4
Annotations    :
	FreeBSD_version: 1200086
Flat size      : 1.44MiB
Description    :
RSYSLOG is the rocket-fast system for log processing.

It offers high-performance, great security features and a modular
design. While it started as a regular syslogd, rsyslog has evolved
into a kind of swiss army knife of logging, being able to accept
inputs from a wide variety of sources, transform them, and output to
the results to diverse destinations.

RSYSLOG can deliver over one million messages per second to local
destinations when limited processing is applied (based on v7, December
2013). Even with remote destinations and more elaborate processing the
performance is usually considered "stunning".

WWW: https://www.rsyslog.com/

# /usr/local/etc/rc.d/rsyslogd stop
Stopping rsyslogd.
Waiting for PIDS: 83694.

# /usr/local/etc/rc.d/rsyslogd start
Starting rsyslogd.

# logger message1

# tail /var/log/messages
2019-07-23T01:04:15.270242+09:00 host1 1 2019-07-23T01:04:15.270185+09:00 host1 pkg 84161 - - rsyslog-8.1907.0 installed
2019-07-23T01:05:22.440819+09:00 host1 rsyslogd[85013] environment variable TZ is not set, auto correcting this to TZ=/etc/localtime [v8.1907.0 try https://www.rsyslog.com/e/2442 ]
2019-07-23T01:05:26.355931+09:00 host1 bokutin[85553] message1
Comment 3 Matthew Seaman freebsd_committer freebsd_triage 2019-07-22 20:46:47 UTC
Yes, you are seeing a real bug in 8.1905.0 that looks like it's been fixed in 8.1907.0.  However, this isn't to do with how enabling RFC5424 timestamp formats is controlled, but seems to be down to some of the other changes between those two releases.

This is a relatively innocuous bug, which neither has any security implications nor does it prevent rsyslogd working in general.  In this case, I'm afraid the policy is not to update the 2019Q3 branch. (Otherwise, what's the point of having a quarterly branch?  If we applied every available update, it would be indistinguishable from HEAD.)

If you can't continue without a fix, the workaround here would be to just use the 8.1907.0 package from the HEAD branch.  

Thank you very much indeed for the report.  I'm sorry I couldn't do much with it.  Will definitely be fixed in 2019Q4 at the latest.
Comment 4 Tomohiro Hosaka 2019-07-22 22:42:34 UTC
The syslog api in libc is the root for logging.

It will be confusing even for /usr/ports/security/denyhosts.

"tail -f /var/log/message | my_audit_program" was also confused.

I accept the quarterly branch policy and its judgments. And I think when it comes to learning.

Thank you very much for your reply.