Created attachment 183758 [details] Resolving patch When sending messages to a remote host syslogd omits the hostname field required by RFC 3164. This affects anyone sending logs from a FreeBSD host to a central logging server that expects RFC 3164-conformant messages (Logstash, fluentd) - it breaks the remote server's ability to parse the FreeBSD system's log messages. This issue is present on 10.3-RELEASE and 11.0-RELEASE. The attached patch corrects the behavior of syslogd when sending messages. Following the logic laid out in the existing code I preserved the previous hostname for forwarded messages, but made the resulting message conform with the RFC. This fix has been verified against Logstash & fluentd. I have not deliberately thrown any pathological input at it so there should be some scrutiny. NOTE: Related standards bug 200933 deals with receiving RFC 3164-conformant messages. There is a patch there which should be reviewed and either applied or adapted as appropriate.
CC'ing ed@, who has done quite a bit with syslogd over the past year.
Hi there, Thanks for the patch! The syslogd code has been refactored a lot lately to support the RFC 5424 message format. Looking at the code, the issue still applies. That said, I'm a bit hesitant to change anything for the RFC 3164 format support. People expect that it works in a certain way and changing that seems like a bad idea. Could you please give the RFC 5424 support that's present in 11-STABLE and HEAD a try? Just add "-O rfc5424" to syslogd_flags in rc.conf. Please let me know whether that addresses the issue for you sufficiently.
I have not forgotten about this PR :) We're currently in a QA cycle for 11.2 (which I don't believe includes the RFC5424 support?), but once that's done I'll spin up a test machine on -STABLE and test it against logstash. Our plan is to switch to RFC 5424 style messages anyway when we move to 11.3 or 12.0 & that eliminates the need for this fix on our end & I'm fine hotfixing our local syslogd until then. I do see the logic in keeping RFC 3164 support as-is (since most folks will probably move to the new format) or making it a 12.0 item (since it introduces incompatibility with old syslogd). If it stays as-is that probably merits a note/erratum in the manpage for the next person who stumbles on this when they set up centralized logging in a heterogeneous environment.
Any updates here?
Created attachment 233882 [details] Patch against 14-CURRENT Attached is a patch for latest version of the code. I did not modify the "Forwarded from" case.
Created attachment 247881 [details] patch with added rfc3164-strict format, against 15-CURRENT Hi, I had a look at RFC3164 and found a couple of interesting items: - Section 4.2 of RFC3164 states there are no set requirements for the contents of the syslog packet. So the existing format is technically valid. - The format described in RFC3164 (i.e. "<PRI>TIMESTAMP HOSTNAME MSG") is only RECOMMENDED to be used. It's not enforced. - The "Forwarded from HOSTNAME:" field is not mentioned anywhere in RFC3164. I found it originally came about in FreeBSD PR 7055, back in 1998. - Section 4.3.1 of RFC3164 states syslog relay servers that forward messages MUST do so without making any changes to the packet. Given this, I think the existing bsd/rfc3164 format should be kept as-is for backward compatibility, but a new optional format should be added that strictly conforms to the recommended format defined in RFC3164. I have attached a new patch based on Bryan Drewery's patch. It adds the third optional format to the -O flag, "strict" or "rfc3164-strict", which adds the hostname and excludes "Forwarded from" field in the network syslog message, and also updates the man page.
^Triage: clear unneeded flags. Nothing has yet been committed to be merged.
Hi, I took my last patch, added a bunch of tests and created a pull request on github with the changes. https://github.com/freebsd/freebsd-src/pull/1536