syslog daemon expects the following format when receiving remote UDP messages: date tag: message from man syslog.conf: If a received message matches the specified facility and is of the speci- fied level (or a higher level), and the first word in the message after the date matches the program, the action specified in the action field will be taken. however RFC 3164 (and later ones) specify the following format: date hostname tag: message this means syslog can not properly filter messages based on tags, which breaks rfc-compliant software like nginx. an excerpt of what is being sent by nginx, with tag "nginxorg" from the hostname of "www.nginx.org": 13:28:23.422156 IP xx.yy.zz.nn.11312 > 192.168.1.1.514: SYSLOG local7.info, length: 220 E.......@..<...?..+!,0......<190>Jun 17 13:28:23 www.nginx.org nginxorg: 12.34.56.78 - - [17/Jun/2015:13:28:23 +0000] "GET / HTTP/1.1" 404 4075 "-" "Debian APT-HTTP/1.3 (0.9.7.7ubuntu4)" "10.10.20.121"
Created attachment 163083 [details] The patch should deal with the hostname in the message. The patch should deal with the hostname in the message. Haven't test it thoroughly.
This is to confirm the patch works fine.
A commit references this bug: Author: glebius Date: Wed Nov 8 16:45:53 UTC 2017 New revision: 325558 URL: https://svnweb.freebsd.org/changeset/base/325558 Log: When parsing UDP messages skip optional hostname as described by RFC 3164. PR: 200933 Submitted by: maxim Reported by: Konstantin Pavlov <thresh nginx.com> MFC after: 2 weeks Changes: head/usr.sbin/syslogd/syslogd.c
A commit references this bug: Author: glebius Date: Wed Nov 22 21:20:26 UTC 2017 New revision: 326103 URL: https://svnweb.freebsd.org/changeset/base/326103 Log: MFC r325558: When parsing UDP messages skip optional hostname as described by RFC 3164. PR: 200933 Submitted by: maxim Reported by: Konstantin Pavlov <thresh nginx.com> Changes: _U stable/11/ stable/11/usr.sbin/syslogd/syslogd.c
Re-opening as patch appeared to be not correct.
(In reply to Gleb Smirnoff from comment #5) > Re-opening as patch appeared to be not correct. What's the problem?
Syslogd treats local and remote messages differently. I'm now implementing proper parser for remote messages and I'd probably leave the local one as is.
Created attachment 188379 [details] a more complex patch
Konstantin, can you please test attached patch?
(In reply to Gleb Smirnoff from comment #9) So I have the following on nginx.conf: access_log syslog:server=192.168.128.11,facility=local7,tag=nginxorg,severity=info main; The on-the-wire message is as follows, taken from a tcpdump (the dates will be of course different in the following log files, as I've made multiple requests throughout the time): 13:47:22.559657 IP 192.168.128.55.34947 > 192.168.128.11.514: SYSLOG local7.info, length: 132 E...;6@.@.}....7...........1<190>Dec 4 13:47:22 deb9-test1 nginxorg: 127.0.0.1 - - [04/Dec/2017:13:47:22 +0000] "GET /foo HTTP/1.1" 200 3 "-" "curl/7.52.1" "-" /etc/syslog.conf contents: !nginxorg local7.* -/data/logs/nginx.org.log I've tested three scenarios: 1/ syslogd from FreeBSD 11.0-RELEASE-p1 to have a baseline. As expected, this one does not print anything to the file. 2/ syslogd from HEAD with a patch from comment #8 (attachment 188379 [details]), using syslogd_flags="-4 -n", prints the following to the file: Dec 4 13:49:05 192.168.128.55 nginxorg: 127.0.0.1 - - [04/Dec/2017:13:49:05 +0000] "GET /foo HTTP/1.1" 200 3 "-" "curl/7.52.1" "-" So as we can see, the hostname from the syslog message became an IP address. Otherwise, the line is fine. 2/ syslogd from HEAD with a patch from comment #8 (attachment 188379 [details]), using syslogd_flags="-4 -n -H", to have an actual hostname as received in the syslog message. Now this is where the line gets corrupted, as I see it's almost doubled in the output file - the string after the hostname gets printed twice: Dec 4 13:51:05 deb9-test1 nginxorg: 127.0.0.1 - - [04/Dec/2017:13:51:05 +0000] "GET /foo HTTP/1.1" 200 3 "-" "curl/7.52.1" "-" nginxorg: 127.0.0.1 - - [04/Dec/2017:13:51:05 +0000] "GET /foo HTTP/1.1" 200 3 "-" "curl/7.52.1" "-"
Created attachment 188530 [details] doubled logging fixed
The doubled logging is fixed with new patch. Regarding the IP address instead of hostname. Syslogd never resolved IP addresses, and it shouldn't do that.
(In reply to Gleb Smirnoff from comment #12) Well, I didnt mean to imply there was a name resolution attempt, just sharing my observation on the message change from the wire to the file. I've tested the patch from comment #11, and everything is now fine. Thank you!
A commit references this bug: Author: glebius Date: Tue Dec 5 19:54:55 UTC 2017 New revision: 326573 URL: https://svnweb.freebsd.org/changeset/base/326573 Log: When parsing remote messages, require them to have standard timestamp field, and support properly parse out the hostname as described by RFC3164, which wasn't done before. However, don't discard message if it doesn't have hostname, for compatibility. Enable logging of the message supplied hostname instead of real hostname with -H switch. PR: 200933 Reported by: Konstantin Pavlov <thresh nginx.com> MFC after: 2 months Changes: head/usr.sbin/syslogd/syslogd.8 head/usr.sbin/syslogd/syslogd.c
batch change of PRs untouched in 2018 marked "in progress" back to open.
Bug is in progress, since it is fixed in head, but not in a stable branch. The overall diff in syslogd is too big to be merged to stable (imho). Until 12.0-RELEASE, bug should stay in in progress status.
Fixed in 12.0-RELEASE.