Bug 234968 - syslogd remote logging doesn't work (regression ?)
Summary: syslogd remote logging doesn't work (regression ?)
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 12.0-RELEASE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-bugs mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-01-15 10:45 UTC by Julien Cigar
Modified: 2019-01-15 12:46 UTC (History)
1 user (show)

See Also:


Attachments
logging jail syslogd (6.79 KB, text/plain)
2019-01-15 10:45 UTC, Julien Cigar
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Julien Cigar 2019-01-15 10:45:56 UTC
Created attachment 201159 [details]
logging jail syslogd

Hello,

I have a JAIL (logging, 10.209.1.31) which is used as a centralized log host. The JAIL and the HOST are running FreeBSD 12.0-RELEASE.

The JAIL has an unmodified syslog.conf with one extra file in /usr/local/etc/syslog.d:

root@logging:~ # ls -l /usr/local/etc/syslog.d
total 5
-rw-r--r--  1 root  wheel  312 Jan 15 10:45 saltstack.conf

root@logging:~ # cat /usr/local/etc/syslog.d/saltstack.conf 
+router1.lan,router2.lan,router.lan
local6.=info    /var/log/haproxy/http-access.log
local6.=err     /var/log/haproxy/http-error.log
local5.*        /var/log/haproxy/smtp.log
+dev.lan,antabif.lan,gitlab.lan,sandbox.lan,www1.prod.lan,www2.prod.lan
local6.=info    /var/log/httpd/access.log
local6.=err     /var/log/httpd/error.log

DNS and reverse DNS are working properly. If I'm tcpdumping on the HOST everything look OK, packets are properly sent from remote hosts:

root@HOST:~/ sudo tcpdump -n -i bge0 -p udp port 514 and host logging.lan
Password:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bge0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:31:37.273760 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 207
11:31:37.501015 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 208
11:31:38.074736 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local5.info, length: 151
11:31:38.501954 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 208
11:31:38.504479 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 205
11:31:38.586405 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 207
11:31:38.943227 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local5.info, length: 151
11:31:39.378678 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 177
11:31:39.500904 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 208
11:31:39.680232 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 177
11:31:39.694193 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 219
11:31:39.906661 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local5.info, length: 151
11:31:40.134680 IP 192.168.10.34.514 > 10.209.1.31.514: SYSLOG local6.error, length: 246
11:31:40.276486 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 207
11:31:40.494038 IP 192.168.10.34.514 > 10.209.1.31.514: SYSLOG local6.error, length: 246
11:31:40.501695 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 208
11:31:40.612300 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 241
11:31:40.745679 IP 10.209.1.252.43055 > 10.209.1.31.514: SYSLOG local6.info, length: 236
(...)

I've launched syslogd in debug mode in the JAIL and as you can see syslog messages arrive properly to the syslogd (see attachment). For some unknown reasons nothing gets logged to /var/log/haproxy/*.log

DNS and reverse DNS are working properly:

root@logging:~ # host 10.209.1.252
252.1.209.10.in-addr.arpa domain name pointer router1.lan.

root@logging:~ # host router1.lan
router1.lan has address 10.209.1.252

syslogd on the HOST is binded to the HOST ip, files exist in /var/log/haproxy in the JAIL, etc:

root@logging:~ # ls -l /var/log/haproxy/
total 2
-rw-------  1 root  wheel  0 Jan 15 10:48 http-access.log
-rw-------  1 root  wheel  0 Jan 15 10:48 http-error.log
-rw-------  1 root  wheel  0 Jan 15 10:48 smtp.log

Note that the same config worked in 10.4-RELEASE as expected and I'm out of ideas why it doesn't work on 12.0-RELEASE
Comment 1 Julien Cigar 2019-01-15 12:17:57 UTC
Ok I found the issue, it works if I'm removing the local domain part (router1 instead of router1.lan) although both "router1" and "router1.lan" return the same ip address (10.209.1.252):


WORKS:

root@logging:~ # cat /usr/local/etc/syslog.d/saltstack.conf 
+router1,router2,router
local6.=info    /var/log/haproxy/http-access.log
local6.=err     /var/log/haproxy/http-error.log
local5.*        /var/log/haproxy/smtp.log

DOESN'T WORK:

root@logging:~ # cat /usr/local/etc/syslog.d/saltstack.conf 
+router1.lan,router2.lan,router.lan
local6.=info    /var/log/haproxy/http-access.log
local6.=err     /var/log/haproxy/http-error.log
local5.*        /var/log/haproxy/smtp.log

root@logging:/var/log/haproxy # host router1
router1.lan has address 10.209.1.252
root@logging:/var/log/haproxy # host router1.lan
router1.lan has address 10.209.1.252
root@logging:/var/log/haproxy # host 10.209.1.252
252.1.209.10.in-addr.arpa domain name pointer router1.lan.

is it an expected behavior ? if yes feel free to close it, but I think it should be mentioned somewhere in the syslogd manpage.
Comment 2 Julien Cigar 2019-01-15 12:46:27 UTC
maybe related to base r332110