Bug 282639

Summary: newsyslog may "double rotate" logs with duplicate rules
Product: Base System Reporter: Josef 'Jeff' Sipek <jeffpc>
Component: binAssignee: freebsd-bugs (Nobody) <bugs>
Status: New ---    
Severity: Affects Only Me CC: michaelo, pat
Priority: ---    
Version: 14.1-RELEASE   
Hardware: Any   
OS: Any   

Description Josef 'Jeff' Sipek 2024-11-09 01:42:35 UTC 
Recently, I set up a new jail to handle my email and so I wanted to increase the number of rotated maillogs to 60 days (from the default 7).  I tried to be clever by creating /usr/local/etc/newsyslog.conf.d/maillog.conf with the following line in it:

/var/log/maillog			640  60	   *	@T00  JC

and I keeping the maillog entry in /etc/newsyslog.conf alone.

I assumed that the .d snippet would *override* the line from the main config file.

What ended up happening was that *every* rotated maillog would look like:

# bzcat maillog.20241019T000000.bz2
Oct 19 00:00:00 smtp newsyslog[33052]: logfile turned over
Oct 19 00:00:00 smtp newsyslog[33052]: logfile turned over

I believe that what happened is that newsyslog rotated the log twice every night - first based on the /etc/newsyslog.conf rule and then based on the /usr/local/etc/newsyslog.conf.d/maillog.conf rule.  Since there were no log messages from postfix during the few milliseconds between rotations, the log only had the newsyslog markers when it got rotated for the second time and the actual log contents from the first log rotation of the night got lost.

(I like timestamp-based log rotation, and so my /etc/crontab has `newsyslog -t DEFAULT`.  It is possible that using the regular integer suffixes would have resulted in every-other-log still containing the log messages from the day.  I haven't tested it.)
Comment 1 Michael Osipov freebsd_committer freebsd_triage 2024-11-09 20:47:34 UTC 
What is the actual bug you are trying to report?
Comment 2 Josef 'Jeff' Sipek 2024-11-09 21:02:36 UTC 
Specifying the same log file name multiple times (across newsyslog.d/*.conf snippets) doesn't result in a single log rotation - it results in the same log getting rotated multiple times resulting in log data loss since the previous rotated log is overwritten by the second/third/etc. rotation.
Comment 3 Josef 'Jeff' Sipek 2024-11-09 21:06:40 UTC 
Btw, this multi-rotation happens completely silently and there is no indication that something like that is happening.

I expected one of these instead:

1. take the last entry for each log file name
2. warn if there are multiple instances of the same log file name
Comment 4 Michael Osipov freebsd_committer freebsd_triage 2024-11-09 21:38:30 UTC 
(In reply to Josef 'Jeff' Sipek from comment #3)

Well, neither is documented in the manpage, I'd say that the behavior you have is undefined.
Comment 5 Josef 'Jeff' Sipek 2024-11-09 22:05:13 UTC 
Agreed.  It is currently undefined.

IMO it'd be a good thing to allow the "overriding" behavior to make config management easier - rewriting existing files is always more painful than being able to drop a fragment into a .d directory.

So, I suppose this ticket is a request to define a behavior - I don't really care how it is defined, but (the current undefined behavior of) silent log file loss is a bad.