Bug 255427

Summary: net-mgmt/librenms: Install a newsyslog config file to rotate Librenms logs.
Product: Ports & Packages Reporter: James French <james>
Component: Individual Port(s)Assignee: Dan Langille <dvl>
Status: Closed FIXED    
Severity: Affects Only Me Flags: bugzilla: maintainer-feedback? (dvl)
Priority: ---    
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Add a newsyslog config
none
Poudriere build log
none
Add a newsyslog config - changes as discussed none

Description James French 2021-04-26 16:37:16 UTC
Created attachment 224451 [details]
Add a newsyslog config

The librenms port does not currently supply a newsyslog config file. The out of the box configuration for a distributed poller is quite chatty to the log file - one of my hosts had a multiple gigabyte log file.

Attached is a patch to include a newsyslog config for log rotation, configured to be installed as a sample to avoid overwriting user edits (eg, to disable compression if using ZFS). The reference file gets installed in the contrib folder in the librenms install. It may be better under /usr/local/share/librenms, which currently doesn't exist.

Poudriere log to follow.
Comment 1 James French 2021-04-26 16:42:30 UTC
Created attachment 224452 [details]
Poudriere build log
Comment 2 Dan Langille freebsd_committer 2021-04-26 17:46:35 UTC
If memory serves:

* This should go in as a .sample file, with everything commented out, otherwise, we would overwrite the users's local changes.

* It needs to be commented out because syslog pulls in everything, but in FreeBSD 13.x it will ignore non *.conf files.
Comment 3 Dan Langille freebsd_committer 2021-04-26 17:49:34 UTC
How do you feel about a daily rotation?

Are you using ZFS? Is compression on by default? If so, how about using B instead of J? That will avoid compression and means you can examine the log with grep or less instead of bzgrep or bzless.
Comment 4 James French 2021-04-26 19:16:58 UTC
Hi Dan,

> This should go in as a .sample file
It is, the sample is installed at:
/usr/local/www/librenms/contrib/newsyslog.conf

The file it installs to is:
/usr/local/etc/newsyslog.conf.d/librenms.conf

It's installed via @sample, so user changes will be preserved in newsyslog.conf.d. The file newsyslog.conf, could be renamed newsyslog.conf.sample to keep it clear if you prefer, although it isn't strictly necessary to keep ports/pkg happy.

>  It needs to be commented out because syslog pulls in everything, but in FreeBSD 13.x it will ignore non *.conf files.
This is exactly why the sample file is *not* installed to /usr/local/etc/newsyslog.conf.d in this patch. It's not the cleanest solution, but other ports that install newsyslog configs typically do the same thing - the sample file is shipped in a documentation or examples directory away from the live config file. In this case, I figured the best spot was probably the contrib folder.

This allows you to deploy a default config that actually does something rather than a commented out file on FreeBSD < 13.x.

See net-mgmt/net-snmpd for another example where this is done, but there are others.

> How do you feel about a daily rotation?
Works for me, the numbers I picked are roughly daily for my use case. You're right though in that time based probably makes more sense than size based if you have a busy poller.

On the other questions which are probably more specific to my setup than to the patch itself.

>Are you using ZFS?
Yes

>Is compression on by default?
Yes, and for my log folders are set for filesystem level gzip.

>If so, how about using B instead of J?
In my own configurations I either remove the J (multiple options), or replace it with a - (only option). B I keep for true binary data. However, compression on is probably still the sanest option for most people, hence J in the file.

> That will avoid compression and means you can examine the log with grep or less instead of bzgrep or bzless.
It also means files get moved rather than being rewritten (compressed) when they're rotated. If there are snapshots of the log file system, and logs are set to be compressed you end up with a compressed and uncompressed copy of every log on disk.
Comment 5 James French 2021-04-27 01:55:16 UTC
Created attachment 224460 [details]
Add a newsyslog config - changes as discussed

Changes as per discussion:
* Sample file renamed to newsyslog.conf.sample
* Rotation changed from size based to daily at midnight.
Comment 6 commit-hook freebsd_committer 2021-04-28 21:27:14 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=76bc66df08d29254a5096c1b5865b39bced9c3ee

commit 76bc66df08d29254a5096c1b5865b39bced9c3ee
Author:     Dan Langille <dvl@FreeBSD.org>
AuthorDate: 2021-04-28 20:55:11 +0000
Commit:     Dan Langille <dvl@FreeBSD.org>
CommitDate: 2021-04-28 21:26:24 +0000

    Install a newsyslog config file to rotate Librenms logs.

    PR:             255427
    Reported by:    James French <james@french.id.au>

 net-mgmt/librenms/Makefile                          | 6 ++++++
 net-mgmt/librenms/files/newsyslog.conf.sample (new) | 7 +++++++
 net-mgmt/librenms/files/pkg-message.in              | 7 +++++++
 3 files changed, 20 insertions(+)
Comment 7 Dan Langille freebsd_committer 2021-04-28 21:27:39 UTC
Thank you.
Comment 8 Dan Langille freebsd_committer 2021-07-02 18:13:55 UTC
Seems this work was done months ago.