Bug 217081 - sysutils/beats port creates logs in /usr/local/sbin/
Summary: sysutils/beats port creates logs in /usr/local/sbin/
Status: Closed DUPLICATE of bug 228519
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Palle Girgensohn
Depends on:
Reported: 2017-02-13 18:52 UTC by Dmitry Sivachenko
Modified: 2018-05-26 20:35 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (girgen)


Note You need to log in before you can comment on or make changes to this bug.
Description Dmitry Sivachenko freebsd_committer 2017-02-13 18:52:37 UTC

I installed filebeat port version 5.2.0.

After startup, it created /usr/local/sbin/logs/ directory (and writes log file there) and /usr/local/sbin/data directory (with registry file inside).

Probably startup script should provide sane defaults for logs and data directories to respect hier(7).

Also, there is no reason for this daemon to run as root, so I'd execute it under "nobody" user.
Comment 1 Palle Girgensohn freebsd_committer 2017-02-14 10:09:07 UTC

Yeah, thanks for noticing this about the log files. I had some config that hid this problems.

About running as non-root, filebeat in its standard setup needs read access to log files that are not accessible as non-root, so running as root should be an optional, IMO.

Comment 2 Dmitry Sivachenko freebsd_committer 2017-02-14 10:13:33 UTC
Okay, we can make running as root optional (may be use daemon's -u option?)

Consider this:
command_args="-rP ${pidfile} -u nobody /usr/local/sbin/${name} -path.home /usr/local -path.config /usr/local/etc -path.data /var/db/filebeat -path.logs /var/log/filebeat"

(and also create relevant directories from pkg-plist).

(we can override this hardcoded "nobody").

Also, Debian's package provides shell wrapper to facilitate one-shot executions of filebeat with proper parameters:

# cat /usr/bin/filebeat.sh 

# Script to run filebeat in foreground with the same path settings that
# the init script / systemd unit file would do.

/usr/share/filebeat/bin/filebeat \
  -path.home /usr/share/filebeat \
  -path.config /etc/filebeat \
  -path.data /var/lib/filebeat \
  -path.logs /var/log/filebeat \

It would be convenient to provide the same one :)
Comment 3 Palle Girgensohn freebsd_committer 2017-02-14 10:28:34 UTC
(In reply to Dmitry Sivachenko from comment #2)

Yes, that is very similar to what I've just configured in my test environment. I set it up with syslog instead of file logging, since that makes it a bit simpler. Dunno which is best, really.

An extra filebeat.sh file is not really customary in FreeBSD's hier(7). perhaps it would be better to patch the go binary to set up proper defaults? Or would that confuse users more than it helps, what do you think?
Comment 4 Dmitry Sivachenko freebsd_committer 2017-02-14 10:30:20 UTC
I vote for patching go binary in order to ship it with proper defaults suitable for FreeBSD.  And do not implement shell wrapper.
Comment 5 Dmitry Sivachenko freebsd_committer 2017-02-15 13:06:56 UTC
Also, something is wrong with terminal handling:

% ssh root@smetanka5b '/usr/local/etc/rc.d/filebeat onestart'
Starting filebeat.
<hung here>

It is solved by adding "> /dev/null" at the end of command_args=""
Comment 6 Palle Girgensohn freebsd_committer 2017-02-19 11:21:18 UTC
(In reply to Dmitry Sivachenko from comment #5)

This usually happens if only stderr is redirected and not stdout (or the opposite). Was stderr redirected?
Comment 7 Palle Girgensohn freebsd_committer 2017-02-19 11:30:31 UTC
There is a new beat binary as well, heartbeat. Its name clashes with another port sysutils/heartbeat, and while the elastic heartbeat is arguably better placed in net-mgmt/, it would still be confusing. I can either prefix it with "elastic-", but then all beats ports should have this prefix, or I'll just refactor then into one single port "beats". The single port could have options to opt out unwanted beats, but would by default build all of them.

There is also some common files, i.e. a migrate script and a binary "import_dasboards", that elastic themselves package verbatim copies for all for beats packages. With separate ports, this would require yet another package libbeat that all four other *beat packages would depend on. 

Does anyone following this thread have a preference between the proposed strategies? :)
Comment 8 Dmitry Sivachenko freebsd_committer 2017-02-19 12:12:41 UTC
With that said, a single port with OPTIONS for enabling the desired *beats sounds reasonable for me.  I would not rename beats to include elastic- prefix, i think CONFLICT would be enough, the probability for people to use both ports is rather low?
Comment 9 Palle Girgensohn freebsd_committer 2017-02-19 12:24:53 UTC
(In reply to Dmitry Sivachenko from comment #8)
With a single port, CONFLICT is not necessary. It does onöy conflict WRT the port name, there is no clash of installed files.
Comment 10 commit-hook freebsd_committer 2017-02-21 01:28:01 UTC
A commit references this bug:

Author: girgen
Date: Tue Feb 21 01:27:09 UTC 2017
New revision: 434496
URL: https://svnweb.freebsd.org/changeset/ports/434496

  Update to 5.2.1

  The three elasticsearch beats ports have merged into one single port
  systuils/beats that installs all of the above and some more. The reason is to
  make the port more maintainable and also that the new heartbeat beat would
  conflict with sysutils/heartbeat.

  Add heartbeat beat.

  Install utilities

  Fix problems with the default configurations [1]

  PR:	217081 [1]

Comment 11 Palle Girgensohn freebsd_committer 2017-02-21 01:33:08 UTC
(In reply to Dmitry Sivachenko from comment #5)
This should only happen if there is a bug in the configuration so the program has to produce output to the terminal. You can also get this problem if not all of stderr and stdout are equally redirected.

I made the port use syslog by default, so you should not se this once config is alright.
Comment 12 Palle Girgensohn freebsd_committer 2017-02-21 01:35:21 UTC
(In reply to Dmitry Sivachenko from comment #4)
In the end, I removed the patches to adhere to hier(7) since that would probly confuse more than it would help, since all elastic's docs expect it to be unpatched. The rc-scripts now use -path.xxx properly, this should suffice.
Comment 13 Dmitry Sivachenko freebsd_committer 2017-02-21 04:18:24 UTC

What about the ability to run as non-root user?
Comment 14 Palle Girgensohn freebsd_committer 2017-03-03 15:39:27 UTC
(In reply to Dmitry Sivachenko from comment #13)

With the default config, reading log files from /var/log requires filebeat to run as root. 

# find /var/log ! -perm +o+r ! -name '*.bz2' | wc -l 

That makes me a bit reluctant to add a knob for this. Please feel free to argue that I'm wrong though... :)
Comment 15 Dmitry Sivachenko freebsd_committer 2017-03-03 16:50:31 UTC
Well, nobody force you to collect system log files from /var/log.

I use filebeat to send nginx/haproxy logs to ElasticSearch cluster.
I need nothing from /var/log.

I think there are a lot of use cases when people would want to collect their application log files, not system logs from /var/log.
Comment 16 Dmitry Sivachenko freebsd_committer 2017-03-03 16:51:20 UTC
Also, you can make "root" a default user, just allow to easily custimise it via rc.conf variable.
Comment 17 Palle Girgensohn freebsd_committer 2017-03-03 17:21:38 UTC
(In reply to Dmitry Sivachenko from comment #15)

OK. cool. I was easily persuaded. I'll add the knob. :)
Comment 18 Mark Linimon freebsd_committer freebsd_triage 2017-07-19 11:35:31 UTC
Port was renamed.

Is this PR still valid?
Comment 19 Walter Schwarzenfeld freebsd_triage 2018-02-12 14:52:50 UTC
Comment 20 Palle Girgensohn freebsd_committer 2018-02-12 15:03:07 UTC
I seem to have forgotten about this. The logging is fixed, but not running as user != root.

So yes, still in progress, at least partly.
Comment 21 Palle Girgensohn freebsd_committer 2018-05-26 20:35:33 UTC
I'm closing this PR and open a separate one for the issue of *Beats daemons running as root.

*** This bug has been marked as a duplicate of bug 228519 ***