Bug 236840 - textproc/elasticsearch6: Elasticsearch 6 service doesn't start at boot time with clear_tmp_enable="YES" in /etc/rc.conf
Summary: textproc/elasticsearch6: Elasticsearch 6 service doesn't start at boot time w...
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: Juraj Lutter
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-27 22:44 UTC by Nemesis11
Modified: 2021-01-08 22:28 UTC (History)
4 users (show)

See Also:
bugzilla: maintainer-feedback? (elastic)


Attachments
Images showing the bug (98.80 KB, application/x-zip-compressed)
2019-03-27 22:44 UTC, Nemesis11
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nemesis11 2019-03-27 22:44:50 UTC
Created attachment 203199 [details]
Images showing the bug

I created a machine with FreeBSD 12.0 and the Elasticsearch 6 and, when we have the option "Clean the /tmp filesystem on system startup" (clear_tmp_enable="YES" in /etc/rc.conf) on, the Elasticsearch 6 service doesn’t start at boot time.
"Clean the /tmp filesystem on system startup" is a security option at install time, that can be enabled/disabled.
The problem happens because the Elasticsearch service starts soon after the Clean /tmp service. The java process tries to write on /tmp at the same time the clean /tmp service is deleting everything in /tmp.

Most of the time, there is no error on the console, at boot, and nothing is written in the logs. But one of the time, i saw the java process giving the error that it couldn't open file /tmp/elasticsearch.xxxxxxxx/gc.log in the console, at boot.

-------------------------------------------------------
Clearing /tmp.
OpenJDK 64-Bit Server VM Warning: Cannot open file /tmp/elasticsearch.wzc6rYAS/gc.log
Updating motd:
-------------------------------------------------------

Even when there is no error in the boot console, the Elasticsearch 6 service doesn't start at boot time with clear_tmp_enable="YES" in /etc/rc.conf.
After the boot sequence, i looked the status of the service, and the Elasticsearch 6 service was not running.

-------------------------------------------------------
felix@elasticforum2:~ % doas service elasticsearch status
elasticsearch is not running.
-------------------------------------------------------

After boot, i started the Elasticsearch 6 service manually, with "service elasticsearch start", it starts with no problem.
If, in /etc/rc.conf, if i comment the Clean /tmp service ( with #clear_tmp_enable="YES" ), the Elasticsearch 6 service starts at boot without problem.

-------------------------------------------------------
felix@elasticforum2:~ % doas service elasticsearch status
elasticsearch is running as pid 16306.
-------------------------------------------------------

I run the "rcorder /etc/rc.d/* /usr/local/etc/rc.d/*" command and it looks like the problem is that the Elasticsearch 6 service starts before the Clean /tmp service, but I’m not absolutely sure.

-------------------------------------------------------
/etc/rc.d/SERVERS
/usr/local/etc/rc.d/elasticsearch
/etc/rc.d/zfsd
/etc/rc.d/ntpdate
/etc/rc.d/rpcbind
/etc/rc.d/nisdomain
/etc/rc.d/ypserv
/etc/rc.d/ypxfrd
/etc/rc.d/ypupdated
/etc/rc.d/ypbind
/etc/rc.d/ypset
/etc/rc.d/yppasswdd
/etc/rc.d/ypldap
/etc/rc.d/wpa_supplicant
/etc/rc.d/virecover
/etc/rc.d/accounting
/etc/rc.d/nfsclient
/etc/rc.d/amd
/etc/rc.d/auditd
/etc/rc.d/auditdistd
/etc/rc.d/automountd
/etc/rc.d/automount
/etc/rc.d/autounmountd
/etc/rc.d/tmp
/etc/rc.d/cleartmp
-------------------------------------------------------

Thank you very much.
Comment 1 Juraj Lutter freebsd_committer 2020-10-27 07:35:24 UTC
Thanks for the report, I (the maintainer) will review and work on this. It slipped through my finger somehow last time I did the elastic components review.
Comment 2 Nemesis11 2020-11-03 16:08:18 UTC
Thank you for reviewing this bug.

Just an update. I had this machine with FreeBSD 12.1 with the latest updates and I upgraded it to version 12.2 and I can confirm that this bug is still valid.

If I have the clear_tmp_enable="YES" service enabled in /etc/rc.conf, the elasticcsearch 6 service will fail to start.

After the boot, if I start the elasticsearch 6 service manually, it will start with no problem.

If I comment clear_tmp_enable="YES" service, the elasticsearch 6 service will start automatically at boot, with no problem.

I can replicate the problem in a second machine, that has more services then this one (nginx, mariadb104-server, etc), and it only happens with the elasticsearch 6 service.

I'm using the latest trimestral elasticsearch 6 package. Version elasticsearch6-6.8.12.

Output before and after the FreeBSD 12.2 update, with the clear_tmp_enable="YES" service enabled, at boot:


felix@elasticforum2:~ % doas uname -a
FreeBSD elasticforum2 12.1-RELEASE-p10 FreeBSD 12.1-RELEASE-p10 GENERIC  amd64
felix@elasticforum2:~ % doas service elasticsearch status
elasticsearch is not running.


felix@elasticforum2:~ % doas uname -a
FreeBSD elasticforum2 12.2-RELEASE FreeBSD 12.2-RELEASE r366954 GENERIC  amd64
felix@elasticforum2:~ % doas service elasticsearch status
elasticsearch is not running.


Thank you and Best Regards.
Comment 3 Juraj Lutter freebsd_committer 2021-01-08 22:28:41 UTC
For the record, this bug applies to both elasticsearch6 and elasticsearch7.
It is, indeed, caused by wrong order in which the services are being started up.