Bug 226907 - databases/mongodb36: failed to start mongod due to "limits: NO: No such file or directory"
Summary: databases/mongodb36: failed to start mongod due to "limits: NO: No such file...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Kirill Ponomarev
URL:
Keywords: patch-ready, regression
Depends on:
Blocks: 227205
  Show dependency treegraph
 
Reported: 2018-03-25 02:04 UTC by Mateusz Piotrowski
Modified: 2019-10-18 17:42 UTC (History)
9 users (show)

See Also:
0mp: maintainer-feedback+


Attachments
Patch adding support for the new ${name}_limits mechanics (databases/mongodb36) (2.56 KB, patch)
2018-04-02 21:10 UTC, Mateusz Piotrowski
0mp: maintainer-approval+
Details | Diff
192119: Patch adding support for the new ${name}_limits mechanics (databases/mongodb36, revision: 1) (2.56 KB, patch)
2018-04-04 18:02 UTC, Mateusz Piotrowski
no flags Details | Diff
Patch adding support for the new ${name}_limits mechanics (databases/mongodb36, revision: 4) (2.50 KB, patch)
2018-04-11 00:07 UTC, Mateusz Piotrowski
0mp: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mateusz Piotrowski freebsd_committer 2018-03-25 02:04:20 UTC
When I try to start mongod with:

    sudo service mongod onestart

I get the following output:

> Starting mongod.
> limits: NO: No such file or directory
> /usr/local/etc/rc.d/mongod: WARNING: failed to start mongod

I've not modified any configuration files yet.

It looks like it is a problem with the `mongod_limits` knob, which has the following description: "Set to "NO" by default. Set it to yes to run `limits -e -U mongodb`".

I decided to add `set -x` to /usr/local/etc/rc.d/mongod. It turns out that `mongod_limits` is an unfortunate name because of the line 1078 in /etc/rc.subr:

    _doit="$_cd limits -C $_login_class $_limits $_doit"

`_limits` is a variable set in line 957:

    _limits=\$${name}_limits    _oomprotect=\$${name}_oomprotect

whereas `${name}_limits` is described as follows: 

    #	Name		Needed	Purpose
    #	${name}_limits	n	limits(1) to apply to ${command}.

I believe that we have a name collision here as with `set -x` we can see that following command is executed (resulting in "limits: NO: No such file or directory"):

    limits -C daemon NO su -m mongodb -c 'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"'

After renaming `mongod_limits` to `mongod_run_limits` in /usr/local/etc/rc.d/mongod the service runs just fine.
Comment 1 dev 2018-03-25 08:24:40 UTC
Hi Mateusz, thank you for the report. I can't reproduce that exact behaviour, I'm afraid.

mongod_limits="YES" (set -x shows mongod_prestart calling /usr/bin/limits)
-- cut here --
$ sudo service mongod onestart
/usr/local/etc/rc.d/mongod: DEBUG: pid file (/var/db/mongodb/mongod.lock): no pid in file.
/usr/local/etc/rc.d/mongod: DEBUG: checkyesno: mongod_enable is set to YES.
/usr/local/etc/rc.d/mongod: DEBUG: run_rc_command: start_precmd: mongod_prestart 
/usr/local/etc/rc.d/mongod: DEBUG: checkyesno: mongod_limits is set to YES.
Starting mongod.
/usr/local/etc/rc.d/mongod: DEBUG: run_rc_command: doit: limits -C daemon su -m mongodb -c 'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"'
$ 
-- and here --

mongod_limits="NO" (set -x shows mongod_prestart not calling /usr/bin/limits)
-- cut here --
$ sudo service mongod onestart
/usr/local/etc/rc.d/mongod: DEBUG: pid file (/var/db/mongodb/mongod.lock): no pid in file.
/usr/local/etc/rc.d/mongod: DEBUG: checkyesno: mongod_enable is set to YES.
/usr/local/etc/rc.d/mongod: DEBUG: run_rc_command: start_precmd: mongod_prestart 
/usr/local/etc/rc.d/mongod: DEBUG: checkyesno: mongod_limits is set to NO.
Starting mongod.
/usr/local/etc/rc.d/mongod: DEBUG: run_rc_command: doit: limits -C daemon su -m mongodb -c 'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"'
$
-- and here --

Running 11.1-p8. Not sure how to take this further.
Comment 2 dev 2018-03-25 08:35:11 UTC
Full set -x output

mongod_limits="NO":

-- cut here --
+ debug 'run_rc_command: start_precmd: mongod_prestart '
+ eval 'mongod_prestart '
+ mongod_prestart
+ [ ! -d /var/db/mongodb ]
+ checkyesno mongod_limits
+ eval '_value=$mongod_limits'
+ _value=NO
+ debug 'checkyesno: mongod_limits is set to NO.'
+ return 1
+ return 0
+ _return=0
+ [ 0 -ne 0 ]
+ check_required_after start
+ local _f _args
+ return 0
+ return 0
+ check_startmsgs
+ [ -n '' ]
+ return 0
+ echo 'Starting mongod.'
Starting mongod.
+ [ -n '' ]
+ _doit='/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null'
+ [ -n mongodb ]
+ _doit=$'su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ [ -n '' ]
+ [ -n '' ]
+ _doit=$'limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ _run_rc_doit $'limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ debug $'run_rc_command: doit: limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ eval $'limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ limits -C daemon su -m mongodb -c 'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"'
+ _return=0
-- and here --


mongod_limits="YES":

-- cut here --
+ debug 'run_rc_command: start_precmd: mongod_prestart '
+ eval 'mongod_prestart '
+ mongod_prestart
+ [ ! -d /var/db/mongodb ]
+ checkyesno mongod_limits
+ eval '_value=$mongod_limits'
+ _value=YES
+ debug 'checkyesno: mongod_limits is set to YES.'
+ return 0
+ /usr/bin/limits -e -U mongodb
+ eval ulimit -t 'unlimited;' ulimit -f 'unlimited;' ulimit -d 'unlimited;' ulimit -s 'unlimited;' ulimit -c 'unlimited;' ulimit -m 'unlimited;' ulimit -l '64;' ulimit -u 'unlimited;' ulimit -n 'unlimited;' ulimit -b 'unlimited;' ulimit -v 'unlimited;' ulimit -p 'unlimited;' ulimit -w 'unlimited;' ulimit -k 'unlimited;' ulimit -o 'unlimited;'
+ _return=0
+ [ 0 -ne 0 ]
+ check_required_after start
+ local _f _args
+ return 0
+ return 0
+ check_startmsgs
+ [ -n '' ]
+ return 0
+ echo 'Starting mongod.'
Starting mongod.
+ [ -n '' ]
+ _doit='/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null'
+ [ -n mongodb ]
+ _doit=$'su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ [ -n '' ]
+ [ -n '' ]
+ _doit=$'limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ _run_rc_doit $'limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ debug $'run_rc_command: doit: limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ eval $'limits -C daemon su -m mongodb -c \'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"\''
+ limits -C daemon su -m mongodb -c 'sh -c "/usr/local/bin/mongod --logpath /var/db/mongodb/mongod.log --logappend --config /usr/local/etc/mongodb.conf --dbpath /var/db/mongodb --fork >/dev/null 2>/dev/null"'
+ _return=0
-- and here --
Comment 3 Mateusz Piotrowski freebsd_committer 2018-03-25 10:31:55 UTC
(In reply to Vlad Galu from comment #1)

> Running 11.1-p8. Not sure how to take this further.

Actually, this is the key! :) This change was introduced 2 months ago in r328331[1][2], so effectively only 12.0-CURRENT is affected.

I'm running 12.0-CURRENT r330529 here, so I was able to spot the problem.

[1]: https://reviews.freebsd.org/D14015
[2]: https://svnweb.freebsd.org/base?view=revision&revision=r328331
Comment 4 dev 2018-03-25 14:22:55 UTC
Ah I see, that makes sense. I have no issues with renaming the variable in the port provided rc script, you have my blessing :-)

As many other ports are affected in a similar way, am I right to assume that _run_limits is proposed as the universal variable suffix and that UPDATING notes will be provided as each port is updated?
Comment 5 Mateusz Piotrowski freebsd_committer 2018-03-26 00:35:01 UTC
(In reply to Vlad Galu from comment #4)

There are much more ports which are affected by this change from what I see. I'll keep you updated on a solution to this problem.
Comment 6 Beeblebrox 2018-03-28 18:29:42 UTC
databases/mariadb100-server (10.0.34) is one such port affected on 12-CURRENT

> Starting mysql.
> limits: NO: No such file or directory

mysqld_safe does start though.
Comment 7 Mateusz Piotrowski freebsd_committer 2018-04-02 21:03:49 UTC
(In reply to Beeblebrox from comment #6)

Thanks for reporting this :)

I'll open an issue for mariadb100-server soon. For the time being, there's a general meta-issue.[1]

[1]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227205
Comment 8 Mateusz Piotrowski freebsd_committer 2018-04-02 21:10:06 UTC
Created attachment 192119 [details]
Patch adding support for the new ${name}_limits mechanics (databases/mongodb36)

Here's a patch, which should fix the issue for the systems with the new behavior. It also should preserve the old behavior for the old systems (10.4, 10-STABLE, 11.1).

I've built this port on my dev machine running 12.0-CURRENT r331748 and it seems to build just fine. The service script looks fine as well.

Could you please review this patch and test it on the systems with the legacy behavior (10.4, 10-STABLE, 11.1)?

Thanks!
Comment 9 dev 2018-04-02 22:24:22 UTC
Comment on attachment 192119 [details]
Patch adding support for the new ${name}_limits mechanics (databases/mongodb36)

Thumbs up.
Comment 10 Mateusz Piotrowski freebsd_committer 2018-04-04 18:02:04 UTC
Created attachment 192218 [details]
192119: Patch adding support for the new ${name}_limits mechanics (databases/mongodb36, revision: 1)

The previous patch would fail to trigger the new ${name}_limits semantics for __FreeBSD_version 1101514. The new patch fixes it (thanks to mmokhi@).
Comment 11 Mateusz Piotrowski freebsd_committer 2018-04-11 00:07:27 UTC
Created attachment 192404 [details]
Patch adding support for the new ${name}_limits mechanics (databases/mongodb36, revision: 4)

Due to an increasing number of problems caused by the new ${name}_limits behavior, the revision, which was backported from 12-CURRENT to 11-STABLE, has now been reverted.[1] As a result, the new behavior will not be present in any FreeBSD version < 12. 

This new patch adapts the port to those latest changes.

[1]: https://svnweb.freebsd.org/base?view=revision&revision=r332363
Comment 12 Mateusz Piotrowski freebsd_committer 2018-04-21 20:36:33 UTC
Vlad, it looks like you've submitted a part of an obsolete ${name}_limits patch in the changes upgrading databases/mongodb36 to 3.6.4.[1]: the Makefile parts are outdated and mongod changes are missing entirely. 

Could you please review the most recent patch I uploaded[2] so that it could be incorporated into the ports tree? At the moment databases/mongodb36 is still broken on 12.0-CURRENT.

Cheers :)

[1]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227636
[2]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226907#c11
Comment 13 dev 2018-04-22 05:38:02 UTC
(In reply to Mateusz Piotrowski from comment #12)

Ah, sorry Mateusz, I missed that. It looks good, please feel free to go ahead with the changes. Thanks!

Vlad
Comment 14 commit-hook freebsd_committer 2018-05-03 17:53:41 UTC
A commit references this bug:

Author: krion
Date: Thu May  3 17:52:41 UTC 2018
New revision: 468952
URL: https://svnweb.freebsd.org/changeset/ports/468952

Log:
  Add support for the new ${name}_limits mechanics on CURRENT and
  preserve the old behavior for the old systems.

  PR:		226907
  Submitted by:	mp0@FreeBSD.org
  Approved by:	maintainer

Changes:
  head/databases/mongodb36/Makefile
  head/databases/mongodb36/files/mongod.in
Comment 15 Bryan Drewery freebsd_committer 2019-10-18 17:42:29 UTC
This isn't right.
mongod_limits is considered to be the USER that will be set by rc.subr.

~ # grep _limits /etc/rc.subr
#       ${name}_limits  n       limits(1) to apply to ${command}.
            _limits=\$${name}_limits    _oomprotect=\$${name}_oomprotect \
                        _doit="$_cd limits -C $_login_class $_limits $_doit"

~ # service mongod onestart
Starting mongod.
limits: NO: No such file or directory
/usr/local/etc/rc.d/mongod: WARNING: failed to start mongod

Changing all mongod_limits to mongod_limits_enable fixes it.