Bug 272162 - sysutils/nomad: nomad is ignoring the etc directory
Summary: sysutils/nomad: nomad is ignoring the etc directory
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: John Hixson
URL: https://github.com/hashicorp/nomad
Keywords:
Depends on:
Blocks:
 
Reported: 2023-06-23 12:41 UTC by Seyed Pouria Mousavizadeh Tehrani
Modified: 2023-09-22 18:03 UTC (History)
3 users (show)

See Also:
jhixson: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Seyed Pouria Mousavizadeh Tehrani 2023-06-23 12:41:25 UTC
Hi,

it seems like Nomad does not read the `/usr/local/etc/nomad/` directory for its configuration or using it as its '-config' flag.

So by default nomad will not run/start with `service nomad start` command and we have to edit the rc.d/nomad file read its etc config files.

I see various options to correct this:
1. Make it to read files inside /usr/local/etc/nomad/ directory as its config files.
2. Add a ${nomad_config} variable to use it as -config flag if available.
3. Add a ${nomad_mode} variable to determine whether it's a server, client, or dev mode. furthermore completely ignore etc directory and make users to use the ${nomad_args} variable for hcl config files if they want.

IMHO, the first option will be compatible with the principle of least surprise (POLA).
Comment 1 John Hixson freebsd_committer freebsd_triage 2023-07-05 20:00:58 UTC
I'll get this updated on the next port update.
Comment 2 Michael Gmelin freebsd_committer freebsd_triage 2023-09-13 14:33:57 UTC
(In reply to John Hixson from comment #1)

Just as a datapoint:

We're currently using something like this:

  nomad_enable="YES"
  nomad_args="-config=/opt/cluster/etc/nomad/client.hcl -network-interface=nomad- pseudo"
  nomad_user="root"
  nomad_group="wheel"
  nomad_env="PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin"
  nomad_debug="YES"
  nomad_dir="/opt/cluster/nomad"

So in case nomad gains some sort of default config mechanism, a big fat warning in UPDATING/pkg-message would be nice, so our setup won't break (it's in our server configs, playbooks and image builders).

Note that nomad currently has some issues with unregistering jobs, which is supposed to be fixed in 1.56 (and reappeared recently in 1.6.1), see also https://github.com/hashicorp/nomad/issues/16616
Comment 3 Michael Gmelin freebsd_committer freebsd_triage 2023-09-14 17:26:52 UTC
(In reply to Michael Gmelin from comment #2)

Maybe adding `nomad_config` without a default would be a backwards-compatible way of introducing this, so that setups using -config <filename> in nomad_args won't break. It would also make sense, since one would usually either use client.hcl or server.hcl (if any examples are given, it would make sense to provide two for these specific differences in config).
Comment 4 Michael Gmelin freebsd_committer freebsd_triage 2023-09-16 09:39:28 UTC
(In reply to Seyed Pouria Mousavizadeh Tehrani from comment #0)

Repeating myself, but directed to you:

You can always set nomad_args like in:

  nomad_args="-config=/opt/cluster/etc/nomad/client.hcl -network-interface=nomad- pseudo"
Comment 5 commit-hook freebsd_committer freebsd_triage 2023-09-22 16:41:27 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=1e8c5858e52fcb3b1664e67740b1f1396ea6ac5d

commit 1e8c5858e52fcb3b1664e67740b1f1396ea6ac5d
Author:     John Hixson <jhixson@FreeBSD.org>
AuthorDate: 2023-09-22 16:36:45 +0000
Commit:     John Hixson <jhixson@FreeBSD.org>
CommitDate: 2023-09-22 16:40:25 +0000

    sysutils/nomad: Update to v1.6.2

    Add 2 new configuration knobs:
      nomad_mode (default to agent for backward compatibility)
      nomad_args (default to -data-dir=${nomad_dir} for backward compatibility)

    PR:     272162

 sysutils/nomad/Makefile       | 122 +++++++++++-----------
 sysutils/nomad/distinfo       | 234 +++++++++++++++++++++---------------------
 sysutils/nomad/files/nomad.in |   8 +-
 3 files changed, 182 insertions(+), 182 deletions(-)
Comment 6 John Hixson freebsd_committer freebsd_triage 2023-09-22 16:43:00 UTC
I've updated the port to v1.6.2. 

I've added 2 new knobs for configuration:

nomad_mode (defaults to agent)
nomad_args (defaults to -data-dir=${nomad_dir})

This keeps backward compatibility while also allowing you to tweak it how you want. I hope this is acceptable. If I missed something, please let me know. 

Thanks for the feedback!
Comment 7 Michael Gmelin freebsd_committer freebsd_triage 2023-09-22 18:03:31 UTC
(In reply to John Hixson from comment #6)

Thanks for putting thought into this. Unfortunately this will basically break our setup on update, as we use

    nomad_dir=/path/to/our/datadir
    nomad_args=paramstonomad

(see above for the real-life example).

So on update, we will lose the datadir parameter in our setup (we will need to set a big fat reminder for that). Maybe adding a pkg-message- or an UPDATING entry would make sense.

Did you see bug #273786? While updating to 1.5.6 is not necessary anymore, the conversion to the new go framework makes Makefile/distinfo much simpler and reduces the overhead when updating the port (assuming you don't have something clever I'm not aware of, as I don't deal with go ports that often).