Bug 279003 - [NEW PORT] sysutils/kleene-daemon Kleene backend component for managing containers on FreeBSD.
Summary: [NEW PORT] sysutils/kleene-daemon Kleene backend component for managing conta...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Dave Cottlehuber
URL: https://reviews.freebsd.org/D49018
Keywords:
Depends on:
Blocks:
 
Reported: 2024-05-15 09:23 UTC by Lasse Andersen
Modified: 2025-02-18 18:19 UTC (History)
4 users (show)

See Also:


Attachments
Patch-file created according to the instructions in the Porter's Handbook (8.74 KB, patch)
2024-05-15 09:23 UTC, Lasse Andersen
no flags Details | Diff
Update (8.48 KB, patch)
2024-07-04 08:45 UTC, Lasse Andersen
no flags Details | Diff
Updated kleene-cli port (8.69 KB, patch)
2024-08-19 15:18 UTC, Lasse Andersen
no flags Details | Diff
Updated kleene-cli port (8.66 KB, patch)
2025-01-05 14:24 UTC, Lasse Andersen
no flags Details | Diff
c37936ff4e616804b7cefecd48d0e7f4a672238e from D49018 (9.14 KB, patch)
2025-02-16 15:09 UTC, Dave Cottlehuber
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Lasse Andersen 2024-05-15 09:23:53 UTC
Created attachment 250675 [details]
Patch-file created according to the instructions in the Porter's Handbook

Kleene is a new tool for managing containers/jails on FreeBSD, and this is the port for the backend componenent.

See https://kleene.dev/get-started/overview/ for an introduction to Kleene.
Comment 1 Lasse Andersen 2024-07-04 08:45:07 UTC
Created attachment 251860 [details]
Update

After some help/review on IRC, the port has been updated with the following:
- Removed '@' prefix on commands in do-install
- Removed hardcoding of the path to the dependency 'mix' binary
- Removed unnecessary usage of ${FIND} in do-install

Atm. the package file list is dynamically generated. The reason for this is because the erlang runtime contains internal packages in versioned folders such as 'lib/asn1-<version>', 'lib/crypto-<another version>' etc.
These folders basically change everytime the erlang-port is updated and
this would in turn break this port.

An alternative to a dynamically generated package list would be
to use the PLIST_SUB sub mechanism and then manually define
CRYPTO_VERSION variable etc. in the Makefile that can be changed manually. The drawback of this solution, however, is that a new port-update (updating CRYPTO_VERSION etc.) needs to be made every time this happens, which in my experience has been quite often. As I understand it, this would require a new review here every time, which would be time consuming for some very busy reviewers.

I am unsure which would make sense to the reviewers so let me know if the present solution should be changed.

Cheers!
Comment 2 Lasse Andersen 2024-08-19 15:18:16 UTC
Created attachment 252919 [details]
Updated kleene-cli port

Updated to adhere to hier(7).
Comment 3 Lasse Andersen 2024-08-20 13:59:06 UTC
(In reply to Lasse Andersen from comment #2)
Err, updated kleene-daemon port.
Comment 4 Robert Clausecker freebsd_committer freebsd_triage 2024-10-25 16:29:39 UTC
Sorry for the delay.

Patch needs some improvements.  COMMENT should perhaps better read:

COMMENT= Kleene backend daemon for managing containers on FreeBSD

This is to avoid repetition in the comment, which is usually printed like

kleene-daemon  Kleene backend daemon for managing containers on FreeBSD

and with your old COMMENT it would have been

kleene-daemon  Kleened, the Kleene backend daemon for managing containers on FreeBSD

repeating itself.

Please fix the indentation.  When editing ports Makefiles, make sure to set your editor to hard tabs, 8 stops per tab.  Indent with at least two tabs for make macros, one tab for make rules.  It's okay if you indent entire blocks of macros deeper if that is needed to give all values the same indentation.

Please use USES=elixir if possible at all.  I have added olgeni@ as a reviewer, who is more familiar with this stuff.

Do not install binaries with COPYTREE_SHARE, use COPYTREE_BIN instead (this strips the binaries if needed).  If you do this right, you won't need to explicitly strip any files.

libexec/kleened/releases does not seem to be an appropriate path for the erlang files.  How do other erlang ports handle this?  Try to copy what they do.

Do not auto-generate the plist in post-install unless you have a very good reason to do so.  "It's a lot of files" is not a good reason usually.  What is your reason here?
Comment 5 Lasse Andersen 2025-01-05 14:24:41 UTC
Created attachment 256439 [details]
Updated kleene-cli port

Hi Robert,

1) The port patch have been updated to accomodate your comments regarding:

- Phrase used in COMMENT
- Indentation

I hope that helps.

2) Regarding 'USES=elixir' I started using it, but ended up hacking the Makefile
a lot to fit my needs. The details are lost in my memory.
@olgeni: If you can provide any help/insights into this, please let me know.

3) I have tried to use COPYTREE_BIN but then I get strip-warnings under 
Q/A-testing in Poudriere. I have brute-forced some workarounds but nothing
have worked. If you have any hints, please let me know.

4) The reason for auto-generating the plist is because there are directory-names
in the list that changes, depending on the version of the Erlang port. If I
hardcode the names into the Makefile and the directory-names are updated before
the pkg-package site is, this port breaks.

5) I have tried to follow the way Elixir organises the files of a release.
This is not entirely like Erlang - also I use releases. I have previously been
advised to use /usr/local/libexec/kleened.
Comment 6 Robert Clausecker freebsd_committer freebsd_triage 2025-02-06 11:16:30 UTC
I don't think I can work on this patch any further as I don't know enough about elixir to judge if this is correct.  Someone else will have to assist.
Comment 7 Dave Cottlehuber freebsd_committer freebsd_triage 2025-02-15 12:02:42 UTC
Thanks Lasse, see URL for a few improvements:

- `portfmt -i Makefile`
- `portclippy --strict Makefile`
- finagle hexpm to work in poudriere env
- add other dependencies
- use `MAKE_ENV` similar to other port builds
- might be worth adding a Makefile target just for the builds to upstream
- simplify stripping a bit
- not fully sold autoplist just yet but lets roll with it


rc.d script TODO:

- does kleene need a daemon user, or will it run entirely as root?
- modifying env in rc.d feels wrong to me, lets see if we can avoid that
Comment 8 Dave Cottlehuber freebsd_committer freebsd_triage 2025-02-15 12:14:10 UTC
some comments for fuz@ along the way.

Elixir and Erlang ports (its the same VM underneath) build a self-contained
"release" which comprises the required dependencies from the erlang runtime
(lang/erlang) and additional modules compiled in from the port itself.

Each time the erlang port gets updated, a few of these /usr/local/lib/erlang/lib/
bits change. The kleene port will get rebuilt, but the names change, breaking the
plist. We also have slightly different versions of erlang runtime in quarterly &
latest, with patch-versions of these libraries.

e.g.:
/usr/local/lib/erlang27/lib/stdlib-6.1.2/ebin/maps.beam
/usr/local/lib/erlang27/lib/tools-4.1/ebin/xref_scanner.beam
/usr/local/lib/erlang27/lib/xmerl-2.1/ebin/xmerl.beam
...

So Lasse's idea to roll erts and lib/ in dynamically is a reasonable approach.

It's worth noting there are no other consumer ports of Mk/Uses/elixir.mk yet in the
tree, but perhaps we can tidy this up once we have kleene-daemon working.

In the past, I tried relying on using the erlang port as a dependency, it's 
technically feasible but goes against the grain of the erlang runtime, and
ends up breaking subtly for end users.

e.g. kleene is at 1.0.0
erlang gets updated from 26.0.0 -> 26.1.3 
this pulls in updated erlang libs

now the plist for kleene doesn't match
*but* as kleene itself hasn't version bumped, it doesn't get rebuilt
people update erlang and next reboot, kleene would fail.
Comment 9 Dave Cottlehuber freebsd_committer freebsd_triage 2025-02-16 15:09:16 UTC
Created attachment 257580 [details]
c37936ff4e616804b7cefecd48d0e7f4a672238e from D49018

Only remaining change would be to use a `version: "0.1.0",` consistently
in mix.exs and rely on ports framework with DISTVERSIONSUFFIX to handle
-rc.3 type names during release candidates. BEAM releases are probably
best left as simple X.Y.Z numeric names, nobody will be installing
different OTP releases adjacent from ports, so there is no conflict.
Comment 10 commit-hook freebsd_committer freebsd_triage 2025-02-18 18:17:51 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=92eeb2761b015c6da8a8cdbe9f28484b630b6af9

commit 92eeb2761b015c6da8a8cdbe9f28484b630b6af9
Author:     Dave Cottlehuber <dch@FreeBSD.org>
AuthorDate: 2025-02-18 18:17:42 +0000
Commit:     Dave Cottlehuber <dch@FreeBSD.org>
CommitDate: 2025-02-18 18:17:42 +0000

    sysutils/kleene-daemon: new port for container management

    PR:             279003
    Sponsored by:   SkunkWerks, GmbH
    Differential Revision:  https://reviews.freebsd.org/D49018

 sysutils/Makefile                                 |  1 +
 sysutils/kleene-daemon/Makefile (new)             | 88 +++++++++++++++++++++++
 sysutils/kleene-daemon/distinfo (new)             |  5 ++
 sysutils/kleene-daemon/files/kleened.in (new)     | 74 +++++++++++++++++++
 sysutils/kleene-daemon/files/pkg-message.in (new) | 33 +++++++++
 sysutils/kleene-daemon/pkg-descr (new)            |  3 +
 sysutils/kleene-daemon/pkg-plist (new)            | 27 +++++++
 7 files changed, 231 insertions(+)