Bug 280619 - security/openbao: New port: open source, community-driven fork of Vault
Summary: security/openbao: New port: open source, community-driven fork of Vault
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Vladimir Druzenko
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-08-04 21:30 UTC by jake
Modified: 2024-10-15 23:09 UTC (History)
4 users (show)

See Also:


Attachments
The shar file for the new port (8.43 KB, text/plain)
2024-08-04 21:30 UTC, jake
no flags Details
OpenBao Patch (7.85 KB, patch)
2024-08-31 02:58 UTC, jake
no flags Details | Diff
0002-added-SOURCEDATE-use-variables-for-user-groups.patch (11.77 KB, patch)
2024-09-01 16:42 UTC, jake
no flags Details | Diff
use GO_CMD var for build; fetch go as a dep properly with USES= (14.25 KB, patch)
2024-09-01 17:32 UTC, jake
no flags Details | Diff
Makefile v1 (1.13 KB, patch)
2024-09-01 22:48 UTC, Vladimir Druzenko
no flags Details | Diff
fixed GID missing : also using the new Makefile. (18.89 KB, patch)
2024-09-02 01:11 UTC, jake
no flags Details | Diff
0005-fixed-to-work-with-GO_MODULE.patch (25.19 KB, patch)
2024-09-02 05:05 UTC, jake
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description jake 2024-08-04 21:30:33 UTC
Created attachment 252503 [details]
The shar file for the new port

OpenBao exists to provide a software solution to manage, store, and distribute sensitive data including secrets, certificates, and keys. The OpenBao community intends to provide this software under an OSI-approved open-source license, led by a community run under open governance principles.
Comment 1 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-06 21:45:01 UTC
PORTVERSION=2.0.0 => DISTVERSION=2.0.0

Can you attach port in tar (gz/xz)?
Comment 2 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-06 21:52:32 UTC
2. For MAINTAINER use email you set in profile here: jake@metalrip.com.
Bugzilla can CC you if somebody create PR about your port.

3. By default GH_ACCOUNT=${PORTNAME} - you can remove GH_ACCOUNT=openbao.

4. Perhaps port don't need MASTER_SITES=https://raw.githubusercontent.com/openbao/openbao/${DISTVERSIONFULL}/ if it have USE_GITHUB=yes, but it can require DISTVERSIONPREFIX=v.
Comment 3 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-07 01:29:59 UTC
5. The port uses the user and group "openbao", but the patch does not change the UIDs and GIDs files.

6. Also need changes to security/Makefile.
Comment 4 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-07 03:12:46 UTC
7. This part:
do-build:
	@echo "Building ${GO_MODULE}..."
	@cd ${WRKSRC} && ${GO} build ${GO_BUILDFLAGS} -o bin/bao
	@ls -l ${WRKSRC}/bin/

# Custom do-install to handle the binary name change and directory
do-install:
	@echo "Installing ${PORTNAME} as bao"
	${MKDIR} ${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/bin/bao ${STAGEDIR}${PREFIX}/bin/bao

can be replaced by this line:
GO_TARGET=      :bao
Comment 5 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-15 11:40:47 UTC
PORTNAME=       openbao
DISTVERSIONPREFIX=      v
DISTVERSION=    2.0.0
CATEGORIES=     security
MASTER_SITES=   https://raw.githubusercontent.com/openbao/openbao/${DISTVERSIONFULL}/

DISTFILES+=     go.mod \
                api/go.mod \
                api/auth/approle/go.mod \
                api/auth/kubernetes/go.mod \
                api/auth/userpass/go.mod \
                sdk/go.mod

MAINTAINER=     github@metalrip.com
COMMENT=        Tool for securely accessing secrets
WWW=            https://openbao.org/

LICENSE=        MPL20
LICENSE_FILE=   ${WRKSRC}/LICENSE

USES=           cpe go:modules
CPE_VENDOR=     openbao
USE_GITHUB=     yes
USE_RC_SUBR=    ${PORTNAME}

GO_MODULE=      github.com/${PORTNAME}/${PORTNAME}
GO_TARGET=      :bao
GO_BUILDFLAGS=  -ldflags="-s \
                -X ${GO_MODULE}/version.GitCommit=700fe3f27ab1f0ec39ce20c36f6d9d97c9fe6ac3 \
                -X ${GO_MODULE}/version.BuildDate=${SOURCE_DATE_EPOCH:U${SOURCE_DATE_EPOCH_CMD:sh}} \
                -X ${GO_MODULE}/version.fullVersion=${DISTVERSION}"

SUB_FILES=      pkg-message
SUB_LIST=       GROUP=${GROUPS} USER=${USERS}
USERS=          ${PORTNAME}
GROUPS=         ${PORTNAME}

PLIST_FILES=    bin/bao

.include <bsd.port.pre.mk>

# Bring DISTINFO_FILE into scope so we can get the timestamp.
SOURCE_DATE_EPOCH_CMD=  /bin/date -ur \
                        $$(${GREP} -m1 TIMESTAMP ${DISTINFO_FILE} | ${SED} -e 's/[^0-9]//g') \
                        '+%Y-%m-%dT%H:%M:%SZ'

.include <bsd.port.post.mk>

Why we need so complicated SOURCE_DATE_EPOCH_CMD?
Comment 6 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-15 11:41:58 UTC
files/openbao.in:
: ${openbao_user:="%%USER%%"}
: ${openbao_group:="%%GROUP%%"}
Comment 7 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-20 21:59:41 UTC
ping
Comment 8 jake 2024-08-21 13:58:39 UTC
Thank you for the review Vladmir. I'll have time to get back to this and submit some updates this week.
Comment 9 jake 2024-08-31 02:20:10 UTC
Having trouble getting this working using GO_MODULE
Getting this on make makesum (go.mod is binary data for some reason):
go: errors parsing go.mod:
go.mod:1: unexpected input character '\x1f'
*** Error code 1

Will upload a working version soon.
Comment 10 jake 2024-08-31 02:58:22 UTC
Created attachment 253196 [details]
OpenBao Patch

working patch for openbao attached
Comment 11 Vladimir Druzenko freebsd_committer freebsd_triage 2024-08-31 07:58:21 UTC
(In reply to jake from comment #10)
I don't see declaration for SOURCE_DATE_EPOCH.

Also better to replace:
# openbao_user (user):	        Set user to run openbao.
#			        Default is "openbao".
# openbao_group (group):	Set group to run openbao.
#			        Default is "openbao".
: ${openbao_user:="openbao"}
: ${openbao_group:="openbao"}
with
# openbao_user (user):	        Set user to run openbao.
#			        Default is "%%USER%%".
# openbao_group (group):	Set group to run openbao.
#			        Default is "%%GROUP%%".
: ${openbao_user:="%%USER%%"}
: ${openbao_group:="%%GROUP%%"}
in files/openbao.in.
Comment 12 Yonas Yanfa 2024-08-31 09:09:07 UTC
(In reply to jake from comment #9)

I've seen this "go.mod:1: unexpected input character '\x1f'" error before as well. How did you fix it?
Comment 13 jake 2024-09-01 16:06:16 UTC
(In reply to Yonas Yanfa from comment #12)
I worked around it by manually with do-build and do-install.
For whatever reason it just wasn't playing well with the built in GO_MODULE functionality. The go.mod was being downloaded as binary data for some reason.
Comment 14 jake 2024-09-01 16:42:45 UTC
Created attachment 253246 [details]
0002-added-SOURCEDATE-use-variables-for-user-groups.patch

Uploaded new patch with all changes made.
Comment 15 jake 2024-09-01 17:32:43 UTC
Created attachment 253252 [details]
use GO_CMD var for build; fetch go as a dep properly with USES=
Comment 16 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-01 22:48:09 UTC
Created attachment 253261 [details]
Makefile v1

Build error:
> go: go: could not create module cache: mkdir /nonexistent: permission denied

Can you please use this Makefile as base for next patches?

Also you missed ":" in GIDs.
Comment 17 jake 2024-09-02 01:11:47 UTC
Created attachment 253262 [details]
fixed GID missing :   also using the new Makefile.
Comment 18 jake 2024-09-02 01:15:10 UTC
(In reply to Vladimir Druzenko from comment #16)
I am not getting that error. Can you help me understand it?
If you are building as the openbao user, that might be your issue since the home directory is /nonexistent. The default GO_PATH and GOCACHE both point to a path within the home directory by default. So if that's what did, I would expect that error.
Comment 19 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-02 01:29:06 UTC
(In reply to jake from comment #18)
I just build port with "my" Makefile in poudriere.
Comment 20 jake 2024-09-02 02:59:10 UTC
(In reply to Vladimir Druzenko from comment #19)
Ok I will test this on my poudriere tomorrow. Manually running the make commands works for me.
Is there anymore information you can provide me? Is poudriere building ports as non-root with a user who has /nonexistent as the home directory?

You are able to build other GO ports such as sysutils/hcloud?
Comment 21 jake 2024-09-02 03:52:16 UTC
Same issue here... looking into it
Comment 22 jake 2024-09-02 05:05:08 UTC
Created attachment 253267 [details]
0005-fixed-to-work-with-GO_MODULE.patch

With this patch, I've been able to remove the custom build steps and get it to work with GO_MODULE by specifying the MASTER_SITES and DISTFILES

I'm fairly sure all issues have been resolved now.
Comment 23 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-02 14:13:58 UTC
(In reply to jake from comment #22)
[/usr/ports/security/openbao]# make makesum
===>  License MPL20 accepted by the user
===>  License MPL20 accepted by the user
===>   openbao-2.0.0 depends on file: /usr/local/sbin/pkg - found
===>   openbao-2.0.0 depends on file: /usr/local/bin/go121 - found
===>   openbao-2.0.0 depends on package: ca_root_nss>0 - found
=> api/go.mod doesn't seem to exist in /usr/ports/distfiles/go/security_openbao/openbao-openbao-v2.0.0_GH0.
=> Attempting to fetch https://raw.githubusercontent.com/openbao/openbao/v2.0.0/api/go.mod
api/go.mod                                            1533  B   10 MBps    00s
=> api/auth/approle/go.mod doesn't seem to exist in /usr/ports/distfiles/go/security_openbao/openbao-openbao-v2.0.0_GH0.
=> Attempting to fetch https://raw.githubusercontent.com/openbao/openbao/v2.0.0/api/auth/approle/go.mod
api/auth/approle/go.mod                                157  B 1345 kBps    00s
=> api/auth/kubernetes/go.mod doesn't seem to exist in /usr/ports/distfiles/go/security_openbao/openbao-openbao-v2.0.0_GH0.
=> Attempting to fetch https://raw.githubusercontent.com/openbao/openbao/v2.0.0/api/auth/kubernetes/go.mod
api/auth/kubernetes/go.mod                             160  B 1434 kBps    00s
=> api/auth/userpass/go.mod doesn't seem to exist in /usr/ports/distfiles/go/security_openbao/openbao-openbao-v2.0.0_GH0.
=> Attempting to fetch https://raw.githubusercontent.com/openbao/openbao/v2.0.0/api/auth/userpass/go.mod
api/auth/userpass/go.mod                               158  B 1441 kBps    00s
=> sdk/go.mod doesn't seem to exist in /usr/ports/distfiles/go/security_openbao/openbao-openbao-v2.0.0_GH0.
=> Attempting to fetch https://raw.githubusercontent.com/openbao/openbao/v2.0.0/sdk/go.mod
sdk/go.mod                                            4671  B 1048 kBps    00s
===> Fetching all distfiles required by openbao-2.0.0 for building
===> Fetching github.com/openbao/openbao dependencies
go: errors parsing go.mod:
go.mod:1: unexpected input character '\x1f'
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/security/openbao
*** Error code 1

Stop.
make: stopped in /usr/ports/security/openbao
Comment 24 jake 2024-09-07 16:53:33 UTC
(In reply to Vladimir Druzenko from comment #23)
Strange, I am not able to replicate the same error as you on either of my FreeBSD machines. I also tried it in a fresh jail on 13.3-RELEASE and did not see the issue. Poudriere builds it fine as well (14.1-RELEASE).
Is it possible that you might have something cached that is messing up the makesum/build?
Comment 25 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-07 17:05:14 UTC
(In reply to jake from comment #24)
It's error during "make makesum", not during build.
Comment 26 jake 2024-09-07 17:28:08 UTC
(In reply to Vladimir Druzenko from comment #25)
I'm unable to replicate
Comment 27 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-07 17:35:55 UTC
(In reply to jake from comment #26)
Attach please your Makefile.
Comment 28 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-07 17:42:40 UTC
1st of all it require go 1.22:
USES=           go:1.22,modules
Comment 29 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-07 18:23:56 UTC
This error has disappeared…

Current Makefile:
PORTNAME=       openbao
DISTVERSIONPREFIX=      v
DISTVERSION=    2.0.0
CATEGORIES=     security
MASTER_SITES+=  https://raw.githubusercontent.com/${PORTNAME}/${PORTNAME}/${DISTVERSIONFULL}/
DISTFILES=      go.mod \
                api/go.mod \
                api/auth/approle/go.mod \
                api/auth/kubernetes/go.mod \
                api/auth/userpass/go.mod \
                sdk/go.mod

MAINTAINER=     jake@metalrip.com
COMMENT=        Tool for securely accessing secrets
WWW=            https://openbao.org/

LICENSE=        MPL20
LICENSE_FILE=   ${WRKSRC}/LICENSE

USES=           go:1.22,modules
USE_GITHUB=     yes
USE_RC_SUBR=    ${PORTNAME}

GO_MODULE=      github.com/${PORTNAME}/${PORTNAME}
GO_TARGET=      :${BIN_NAME}
GO_BUILDFLAGS=  -ldflags="-s \
                -X ${GO_MODULE}/version.GitCommit=${GITID} \
                -X ${GO_MODULE}/version.BuildDate=${SOURCE_DATE_EPOCH} \
                -X ${GO_MODULE}/version.fullVersion=${DISTVERSION}"

SUB_FILES=      pkg-message
SUB_LIST=       GROUP=${GROUPS} USER=${USERS}
USERS=          ${PORTNAME}
GROUPS=         ${PORTNAME}

PLIST_FILES=    bin/${BIN_NAME}

BIN_NAME=               bao
GITID=                  700fe3f27ab1f0ec39ce20c36f6d9d97c9fe6ac3
SOURCE_DATE_EPOCH=      ${TIMEEPOCHNOW:gmtime}
TIMEEPOCHNOW=           %Y-%m-%dT%H:%M:%SZ

.include <bsd.port.mk>
Comment 30 commit-hook freebsd_committer freebsd_triage 2024-09-19 01:03:02 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=a9cd810269d14464f96a966c1fb9ee8fb46e937c

commit a9cd810269d14464f96a966c1fb9ee8fb46e937c
Author:     jake <jake@metalrip.com>
AuthorDate: 2024-09-19 01:00:38 +0000
Commit:     Vladimir Druzenko <vvd@FreeBSD.org>
CommitDate: 2024-09-19 01:00:38 +0000

    security/openbao: New port: open source, community-driven fork of Vault

    OpenBao exists to provide a software solution to manage, store, and
    distribute sensitive data including secrets, certificates, and keys.
    The OpenBao community intends to provide this software under an
    OSI-approved open-source license, led by a community run under open
    governance principles.

    https://openbao.org
    https://github.com/openbao/openbao

    PR:     280619

 GIDs                                        |  2 +-
 UIDs                                        |  2 +-
 security/Makefile                           |  1 +
 security/openbao/Makefile (new)             | 43 ++++++++++++++
 security/openbao/distinfo (new)             | 15 +++++
 security/openbao/files/openbao.in (new)     | 89 +++++++++++++++++++++++++++++
 security/openbao/files/pkg-message.in (new) | 25 ++++++++
 security/openbao/pkg-descr (new)            |  4 ++
 8 files changed, 179 insertions(+), 2 deletions(-)
Comment 31 Vladimir Druzenko freebsd_committer freebsd_triage 2024-09-19 01:05:28 UTC
2.0.1 released - I updated port to new version and committed.

Thanks!
Comment 32 commit-hook freebsd_committer freebsd_triage 2024-09-19 01:44:08 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=d0c3c2c020b8a939227b229dcfef41d40ffb2962

commit d0c3c2c020b8a939227b229dcfef41d40ffb2962
Author:     Vladimir Druzenko <vvd@FreeBSD.org>
AuthorDate: 2024-09-19 01:41:29 +0000
Commit:     Vladimir Druzenko <vvd@FreeBSD.org>
CommitDate: 2024-09-19 01:41:29 +0000

    security/openbao: Update GITID

    GITID was from 2.0.0 - update it to 2.0.1.

    PR:     280619
    Fixes:  a9cd810269d1 (New port: open source, community-driven fork of Vault)

 security/openbao/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)