Created attachment 245023 [details] patch for crowdsec 1.5.4 Upstream changes are in the github release pages. This version of the port applies vendor dependencies as a separate file, and builds with the re2 C++ library to improve performance when parsing logs. Service management is more resilient with a timeout in case the application is not responsive to a termination signal. A new notification plugin for sentinel is included.
The quarterly branch is still on Go 1.19. Is this version of the Go toolchain still supported? The quarterly branch should ideally only get bug fixes, not new feature releases, but I'll leave it up to you. Please use ${TAR} instead of tar. I see that you have removed -mod=vendor from the build options. Is this option now supplied by some other means? FreeBSD ports must not fetch dependencies from the web during build and may not assume that internet access is possible. Please also check if you can switch to verbose builds (pass -v to go build). I really wonder why you can't just use the standard USES=go build target. All your port does is build a bunch of Go binaries. This would also remove the need for your custom vendor tarball and much of the other weird stuff in it. Stage QA says: ====> Running Q/A tests (stage-qa) Warning: you might not need LIB_DEPENDS on libabsl_base.so Warning: you might not need LIB_DEPENDS on libre2.so While I was able to find a binary that links to libre2.so, I didn't find one that links with libabsl_base.so. Is this LIB_DEPENDS correct?
(In reply to Robert Clausecker from comment #1) Hi, thanks! I'll update the patch tomorrow in light of your review > The quarterly branch is still on Go 1.19. Is this version of the Go toolchain still supported? This needs Go 1.20, so no quarterly, it's ok. I'll stop proposing that, we seldom have bugfix-only releases. > I see that you have removed -mod=vendor from the build options. Is this option now supplied by some other means? Yes, dependencies are now read from github before the build, there is a vendor.tgz file created at release time by a github action. I tested in poudriere. > I really wonder why you can't just use the standard USES=go build target. I'd like to do that, and have done so for the crowdsec bouncers already. Maybe for the next release, until this version crowdsec had 5 go.mod files - one for each notification plugin. Removing gmake should be easier but I haven't gotten around to do it. > While I was able to find a binary that links to libre2.so, I didn't find one that links with libabsl_base.so. Is this LIB_DEPENDS correct? It may depend on the version of re2, because abseil is a new dependency. I certainly have it in my binaries when building against the main branch. Not libabsl_base.so, but everything else, I'm not sure what's the right file to put there. ldd /usr/local/bin/crowdsec /usr/local/bin/crowdsec: libthr.so.3 => /lib/libthr.so.3 (0x183fe90cd000) libre2.so.11 => /usr/local/lib/libre2.so.11 (0x183fe8301000) libabsl_spinlock_wait.so.2301.0.0 => /usr/local/lib/libabsl_spinlock_wait.so.2301.0.0 (0x183fe913f000) libabsl_flags.so.2301.0.0 => /usr/local/lib/libabsl_flags.so.2301.0.0 (0x183fe9ee1000) libabsl_bad_optional_access.so.2301.0.0 => /usr/local/lib/libabsl_bad_optional_access.so.2301.0.0 (0x183fea5cb000) libabsl_strings_internal.so.2301.0.0 => /usr/local/lib/libabsl_strings_internal.so.2301.0.0 (0x183febe96000) libabsl_flags_program_name.so.2301.0.0 => /usr/local/lib/libabsl_flags_program_name.so.2301.0.0 (0x183feb0b1000) libabsl_graphcycles_internal.so.2301.0.0 => /usr/local/lib/libabsl_graphcycles_internal.so.2301.0.0 (0x183fecc11000) libabsl_civil_time.so.2301.0.0 => /usr/local/lib/libabsl_civil_time.so.2301.0.0 (0x183fee5e5000) libabsl_time_zone.so.2301.0.0 => /usr/local/lib/libabsl_time_zone.so.2301.0.0 (0x183fed953000) libabsl_symbolize.so.2301.0.0 => /usr/local/lib/libabsl_symbolize.so.2301.0.0 (0x183feea4f000) libabsl_debugging_internal.so.2301.0.0 => /usr/local/lib/libabsl_debugging_internal.so.2301.0.0 (0x183ff04a3000) libabsl_demangle_internal.so.2301.0.0 => /usr/local/lib/libabsl_demangle_internal.so.2301.0.0 (0x183feec44000) libabsl_malloc_internal.so.2301.0.0 => /usr/local/lib/libabsl_malloc_internal.so.2301.0.0 (0x183fef573000) libabsl_time.so.2301.0.0 => /usr/local/lib/libabsl_time.so.2301.0.0 (0x183ff10a2000) libabsl_flags_internal.so.2301.0.0 => /usr/local/lib/libabsl_flags_internal.so.2301.0.0 (0x183ff1245000) libabsl_flags_marshalling.so.2301.0.0 => /usr/local/lib/libabsl_flags_marshalling.so.2301.0.0 (0x183ff2ec2000) libabsl_log_severity.so.2301.0.0 => /usr/local/lib/libabsl_log_severity.so.2301.0.0 (0x183ff14a6000) libabsl_str_format_internal.so.2301.0.0 => /usr/local/lib/libabsl_str_format_internal.so.2301.0.0 (0x183ff2318000) libabsl_flags_reflection.so.2301.0.0 => /usr/local/lib/libabsl_flags_reflection.so.2301.0.0 (0x183ff4345000) libabsl_flags_private_handle_accessor.so.2301.0.0 => /usr/local/lib/libabsl_flags_private_handle_accessor.so.2301.0.0 (0x183ff3d93000) libabsl_flags_commandlineflag.so.2301.0.0 => /usr/local/lib/libabsl_flags_commandlineflag.so.2301.0.0 (0x183ff52cc000) libabsl_flags_commandlineflag_internal.so.2301.0.0 => /usr/local/lib/libabsl_flags_commandlineflag_internal.so.2301.0.0 (0x183ff6222000) libabsl_flags_config.so.2301.0.0 => /usr/local/lib/libabsl_flags_config.so.2301.0.0 (0x183ff6e81000) libabsl_cord.so.2301.0.0 => /usr/local/lib/libabsl_cord.so.2301.0.0 (0x183ff7c75000) libabsl_crc32c.so.2301.0.0 => /usr/local/lib/libabsl_crc32c.so.2301.0.0 (0x183ff833a000) libabsl_crc_internal.so.2301.0.0 => /usr/local/lib/libabsl_crc_internal.so.2301.0.0 (0x183ff97bb000) libabsl_crc_cpu_detect.so.2301.0.0 => /usr/local/lib/libabsl_crc_cpu_detect.so.2301.0.0 (0x183ff8410000) libabsl_cordz_functions.so.2301.0.0 => /usr/local/lib/libabsl_cordz_functions.so.2301.0.0 (0x183ff8f0a000) libabsl_cordz_handle.so.2301.0.0 => /usr/local/lib/libabsl_cordz_handle.so.2301.0.0 (0x183ffa033000) libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x183ffb0de000) libabsl_stacktrace.so.2301.0.0 => /usr/local/lib/libabsl_stacktrace.so.2301.0.0 (0x183ffb01b000) libabsl_cord_internal.so.2301.0.0 => /usr/local/lib/libabsl_cord_internal.so.2301.0.0 (0x183ffb91c000) libabsl_cordz_info.so.2301.0.0 => /usr/local/lib/libabsl_cordz_info.so.2301.0.0 (0x183ffce65000) libabsl_crc_cord_state.so.2301.0.0 => /usr/local/lib/libabsl_crc_cord_state.so.2301.0.0 (0x183ffb988000) libabsl_hash.so.2301.0.0 => /usr/local/lib/libabsl_hash.so.2301.0.0 (0x183ffbf3d000) libabsl_city.so.2301.0.0 => /usr/local/lib/libabsl_city.so.2301.0.0 (0x183ffdd80000) libabsl_bad_variant_access.so.2301.0.0 => /usr/local/lib/libabsl_bad_variant_access.so.2301.0.0 (0x183ffe2da000) libabsl_low_level_hash.so.2301.0.0 => /usr/local/lib/libabsl_low_level_hash.so.2301.0.0 (0x183ffed18000) libabsl_int128.so.2301.0.0 => /usr/local/lib/libabsl_int128.so.2301.0.0 (0x183ffedc8000) libabsl_hashtablez_sampler.so.2301.0.0 => /usr/local/lib/libabsl_hashtablez_sampler.so.2301.0.0 (0x183fffa7f000) libabsl_exponential_biased.so.2301.0.0 => /usr/local/lib/libabsl_exponential_biased.so.2301.0.0 (0x184000372000) libabsl_raw_logging_internal.so.2301.0.0 => /usr/local/lib/libabsl_raw_logging_internal.so.2301.0.0 (0x184001f8e000) libabsl_throw_delegate.so.2301.0.0 => /usr/local/lib/libabsl_throw_delegate.so.2301.0.0 (0x184000a2c000) librt.so.1 => /usr/lib/librt.so.1 (0x184000de8000) libabsl_base.so.2301.0.0 => /usr/local/lib/libabsl_base.so.2301.0.0 (0x1840018c0000) libabsl_raw_hash_set.so.2301.0.0 => /usr/local/lib/libabsl_raw_hash_set.so.2301.0.0 (0x18400278a000) libabsl_strings.so.2301.0.0 => /usr/local/lib/libabsl_strings.so.2301.0.0 (0x184003eff000) libabsl_synchronization.so.2301.0.0 => /usr/local/lib/libabsl_synchronization.so.2301.0.0 (0x184002c7b000) libc++.so.1 => /usr/lib/libc++.so.1 (0x184002d4e000) libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x1840038d2000) libm.so.5 => /lib/libm.so.5 (0x1840050a7000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x184004595000) libc.so.7 => /lib/libc.so.7 (0x184006dd7000) libelf.so.2 => /lib/libelf.so.2 (0x18400548a000) [vdso] (0x7ffffffff5d0)
Thank you. I just checked again and it appears go 1.20 is available on quarterly, though it is not the default. Will check and possibly modify the patch for MFH. As for the library dependency, I didn't see all these dependencies when I tested the port on arm64 FreeBSD 13.2. Could it be architecture dependent?
(In reply to Robert Clausecker from comment #3) Hi, I am testing a version that uses go:modules and I'm happy with the result, will upload it asap, thanks for the motivation! For the dependencies, a few months ago a new version of re2 came out that requires abseil too. Some architectures may be behind indeed. The last one without abseil was released on 2023-03-01
Created attachment 245062 [details] improved patch for 1.5.4 - This is now using go:modules - removed the direct dependency on abseil which is already provided by re2 - use ${TAR} instead of tar - pass -v to go build
LGTM. Note that -mod=vendor and friends are already the default when using the default build targets. It's only if you have custom build rules for Go projects that you need to add them. However, the port seems to build fine, so if you have no objections, I think I can just commit it as is.
(In reply to Robert Clausecker from comment #6) > Note that -mod=vendor and friends are already the default when using the default build targets. [...] However, the port seems to build fine, so if you have no objections, I think I can just commit it as is. Yes, this patch is _removing_ mod=vendor indeed.. thanks for merging!
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=6b1a624911903f4cd19774381169d5ac3c92240d commit 6b1a624911903f4cd19774381169d5ac3c92240d Author: Marco Mariani <marco@crowdsec.net> AuthorDate: 2023-09-19 13:42:58 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-09-22 04:56:47 +0000 security/crowdsec: update to 1.5.4 new upstream version use go:modules instead of gmake link with re2 for faster log parsing include sentinel plugin force termination if process is not responsive configurable local machine name Changelog: https://github.com/crowdsecurity/crowdsec/releases/tag/v1.5.4 PR: 273944 MFH: 2023Q3 security/crowdsec/Makefile | 59 +++++++++++++++++------------- security/crowdsec/distinfo | 8 +++-- security/crowdsec/files/crowdsec.in | 66 +++++++++++++++++++++++++++------- security/crowdsec/files/pkg-message.in | 8 +++++ security/crowdsec/files/upgrade-hub.in | 8 ++--- security/crowdsec/pkg-plist | 2 ++ 6 files changed, 107 insertions(+), 44 deletions(-)
A commit in branch 2023Q3 references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=7cbfa3c37561008e0b8486589cc2a4bbc398732c commit 7cbfa3c37561008e0b8486589cc2a4bbc398732c Author: Marco Mariani <marco@crowdsec.net> AuthorDate: 2023-09-19 13:42:58 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-09-22 05:32:33 +0000 security/crowdsec: update to 1.5.4 new upstream version use go:modules instead of gmake link with re2 for faster log parsing include sentinel plugin force termination if process is not responsive configurable local machine name Changelog: https://github.com/crowdsecurity/crowdsec/releases/tag/v1.5.4 PR: 273944 MFH: 2023Q3 (cherry picked from commit 6b1a624911903f4cd19774381169d5ac3c92240d) security/crowdsec/Makefile | 59 +++++++++++++++++------------- security/crowdsec/distinfo | 8 +++-- security/crowdsec/files/crowdsec.in | 66 +++++++++++++++++++++++++++------- security/crowdsec/files/pkg-message.in | 8 +++++ security/crowdsec/files/upgrade-hub.in | 8 ++--- security/crowdsec/pkg-plist | 2 ++ 6 files changed, 107 insertions(+), 44 deletions(-)
Thank you for your contribution.