Created attachment 225126 [details] git patch for sysutils/loki New port with Grafana Loki, the log aggregation system. I have prepared a separate ports submission for logcli and am also working on promtail to complete the picture. There is another port devel/loki and I am not sure if sysutils/loki is therefore a good choice. Let me know if that should be changed. Thanks Christopher
How is this work going? Do you need testers or other help?
It would need a version bump, but I am not part of the ports team and thus don't have commit rights. I'd like to have a ports maintainer to help out and let me know if there is interest in such a port.
Is there any way to help getting this forward?
(In reply to Christopher Beppler from comment #2) Hi, Thanks for the submission. If you still interested in maintaining this port, could you please update it (and bug #256031) to the latest version?
One more thing - the rc script has mentions of Grafana. If this port is related to Grafana, can it use "grafana" user instead of "nobody"? If there is no relation, it probably would be better to create a dedicated user/group for it.
It’s from the same company Grafana but the services don’t have to share the user in order to work properly. So having a dedicated user other than nobody would also be good.
And two answer comment #4: Sure thing. I am currently on vacation but once I am back on Monday, I‘ll bump the Version and add a dedicated user. Thanks!
Created attachment 234001 [details] git patch for Loki 2.5.0 Hi, I've bumped the version to 2.5.0. This creates a user and group "loki" instead of using "nobody" like before. Let me know if there is more to do to get this merged. Thanks Christopher
Created attachment 234003 [details] git patch for Loki 2.5.0 I noticed when running the port through poudriere that there were duplicates in distinfo (due to multiple uses in GH_TUPLE) that lead to build failures. This patch works now.
Created attachment 234004 [details] loki-GO_MODULE.diff The build still failed for me at the fetch stage: => go4org-go4-d4a079459e60_GH0.tar.gz doesn't seem to exist in /portdistfiles/. => Attempting to fetch https://codeload.github.com/go4org/go4/tar.gz/d4a079459e60?dummy=/go4org-go4-d4a079459e60_GH0.tar.gz fetch: 150608: No such file or directory fetch: https://codeload.github.com/go4org/go4/tar.gz/d4a079459e60?dummy=/go4org-go4-d4a079459e60_GH0.tar.gz: size unknown fetch: https://codeload.github.com/go4org/go4/tar.gz/d4a079459e60?dummy=/go4org-go4-d4a079459e60_GH0.tar.gz: size of remote file is not known go4org-go4-d4a079459e60_GH0.tar.gz 147 kB 353 kBps 00s => Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/go4org-go4-d4a079459e60_GH0.tar.gz fetch: 150608: No such file or directory fetch: http://distcache.FreeBSD.org/ports-distfiles/go4org-go4-d4a079459e60_GH0.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /portdistfiles/ and try again. *** Error code 1 The culprit is these two lines in GH_TUPLE: + go4org:go4:d4a079459e60:go4org_go4_1/vendor/go4.org/unsafe/assume-no-moving-gc \ + go4org:go4:d4a079459e60:go4org_go4/vendor/go4.org/intern \ They have the same account/project/commit-tag values and this is not currently supported by the ports framework. As there's no easy way around the above, I updated port to use GO_MODULE. One wrinkle with this approach is that upstream doesn't follow Go versioning scheme (their module name doesn't end with /v2 as required by version value "v2.5.0") so it's not listed on pkg.go.dev. So the port currently pulls go.mod from the Github (with MASTER_SITES/DISTFILES). It probably would be a good idea to file an issue with the upstream about this versioning incompatibility. Please review the attached diff, it builds OK for me in Poudriere.
Another thing - as you already noticed, there's another port devel/loki so adding sysutils/loki will create a PKGBASE collision. It could be avoided by adding e.g. PKGBASEPREFIX=grafana-, does this look ok?
Hm... did you take the revised diff (it has the same name but obsoleted the previous two - approx. 1 hour after my first submission)? It built fine on my poudriere installation after that. Multiple GH_TUPLE pairs as you mentioned led to double entries in distfiles which I fixed manually. The error message you are showing is similar to the one I faced before (it was etcd in my case, bug together with otel and go4 those were dups) Either way, the Go Modules variant also looks fine but I'd probably prefer the previous variant because then we're independent from upstream fixes on branch/tag naming. I'd be fine with grafana-loki. By the way, is sysutils the right place or would databases fit better? Will you do the renaming prior to merge or shall I do that?
I tested both diff, the one you uploaded before and the recent one. Both gave me the same error during the fetch. It should be reproducible if you force redownload by cleaning distfiles. This particular issue actually can be fixed by manual editing of GH_TUPLE and some extra hacking on port-extract, but I thought it would be rather annoying doing that after each update. I'll do more testing tonight and see if GH_TUPLE variant can be salvaged. sysutils is probably fine. I'll also add PKGNAMEPREFIX before committing.
OK, interesting. Maybe my poudriere was then spoiled with too many tries ;-). If it's reproducible on your end, the Go Modules variant is the clear winner. It was indeed already quite annoying with the GH_TUPLE today. I had to fix some of the dependencies by hand because they resulted in 404 during fetch. To be honest, I wasn't aware about the Go Modules way. It looks a lot easier for future updates as well. Thanks again. Let me know if I can help further.
I did some more testing and removing duplicate entries from the distinfo indeed fixes the build. So which variant would you prefer as the future maintainer - with GH_TUPLE or with GO_MODULES? :) I'm about to commit, everything else looks good.
GH_TUPLE then :-) Thanks.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=a96c988a29770411da79930d45edc21aef7efe40 commit a96c988a29770411da79930d45edc21aef7efe40 Author: Christopher Beppler <freebsd@funzi.org> AuthorDate: 2022-05-17 21:50:59 +0000 Commit: Dmitri Goutnik <dmgk@FreeBSD.org> CommitDate: 2022-05-17 21:52:29 +0000 sysutils/loki: Add new port Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream. WWW: https://github.com/grafana/loki PR: 256030 GIDs | 2 +- UIDs | 2 +- sysutils/Makefile | 1 + sysutils/loki/Makefile (new) | 348 +++++++++++++ sysutils/loki/distinfo (new) | 539 +++++++++++++++++++++ sysutils/loki/files/loki.in (new) | 68 +++ .../patch-cmd-loki-loki-local-config.yaml (new) | 17 + sysutils/loki/pkg-descr (new) | 6 + sysutils/loki/pkg-plist (new) | 10 + 9 files changed, 991 insertions(+), 2 deletions(-)
Committed, thanks!