Bug 205283
| Summary: | [NEW PORT] devel/gogs: Painless self-hosted Git service | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Yuri Victorovich <yuri> | ||||||||||||||||||||||
| Component: | Individual Port(s) | Assignee: | Yuri Victorovich <yuri> | ||||||||||||||||||||||
| Status: | Closed FIXED | ||||||||||||||||||||||||
| Severity: | Affects Only Me | CC: | bsd, dmgk, douglas, freebsd, freebsd, mfechner, nreilly, sascha.folie, sergey.v.brunov, swills, t.tom, yonas | ||||||||||||||||||||||
| Priority: | --- | Keywords: | feature, patch, patch-ready | ||||||||||||||||||||||
| Version: | Latest | ||||||||||||||||||||||||
| Hardware: | Any | ||||||||||||||||||||||||
| OS: | Any | ||||||||||||||||||||||||
| URL: | https://reviews.freebsd.org/D13141 | ||||||||||||||||||||||||
| Bug Depends on: | 205282, 207206 | ||||||||||||||||||||||||
| Bug Blocks: | |||||||||||||||||||||||||
| Attachments: |
|
||||||||||||||||||||||||
|
Description
Yuri Victorovich
2015-12-13 08:44:56 UTC
User needs to be added to /usr/ports/UIDs: gogs:*:1011:1011::0:0:gogs user:/var/db/gogs/home:/bin/sh Group needs to be added to /usr/ports/GIDs: gogs:*:1011: I only tested gogs with SQLite DB. It passes poudriere build. Typo correction: This is an open source version of GitHub. https://gogs.io/ Created attachment 164175 [details]
shar archive
Great, I already started with a port by my own, but had not really time to continue on it. Nice how you solved the dependencies! I just tested you port but the file /usr/ports/Mk/Uses/go.mk is missing. Would be great if I could help you a little bit and if it is only testing. Thanks! /usr/ports/Mk/Uses/go.mk comes from 205282 (see Depends on). You will need to apply patches from 205282 and from 205032, and to add user/group, see comment#1. Please let me know if something else is a problem. I#, not sure if it is a upstream problem, but the fetch fails: http://pkg.fechner.net/data/102amd64-default/2015-12-13_13h36m17s/logs/errors/gogs-0.151211.log yes, I get the same now, and got this before too. It looks like https://go.googlesource.com produces the different file for the same URL after a while. Temporary workaround: rm /usr/ports/distfiles/* && make makesum && make Looks like https://go.googlesource.com suffers from non-determinic output problem. BSD tar has this problem. It might be that stable tarballs aren't available anywhere. Need the side-hosting to place the stable versions. I added some parts of the port I already started to make webserver and database selectable, I hope it is ok for you, that I push your port to github: https://github.com/idefix6/freebsd-gogs Feel free to include everything you like. I cannot test the modifications as we have the upstream problem with googlesource.com. The initial port I started can be found here: https://github.com/idefix6/gogs-freebsdport Would be nice if we glue both together to get a nice port for gogs. Gogs doesn't need nginx, why do you depend on it? For what I am concerned, SQLite should be sufficient for any practical purposes. They only store some general accounting and log info there. I will make LOCALDB option, and make DB dependencies predicated on it and corresponding options. You can still test it if you ignore checksums: make checksums. Gogs is a web server itself. It doesn't need other web servers. They might bundle it with nginx or apache for linux Docker, so that nginx can work as a port forwarder or load balancer or whatever. But we don't need to do this. Add added a standard option to not rely on a webserver as default. But I would never put gogs to real internet before I do not have a real webserver on front. But this can each user decided by himself. SQLite has some drawbacks, so we should use the decision open for the user. Standard is sqlite. Compiling does not work, it seems that the tarball is damaged delivered from upstream: =======================<phase: extract >============================ ===> gogs-0.151211 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by gogs-0.151211 for building ===> Extracting for gogs-0.151211 => SHA256 Checksum OK for golang-net-d75b190.tgz. => SHA256 Checksum OK for golang-text-435f36e.tgz. => SHA256 Checksum OK for golang-crypto-7b85b09.tgz. tar: Error opening archive: Unrecognized archive format *** Error code 1 You didn't read my previous comment: rm -rf /usr/ports/distfiles/* && make makesum && make -- this is the workaround Or just set NOCHECKSUM=yes for now I agree that it is good to add the real web server in front of gogs. But this isn't the job of the port. This is the job of an admin to set this up. User will choose the database for sure. This is already working. As I build all my ports using poudriere I cannot use such work arounds. The command to test the new port is: poudriere testport -j 102amd64 -s -vv -i -o devel/gogs But I do not have a problem with a checksum but I have a problem that the port does not accept the tar ball with the information: ===> Fetching all distfiles required by gogs-0.151211 for building ===> Extracting for gogs-0.151211 => SHA256 Checksum OK for golang-net-d75b190.tgz. => SHA256 Checksum OK for golang-text-435f36e.tgz. => SHA256 Checksum OK for golang-crypto-7b85b09.tgz. tar: Error opening archive: Unrecognized archive format *** Error code 1 I tested it now without the use of poudriere and got exactly the same problem: rm -rf /usr/ports/distfiles/* && make makesum && make ===> License MIT accepted by the user ===> Found saved configuration for gogs-0.151211 ===> gogs-0.151211 depends on file: /usr/local/sbin/pkg - found => golang-net-d75b190.tgz doesn't seem to exist in /usr/ports/distfiles/. => Attempting to fetch https://go.googlesource.com/net/+archive/d75b190.tar.gz?dummy=/golang-net-d75b190.tgz fetch: https://go.googlesource.com/net/+archive/d75b190.tar.gz?dummy=/golang-net-d75b190.tgz: size of remote file is not known golang-net-d75b190.tgz 700 kB 2660 kBps 00m00s => golang-text-435f36e.tgz doesn't seem to exist in /usr/ports/distfiles/. => Attempting to fetch https://go.googlesource.com/text/+archive/435f36e.tar.gz?dummy=/golang-text-435f36e.tgz fetch: https://go.googlesource.com/text/+archive/435f36e.tar.gz?dummy=/golang-text-435f36e.tgz: size of remote file is not known golang-text-435f36e.tgz 4660 kB 2831 kBps 00m02s => golang-crypto-7b85b09.tgz doesn't seem to exist in /usr/ports/distfiles/. => Attempting to fetch https://go.googlesource.com/crypto/+archive/7b85b09.tar.gz?dummy=/golang-crypto-7b85b09.tgz fetch: https://go.googlesource.com/crypto/+archive/7b85b09.tar.gz?dummy=/golang-crypto-7b85b09.tgz: size of remote file is not known golang-crypto-7b85b09.tgz 876 kB 3183 kBps 00m00s ===> Fetching all distfiles required by gogs-0.151211 for building ===> License MIT accepted by the user ===> Found saved configuration for gogs-0.151211 ===> gogs-0.151211 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by gogs-0.151211 for building ===> Extracting for gogs-0.151211 => SHA256 Checksum OK for golang-net-d75b190.tgz. => SHA256 Checksum OK for golang-text-435f36e.tgz. => SHA256 Checksum OK for golang-crypto-7b85b09.tgz. tar: Error opening archive: Unrecognized archive format *** Error code 1 Created attachment 164229 [details] shar archive Hi idefix@fechner.net , Sorry about your troubles with these distfiles. The host produces changing tarballs. I replaced the source with GitHub mirror. You need both this patch here and the updated go.mk Dear Yuri, I'm not sure if I destroyed my complete build env now. To make sure everything is fine, I started with a clean port. Applied the fresh patch for Mk: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205282 and https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205032 Killed the gogs port and downloaded the new version and extracted it wish "sh attachment..." Removed everything from distfiles. Started with a fresh build: poudriere testport -j 102amd64 -s -vv -i -o devel/gogs But it break again with the same error message. For complete logfile, please see here: http://pkg.fechner.net/data/102amd64-default/2015-12-14_14h13m54s/logs/errors/gogs-0.151211.log It is not really clear what archive is damaged here. Is there a way to get more details while the port is build to see what archive he tries to extract? It looks like your version of the gogs port isn't from the shar because Extracting... phase begins like this: ===> Extracting for gogs-0.151211 => SHA256 Checksum OK for gogits-gogs-v0.151211-4d31eb2_GH0.tar.gz. => SHA256 Checksum OK for codegangsta-cli-0302d39_GH0.tar.gz. ... and goes on to check all 45 distfiles. (look at the distinfo in the attached shar) Created attachment 164251 [details]
shar archive
Tested the package installation on the clean machine - works like a charm.
Created attachment 164256 [details]
shar archive
Removed DB options because all DB support is built-in and doesn't require any external libraries. The desired DB server should be installed by an admin if required.
The checksums of the files I use are: SHA256 (Makefile) = 7620403fadc62c60e476dfc21e9efa48116b652d87df75607d7c9944e9a8351a SHA256 (distinfo) = ddb739f18c00d97d8df6fb8fec33c7f78e664a6aaf237e63e08abab29ebd396e SHA256 (files) = a43055f8bd67768dd864754d821ba89c8379418f1c93347514fad2169d9ee378 SHA256 (pkg-descr) = eaaa085bd15f00de2660cf8fe4e00fd6984c763f51b0ba89e025e4d40ee4ad92 SHA256 (pkg-plist) = 3e1c25d2c06fed863d8292d052ba375afdfe85a5eff0ea1001229a65d9ba0780 I'm on 10.2-RELEASE-p7. Created attachment 164287 [details]
shar archive
idefix@fechner.net, It is very amazing you can't build it. You need: 1. This shar 2. Mk/Uses/go.mk from Bug#205282 3. Patch from Bug#204772 4. Patch from Bug#205032 After all of them are applied, you should be able to build. You need to first do this in the ports tree to eliminate the chance that poudriere machine isn't using the same ports tree: # rm -rf /usr/ports/distfiles/* && cd /usr/ports/devel/gogs && make Could you verify that all these patches are applied, and tell me what error are you getting? Thanks, Bug#204772 was missing in my patch list. I did now a compile using poudriere and it succeeded! The log file can be found here: http://pkg.fechner.net/data/102amd64-default/2015-12-16_11h45m30s/logs/gogs-0.151216.log The only thing I changed was the UID. I added the following lines: gogs:*:209:209::0:0:gogs user:/var/db/gogs/home:/bin/sh and gogs:*:209: the uid used from your entry is not free as everything over 1000 is used for real users. I will give it now a try and make further tests. I hope that all depending patch will quickly go into HEAD so the port could be added. Good! Yes, gogs user number is now taken. I really hope they adopt gogs as a port submission system. This way all poudriere builds could be done automatically, and committer could just press a button to commit the pull request. This can improve the speed of port commits. Created attachment 164341 [details]
shar archive
Version update.
I tested you new build and it works fine. One suggestion: The configfile (/usr/local/etc/gogs.ini*) should have permission 640 as they hold passwords in cleartext. I also tested the mysql backend for the database, works as fine as well. Great work you did here! Thanks! I will change file permissions in the next update. If some one else would like to test the port, the already packaged version can be found here: https://pkg.fechner.net/packages/102amd64-default/All/gogs-0.151218.txz (In reply to idefix from comment #28) Hello! There is 404 Error when trying to download the file: https://pkg.fechner.net/packages/102amd64-default/All/gogs-0.151218.txz. Best regards, Sergey Brunov. @Sergey, This is the new port, not in the tree yet. (In reply to sergey.v.brunov from comment #29) Try https://pkg.fechner.net/packages/102amd64-default/All/gogs-0.151216.txz Hello! Thank you for pointing out! Have just completed smoke testing of the gogs-0.151216.txz package. The gogs server seems to be running fine. Best regards, Sergey Brunov. Created attachment 166317 [details]
shar archive (update to 0.8.25)
There seems to be a problem in the created package. I compiled it and it is available at: http://pkg.fechner.net/packages/102amd64-default/All/gogs-0.8.25.txz I had to install it with pkg install -f gogs as version number logic changes from gogs: 0.151216 -> 0.8.25. But the installation shows an error message: Checking integrity... done (0 conflicting) [1/1] Downgrading gogs from 0.151216 to 0.8.25... You may need to manually remove /usr/local/etc/gogs.ini if it is no longer needed. ==> You should manually remove the "gogs" user. ===> Creating users and/or groups. Using existing group 'gogs'. Using existing user 'gogs'. usage: install [-bCcpSsUv] [-f flags] [-g group] [-m mode] [-o owner] [-M log] [-D dest] [-h hash] [-T tags] [-B suffix] [-l linkflags] [-N dbdir] file1 file2 install [-bCcpSsUv] [-f flags] [-g group] [-m mode] [-o owner] [-M log] [-D dest] [-h hash] [-T tags] [-B suffix] [-l linkflags] [-N dbdir] file1 ... fileN directory install -dU [-vU] [-g group] [-m mode] [-N dbdir] [-o owner] [-M log] [-D dest] [-h hash] [-T tags] directory ... pkg: PRE-INSTALL script failed [1/1] Extracting gogs-0.8.25: 100% Message from gogs-0.8.25: ====================================================================== You installed gogs: Go Git Service. It is recommended to run gogs as a service: # service gogs start To enable the service at startup you should add this line to /etc/rc.conf: gogs_enable="YES" You can also run gogs manually with a command: # gogs Connect to gogs on the default port 3000: http://localhost:3000 Configuration file is /usr/local/etc/gogs.ini. You can only edit it when gogs isn't running. Gogs needs an SSH daemon, so make sure this line is in /etc/rc.conf: sshd_enable="YES" and sshd is configured to listen on the same port that is configured in gogs. ====================================================================== One suggestion for the files/pkg-message.in. Change from: To enable the service at startup you should add this line to /etc/rc.conf: %%PORTNAME%%_enable="YES" To: To enable the service at startup you should execute the command: sysrc gogs_enable="YES" Maybe the same for bottom: %%PORTNAME_FANCY%% needs an SSH daemon, so make sure this line is in /etc/rc.conf: sshd_enable="YES" To: %%PORTNAME_FANCY%% needs an SSH daemon, so make sure you execute: sysrc sshd_enable="YES" The problem with version isn't a real problem because version indeed changed from date-based to the real one. install problem is puzzling, which FreeBSD version do you use? I didn't see the problem on 10.3-PRERELEASE. Thanks for your suggestion for pkg-message.in, will change this in the next update. I use 10.2-RELEASE-p11 (current one that freebsd-update give me). I can reproduce 'install' error with your package. But not with package I create on 10.3
> # pkg add gogs-0.8.25.txz
> Installing gogs-0.8.25...
> ===> Creating users and/or groups.
> Using existing group 'gogs'.
> Using existing user 'gogs'.
> Extracting gogs-0.8.25: 100%
> Message from gogs-0.8.25:
> ======================================================================
This has to do with the version of pkg, mine is 1.6.3 But in any case this has nothing to do with gogs itself, this is likely some bug in pkg. I use pkg 1.6.3 too. Just rebuild the package, but error message is there. Maybe it is really a bug in freebsd 10.2 which is fixed for 10.3. But as 10.3 is not stable yet I suggest we wait how it develops ;) This is clearly not a problem from the gogs port, but maybe a bug in poudriere or pkg. Created attachment 167584 [details]
shar archive (update to 0.8.43)
I tested your new version 0.8.43 but it fails to build: https://pkg.fechner.net/data/102amd64-default/2016-03-12_11h30m49s/logs/errors/gogs-0.8.43.log I have retested it with 10.3 and it fails. I applied the two patches from: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205282 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205032 Error message: https://pkg.fechner.net/data/103amd64-default/2016-04-11_12h30m49s/logs/errors/gogs-0.8.25.log (In reply to idefix from comment #34) Hello! The file (http://pkg.fechner.net/packages/102amd64-default/All/gogs-0.8.25.txz) is not available. Could you please provide it? Best regards, Sergey Brunov. Sorry I cannot. Please see #42, #43, the build fails so I'm not able to provide with a package.(In reply to sergey.v.brunov from comment #44) Hi Yuri, Matthias, Gogs seems like an interesting application. What remaining work is there to have this port committed? I'm not sure if this install guide helps: https://www.codejam.info/2015/03/installing-gogs-on-freebsd.html Gogs depends on the new go.mk module. Due to certain restrictions of the old version of make used in 9.X, it can't work with it, so the new go.mk should wait until 9.X is discontinued (I think this is Jan 1, 2017). Otherwise you can install it by copying go.mk and other supporting patches and building the port. Yuri Thanks! Couldn't we use ${OSVERSION} to check if we're running on 9.x and then prevent the port from building?
It's not so simple, because this patch https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205032 that actually causes the problem changes bsd.port.mk in many places breaking 9.X. Without the need to support 9.X it can be committed any time. What work around are other ports using for GitHub group names that contain a hyphen? They do this manually using multiple MASTER_SITES and then bundle them by hand. But many modern projects have just too many such downloads to handle them manually. (In reply to Yuri Victorovich from comment #50) FreeBSD 9.x is now discontinued. Are there any open issues left or is the port ready to be committed? I would just like to repeat that I believe that this port is misguided in defaulting to a "gogs" user rather than the standard "git" user to run as. The established convention for services which have a multiplexed user for handling git over SSH use "git" as that user, this is the case for Gitosis, Gitolite, and Gitlab which are all set up that way in ports. Also, the documentation for Gogs itself recommends using "git" as the user. I have tried discussing this on the mailing list and another ticket, but it doesn't seem like it went anywhere (which I guess is okay since this port hasn't been included yet): https://lists.freebsd.org/pipermail/freebsd-ports/2016-February/102137.html I still think it would be best to change this port to be in line with the others and use "git" to minimize confusion and surprise. > FreeBSD 9.x is now discontinued. Are there any open issues left or is the port ready to be committed? Another, unrelated and incompatible version of go.mk was committed in the meantime. Actually, this is against the rules because this one was pending. I will bring this up with the offending committer. I will probably have to rename this one into golang.mk while dependencies will be sorted out. > this port is misguided in defaulting to a "gogs" user You are quite possibly right. I didn't give it much thought, and will rename the user. Haven't had time for this, back to the pool. You may want to consider tracking gogs' fork, gitea, instead, as it has bigger momentum and community: https://github.com/go-gitea/gitea I agree, gitea looks like the future. There is already a port for www/gitea. Anyway, that should not stop efforts to port gogs as well. Looking carefully at Gogs and Gitea, it seems there is already quite a bit of difference. It seems healthy and enriching to all if Gogs would be available in ports as well. PS: Sorry for the noise, I missed the www/gitea port earlier on. This seems to be severely out of date at this point and needs an update. AFAICT Gogs also vendors dependencies themselves these days, so the port can probably be a lot simpler than it is now. Thanks! I will update it within a week. Yuri. Created attachment 188086 [details] gogs-0.11.29.patch * Update to 0.11.29 * Switch to DISTVERSION * Add DOCS knob * Change MAINTAINER to dg@syrec.org QA: portlint -ACN: WARN: Makefile: possible use of absolute pathname "/var/db/gogs". WARN: Makefile: possible use of absolute pathname "/var/log/gogs". 0 fatal errors and 2 warnings found. poudriere testport: OK (11a, 10i; DOCS tested) Thanks! A commit references this bug: Author: yuri Date: Mon Nov 20 17:52:36 UTC 2017 New revision: 454574 URL: https://svnweb.freebsd.org/changeset/ports/454574 Log: New port: devel/gogs: Painless self-hosted Git service PR: 205283 Submitted by: Dmitri Goutnik <dg@syrec.org> Approved by: tcberner (mentor), adamw (mentor) Differential Revision: https://reviews.freebsd.org/D13141 Changes: head/devel/Makefile head/devel/gogs/ head/devel/gogs/Makefile head/devel/gogs/distinfo head/devel/gogs/files/ head/devel/gogs/files/gogs-rm-data.in head/devel/gogs/files/gogs-service.in head/devel/gogs/files/gogs.in head/devel/gogs/files/patch-templates_install.tmpl head/devel/gogs/files/pkg-message.in head/devel/gogs/pkg-descr head/devel/gogs/pkg-plist |