Bug 225309 (gitlab_config)

Summary: www/gitlab-ce: Lack of FreeBSD-specific documentation, install bug
Product: Ports & Packages Reporter: Daniel Eischen <deischen>
Component: Individual Port(s)Assignee: Matthias Fechner <mfechner>
Status: Closed Overcome By Events    
Severity: Affects Some People CC: bdrewery, mfechner, pi, swills, w.schwarzenfeld
Priority: --- Flags: bugzilla: maintainer-feedback? (mfechner)
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226610
Bug Depends on: 234807    
Bug Blocks:    

Description Daniel Eischen freebsd_committer freebsd_triage 2018-01-19 02:01:58 UTC 
On our systems, /home is not a link to /usr/home but a directory containing automounts.  Other directories are automounted under it.  Our users (LDAP) homes are located in /home/[sysname]/[username], where [sysname] is a machine hosting their home directories.  So for instance, 100 users (user1 - user100) could be hosted on brightstar and 100 users (user101-user200) could be on sirius.  There'd be 2 mounts associated with them on all our systems.  So an /etc/auto_master would look like this:

  /home    auto_homedirs    -nobrowse

and the matching /etc/auto_homedirs would be:

  brightstar    brightstar:/tank/exports/homes
  sirius        sirius:/tank/exports/homes

The point of all this is that 'pkg install' fails because it can't create /home/git.  When you try to create a directory in a directory controlled by automountd, you get "operation not permitted".  It installs much of gitlab, but stops as soon as it gets this error.

While it may seem more of like an overall pkg/ports issue (which it may be), the FreeBSD gitlab port modifies everything to change the default home directory to /usr/home/git anyway.  If it is doing that, why doesn't the pkg-plist also use /usr/home instead of /home?

I guess py-gitosis creates the 'git' user because the passwd entry shows this:

  git:*:211:211:gitosis user:/usr/local/git:/bin/sh

It is confusing to have /usr/local/git and /usr/home/git.  Which is which?  There is one git user, but which directory belongs to which port?  Can they be the same?  Is the fact that passwd can only be set to one of them affect how the other operates?  Should the gitlab home directory just be called gitlab?

Lastly, there is not enough documentation on how this port tailors things for FreeBSD.  The document pointed to by the install:

  http://gitlab.toco-domains.de/FreeBSD/GitLab-docu/blob/master/install/10.1-freebsd.md

is for installing gitlab from source, not from a package.  It's not really relevant.  What are the rc.conf variables to use for gitlab?  Where is it installed on FreeBSD so that one knows how to hook it into apache or nginx?  What other configuration steps can be culled from the link above?  The pkg-message should list the steps needed to at least get it up and running.  The postgresql-server port does a decent job of doing this.
Comment 1 Matthias Fechner freebsd_committer freebsd_triage 2018-01-21 09:26:12 UTC 
Dear Daniel,

at first thanks a lot for your report!

As I do not have such a setup I would require here help for you.
If you have suggestions to improve the installation manual without breaking it for other users, please feel free to provide with a patch or create a merge request for:
http://gitlab.toco-domains.de/FreeBSD/GitLab-docu/blob/master/install/10.1-freebsd.md

I will include it then.

Thanks a lot!
Comment 2 Daniel Eischen freebsd_committer freebsd_triage 2018-01-21 20:00:42 UTC 
Do you install/use the FreeBSD port/pkg for gitlab, or do you install it from source as described in the taco-domains link?

This is my first time working with gitlab, I don't have it up and running, just the package installed.  I was looking for some basic description of how to set it up once installed.

I would be willing to write something after getting it working, but perhaps you could give me some guidance off-line?

Eventually, I think the port should be changed so that the gitlab home user (or whatever port creates the git user's home directory) should be somewhere in /usr/local (LOCALBASE), not /home or /usr/home.  It seems against port etiquette to place things in /usr.
Comment 3 Matthias Fechner freebsd_committer freebsd_triage 2018-01-22 20:33:36 UTC 
I do it like I described it in the manual, do package installation using pkg.

pkg install gitlab

The manual describes everything you have to do.
Use postgres as database and not mysql.
Comment 4 Bryan Drewery freebsd_committer freebsd_triage 2018-03-16 22:04:40 UTC 
(In reply to Daniel Eischen from comment #2)
> Do you install/use the FreeBSD port/pkg for gitlab, or do you install it
> from source as described in the taco-domains link?
> 
> This is my first time working with gitlab, I don't have it up and running,
> just the package installed.  I was looking for some basic description of how
> to set it up once installed.
> 
> I would be willing to write something after getting it working, but perhaps
> you could give me some guidance off-line?
> 
> Eventually, I think the port should be changed so that the gitlab home user
> (or whatever port creates the git user's home directory) should be somewhere
> in /usr/local (LOCALBASE), not /home or /usr/home.  It seems against port
> etiquette to place things in /usr.

I agree. This is the only port that is trying to install something to /home or
/usr/home. I'm working over in Bug 226610 to make Poudriere lockdown the base system from being written to and had to exempt this directory for this 1 port.
It would be better if this were using something like /var/db/gitlab or anything
not in the homedir.
Comment 5 Bryan Drewery freebsd_committer freebsd_triage 2018-03-16 22:08:34 UTC 
And the git user's homedir is specifically listed as /usr/local/git in UIDs:
git:*:211:211::0:0:gitosis user:/usr/local/git:/bin/sh
So /usr/home/git is not right.
Comment 6 Matthias Fechner freebsd_committer freebsd_triage 2018-03-19 16:15:59 UTC 
Dear all,

I currently wait that my public key got added to the servers so I can start to commit (I just shortly got my commit bit).

I have a pending update for gitlab 10.5.5 in the pipe:
https://gitlab.fechner.net/mfechner/Gitlab/commits/10.5

After the update is in the ports I can start to check if gitlab can work with a different home directory without patching it. If patching would be required I will not do this migration as it will make the port unmaintainable.

But that means gitlab will conflict with other tools like devel/py-gitosis or devel/gogs as all of them are modifying the ~/.ssh/authorized_keys file.
So you can have only one port installed.

So if gitlab uses the standard path /usr/local/git I have to add a CONFLICT to these ports. Do you think this is a good idea to do this?

Using another user as git for gitlab is not supported by gitlab.
Comment 7 Daniel Eischen freebsd_committer freebsd_triage 2018-03-20 18:59:17 UTC 
These should be the only changes necessary to change repo storage.  As for the ssh keys, I suppose you can just change the auth_file setting in gitlab-shell/config.yml.  If 3 ports want /usr/local/git as home directory, I guess you could base gitlab out of a subdirectory of git, like /usr/local/git/gitlab.  Then the auth_file knob could be set to /usr/local/git/gitlab/.ssh/authorized_keys, for instance.

Note that below, there's also a fix for gitaly's path not being correct.



--- /usr/local/www/gitlab/config/gitlab.yml.sample      2018-01-22 21:35:41.000000000 -0500
+++ /usr/local/www/gitlab/config/gitlab.yml     2018-02-07 04:02:09.866011000 -0500
@@ -501,7 +501,7 @@
   # Gitaly settings
   gitaly:
     # Path to the directory containing Gitaly client executables.
-    client_path: /home/git/gitaly
+    client_path: /usr/local/share/gitaly
     # Default Gitaly authentication token. Can be overriden per storage. Can
     # be left blank when Gitaly is running locally on a Unix socket, which
     # is the normal way to deploy Gitaly.
@@ -519,7 +519,7 @@
     # real path not the symlink.
     storages: # You must have at least a `default` storage path.
       default:
-        path: /usr/home/git/repositories/
+        path: /usr/local/git/repositories/
         gitaly_address: unix:/usr/local/www/gitlab/tmp/sockets/private/gitaly.socket # TCP connections are supported too (e.g. tcp://host:port)
         # gitaly_token: 'special token' # Optional: override global gitaly.token for this storage.
 
--- /usr/local/share/gitlab-shell/config.yml.sample     2018-01-04 09:19:17.000000000 -0500
+++ /usr/local/share/gitlab-shell/config.yml    2018-02-06 22:15:00.046361000 -0500
@@ -25,7 +25,7 @@
   self_signed_cert: false
 
 # File used as authorized_keys for gitlab user
-auth_file: "/home/git/.ssh/authorized_keys"
+auth_file: "/usr/local/git/.ssh/authorized_keys"
 
 # File that contains the secret key for verifying access to GitLab.
 # Default is .gitlab_shell_secret in the gitlab-shell directory.
--- /usr/local/share/gitaly/config.toml.sample  2018-01-04 06:42:01.000000000 -0500
+++ /usr/local/share/gitaly/config.toml 2018-02-07 19:37:55.425887000 -0500
@@ -16,7 +16,7 @@
 
 [[storage]]
 name = "default"
-path = "/usr/home/git/repositories"
+path = "/usr/local/git/repositories"
 
 [gitaly-ruby]
 dir = "/usr/local/bin/ruby"
Comment 8 Matthias Fechner freebsd_committer freebsd_triage 2019-01-09 22:04:06 UTC 
I have tested a patch successfully but if you use gitlab-ce now together with e.g. gitolite on one host they will break each other.
As long as pkg cannot handle this conflict correctly I will not apply this patch is it could break gitlab-ce completely and in worst case you can lose data.

The patch can be found here:
https://gitlab.fechner.net/mfechner/Gitlab/commit/eecee886cd5dada4b5c87560e21b7818aa1422d2
Comment 9 Walter Schwarzenfeld freebsd_triage 2019-08-13 11:39:23 UTC 
We have version 12.0.4_1. Does the problem still exist?
Comment 10 Matthias Fechner freebsd_committer freebsd_triage 2019-08-21 08:46:41 UTC 
I think the problem should be fixed, if not, please feel free to comment and/or reopen the ticket.
Thanks.