Bug 202468 - [new port] www/gitlab
Summary: [new port] www/gitlab
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Kurt Jaeger
URL:
Keywords:
Depends on: 202871 203468 203469 203470 205605 205645 205660 205865 205866 205877 206113 206130 206131 206950 206955 207480 207481 207482 207483 207484 207485 207791 207792 207793 207794 207795 207796 207797 207799 207800 207802 208135 208172 208321 208352
Blocks: 203619
  Show dependency treegraph
 
Reported: 2015-08-19 10:59 UTC by Torsten Zühlsdorff
Modified: 2017-12-17 07:11 UTC (History)
12 users (show)

See Also:


Attachments
the gitlab-port (150.02 KB, text/plain)
2015-08-19 10:59 UTC, Torsten Zühlsdorff
no flags Details
poudriere log (103.21 KB, text/plain)
2015-08-21 08:01 UTC, Torsten Zühlsdorff
no flags Details
GitLab 7.14.1 (151.22 KB, text/plain)
2015-09-03 16:03 UTC, Torsten Zühlsdorff
no flags Details
Poudriere log for GitLab 7.14.1 (103.49 KB, text/x-log)
2015-09-03 16:03 UTC, Torsten Zühlsdorff
no flags Details
GitLab 7.14.3 (151.05 KB, text/plain)
2015-10-01 11:53 UTC, Torsten Zühlsdorff
no flags Details
current progress as diff against /usr/ports (230.59 KB, patch)
2016-02-23 11:49 UTC, Torsten Zühlsdorff
no flags Details | Diff
current progress as diff against /usr/ports (231.00 KB, patch)
2016-02-23 12:59 UTC, Torsten Zühlsdorff
no flags Details | Diff
current progress as diff against /usr/ports (232.95 KB, patch)
2016-02-24 10:22 UTC, Torsten Zühlsdorff
no flags Details | Diff
current progress as diff against /usr/ports (234.29 KB, patch)
2016-02-29 15:11 UTC, Torsten Zühlsdorff
no flags Details | Diff
patch with GitLab 8.5.2 and all needed updates against /usr/ports (243.28 KB, patch)
2016-03-03 10:17 UTC, Torsten Zühlsdorff
no flags Details | Diff
shar with GitLab 8.5.4 (211.12 KB, patch)
2016-03-08 11:54 UTC, Torsten Zühlsdorff
no flags Details | Diff
shar with GitLab 8.5.5 (211.17 KB, patch)
2016-03-11 11:09 UTC, Torsten Zühlsdorff
no flags Details | Diff
shar with GitLab 8.5.5 (211.63 KB, patch)
2016-03-11 13:46 UTC, Torsten Zühlsdorff
no flags Details | Diff
shar with working Gitlab 8.5.5 (212.03 KB, text/plain)
2016-03-21 08:19 UTC, Johannes Jost Meixner
no flags Details
shar with working Gitlab 8.5.5, 2016-03-26 edition (212.27 KB, patch)
2016-03-27 09:41 UTC, Johannes Jost Meixner
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Torsten Zühlsdorff 2015-08-19 10:59:51 UTC
Created attachment 160079 [details]
the gitlab-port

GitLab is the most used open source repository management tool for GitLab repositories. It is listed a long time at https://wiki.freebsd.org/WantedPorts and could hopefully removed soon ;)

The port status and all the needed slave ports where discussed at ruby@, mainly with steve@.
Comment 1 Torsten Zühlsdorff 2015-08-21 08:01:35 UTC
Created attachment 160167 [details]
poudriere log

Add poudriere test-log
Comment 2 Philip M. Gollucci freebsd_committer 2015-08-21 21:12:07 UTC
Take.
Comment 3 Torsten Zühlsdorff 2015-08-31 10:52:57 UTC
Hello Philip,

currently there is a new version of GitLab. GitLab releases every month a new version. And in my experience shortly afterwards some bugfix releases (as already happened).

Do you want me to update the port to the newest version, which can include some more dependencies (i did not checked it until now)? Or should i wait until you finished everything and than provide an update?

Greetings,
Torsten
Comment 4 Torsten Zühlsdorff 2015-09-03 16:03:11 UTC
Created attachment 160677 [details]
GitLab 7.14.1

I've updated the new port to the newest version 7.14.1. In the meantime some changes to the portstree make the installation of GitLab not possible. Fixes are already contained in the new version. :)
Comment 5 Torsten Zühlsdorff 2015-09-03 16:03:48 UTC
Created attachment 160678 [details]
Poudriere log for GitLab 7.14.1
Comment 6 Palle Girgensohn freebsd_committer 2015-09-30 19:23:16 UTC
Hi Torsten,

What is the state of this port, does it have a "sponsoring" comitter?

Palle
Comment 7 Torsten Zühlsdorff 2015-10-01 07:32:24 UTC
(In reply to Palle Girgensohn from comment #6)

> What is the state of this port, does it have a "sponsoring" comitter?

Sorry, i don't know the term "sponsoring" committer. But currently Philip (pgollucci@) is looking at it and making the QA. He was quite busy in the last time, but we stand in contact to get this port into the tree. :)

Also there is already a new version of GitLab with more new dependencies. In communication with Philip and ruby@ i decided to wrote the new version as update of this one. The new versions has more dependencies and they can be tested without interfering with this PR.

I don't know if you can help Philip. But the new dependencies for the update do not have an Assignee yet. Currently this are 203408 and 203230. There will be 8 more shortly. If your are interested feel free to grab one and/or contact me. This week am very short of time, but since Monday i will push the update forward.
Comment 8 Torsten Zühlsdorff 2015-10-01 11:53:52 UTC
Created attachment 161595 [details]
GitLab 7.14.3

I've updated the port to the latest version of the 7.14 branch. It just contains some bugfixes.

In this run i've also fixed some dependencies with broke in the meantime.
Comment 9 Yonas Yanfa 2015-11-01 18:51:46 UTC
Hi Torsten, I built a port as well, not knowing this existed :) Doh!

https://github.com/yonas/gitlab-freebsd-port/tree/master


It comes with a setup script that completely sets up GitLab 8.1 and does not hard-code any paths:

https://github.com/yonas/gitlab-freebsd-port/blob/master/files/gitlab-setup.in


If there's anything you'd like to borrow, please feel free.

I like the path you're on, though. Specifying all individual requirements, like Ruby gems, is more work but is more transparent.

Cheers,
Yonas
Comment 10 Torsten Zühlsdorff 2015-11-23 11:33:36 UTC
While adjusting the Gemfile again to keep it in sync with the Portstree, i noticed it becomes hard more often.

So there was a switch from gitlab-linguist to github-linguist. While GitLab 7.14.3 directly depends on gitlab-linguist the new gitlab_git 7.2.15 has a dependency for github-linguist. Second one is the replacement for the first one. So i'm not sure if this really works together. This would require a complete QA-cycle again to be sure. And in case of an error, its not very straight forward to fix it.

Also yesterday was a new major release of GitLab.

And sunpoet@ already started to take a look at the new gems for the update to GitLab 8.0.

pgollucci@ also is very busy.

And it becomes hard to handle the documentation changes between the releases.

Based at this i suggest the following:
- i will write an update to the latest GitLab 8.2.x version and create a new ticket for it
- i will reconsider all the dependencies, merging the needed as blocker-dependencies to the new ticket. Also some of the gems won't be needed anymore - their PRs i would close
- i will make a new QA-cycle and call for testers

I don't know if we can help pgollucci@ on the committer site by further splitting the ports? Philip, can we help you?

Greetings,
Torsten
Comment 11 Michael Moll freebsd_committer 2015-12-06 15:12:18 UTC
Torsten,

GitLab 8.3 will use Rails 4.2 which is great because all the rails41 ports are superfluous then. Also, they raised the versions of some gems GitLab does depend on, which should further bring it closer to what we have in the current ports tree.

Is it an option to wait for the 8.3 release (or RC) and then make a list, what's still to do?

The newly introduced golang stuff shouldn't be a big problem, but I admit I didnt have a closer look, yet.
Comment 12 Torsten Zühlsdorff 2015-12-07 09:09:23 UTC
(In reply to Michael Moll from comment #11)

> Is it an option to wait for the 8.3 release (or RC) and then make a list, 
> what's still to do?

This are great news! :) Yes, it is a option to wait until the switched to Rails 4.2. 

I took a quick look at it and the overall image is very positive. There are some new ports, but most of them are already in the ports as part of PR 203619.

I am in my first vacation of this year, when the version arrives, but i will prepare the update a little, so we can test the port directly at the beginning of the new year. :)
Comment 13 Martin Wilke freebsd_committer 2015-12-19 07:13:40 UTC
Going to take care of this.
Comment 14 Torsten Zühlsdorff 2016-01-04 14:41:12 UTC
Hello,

as you may noticed i removed a long list of dependencies.

GitLab 8.3 now uses Rails 4.2, which obsoletes my complete recreation of Rails 4.1 and a number of slave ports.

The good news: the currently 5 (!) dependencies left is all of needed updates or new ports!

More work is needed to get the new configuration, patches and plist right and of course a new QA is needed, but now we are on a straight road forward again! :)

Greetings,
Torsten
Comment 15 Hans 2016-01-08 23:21:39 UTC
(In reply to Torsten Zühlsdorff from comment #14)

Sorry if you wasted hours... :/

But at least let me thank you for the time spend on getting gitlab working on FreeBSD! I am truly looking forward to have an easy way to spin up gitlab in any not to distant future!

Thanks!
Hans
Comment 16 Torsten Zühlsdorff 2016-01-11 13:57:56 UTC
(In reply to Hans from comment #15)

> Sorry if you wasted hours... :/
> But at least let me thank you for the time spend on getting gitlab working 
> on FreeBSD! I am truly looking forward to have an easy way to spin up gitlab 
> in any not to distant future!

You are very welcome! :)

There is nothing to be sorry about. Obsoleting this number of PRs seems like a big waste, but it isn't. I gained a big number of experience, motivated Michael Fausten (which is very important to him) and also we do a good groundwork with the other 50 PRs which were not obsoleted, but committed. I even was nominated as committer by pgollucci, thought nothing more happened in this direction.

Back to the good news: today i added the last 3 dependencies. This statement is a little vague and could be repeated, because rubygems are a little fragile. :D But the statement is correct for the portstree in revision 405771.

I'm currently reworking the documentation and check some indirect dependencies. It seems the project introduced a new proxy server (gitlab-workhorse) which needs special attention. At the moment i am not sure how to handle this one, but this seems to be the last barrier to take.

Greetings,
Torsten
Comment 17 Torsten Zühlsdorff 2016-01-20 10:39:39 UTC
A short status report:

I found a way to support gitlab-workhorse. I've created a new port for this.

Also i stumped across an error which seems to hunt me since my first start. But today i could fix it.

Now there are a bunch of permissions problems left. Gitlab wants to store files in the home-directory of git-user but the code is in /usr/local/www and owned by root. This will need some careful rework at different areas.

The rc-script works basically and starts unicorn and sidekiq-demons.

Precompilation of assets fails because of a jquery-ui problem. Currently there is jquery-ui 5.0.5 in the ports but GitLab works only with 4.x. -.-
After some research i decided to wait 2 days. The next GitLab release at friday will fix this problem. :)

The documentation is mostly updated. Just the gitlab-shell part needs to be removed:
https://github.com/t-zuehlsdorff/gitlabhq/blob/8-3-docu/doc/install/installation-freebsd.md

Greetings,
Torsten
Comment 18 Torsten Zühlsdorff 2016-02-02 15:56:24 UTC
I continued my work (again) and moved them for easier testing and helping to Github:
https://github.com/t-zuehlsdorff/freebsd-ports/tree/gitlab

The branch "gitlab" contains all what currently is missing in the ports-tree (needed new ports, needed updates, etc.).

At the moment it is possible to just build and install everything. The documentations needs an update; also the rc scripts needs one. I expect some major work needed to get the permissions and configuration right, but now we have a solid base.

Feel free to test and reply. :)
Comment 19 Torsten Zühlsdorff 2016-02-09 14:51:59 UTC
Today i finished my update to GitLab 8.4.3! :)

Have a look at:
https://github.com/t-zuehlsdorff/freebsd-ports/tree/gitlab

gitlab-shell und gitlab-workhorse are now separate ports. I revisited the rc-script which should work. My biggest problem is currently gitlab-shell, which needs some improvement. GitLab expects its installation in /home/git/gitlab-shell, but i am not happy with that. In the current state gitlab-shell is not found (even with changed configuration) and the installation aborts. If somebody has any idea, please feel free to suggest.
Comment 20 Johannes Jost Meixner freebsd_committer 2016-02-22 09:21:34 UTC
Is there a way you could patch gitlab so that it can be deployed to 

/usr/local/www/gitlab,

and gitlab-shell to

/usr/local/share/gitlab-shell ?
Comment 21 Torsten Zühlsdorff 2016-02-22 10:33:16 UTC
(In reply to Johannes Jost Meixner from comment #20)

> Is there a way you could patch gitlab so that it can be deployed to 
> /usr/local/www/gitlab,

This is already done :)

> and gitlab-shell to
> /usr/local/share/gitlab-shell ?

This did not work the last time, but is my goal. It seems to be a configuration issue.

But before i could solve this issue, i need to solve a bug in GitLab itself. After nearly a month a developer of GitLab finally replies, so i get a small step further. This (type of) error seems to be the last blocker!
Comment 22 Torsten Zühlsdorff 2016-02-23 11:49:14 UTC
Created attachment 167322 [details]
current progress as diff against /usr/ports

Hello,

i spend several hours and chocolate for more progress. Attached a patch with my current state. It is created against /usr/ports and - of course - very experimental. 

It contains several updates and some new ports. Also it contains GitLab in its newest version 8.5.0.

After applying the patch you can:
# cd /usr/ports/www/gitlab
# make install

There is a documentation to follow:
https://github.com/t-zuehlsdorff/gitlabhq/blob/8-5-docu/doc/install/installation-freebsd.md

If somebody is interested in helping, there are several open tasks:
- Testing of the build-process and the installation. Also: is the documentation useful?
- the patch contains some updates and new ports which also needs reviews (and PRs)

There are known issues left:
- the rc script starts gitlab on every boot, regardless of /etc/rc.conf
- currently there is a right problem with the cache right after the start. any idea is appreciated. I will update the patch if the error is gone and GitLab itself could be tested :)
Comment 23 Torsten Zühlsdorff 2016-02-23 12:10:19 UTC
Just a short notice:
The cache problem seems to occur just on another server. So there is good chance it will work for you.

At my current server i could log into GitLab and start testing :)
Comment 24 Torsten Zühlsdorff 2016-02-23 12:59:02 UTC
Created attachment 167325 [details]
current progress as diff against /usr/ports

Updated the patch - now it is possible to create own repositories.

Have fun to test! :)
Comment 25 Torsten Zühlsdorff 2016-02-24 10:22:46 UTC
Created attachment 167357 [details]
current progress as diff against /usr/ports

Attached a patch which:
- adds rubygem-html-pipeline1 because the existing one did not work together with GitLab
- patches the Gemfile to include rinku -> this and the previous one fixes different 500 errors and project pages and admin area
- patch gitlab-shell to create a logfile in /var/log and use this -> this reenables creation of own projects
Comment 26 Torsten Zühlsdorff 2016-02-25 11:14:00 UTC
Hello,

as you may noticed i have created some more PRs with updates and new ports. This is *not* the full dependency stack needed for www/gitlab.
Most of the PRs are updates which will bring the ports to its current version and enables gitlab to run. A clear win-win ;)

Also it will hopefully bring the diff under 5.000 lines to make it much better reviewable...

As a side-note: there was a new GitLab minor release. The current version is 8.5.1. I already checked and can confirm, that the update is possible without any problems.
But there is clearly a missing documentation about the upgrade process, which i will wrote before the update.

Greetings,
Torsten
Comment 27 Torsten Zühlsdorff 2016-02-29 15:11:49 UTC
Created attachment 167569 [details]
current progress as diff against /usr/ports

Yet another update. This updates GitLab to its current version 8.5.1. The documentation for the update process is reviewed at the moment. I will post the link if done. :)

And after that i will wrote the last prs and reviews, to get it finally into the ports :)
Comment 28 Torsten Zühlsdorff 2016-03-03 10:17:25 UTC
Created attachment 167679 [details]
patch with GitLab 8.5.2 and all needed updates against /usr/ports

Hello,

attached a patch to upgrade GitLab to its current version 8.5.2. 
This patch also includes the needed Rails update from 4.2.5.1 to 4.2.5.2.

After installation you can notice a new pointer to the update instruction for such a patch version:
https://github.com/t-zuehlsdorff/gitlabhq/blob/8-5-docu/doc/update/freebsd_patch_versions.md

If you already installed GitLab 8.5.0 from here please follow this guide to try the update. :)

Based on the positive feedback so far i will start next week to finish the work at the project by creating the last needed PRs. And of course the needed shars. :)

Greetings,
Torsten
Comment 29 Torsten Zühlsdorff 2016-03-08 11:54:45 UTC
Created attachment 167849 [details]
shar with GitLab 8.5.4

Hello,

here we go! Attached a shar with the current version 8.5.4. All other PRs are written and waiting for committment. Now we can have a working GitLab in the portstree :)

Greetings,
Torsten
Comment 30 Hans 2016-03-09 09:06:52 UTC
(In reply to Torsten Zühlsdorff from comment #29)

Once again Torsten - Thanks a ton for all of your work! Can't wait for it to get part of the portstree and even better a simple pkg install away!

When it hits the portstree ill try to get it up and running here on my end and report back if I find any problems.


Thanks,
Hans
Comment 31 Johannes Jost Meixner freebsd_committer 2016-03-09 11:17:18 UTC
I've only tried with the 8.5.0 patch of this PR, but there weren't any. My FreeBSD/iocage-powered Gitlab instances are running happily.

Keep up the good work Torsten!
Comment 32 Torsten Zühlsdorff 2016-03-11 11:09:44 UTC
Created attachment 168007 [details]
shar with GitLab 8.5.5

Update to GitLab 8.5.5 :)
Comment 33 Torsten Zühlsdorff 2016-03-11 13:46:04 UTC
Created attachment 168010 [details]
shar with GitLab 8.5.5

Yet another update. After upgrading my installed rubygems i noticed that more patches of GitLabs Gemfile is needed. Attached a shar which works with the current portstree :)
Comment 34 Johannes Jost Meixner freebsd_committer 2016-03-19 10:54:27 UTC
Patch attached fails to work with current portstree:

ports rails4 is at 4.2.6 whereas patch attached presupposes 4.2.5.2.
Comment 35 Johannes Jost Meixner freebsd_committer 2016-03-21 07:46:51 UTC
Sunpoet's most recent commit to devel/rubygem-gitlab_git broke this patch even further:

gitlab's gemspec currently presupposes version ~>8.2.
He committed 10.0, which will obviously make bundler fail to rake GitLab into existence, as well as make the startup service moot.

https://svnweb.freebsd.org/ports?view=revision&revision=411496


Can we please start considering this "critical infrastructure" and stop breaking it?

Thanks.
Comment 36 Johannes Jost Meixner freebsd_committer 2016-03-21 08:03:38 UTC
Same for:
* security/rubygem-net-ssh update to 3.1.0
* textproc/rubygem-github-linguist update to 4.8.0

What's more, the update to rubygem-github-linguist breaks rubygem-gitlab_git, because it explicitly depends on rubygem-github-linguist version 4.7.0.

I recall that when I was committer and part of the ruby team, the rule was to check if an update breaks any reverse dependencies BEFORE committing said update.

What happened to that?
Comment 37 Johannes Jost Meixner freebsd_committer 2016-03-21 08:19:39 UTC
Created attachment 168446 [details]
shar with working Gitlab 8.5.5

Patch attached fixes aforementioned Gemfile problems.

Tested in poudriere (builds), with rake (gitlab:setup) and service gitlab start.
Comment 38 Torsten Zühlsdorff 2016-03-21 14:40:14 UTC
Thanks for the patch Johannes!

Please note: the current version is 8.5.8. It did not wrote a patch for two reasons:
1. it depends on git 2.7.4, which is currently not in the ports
2. tomorrow GitLab 8.6 will be released


> Can we please start considering this "critical infrastructure" 
> and stop breaking it?

That would be really fine. Current commit behavior makes gitlab a moving target and you could easily end up writing patches multiple days a week :(
Comment 39 Johannes Jost Meixner freebsd_committer 2016-03-22 05:23:59 UTC
(In reply to Torsten Zühlsdorff from comment #38)
ports devel/git has been at 2.7.4 since Friday :)

https://svnweb.freebsd.org/ports/head/devel/git/Makefile?r1=411338&r2=411339&
Comment 40 Johannes Jost Meixner freebsd_committer 2016-03-27 09:41:47 UTC
Created attachment 168675 [details]
shar with working Gitlab 8.5.5, 2016-03-26 edition
Comment 41 Steve Wills freebsd_committer 2016-03-28 02:44:21 UTC
(In reply to Johannes Jost Meixner from comment #40)
This references a net/rubygem-omniauth-cas3 but I see no port or PR for this.
Comment 42 Johannes Jost Meixner freebsd_committer 2016-03-28 08:58:12 UTC
(In reply to Steve Wills from comment #41)
Thanks, I've created https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=208352 for it.
Comment 43 Torsten Zühlsdorff 2016-03-29 13:55:05 UTC
Hello,

since GitLab 8.6.1 is the current version i am already working on an update.

But the next update will bring up 4 new ports. Considering already the work done till here this brings up two options:

1) finish #208135 and commit GitLab 8.5.5 into the portstree
2) wait for finishing 8.6.1 and its dependencies, than going through them

Personally i would opt for number 1. This port took nearly a year till now and seems to eaten committer. :( We have a working version attached to the PR and this could provide a good basic for further updates.

Of course the committed version would be outdated, but GitLab releases a new major version every month and multiple minor versions in between. A patch must be written in every case.
Comment 44 Kurt Jaeger freebsd_committer 2016-03-29 14:00:41 UTC
My suggestion: Lets get 8.5.5 in the tree in time for the quarterly, if 8.6.1 is not too difficult to upgrade.
Comment 45 Torsten Zühlsdorff 2016-03-29 15:33:08 UTC
(In reply to Kurt Jaeger from comment #44)

> My suggestion: Lets get 8.5.5 in the tree in time for the quarterly, 
> if 8.6.1 is not too difficult to upgrade.

At least it is not that easy. We need:
- 4 new ports (already written, but we should test them of course)
- 1 update of an existing port (outstanding)
- an updated documentation fitting the new version
- more runtime-tests

Also i found some changes done in the installation process, which needs some attention. For example it is now expected, that the database user is an superuser. This is not described in the manual as far as i can see.

The quarter ends in two days. I would say we could not handle 8.6.1 properly before the quarter. So lets bring 8.5.5 into the tree. :)

I will have a look at 208135 tomorrow; hopefully also at the shar provided by Johannes.
Comment 46 Torsten Zühlsdorff 2016-03-30 12:24:57 UTC
I approved the extended patch in PR 208135. :) This is the last blocker, so from my point of view GitLab is ready to be committed :)
Comment 47 Kurt Jaeger freebsd_committer 2016-03-31 18:21:17 UTC
Committed in r412280. Dang, I missed the PR line 8-(

Torsten and all involved, thanks for this tireless work and the test of the bugzilla depends-PR code 8-}
Comment 48 vali gholami 2017-12-17 07:11:53 UTC
MARKED AS SPAM