I've been working on something similar, let me know if you need any help testing or with the rc.d script.

Comment 2 Dave Cottlehuber 2017-04-24 19:31:05 UTC

(In reply to Reshad from comment #1)
reshad yes please lets share notes. I'm on freenode in #couchdb-dev channel most of the time.

Created attachment 182233 [details]
v0.2 patch builds, rc.d start/stop, still needs refinement

Hey,

Went through your port and merged it whit what I had.

The port seems to work now, but still needs some polish:

1. rc script to start and stop works, but is a hack.
The couchdb command(shell script) daemonises  without giving us a pid.
Alos logging (pipe stdout and stderr to file) is a hack.

2. Configs used are in /usr/local/lib/couchdb/etc.
Need to find a way to have couchdb take configuration files in /usr/local/etc/couchdb

3. pkg plist/files installed need to be trimmed down

Possibly re-writing the script to start couchdb which takes some parameters such as config files, log location, etc would be the way to go.

Created attachment 182288 [details]
v0.3 patch builds small refinement to rc.d stop start logic needed

Logs to /var/log/couchdb/couchdb.log
Reads config files from /usr/local/etc/couchdb/*

TODO:
1. PID file
2. reduce pkg-plist (installed files)

Created attachment 182420 [details]
v0.4 Fix permissions issue, Fauxton self test passes, more refined rc.d script

This patch fixes some permissions issue with the previous version.

The rc.d script is also cleaner and should handle better.

Fauxton self test for DB passes.

TODO:
- Stop should stop all processes running as couchdb (currently, it only kills the beam process, which is usually good enough, but sometimes it does not die)
- Cleanup make file a bit.
- Generate pidfile (nice to have).

Created attachment 182429 [details]
v0.5 Fix rc.d script, cleanup Makefile, set permissions for /var

New patch updates the old one with these changes:
- Cleanup echos and testing comments in Makefile.
- rc.conf erl_flags configurable as an rc_var
- rc.d start daemon now uses -f to pipe unwanted output to /dev/null
- fix bug in rc.d stop (status_quiet)
- fix make package warning about etc/rc.d/couchdb declared twice
- set permissions to 740 for /var/{log,db,run}/couchdb

Issues 
- pidfile still not generated

The pidfile works with the current revision (v0.5)

@Dave, let me know if there is any other change you would like to make.

I have this running in a testing environment at the moment and its working fine so far.

I think this is ready to be committed to the tree.

Created attachment 182475 [details]
v6 uses WTFPL License just patched in ports tree

Updated with WTFPL

Created attachment 182543 [details]
v7 config file cleanup + portlint

Cleanup the etc/couchdb dir by putting sample files in the lib/couchdb/etc/ directory.

adhere to portlint -A

Created attachment 182800 [details]
v8 update versions in plist

Updated erts and lib depend versions in plist.
This can be a problem in the future.

Hi, 
I'm trying to install your port (version v8) but `make install` fails; do you have any idea on how to fix this?

My test environment is a fresh FreeBSD 11 installation running on a virtual machine.

Thanks



Installing CouchDB into rel/couchdb/ ...
==> rel (generate)
WARN:  'generate' command does not apply to directory /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0
... done

    You can now copy the rel/couchdb directory anywhere on your system.
    Start CouchDB with ./bin/couchdb from within that directory.

gmake[1]: Leaving directory '/usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0'
===>  Staging for couchdb2-2.0.0
===>   couchdb2-2.0.0 depends on package: erlang>=19 - found
===>   Generating temporary packing list
===> Creating groups.
===> Creating users
install  -m 0644 /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/share/docs/pdf/CouchDB.pdf  /usr/ports/databases/couchdb2/work/stage/usr/local/share/doc/couchdb2
install  -m 444 /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/share/docs/man/apachecouchdb.1  /usr/ports/databases/couchdb2/work/stage/usr/local/man/man1
/bin/mv /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/rel/couchdb/share /usr/ports/databases/couchdb2/work/stage/usr/local/share/couchdb
/bin/mv /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/rel/couchdb/etc/vm.args   /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/rel/couchdb/etc/vm.args.sample #/usr/ports/databases/couchdb2/work/stage/usr/local/etc/couchdb/vm.args.sample
/bin/mv /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/rel/couchdb/etc/local.ini /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/rel/couchdb/etc/local.ini.sample #/usr/ports/databases/couchdb2/work/stage/usr/local/etc/couchdb/local.ini.sample
cp -f -R -p /usr/ports/databases/couchdb2/work/apache-couchdb-2.0.0/rel/couchdb /usr/ports/databases/couchdb2/work/stage/usr/local/lib/
====> Compressing man pages (compress-man)
===> Staging rc.d startup script(s)
===>  Installing for couchdb2-2.0.0
===>  Checking if couchdb2 already installed
===>   Registering installation for couchdb2-2.0.0
pkg-static: Unable to access file /usr/ports/databases/couchdb2/work/stage/usr/local/root@FreeBSD12-20-33:/usr/ports/databases/couchdb2 # cat pkg-plist:No such file or directory
pkg-static: duplicate file listing: /usr/local/lib/couchdb/etc/local.ini.sample, ignoring
pkg-static: duplicate file listing: /usr/local/lib/couchdb/etc/vm.args.sample, ignoring
*** Error code 74

Stop.
make: stopped in /usr/ports/databases/couchdb2

Created attachment 183273 [details]
v9 update versions in plist

update versions for erlang-19.3.5,3

(In reply to sgironella from comment #11)

Hi,

Sorry v8 had an error in the pkg-plist file.

This is fixed in v9.

v9 also takes into account an updated version of erlang committed 2 days back, it may not be in packages yet.

Let me know how it goes, I have only tested with 12-CURRENT and limited testing on 10.3-RELEASE.

(In reply to Reshad Patuck from comment #13)

Hi, now it's running fine. Thanks!

Comment 15 Kubilay Kocak 2017-07-11 07:40:42 UTC

Comment on attachment 182041 [details]
v0.1 patch builds successfully but needs rc.d script and BSD-friendly file locations

Obsoleted by Reshads patch (Discussed via IRC)

Comment 16 Kubilay Kocak 2017-07-11 07:44:24 UTC

Bug 212846 is not blocked by this, it in fact is a proposed change that would deprecate/supersede this issue.

Opinions aside, one of these issues needs to be closed in favour of the other, depending on whether the existing port will cross a major boundary from 1.x -> 2.x, or a new port (databases/couchdb2) is created.

With that in mind, this is existing maintainer related (request feedback accordingly)

Comment 17 Dave Cottlehuber 2017-07-11 10:11:59 UTC

@rap aside from the OTP 20.0 vs OTP 19.x clashes discussed on IRC there are a few changes needed for pkg-plist yet:

- /usr/local/lib/* should be read only in general and root:wheel (default) groups
- 0755 for dirs, 0644 for files (defaults)
- sensitive files should be couchdb:couchdb 0600 or root:couchdb 0660 (local.ini, vm.args, sys.config)
- default.ini should be 0644 root:wheel as its part of the distribution
- directories should be root:wheel 0755 apart from /var/*/couchdb which should be  root:couchdb 0770 to prevent information disclosure
- .o, .a .c intermediate files should be removed
- the port should install to /usr/local/lib/couchdb2 so that people who wish to migrate from v1.x to v2.x can have them concurrently installed, at least via `pkg install -f couchdb couchdb2`
- this would mean you can use prefixes instead of custom locations in the pkg-plist:

DATADIR=/usr/local/share/couchdb2
DOCSDIR=/usr/local/share/doc/couchdb2
EXAMPLESDIR=/usr/local/share/examples/couchdb2
WWWDIR=/usr/local/www/couchdb2
ETCDIR=/usr/local/etc/couchdb2

using https://pkg.skunkwerks.at/poudriere/build.html?mastername=11_amd64-default&build=2017-07-11_09h40m41s as reference for errors, which is your latest patch + an updated erts version spliced in.

Created attachment 184308 [details]
v10 Multiple updates to plist, rc.d script and Makefile

Hi,

I have updated the port with the following changes:
- Changed dependency on 'erlang' to 'erlang-runtime19'.
- Change 'couchdb' to 'couchdb2' allowing for both the version 1 port and version 2 port to coexist if needed.
- Cleanup source/staging (*.a, *.c and *.o) files.
- Use wait_for_pids in the couchdb2 rc stop script to wait till the couchdb pid is dead.
- Change permissions of the default config files to couchdb:couchdb 0660.
- Bumped erts version to 8.3.5.1.
- Changed perms in DATADIR (www) from 0755 to 0744.
- Changed perms of var dirs to couchdb:couchdb 0770.

@dch
I have NOT changed the perms in /usr/local/lib/*.
Most of the files are already root:wheel 0644.
The last time I tried changing the perms (some months back) in this dir couch refused to start.
That is why I am using a 'cp' to copy them over.

Do let me know if there are any other changes you would like.

Best,

Reshad

Created attachment 184911 [details]
v11 go back to erlang insted of erlang-runtime19

Changed dependency from 'erlang-runtime19' back to 'erlang'.

erlang-runtime19 installs to its own directory which needs to be in path.

@Dave, if you have a way to point couchdb to a particular path for erlang then we can use erlang-runtime19.
I don't see any ./configure options to do this.

Comment 20 Kubilay Kocak 2017-08-02 04:09:45 UTC

Pending QA confirmation (this applies to each newly proposed patch/attachment)

Created attachment 184939 [details]
v12 updated erts version

Bumped erts version from 'erts-8.3.5.1' in pkg-plist to 'erts-8.3.5.2'

Created attachment 184940 [details]
v12 - poudriere log

poudriere log for v12 patch

Created attachment 185036 [details]
v13 - patch - update to couchdb 2.1.0

Updates the patch to version 2.1.0 of couchdb

Files changed/updated since last patch:
- Makefile (new version + removed line for pdf docs that do not exist in new version)
- distinfo
- files/patch-* (slight changes to patch files, just moving lines up or down as needed)
- pkg-message (point to docs for 2.1.0 instead of 2.0.0)
- pkg-plist (updated)

Created attachment 185037 [details]
v13 - poudriere log

Comment 25 Richard Gallamore 2017-08-11 17:40:54 UTC

* Uses ssl is being used. In order to link ssl to the correct branch the variables OPENSSLBASE, OPENSSLDIR, OPENSSLLIB, OPENSSLINC, OPENSSLRPATH maybe needed. Please check how the port is linking to ssl and adapt to respect the variables required.

* The rc script has many hardcoded paths this should be changed, for example %%PREFIX%% for /usr/local. Also, consider adding more configuration variables for default.ini/local.ini.

* Why are executables/docs/includes/misc? being installed in %%PREFIX%%/lib/couchdb2? This should probably be changed to the appropriate directories. If for whatever reason the software can't run in a different hierarchy, which I doubt, it should probably have its home directory somewhere like /usr/local/couchdb2.

Created attachment 186144 [details]
v14 - patch - move files to %%PREFIX%%/couchdb2

Sorry for the late update, Ive been unavailable for the last month or so.

As per @Richard's recommendations I have:
- Moved files from %%PREFIX%%/lib/couchdb2 to %%PREFIX%%/couchdb2.
- Removed hardcoded paths in the rc script in favour of templates

@Richard
I have tried to seperate the couchdb files out so that it runs from the appropriate directories, but have not gotten it to run.
Further when speaking with the couchdb devs about this they recommend it stays altogether.

I am not sure how this port is linking to ssl. Any help here would be appreciated

Created attachment 186145 [details]
v14 - poudriere log

Created attachment 193529 [details]
v15 - patch - couchdb 2.1.1 port

Here are some improvements

1. installs couchdb version 2.1.1
2. installs into /usr/local/libexec/couchdb2
3. builds most of pkg-plist dynamically so it works with minor OPT version changes and other libs version changes like mochiweb
4. adds some configuration options

Created attachment 193540 [details]
v16 - patch couchdb 2.1.1 port - portlint happy

same as v15 but portlint is happy

Is a complete Port of CouchDB 2.1.1 in the making?

Comment 31 Mathieu Arnold 2018-06-14 15:38:59 UTC

This needs a bit more cleanup, which can be done by the committer taking it on.

Many bits are in the wrong order, See Chapter 15. Order of Variables in Port Makefiles, https://www.freebsd.org/doc/en/books/porters-handbook/porting-order.html from the porter's handbook.

In BUILD_DEPENDS the ${LOCALBASE}/bin/ should be removed.

PLIST should not be changed from its default, and all the `find >> ${PLIST}` should be changed to `find >> ${TMPPLIST}`.

bsd.port.options.mk should not be included, all the following .if should use options helpers instead. http://www.freebsd.org/doc/en/books/porters-handbook/makefile-options.html

pre-install and do-install should be merged.  Files MUST NOT be installed with MV or CP, they MUST be installed with one of the INSTALL_* macro or using the COPYTREE_* macros. https://www.freebsd.org/doc/en/books/porters-handbook/install.html

The man page MUST always be installed, and not be dependant on a DOCS option.

Comment 32 Mathieu Arnold 2018-06-14 15:40:31 UTC

Oh, and in pkg-plist, `@dir %%ETCDIR%%` should be removed. (Or better, the plist should be generated using make makeplist, and not be made dynamic.)

Comment 33 Larry Rosenman 2018-06-14 16:03:21 UTC

Take.  I'll work on this in the next few days.

Comment 34 Kurt Jaeger 2018-06-14 18:10:11 UTC

Created attachment 194254 [details]
patch-v17

New version, hopefully covering all comments.

Testbuilds on 12a, 11.2a, 11.1a, 10.4i are fine.

Comment 35 Kurt Jaeger 2018-06-14 18:27:18 UTC

Take, in coordination with ler. v17 looks close enough to be almost ready.

Comment 36 Kurt Jaeger 2018-06-14 18:35:38 UTC

Is there a good reason to support both erlang runtimes ? This causes issues with the pkg-plist. It would be much easier if we stay with erlang-runtime20, only.

Comment 37 Kurt Jaeger 2018-06-14 19:35:35 UTC

Created attachment 194257 [details]
patch-v18

Tested most options. erlang19 is still broken because the pkg-plist is wrong if 19 is selected.

I'm unsure about conflicts with couchdb1 -- any hints ?

Comment 38 Dave Cottlehuber 2018-06-14 20:15:40 UTC

*** Bug 212846 has been marked as a duplicate of this bug. ***

Comment 39 Kurt Jaeger 2018-06-14 20:17:31 UTC

Created attachment 194259 [details]
patch-v19

So, that should be the final version.

Comment 40 Dave Cottlehuber 2018-06-14 21:21:07 UTC

some comments; super delighted to see this moving again & LGTM.

- there is no direct "copy the files" migration from couchdb 1.x -> 2.x so ideally you'd be able to run both versions side by side. It looks like this is sorted - nice!

- I see no reason at all to build against OTP 20 as it's not supported upstream, and the build produces some very reasonable errors in the multipart mime parser,  and http session handling:

/tmp/usr/ports/databases/couchdb2/work/apache-couchdb-2.1.1/src/mochiweb/src/mochiweb_multipart.erl:41: Warning: call to crypto:rand_bytes/1 will fail, since it was removed in 20.0; use crypto:strong_rand_bytes/1
/tmp/usr/ports/databases/couchdb2/work/apache-couchdb-2.1.1/src/mochiweb/src/mochiweb_session.erl:125: Warning: call to crypto:rand_bytes/1 will fail, since it was removed in 20.0; use crypto:strong_rand_bytes/1
/tmp/usr/ports/databases/couchdb2/work/apache-couchdb-2.1.1/src/couch_epi/src/couch_epi_codegen.erl:78: Warning: erl_scan:token_info/2: removed in 19.0; use erl_scan:{category,column,line,location,symbol,text}/1 instead

- fauxton doesn't need to be a build time option, it's only required if you intend to download and build the nodejs UI from scratch, and this is already provided by couchdb release tarball. Any production couch instance should have fauxton ui available.

- curl is not a production requirement, only to run the javascript test suite. the couchjs binary should not have this included in production as it might allow a js function to use it to escape its sandbox

The daemon doesn't start for me, looks like too aggressive pruning of executable permissions. This should fix it:

chmod +x /usr/local/libexec/couchdb2/erts-*/bin/* \
  /usr/local/libexec/couchdb2/lib/*/priv/bin/* \
  /usr/local/libexec/couchdb2/bin/* 

But there are a few others inside ERTS as well I may have missed.

Comment 41 Kurt Jaeger 2018-06-15 03:38:33 UTC

(In reply to Dave Cottlehuber from comment #40)
Thanks for your comments and testing.

I've tested the startup now and find that it looks like the database
is not starting at all, as the rc.d script looks for

pgrep beam

but that process is not started. Maybe it's because there's no
default.ini file ? Those are only installed if OTP20 is selected (?).

It seems there's still some open issue somehow ?

Comment 42 Kurt Jaeger 2018-06-15 08:51:57 UTC

Building it with OTP20 allows the beam* processes to run, but:
if I then proceed as explained in the pkg-message, the curl calls work.
Proceeding as described in

http://docs.couchdb.org/en/2.1.1/install/setup.html#single-node-setup

to the URL

http://127.0.0.1:5984/_utils#setup

it works until a point and some errors pop up. I have no idea about couchdb2,
so if anyone has more experience... ?

"make install" is already broken again. At least for OTP20. Same will happen with OTP19.

Cause is the *static* pkg plist. 

Couchdb2 builds as an erlang application package which self-contains the complete OTP which is taken from the currently installed erlang-runtime20-20.x.y.z. For example pkg-plist wants to copy files from "erts-9.3.2" but OPT20 is already at "erts-9.3.3".

So the port will break with each minor OTP update.

Comment 44 Kurt Jaeger 2018-06-29 12:54:41 UTC

Ok, now I understand your comment about the dynamic plist. We still have
issues with OTP20 not working completly and OTP19 not working at all, no ?

Comment 45 Dave Cottlehuber 2018-06-29 20:39:28 UTC

I'm unable to create users in the OTP20 version, does this work for anybody else?

curl -X PUT http://admin:passwd@localhost:5984/_users/org.couchdb.user:beastie -H "Accept: application/json" -H "Content-Type: application/json" -d '{"name": "beastie", "password": "whatiscoredoingaboutit", "roles": [], "type": "user"}'

BTW wearing my "upstream" hat we should be fine with OTP20 and the 3 small patches, but it's a bit odd nobody else has mentioned/noticed this already; I'll not get to submitting them until monday earliest though as I borked my ports computer.

Neither OTP19 nor OTP20 works.
As already mentioned OTP20 fails at "make install" with file not found erros. pkg-plist points to "%%OTP20%%libexec/couchdb2/erts-9.3.2/" but the "make" process copies the erts directory "from /usr/local/lib/erlang20/erts-9.3.3/" which is installed when using the latest ports tree with svn update. Btw. "pkg install erlang-runtime20" installs erts currently in "/usr/local/lib/erlang20/erts-9.3/"

So I think it is impossible to build an always working port for couchdb app with self-contained erts when using a static pkg-plist.

For OTP19 "make install" works, because pkg-plist still matches the erlang runtime system in "/usr/local/lib/erlang19/erts-8.3.5.4/".

But you can't start it (service start couchdb2) because none of the installed files have +x permission. Reason is the use of ${COPYTREE_SHARE} instead of ${CP}.

@Dave: Yes, I can create users with OTP20 without any problems. (provided that I've got a working couchdb2 installation like from patch v16)

There is an option "excl_lib" in erlang reltool (http://erlang.org/doc/man/reltool.html), but it's marked experimental.
Setting excl_lib to otp_root in work/.../rel/reltool.config prevents embedding of OTP. Also some modifications in bin/couchdb are needed to get it to work. So this would be a possibility to stick on a static plist, because OTP hast not to be copied at "make install".
But this feature is experimental and not the way couchdb2 should be built. 
At most this could be offered as experimental feature in "make config" which  could be selected to save some disk space.

Comment 48 Kubilay Kocak 2018-08-13 02:40:54 UTC

Request feedback from MAINTAINER listed in latest patch.

If someone else wants to progress this, please provide an updated patch addressing any remaining issues (and updating the MAINTAINER line if needed)

Comment 49 Dave Cottlehuber 2018-08-20 21:07:07 UTC

Comment on attachment 194259 [details]
patch-v19

I moved discussion to https://reviews.freebsd.org/D16819 with proposed patch
that complies with expected permissions, file locations, allows running couch 
1.7.x and 2.x concurrently to ease migration, and depends on the OTP files from
lang/erlang-runtime* to make port maintenance easier. It's missing the ability
to split out the HTML docs, as previous ports removed fauxton (IMO required),and also some of the javascripts needed for views.

I'll look at adding them back, and probably removing OTP19 support as I see no
real-world use case for it.

comments welcomed & many thanks for everybody's contributions.

Comment 50 commit-hook 2018-09-21 15:21:31 UTC

A commit references this bug:

Author: dch
Date: Fri Sep 21 15:21:02 UTC 2018
New revision: 480279
URL: https://svnweb.freebsd.org/changeset/ports/480279

Log:
  databases/couchdb2: new port via PR#218844

  Summary: JSON document database with HTTP API & scalable multi-master sync

  A number of people provided signficant assistance while developing this
  port. Thank-you for your perseverance and assistance.

  PR:		 218844
  Submitted by:	dch
  Reviewed by:	Reshad Patuck, Richard Gallamore, Tobias Herre, mat, ler, pi
  Approved by:	jrm (mentor)
  Differential Revision:	https://reviews.freebsd.org/D16819

Changes:
  head/databases/Makefile
  head/databases/couchdb2/
  head/databases/couchdb2/Makefile
  head/databases/couchdb2/distinfo
  head/databases/couchdb2/files/
  head/databases/couchdb2/files/couchdb2.in
  head/databases/couchdb2/files/patch-configure
  head/databases/couchdb2/files/patch-rel_reltool.config
  head/databases/couchdb2/files/patch-src_ibrowse_src_ibrowse__http__client.erl
  head/databases/couchdb2/files/pkg-message.in
  head/databases/couchdb2/pkg-descr
  head/databases/couchdb2/pkg-plist