Bug 226377 - [NEW PORT] databases/cassandra3: Highly scalable distributed database
Summary: [NEW PORT] databases/cassandra3: Highly scalable distributed database
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Yuri Victorovich
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-03-05 20:57 UTC by Angelo Polo
Modified: 2019-01-28 06:12 UTC (History)
4 users (show)

See Also:


Attachments
New port databases/cassandra3 shar archive (29.11 KB, text/plain)
2018-03-05 20:57 UTC, Angelo Polo
no flags Details
Updated shar archive (30.64 KB, text/plain)
2018-03-08 02:20 UTC, Angelo Polo
no flags Details
Updated shar with rc script (31.85 KB, text/plain)
2018-03-14 18:33 UTC, Angelo Polo
no flags Details
Updated shar (36.07 KB, text/plain)
2018-07-17 15:44 UTC, Angelo Polo
no flags Details
Updated shar (36.07 KB, text/plain)
2018-08-29 22:04 UTC, Angelo Polo
no flags Details
GIDs diff taking ID 236 (45 bytes, patch)
2018-08-29 22:06 UTC, Angelo Polo
no flags Details | Diff
UIDs diff taking ID 236 (106 bytes, patch)
2018-08-29 22:06 UTC, Angelo Polo
no flags Details | Diff
Updated shar (39.06 KB, text/plain)
2018-09-07 23:11 UTC, Angelo Polo
no flags Details
Updated shar (38.95 KB, text/plain)
2018-09-24 19:59 UTC, Angelo Polo
no flags Details
Updated shar (38.94 KB, text/plain)
2018-10-08 23:06 UTC, Angelo Polo
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Angelo Polo 2018-03-05 20:57:04 UTC
Created attachment 191234 [details]
New port databases/cassandra3 shar archive

New port for Cassandra 3.11.2

Apache Cassandra is a free and open-source distributed database.
It is a Java-based, highly scalable, eventually consistent, wide
column store.
Comment 1 Yuri Victorovich freebsd_committer 2018-03-06 05:36:29 UTC
For the record, cassandra has been in the tree, 2 versions have been deleted before:
> $ grep cassandra MOVED
> databases/cassandra||2016-03-29|Has expired: 1.2.x has reached EOL, upgrade to a supported version of Cassandra
> databases/cassandra2||2017-11-12|Has expired: Broken for more than 6 months
Comment 2 Angelo Polo 2018-03-08 02:20:53 UTC
Created attachment 191294 [details]
Updated shar archive

Updated shar archive fixes external lib issues.
Comment 3 Yuri Victorovich freebsd_committer 2018-03-10 07:07:39 UTC
Hi Angelo,

cassandra2 had the rc script (in files/cassandra.in).
Could you please add one too?

Thanks!
Yuri
Comment 4 Angelo Polo 2018-03-14 18:33:07 UTC
Created attachment 191502 [details]
Updated shar with rc script

Yuri,

Please see the new shar, including rc script.

Angelo
Comment 5 Yuri Victorovich freebsd_committer 2018-03-14 20:26:00 UTC
(In reply to Angelo Polo from comment #4)

Thank you Angelo,

Yuri
Comment 6 Sergey Akhmatov 2018-06-25 14:38:53 UTC
I've tried the port, unfortunately it fails to build in poudriere:

maven-ant-tasks-download:
     [echo] Downloading Maven ANT Tasks...
      [get] Getting: http://repo2.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.3/maven-ant-tasks-2.1.3.jar
      [get] To: /wrkdirs/usr/ports/kdp/cassandra3/work/apache-cassandra-3.11.2-src/build/maven-ant-tasks-2.1.3.jar
      [get] Error getting http://repo2.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.3/maven-ant-tasks-2.1.3.jar to /wrkdirs/usr/ports/kdp/cassandra3/work/apache-cassandra-3.11.2-src/build/maven-ant-tasks-2.1.3.jar

BUILD FAILED
/wrkdirs/usr/ports/kdp/cassandra3/work/apache-cassandra-3.11.2-src/build.xml:296: java.net.SocketException: Permission denied (connect failed)


AFAIK ports should NEVER download anything during build process. All downloads must be happening on fetch stage and be checked with distfiles.
We should either do all downloads before build, or consider using tarball with prebuild binary distribution which is available on official download page.
Comment 7 Angelo Polo 2018-06-26 13:27:01 UTC
Thanks Sergey,

Taking a look into it.

Angelo
Comment 8 Angelo Polo 2018-07-17 15:44:22 UTC
Created attachment 195202 [details]
Updated shar

Uses a pre-assembled Maven repository. Removed duplicate copies of tools/bin scripts from staging.
Comment 9 Yuri Victorovich freebsd_committer 2018-07-17 17:24:47 UTC
Ok, thanks, I will take a look.
Comment 10 Angelo Polo 2018-08-13 22:11:49 UTC
If anyone wants to use or test the port, for now you can manually download the Maven repository "apache-cassandra-3.11.2-repo.tar.gz" to /usr/ports/distfiles from here:

https://drive.google.com/open?id=1U81megQnir68r3HbFvrTQTqTjgWNj7wg
Comment 11 Lapo Luchini 2018-08-29 15:20:29 UTC
Strangely enough I could add the file to my GDrive, but when I attempt download it stucks indefinitely or fails with a 409 error.
Comment 12 Lapo Luchini 2018-08-29 15:24:28 UTC
…and of course I tried 4 times, I wrote that comment, then I tried again "for the sake of it" and it finally worked perfectly.
Comment 13 Lapo Luchini 2018-08-29 16:15:17 UTC
BTW: the rc.d uses a "cassandra" user by default, but it's not created during install.
Comment 14 Angelo Polo 2018-08-29 22:04:33 UTC
Created attachment 196689 [details]
Updated shar
Comment 15 Angelo Polo 2018-08-29 22:06:04 UTC
Created attachment 196690 [details]
GIDs diff taking ID 236
Comment 16 Angelo Polo 2018-08-29 22:06:25 UTC
Created attachment 196691 [details]
UIDs diff taking ID 236
Comment 17 Angelo Polo 2018-08-29 22:07:52 UTC
Thanks Lapo, updated Makefile now creates the cassandra user and group during install.
Comment 18 Lapo Luchini 2018-08-30 12:51:54 UTC
Would also be nice to change "/var/lib/cassandra" to "/var/db/cassandra" (and uncomment those lines) in $PREFIX/etc/cassandra/cassandra.yaml to have the default database directory similar to other Ports. (/var/db/mysql, /var/db/mongodb, /var/db/pgsql)

Also logs should (probably) be adapted to FreeBSD hierarchy, something like /var/log/cassandra (?)
Can bechanged in $PREFIX/etc/cassandra/logback.xml

PS: sorry about trickling comments, I'm writing as I'm interested in it as a user and noticing one thing at a time.
Comment 19 Angelo Polo 2018-08-31 12:37:00 UTC
These are good ideas. I'll update this too.
Comment 20 Angelo Polo 2018-09-07 23:11:17 UTC
Created attachment 196954 [details]
Updated shar

Default data and log directories set.
Comment 21 Lapo Luchini 2018-09-14 07:41:28 UTC
Oh, nice, changing -Dcassandra.logdir is better than logback.xml as I did locally.
Only bug I can notice right now (and I consider it marginal) is that the cqlsh client doesn't seem to find readline and does not allow line editing (e.g. pressing left arrow and right arrow prints "^[[D^[[C").
I hope this gets committed soon.
Comment 22 Lapo Luchini 2018-09-21 08:44:49 UTC
Meanwhile, 3.11.3 is out. =)
Comment 23 Lapo Luchini 2018-09-24 09:28:08 UTC
PS: the readline bug I mentioned in comment 21 was produced by an old libreadline I had still in /lib/, it works perfectly.
Comment 24 Angelo Polo 2018-09-24 19:59:32 UTC
Created attachment 197462 [details]
Updated shar

Don't pack the broken ohc libs.
Comment 25 Angelo Polo 2018-09-24 20:01:09 UTC
To use the (normally default) off-heap row cache, you can build this pending port:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227392

Then link the following two files from $JAVASHAREDIR/classes into $JAVASHAREDIR/cassandra/lib:
ohc-core-0.4.4.jar
ohc-core-j8-0.4.4.jar

And set:
row_cache_class_name: org.apache.cassandra.cache.OHCProvider

When the OHC port gets committed I will update the Cassandra port to depend on it and reinstate the default row cache.
Comment 26 Angelo Polo 2018-10-08 23:06:00 UTC
Created attachment 197936 [details]
Updated shar

Update to Cassandra 3.11.3.

The updated Maven repository is available here:
https://drive.google.com/open?id=1ICXEf4D5_XBQSQz_OuJHoShb37yXA7Ms
Comment 27 Lapo Luchini 2018-10-24 14:14:23 UTC
Thanks! I'm deploying 3.11.3 and seems to be working fine.
Comment 28 Yuri Victorovich freebsd_committer 2019-01-28 06:12:27 UTC
Committed with changes.

Thank you for your submission!

Sorry again for the delay!
Comment 29 commit-hook freebsd_committer 2019-01-28 06:12:45 UTC
A commit references this bug:

Author: yuri
Date: Mon Jan 28 06:12:22 UTC 2019
New revision: 491455
URL: https://svnweb.freebsd.org/changeset/ports/491455

Log:
  New port: databases/cassandra3: Highly scalable distributed database

  Versions 1.X and 2.X existed previously, but were removed due to build issues.

  PR:		226377
  Submitted by:	Angelo Polo <language.devel@gmail.com>

Changes:
  head/GIDs
  head/UIDs
  head/databases/Makefile
  head/databases/cassandra3/
  head/databases/cassandra3/Makefile
  head/databases/cassandra3/distinfo
  head/databases/cassandra3/files/cassandra.in
  head/databases/cassandra3/files/patch-bin_cassandra
  head/databases/cassandra3/files/patch-bin_cassandra.in.sh
  head/databases/cassandra3/files/patch-build.xml
  head/databases/cassandra3/files/patch-conf_cassandra-env.sh
  head/databases/cassandra3/files/patch-conf_cassandra.yaml
  head/databases/cassandra3/files/patch-doc_Makefile
  head/databases/cassandra3/files/patch-src_java_org_apache_cassandra_config_Config.java
  head/databases/cassandra3/files/patch-src_java_org_apache_cassandra_service_StartupChecks.java
  head/databases/cassandra3/files/patch-src_java_org_apache_cassandra_utils_FastByteOperations.java
  head/databases/cassandra3/files/patch-test_conf_cassandra.yaml
  head/databases/cassandra3/files/patch-tools_bin_cassandra.in.sh
  head/databases/cassandra3/files/repaircluster.in
  head/databases/cassandra3/pkg-deinstall
  head/databases/cassandra3/pkg-descr
  head/databases/cassandra3/pkg-install
  head/databases/cassandra3/pkg-message
  head/databases/cassandra3/pkg-plist