Created attachment 148379 [details] glusterfs port shar port for GlusterFS (gluster.org) GlusterFS is a unified, poly-protocol, scale-out filesystem serving many PBs of data. I originally lifted this from the FreeNAS team and updated to the latest upstream release. I am also looking to maintain the FreeNAS port as well to stop double touching the same codebase. Credit and hats off to Baptiste Daroussin <bapt@freebsd.org> and the FreeNAS team for the original work. I have emailed Baptiste regarding maintainership, happy for me to go ahead. Regards Craig Butler
I will handle this.
Created attachment 149233 [details] updated glusterfs shar bump PR, and sync with latest improvements from github as suggested by Harshavardhana <harsha@harshavardhana.net> with thanks.
Created attachment 149791 [details] Commit candidate of GlusterFS port Hi, I reviewed the patch and fixed several issues of them. Can you try the attached one? Changes include: - Fix package name. beta3 is not allowed in the version string. - Add more dependency. GlusterFS depends on glib-2.0, gettext, and readline. - Use libargp from ports as dependency. - Use USES=python instead of USE_PYTHON. - Fix Python.h detection. - Add etc/rc.d/glusterd startup script. Just putting the line glusterd_enable="YES" into /etc/rc.conf will enable a GlusterFS server. I tried some simple configurations and they worked fine so far. I am double-checking the functionality by using more complex ones now. A bug I noticed at this moment was that NFS export did not unregister RPC entries of mountd, nfs, and nfs_acl upon shutdown. One question: I removed autotools because the stock configure scripts worked without a problem. Is there any speicific reason to rebuild them?
http://bsdtec.com/freebsd/poudriere/logs/bulk/latest-per-pkg/glusterfs/3.6.1/ Summary; 10.1-RELEASE amd64 -- pass 10.1-RELEASE x86 -- fail 10.1-RELEASE sparc64 -- fail 9.3-RELEASE amd64 -- fail 9.3-RELEASE x86 -- fail
Created attachment 153091 [details] unified diff to bring port candidate upto glusterfs 3.6.2 updated Makefile, pkg-plist and distinfo to reflect newer glusterfs release.. modified patch-configure as some changes have made it upstream
http://bsdtec.com/freebsd/poudriere/logs/bulk/latest-per-pkg/glusterfs/3.6.2/ Summary; 10.1-RELEASE amd64 -- pass testing continues...
GlusterFS 3.6.2 port candidate builds fine on FreeBSD 10.1-RELEASE amd64. However the GlusterFS daemon (glusterd) and management tool (gluster) crash at startup because parts of the Python code rely on the ENODATA syscall error return code (errno), which FreeBSD does not implement. References to ENODATA should be replaced by ENOMSG. Detailed debug log of glusterd startup is available on the FreeBSD forums: https://forums.freebsd.org/threads/freebsd-support-being-added-to-glusterfs.46923/page-2#post-282793
Something broke between beta3 and 3.6.2 (and 3.6.1)... after contacting Harsha and Justin I was advised to open up a bug report on their redhat bugzilla; https://bugzilla.redhat.com/show_bug.cgi?id=1195053
Created attachment 153404 [details] unified diff to bring port candidate upto gluster 3.6.2 with georeplication disabled disable georeplication until portability issues have been fixed upstream.
It seems that an user has no privileges to access the glusterfs mount point: --------------------------------------------------------------------------------- root@aldi-nas:/root/#ls -la /dev/fuse crw-rw-rw- 1 root operator 0x7d Mar 26 10:26 /dev/fuse root@aldi-nas:/root/#su -m nobody -c 'ls -la /mnt/gv0' total 2 drwxr-xr-x 2 root wheel 2 Mar 25 09:17 . drwxr-xr-x 11 root wheel 12 Mar 25 09:17 .. root@aldi-nas:/root/#mount_glusterfs 10.0.0.134:/gv0 /mnt/gv0 mountpoint: /mnt/gv0 fsname: 10.0.0.134:/gv0 mnt_param: default_permissions,allow_other,max_read=131072 root@aldi-nas:/root/#su -m nobody -c 'ls -la /mnt/gv0' total 0 ls: /mnt/gv0: Operation not permitted root@aldi-nas:/mnt/#ls -la /mnt/gv0 total 5 drwxrwxrwx 5 root wheel 5 Oct 21 08:00 . drwxr-xr-x 11 root wheel 12 Mar 25 09:17 .. drwxrwxrwx 3 nobody wheel 40 Mar 17 14:14 4all drwxr-xr-x 12 root wheel 12 Mar 11 14:42 homes --------------------------------------------------------------------------------- Any ideas how to fix that? '/mnt/gv0' will be a storage area for SAMBA, and the share "4all" is being guest accessed as the UNIX user "nobody"...
Hello Nils I am working on the permissions issue now. I can confirm that I am experiencing the same. I am currently rolling back to beta3 to see if it is a code change that has caused it. Report back soon Regards Craig Butler
strange... I can get freebsd bricks and centos client to work. but freebsd bricks and freebsd client fails.
working on getting 3.7.2 diff.... some new lib depends, and needs to be built with GCC for now.
Created attachment 158002 [details] unified diff to bring port candidate upto gluster 3.7.2 with georeplication disabled
unfortunately 3.7.2 is still not letting users access the mount. I'll open another PR for this and hopefully get some more eye's on the case.
(In reply to craig001 from comment #15) Strictly speaking, it is not a problem of GlusterFS itself, but different behaviour of FreeBSD and Linux implementation of FUSE: by default, Linux adds 'allow_other' mount option to FUSE mounts while FreeBSD does not. So, it is rather simple to spin things around: just add missing mount option. :) diff --git a/contrib/fuse-lib/mount.c b/contrib/fuse-lib/mount.c index 1edde86..521a571 100644 --- a/contrib/fuse-lib/mount.c +++ b/contrib/fuse-lib/mount.c @@ -256,6 +256,7 @@ fuse_mount_sys (const char *mountpoint, char *fsname, build_iovec (&iov, &iovlen, "from", "/dev/fuse", -1); build_iovec (&iov, &iovlen, "volname", source, -1); build_iovec (&iov, &iovlen, "fd", fdstr, -1); + build_iovec (&iov, &iovlen, "allow_other", NULL, -1); ret = nmount (iov, iovlen, mountflags); #else ret = mount (source, mountpoint, fstype, mountflags, That's all, I run GlusterFS on FreeBSD for a quite time.
thank you Andrew !! I'll bump the revision upto 3.7.6 (the latest) and push your patch in.
Created attachment 163544 [details] unified diff to bring port candidate upto gluster 3.7.6 with georeplication disabled and fuse mount patched bump upto glusterfs 3.6.7, add patch as submitted by andrew@rinet.ru
Created attachment 163545 [details] glusterfs 3.7.6 port tarball
will run this through with poudriere shortly when I have upgraded the platform to 10.2-RELEASE
Port build fails on new 10.2-RELEASE: checking for GLIB... yes checking for ibv_get_device_list in -libverbs... no checking for rdma_create_id in -lrdmacm... no checking for ZLIB... yes features requiring zlib enabled: yes checking for SQLITE... no configure: error: pass --disable-tiering to build without sqlite ===> Script "configure" failed unexpectedly. Please report the problem to craig001@lerwick.hopto.org [maintainer] and attach the "/usr/ports/net/glusterfs/work/glusterfs-3.7.6/config.log" including the output of the failure of your make command. Also, it might be a good idea to provide an overview of all packages installed on your system (e.g. a /usr/local/sbin/pkg-static info -g -Ea). *** Error code 1 Stop. make: stopped in /usr/ports/net/glusterfs If I install sqlite3 manually the glusterfs build works.
Created attachment 163805 [details] patch make the port 'poudriere testport' clean Attached port (glusterfs.tgz) does't pass 'poudriere testport'. I fixed this this issue. changes made are: - Added 'sysutils' to CATEGORIES I don't know which is correct, sysutils/glusterfs or net/glusterfs - Use shebangfix for some scripts - Added USE_SQLITE=yes - Fixed version number in PLIST_SUB and remove @commnet from pkg-plist
Created attachment 163811 [details] patch make the port 'poudriere testport' clan and freebsd 9 build support patch updated. add support for freebsd 9. patched port is tested on 10.2-RELEASE amd64 and 9.3-RELEASE amd64.
Excellent. Builds now on 10.2-RELENG and glusterfsd starts on its own with the default install. The original 3.7.6 patch did not.
thanks for the effort folks... I'll check KIKUCHI's diff now. I think it was decided by Kiroki that the port was to fall into net/ and not as originally sysutils/. I think that may be the only thing I end up changing.
Created attachment 163830 [details] unified diff to bring port candidate upto gluster 3.7.6 with georeplication disabled and fuse mount patched applied KIKUCHI Koichiro diff
Created attachment 163831 [details] glusterfs 3.7.6 port tarball applied KIKUCHI Koichiro diff
created phabricator differential review; https://reviews.freebsd.org/D4367 (hopefully correctly and assigned to the right folk, sorry if it's not...)
Thx merging my patch. then I have one question. I use binaries built with clang on my FreeBSD 10 machines and they seem to work fine. So I think we can remove USE_GCC=any line from Makefile. Is there any specific reason to use gcc?
Hello KIKUCHI I suspect it's legacy issue that may now have been resolved. I'll keep it GCC for now and get it into the ports tree then sort it out. I am currently running it through a local poudriere here atm with GCC enabled. Will try with clang after. My biggest worry about clang is that it has not yet supported all archs that I am hoping to get gluster running on.
http://bsdtec.com/freebsd/poudriere/logs/bulk/latest-per-pkg/glusterfs/3.7.6/ Summary; 10.2-RELEASE amd64 -- pass testing continues...
http://bsdtec.com/freebsd/poudriere/logs/bulk/latest-per-pkg/glusterfs/3.7.6/ Summary; 10.2-RELEASE amd64 -- pass 9.3-RELEASE amd64 -- pass
Can confirm that on 10.2-RELENG AMD64 in addition to building and glusterfsd running, we can mount shares via fusefs from a remote 10.2-RELENG AMD64 client.
Is there a reason why this isn't in FreeBSD ports, yet? There's lot's of "experimental"-type ports marked as such in the tree that might or might not deserve actually being in ports. If it was in the tree, I (and probably others) could just build it as part of my regular (quarterly) build and play with it.
I just tried the glusterfs 3.7.6 port tarball (2015-12-04) and it all seemed to work. (I used an end of November FreeBSD11 on an amd64.) The only issue I ran into is that glusterd wouldn't start because there was no /usr/local/etc/glusterfs/glusterd.vol file. I did this trivial patch to the glusterd script for /etc/rc.d, which is one simple way to fix it: --- glusterd.sav 2015-12-09 16:09:50.226225000 -0500 +++ glusterd 2015-12-09 16:14:20.116241000 -0500 @@ -16,6 +16,11 @@ glusterd_prestart() { mkdir -p /var/db/glusterd /var/log/glusterfs + if ! [ -f /usr/local/etc/glusterfs/glusterd.vol ]; then + cp -n /usr/local/etc/glusterfs/glusterd.vol.sample \ + /usr/local/etc/glusterfs/glusterd.vol + echo "Copying sample /usr/local/etc/glusterfs/glusterd.vol" + fi } load_rc_config $name Thanks everyone for working on this, rick.
Is there a holdup on getting this committed? This has been functional for almost 2 months now.
Hiroki -- is there anything else we need to do to get this committed ??
Just fyi, if you are going to use the fuse interface, you probably want the 2 patches found in PR#206238. And, yes, I do feel this can go in ports now and would appreciate that happening. Thanks everyone for your work on this, rick
This has been sitting idle for 3 weeks. Any objections to committing it to ports?
hay Mark... Is there such thing as committer time out ?? Hiroki had claimed it, but has been radio silent via bugzilla and offlist emails. Is there someone else with a commit bit that can get it in for us ?
(In reply to craig001 from comment #40) I am very sorry for not handling this in an appropriate manner. I will commit it as soon as possible this time around...
A commit references this bug: Author: hrs Date: Wed Feb 24 19:44:46 UTC 2016 New revision: 409479 URL: https://svnweb.freebsd.org/changeset/ports/409479 Log: Add net/glusterfs, an open source, distributed file system capable of scaling to several petabytes and handling thousands of clients. Submitted by: Craig Butler (maintainer) PR: 194409 Changes: head/net/Makefile head/net/glusterfs/ head/net/glusterfs/Makefile head/net/glusterfs/distinfo head/net/glusterfs/files/ head/net/glusterfs/files/glusterd.in head/net/glusterfs/files/patch-configure head/net/glusterfs/files/patch-contrib-fuse-lib-mount.c head/net/glusterfs/files/patch-extras-Makefile.in head/net/glusterfs/pkg-descr head/net/glusterfs/pkg-plist
Committed. Thank you all for your contributions, and I am sorry for the looong delay. Please reopen this or open another PR if there is a problem.