Created attachment 162014 [details] s3ql.shar I ported this together with devel/py-dugong and devel/llfuse. --- S3QL is a file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X. S3QL is a standard conforming, full featured UNIX file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees and snapshotting which make it especially suitable for online backup and archival. S3QL is designed to favor simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection and error handling have been included from the very first line, and S3QL comes with extensive automated test cases for all its components. WWW: https://bitbucket.org/nikratio/s3ql/
Don't forget to set the "Depends on" and "Blocks" fields to point to the correct dependent issues :) Also, COMMENT is too long (potlint should pick this issue up) As per my comment in Bug 203759, please run this port through QA tool (portlint/poudriere) and attach results
(In reply to Kubilay Kocak from comment #1) I am amazed by "Depends on" and "Blocks". Really cool feature. :) I've finished updating the dependencies and will update this submission most probably next week.
In the meantime (while you update this submission) - please obsolete the existing attachment 162014 [details] so that it doesnt get accidentally committed as is :)
(In reply to Kubilay Kocak from comment #3) Of course, makes sense. :) Thanks for your patience and introduction concerning the workflow here.
Created attachment 162204 [details] Shar file
Created attachment 162205 [details] Output of `poudriere testport` Looks fine; despite 9 compiler warnings that I will submit upstream. --- cc -DNDEBUG -O2 -pipe -fstack-protector -fno-strict-aliasing -I/usr/local/include -fPIC -I/usr/local/include/python3.4m -c src/s3ql/deltadump.c -o build/temp.freebsd-10.1-RELEASE-p22-amd64-3.4/src/s3ql/deltadump.o -Wall -Wextra -Wconversion -Wsign-compare -Werror=conversion -Werror=sign-compare src/s3ql/deltadump.c:11414:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function] static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { ^ src/s3ql/deltadump.c:9065:32: warning: unused function '__Pyx_GetItemInt_List_Fast' [-Wunused-function] static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, ^ src/s3ql/deltadump.c:9080:32: warning: unused function '__Pyx_GetItemInt_Tuple_Fast' [-Wunused-function] static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, ^ src/s3ql/deltadump.c:9293:26: warning: unused function '__Pyx_PyBytes_Equals' [-Wunused-function] static CYTHON_INLINE int __Pyx_PyBytes_Equals(PyObject* s1, PyObject* s2, int equals) { ^ src/s3ql/deltadump.c:9992:28: warning: unused function '__Pyx_CyFunction_InitDefaults' [-Wunused-function] static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) { ^ src/s3ql/deltadump.c:10001:27: warning: unused function '__Pyx_CyFunction_SetDefaultsTuple' [-Wunused-function] static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) { ^ src/s3ql/deltadump.c:10006:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function] static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) { ^ src/s3ql/deltadump.c:10011:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function] static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) { ^ src/s3ql/deltadump.c:11104:27: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration] static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { ^ 9 warnings generated.
Hi, Any update on this?
Hi Martin, concerning the warnings? Please see the submission of the warnings upstream here: https://bitbucket.org/nikratio/s3ql/issues/168/cc-compiler-warnings-on-freebsd I am in contact with the maintainer. He is in need of a FreeBSD built environment to work on the code. I actually want to offer him such an environment in form of a jail on one of my machines. Because of other projects, I haven't had time to do so yet but I am still keen to put this forward. As far as I can remember, the port worked fine despite the warnings. I will double-check that and work on the warnings (together with the maintainer) at the end of January. Unfortunately, I won't find time for that before. Best, Niklaas
Hi Niklaas, The warnings are ok to me, but the fail of FreeBSD 9.X is not ok, either we mark this port broken for 9X and bring it in, or we wait till there is a fix for 9.X. - Martin
(In reply to Martin Wilke from comment #9) Hi Martin, ah, I'm sorry, I didn't know about the fail of 9.X. It's one of the first ports that I committed. Probably I've overlooked something here. Where can I have a look at the errors? I don't mind whether to bring it in or not. Anyway I would take a closer look on it again in January/February and see that it's working on 9.X too. Unfortunately, I am too busy at the moment. Maybe it makes sense to bring it in now to hope for some further feedback? I could address this in January/February too. But if it's common to wait until a port works on every system before bringing it in, we should stick to that. Best, Niklaas
Hi Martin, probably my comment got lost over Christmas days. Have you had a closer look on this? What's exactly causing the port to fail on 9.x? Best, Niklaas
back to pool
testbuilds@work
Testbuilds all somehow depend on py2.7 and fail, see http://people.freebsd.org/~pi/logs/net__s3ql-10a Yes, USES=python:3.3+ is set in the Makefile. Any ideas what's going wrong ?
Kurt, thanks for taking up on this. The last time I tested this was months ago, so I might be wrong about the following. I'll test it as soon as I can but maybe this will point towards the correct direction. As far as I can remember, poudriere testport cannot cope with the different python version/dependency on its own. I think someone explained this to me on #bsdports, but I am not sure. That's why I had to put DEFAULT_VERSIONS=python=3.4 python3=3.4 in make.conf that poudriere uses to build net/s3ql. When I was told that I was wondering how then it will be possible to build the package later without manual intervention in make.conf. But I assumed that it's just *testport* that cannot cope with the version/dependency. In addition, as far as I can remember, it built fine with bulk (otherwise it wouldn't be here, I guess. :) I'll try to check the port again as soon as I can and keep you updated. Maybe this is already some help? It will be great if we get it committed eventually.
Btw, I tested 2.17.1 (the most recent version). Yes, using DEFAULT_VERSIONS=python=3.4 python3=3.4 in make.conf works, my question is: Is there a way to enforce it with the make.conf settings. From https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=207715#c2 I take it that this 3.x enforcing can't be done without seperate py34-xxx ports.
(In reply to Kurt Jaeger from comment #16) So there is no way/reason to commit this until there are py34-* ports?
I ping'ed koobs and wait for his feedback.
To build a port and it's dependencies using python 3.5 when 2.7 is the default you can adjust the defaults in make.conf or you can use setenv PYTHON_VERSION 3.5; make This currently fails in poudriere as each port is built by itself, so they don't inherit the PYTHON_VERSION from the port triggering the dependency. There is work being done to remedy this - https://www.freebsd.org/news/status/report-2015-10-2015-12.html#Supporting-Variants-in-the-Ports-Framework Personally I use multiple poudriere sets (-x setname) to build python modules, each set name gets a py3x suffix and has it's own mypkgpy3x-make.conf with appropriate defaults set. My config for pkg contains the multiple pkg repo locations and when I pkg install devel/py-tox it finds and installs the packages for each python version. Note that to get python 3.x packages that will install on a system that has 2.7 as default (which is having scripts in bin installed correctly) you need to set - DEFAULT_VERSIONS= python=2.7 python2=2.7 python3=3.5 .if ${.CURDIR:M*/py*} PYTHON_VERSION=3.5 .endif Not every port builds if PYTHON_VERSION is set globally (for lack of py3 support). I have been getting better results with the CURDIR test. Obviously not a solution for the public pkg repos. Although I don't know if we would have any luck getting extra repos setup for multiple python versions. Or syncing py3x packages from another build into the public repo... Until the variant work is completed you may have to consider making users manually install from ports. That could be achieved by adding "NO_PACKAGE=Requires python 3.x to be set as default" - if you have that within an .if ${PYTHON_MAJOR_VER} == 3 then it will still package if the defaults have been set.
(In reply to FreeBSD from comment #19) Thanks for the detailed outline. I'll try to implement and testport the suggestions within the next days and update the shar file correspondingly. This way, at least, the port can make it into the port system.
Created attachment 171384 [details] Shar file for version 2.18 Please excuse, I was occupied with other projects. I attached a new version of the shar file that includes the recommendation concerning NO_PKG (thank you!) and also ports a newer version (2.18 instead of 2.15).
Created attachment 171385 [details] testport log for version 2.18
Created attachment 181390 [details] s3ql.shar version 2.21 This is in progress for some while now. I haven't looked into this for a long time too, but really would like to get the port added to the ports tree. The new shar file is for the most recent version of s3ql: 2.21. Portlint is fine despite some complaints: WARN: Makefile: no ftp/http mirror in MASTER_SITES for users behind a proxy. WARN: Makefile: wrong dependency value for BUILD_DEPENDS. BUILD_DEPENDS requires 2 or 3 colon-separated tuples. WARN: Makefile: wrong dependency value for RUN_DEPENDS. RUN_DEPENDS requires 2 or 3 colon-separated tuples. WARN: Makefile: extra item placed in the *_DEPENDS section, for example, "MY_DEPENDS". 0 fatal errors and 4 warnings found. This should be okay, I guess. I have a bug report running upstream (https://bitbucket.org/nikratio/s3ql/issues/254/chmod-corrupts-file-system). Nevertheless, pushing this in the ports tree maybe eases checking whether this is a general issue on FreeBSD.
A commit references this bug: Author: swills Date: Fri Apr 14 18:58:45 UTC 2017 New revision: 438541 URL: https://svnweb.freebsd.org/changeset/ports/438541 Log: net/s3ql: create port S3QL is a file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X. WWW: https://bitbucket.org/nikratio/s3ql PR: 203760 Submitted by: Niklaas Baudet von Gersdorff <stdin@niklaas.eu> (with changes) Changes: head/net/Makefile head/net/s3ql/ head/net/s3ql/Makefile head/net/s3ql/distinfo head/net/s3ql/pkg-descr
Committed, thanks!