Created attachment 260098 [details] update py-paperless-ngx to 2.14.7 I started using paperless-ngx a few weeks ago and ran into some issues/limitations that were fixed in later versions, so I spent the better part of today trying to update the port to a more recent version. In versions 2.15.0 and later, they replaced gunicorn with another webserver (granian), for which there is no port available. 2.15.3 builds fine without grainan, but the webserver.py file (amongst other parts) is missing, so it's rather useless... If someone can figure out how to get this working, I'm happy to assist/test. For now the diff updates py-paperless-ngx to 2.14.7. Versions >2.11 are using django51, so I had to create new ports ('dj51' variants) for dependencies which are using django. I'm not sure if I have to create PRs against all of those individual ports, so I put those new ports in a second diff. Some of those ports are rather outdated and might benefit from a little attention, i.e. could/should be updated to more recent versions. I didn't have the time yet to look at that and minimum requirements are fulfilled after django-allauth was updated to 65.7.0 (min. requirement by paperless 2.14.7 seems to be 65.2), so for now I just bumped the required django version for those dependencies. The port builds and runs fine for me on 14.2-RELEASE built with poudriere from quarterly and latest ports tree.
Created attachment 260099 [details] update py-paperless-ngx dependencies that need django51
Hi Sebastian, I started working on porting paperless-ngx 2.15.3 today. I already ported granian and will proceed to newer django port versions soon. When checking my Inbox, I stumbled over your contribution - just in time. I'll check it and incorporate as much as possible into what I'm doing, which hopefully reduce the amount of effort I'll have to put into it myself (especially when creating these django port variants). Thank you for your contribution! Best Michael
(In reply to Michael Gmelin from comment #2) Hi Michael, Thanks for that update and your efforts! Let me know if I can be of any assistance. I have 2 instances of paperless-ngx running, one of which isn't in production yet and can be broken (I already used this one for the incremental updates to 1.14.7). I also have two 14.2-RELEASE poudriere bulild hosts at hand for test builds. Thanks, Sebastian
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=22abe5b9370cb4704753a9e02dc0bf00c49a5f39 commit 22abe5b9370cb4704753a9e02dc0bf00c49a5f39 Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-05 14:53:51 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 14:57:11 +0000 www/py-granian: New port Granian is a Rust HTTP server for Python applications built on top of the Hyper crate. PR: 286533 www/Makefile | 1 + www/py-granian/Makefile (new) | 24 +++ www/py-granian/Makefile.crates (new) | 157 +++++++++++++++++ www/py-granian/distinfo (new) | 317 +++++++++++++++++++++++++++++++++++ www/py-granian/pkg-descr (new) | 8 + 5 files changed, 507 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=2dbbaacbadf7c775e331659abde0da139b4dcc99 commit 2dbbaacbadf7c775e331659abde0da139b4dcc99 Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 15:14:37 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 17:59:48 +0000 www/py-dj51-channels-redis: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + www/py-dj51-channels-redis/Makefile (new) | 24 ++++++++++++++++++++++++ www/py-dj51-channels-redis/distinfo (new) | 3 +++ www/py-dj51-channels-redis/pkg-descr (new) | 12 ++++++++++++ 4 files changed, 40 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=0ea179503a96ea442fcbc4eba59ad53c19b9a726 commit 0ea179503a96ea442fcbc4eba59ad53c19b9a726 Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-05 15:27:20 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:00:36 +0000 www/py-dj51-django-allauth: New port Clone of existing port for Django 5.1. Compared to py-django-allauth, this updates to the latest version and enables MFA by default. Did not want to push this onto sunpoet@, so currently I'm the maintainer. Please feel free to take the port from me without asking. PR: 286533 Co-authored-by: Sebastian <sko@rostwald.de> www/Makefile | 1 + www/py-dj51-django-allauth/Makefile (new) | 43 ++++++++++++++++++++++++++++++ www/py-dj51-django-allauth/distinfo (new) | 3 +++ www/py-dj51-django-allauth/pkg-descr (new) | 2 ++ 4 files changed, 49 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=6a335c06edc9f8218e08b79acd8b907cf8037490 commit 6a335c06edc9f8218e08b79acd8b907cf8037490 Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 15:34:18 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:01:05 +0000 www/py-dj51-django-auditlog: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + www/py-dj51-django-auditlog/Makefile (new) | 27 +++++++++++++++++++++++++++ www/py-dj51-django-auditlog/distinfo (new) | 3 +++ www/py-dj51-django-auditlog/pkg-descr (new) | 4 ++++ 4 files changed, 35 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=52b4ab33fb4796f298b7e4d5214c88b2af787a15 commit 52b4ab33fb4796f298b7e4d5214c88b2af787a15 Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 15:39:03 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:01:34 +0000 www/py-dj51-django-celery-results: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + www/py-dj51-django-celery-results/Makefile (new) | 21 +++++++++++++++++++++ www/py-dj51-django-celery-results/distinfo (new) | 3 +++ www/py-dj51-django-celery-results/pkg-descr (new) | 6 ++++++ 4 files changed, 31 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=da3cd037dc3a7b19750d0e620467cfa6ea9403b4 commit da3cd037dc3a7b19750d0e620467cfa6ea9403b4 Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 17:16:57 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:02:06 +0000 www/py-dj51-drf-writable-nested: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + www/py-dj51-drf-writable-nested/Makefile (new) | 23 +++++++++++++++++++++++ www/py-dj51-drf-writable-nested/distinfo (new) | 3 +++ www/py-dj51-drf-writable-nested/pkg-descr (new) | 9 +++++++++ 4 files changed, 36 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=6409804682fa52270bc041fa557a3e0e137c5ae8 commit 6409804682fa52270bc041fa557a3e0e137c5ae8 Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-05 17:20:54 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:02:46 +0000 www/py-dj51-django-extensions: New port Clone of existing port for Django 5.1. Did not want to push this onto ports@caomhin.org without asking, so currently I'm the maintainer. Please feel free to take the port from me without asking. PR: 286533 Co-authored-by: Sebastian <sko@rostwald.de> www/Makefile | 1 + www/py-dj51-django-extensions/Makefile (new) | 22 ++++++++++++++++++++++ www/py-dj51-django-extensions/distinfo (new) | 3 +++ www/py-dj51-django-extensions/pkg-descr (new) | 1 + 4 files changed, 27 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=e874a9b526247e710792d3f3258dc0548dd7fd07 commit e874a9b526247e710792d3f3258dc0548dd7fd07 Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-05 17:25:30 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:03:04 +0000 www/py-dj51-django-guardian: New port Clone of existing port for Django 5.1. Did not want to push this onto python@ without asking, that's why I'm the maintainer. Please feel free to take the port from me without asking. PR: 286533 Co-authored-by: Sebastian <sko@rostwald.de> www/Makefile | 1 + www/py-dj51-django-guardian/Makefile (new) | 22 ++++++++++++++++++++++ www/py-dj51-django-guardian/distinfo (new) | 3 +++ www/py-dj51-django-guardian/pkg-descr (new) | 2 ++ 4 files changed, 28 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=38932cdab02865c03f7f1e7ae29125d2c149b8d6 commit 38932cdab02865c03f7f1e7ae29125d2c149b8d6 Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 17:30:45 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:04:53 +0000 www/py-dj51-django-compression-middleware: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + .../Makefile (new) | 24 ++++++++++++++++++++++ .../distinfo (new) | 3 +++ .../pkg-descr (new) | 6 ++++++ 4 files changed, 34 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=f69193f1d47698c9b2a132b2ac82cfe26a65a16a commit f69193f1d47698c9b2a132b2ac82cfe26a65a16a Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 17:33:26 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:05:26 +0000 www/py-dj51-django-soft-delete: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + www/py-dj51-django-soft-delete/Makefile (new) | 26 ++++++++++++++++++++++++++ www/py-dj51-django-soft-delete/distinfo (new) | 3 +++ www/py-dj51-django-soft-delete/pkg-descr (new) | 1 + 4 files changed, 31 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=b6bfe9745d93dbf3a22737a9d7a21d3eb9d712a8 commit b6bfe9745d93dbf3a22737a9d7a21d3eb9d712a8 Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 17:44:37 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:05:52 +0000 www/py-dj51-djangorestframework-guardian: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + .../Makefile (new) | 24 ++++++++++++++++++++++ .../distinfo (new) | 3 +++ .../pkg-descr (new) | 2 ++ 4 files changed, 30 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=6cd2cb4f2aef1325336656d143cbaa1b345e4e6f commit 6cd2cb4f2aef1325336656d143cbaa1b345e4e6f Author: Sebastian <sko@rostwald.de> AuthorDate: 2025-05-05 17:49:33 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-05 18:06:50 +0000 www/py-dj51-django-multiselectfield: New port Clone of existing port for Django 5.1. PR: 286533 www/Makefile | 1 + www/py-dj51-django-multiselectfield/Makefile (new) | 23 ++++++++++++++++++++++ www/py-dj51-django-multiselectfield/distinfo (new) | 3 +++ .../pkg-descr (new) | 3 +++ 4 files changed, 30 insertions(+)
Created attachment 260203 [details] py-paperless-ngx update 2.11.6 -> 2.15.3 I successfully compiled version 1.15.3 yesterday and after a lot of headdesking I finally got the webserver running via daemon as an unprivileged user (paperless). granian has no way of setting its working directory or dropping privileges after startup - one has to run it from the desired path as the desired user. Hence I had to add '${name}_chdir' to the rc-file to get this working - after I found out about that variable... I also used 'su -l' by habit during testing, which also doesn't work for some reason and the error messages python threw weren't helpful at all. However, the rc.d file is now updated and working and the granian server is running as user paperless. I cleaned up my ports tree and created a diff that incorporates all changes to get from 2.11.6 -> 2.15.3. (In reply to commit-hook from comment #6) Newer allauth also needs py-fido2 as a RUN_DEPENDS; this must have slipped through on my end when moving around and testing the diffs as I had this as a separate commit on one host. I'm really sorry for that, I have added a diff that fixes this. Using the current 'latest' branch with those two patches, everything bulids fine and 'testport' finishes without errors on 14.2-RELEASE. Backporting everything to the quarterly branch also builds/'testoprt's fine and is currently running on my test instance without any apparent issues. I ran the migration, imported and edited some more documents and had no errors in the logs so far.
Created attachment 260204 [details] py-dj51-django-allauth fix missing run dependency
(In reply to Sebastian Oswald from comment #16) Hi, thank you for your work and testing. I already created my own patches for py-paperless-ngx which I'm currently testing locally (there were django 5.1 sqlite issues that needed fixing first). We'll see if/how much it differs from your attached patch. I'm moving step by step, as there were dependencies to add/update/fix as well (new gotenberg client for example). I'll add py-fido2 to allauth later, thanks for catching that. After "battle" testing the new version on my large main document repository I'll share and/or commit.
Comment on attachment 260204 [details] py-dj51-django-allauth fix missing run dependency The patch needs: - bump portrevision - add runtime dependency only in case MFA is selected (made this default) I'll do those changed on commit, thank you.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=11327766cf02919fec79323d03c6674c1d8155ac commit 11327766cf02919fec79323d03c6674c1d8155ac Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-06 12:39:52 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-06 12:41:32 +0000 www/py-dj51-django-allauth: Add missing dependency Wth the MFA option enabled, py-fido2 is also required. PR: 286533 Co-authored-by: Sebastian <sko@rostwald.de> www/py-dj51-django-allauth/Makefile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=3ee568482f52e0fb1b5e001eb3e4d3a8b0ba9126 commit 3ee568482f52e0fb1b5e001eb3e4d3a8b0ba9126 Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-06 17:37:24 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-06 17:40:17 +0000 www/py-dj51-django-multiselectfield: Update to 0.1.13 PR: 286533 www/py-dj51-django-multiselectfield/Makefile | 3 ++- www/py-dj51-django-multiselectfield/distinfo | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=326c7139885fdd62580797fada7cea5eaa4e6a3e commit 326c7139885fdd62580797fada7cea5eaa4e6a3e Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-07 10:04:31 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-07 10:10:17 +0000 deskutils/py-paperless-ngx: Update to 2.15.3 PR: 286533 Co-authored-by: Sebastian <sko@rostwald.de> deskutils/py-paperless-ngx/Makefile | 81 ++- deskutils/py-paperless-ngx/distinfo | 10 +- .../py-paperless-ngx/files/paperless-webui.in | 5 +- .../files/patch-gunicorn.conf.py (gone) | 11 - ...ments_management_commands_document__consumer.py | 4 +- .../files/patch-src_paperless__mail_parsers.py | 8 +- .../files/patch-src_paperless_settings.py | 12 +- deskutils/py-paperless-ngx/files/pkg-message.in | 6 + deskutils/py-paperless-ngx/pkg-plist | 632 ++++++++++++++------- 9 files changed, 495 insertions(+), 274 deletions(-)
Comment on attachment 260203 [details] py-paperless-ngx update 2.11.6 -> 2.15.3 Marking the patch as obsolete, as I landed something different. The input was very helpful though. One aspect I did not incorporate was chdir granian. What exactly is this required for (are you serving static content without a proxy)?
(In reply to Michael Gmelin from comment #23) If this was about finding the python files: We already set PYTHONPATH, which allows granian to locate the python files, see: https://cgit.freebsd.org/ports/tree/deskutils/py-paperless-ngx/files/paperless-webui.in?id=326c7139885fdd62580797fada7cea5eaa4e6a3e#n60 Changing the user worked out of the box (we already did this with gunicorn). Would be great if you could also test the committed version of the port and check if everything is alright for you.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=fbac8b5cf2bb41845810ceab1d26adbeb7fbe132 commit fbac8b5cf2bb41845810ceab1d26adbeb7fbe132 Author: Michael Gmelin <grembo@FreeBSD.org> AuthorDate: 2025-05-09 09:45:44 +0000 Commit: Michael Gmelin <grembo@FreeBSD.org> CommitDate: 2025-05-09 09:48:16 +0000 deskutils/py-paperless-ngx: Fix custom webui endpoint This brings back paperless_webui_listen_address and paperless_webui_listen_port, which were turned into noops by accident during the update. PR: 286533 Reported by: mfechner deskutils/py-paperless-ngx/Makefile | 1 + deskutils/py-paperless-ngx/files/paperless-webui.in | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)
^Triage: assign to committer who resolved.
I'm really sorry for the late reply; I was completely buried in work last week. I just pulled all the updates into my ports tree and built everything via 'poudriere bulk' - which went fine for bare 'latest', but to pull the update to 'quarterly' I needed to also pull a lot of additionall updates to various python ports. I guess you updated all dependencies according to the paperless-ngx 'changelogs' (if you can call them that...), where they just blindly update all dependencies in their docker thingy to the latest available release, regardless if actually required or not. That's why I left all dependency versions as they were unless I ran into errors/warnings. It's probably still the right choice to update all those dependencies and no big deal if building from latest, but makes backporting the update to quarterly not as trivial. (I can share a diff for that if you want) > One aspect I did not incorporate was chdir granian. What exactly is this required for (are you serving static content without a proxy)? I always got various 'permission denied' errors resulting in granian not being able to load modules and start. I suspected it was because 'deamon' was called with '-c' and hence starting granian in the root directory. Thanks to granian having no proper debug output I could just guess and try various things, of which setting the working directory via 'chdir' directive worked. ('--log-level debug' just barfs out pretty much any code it touches, but nothing useful in regards of what actually goes wrong...) However, it now works *without* the 'chdir' - no idea why, but I'm happy if it 'just works'™ In short: everything works for me; no errors and health check reports everything as "OK" and 'paperless check' returns with so issues. The only warning I'm still getting in daemon.log is this: May 12 10:19:42 paperless paperless-webui[78907]: /usr/local/lib/python3.11/site-packages/whitenoise/base.py:115: UserWarning: No directory at: /usr/local/lib/python3.11/site-packages/static/ May 12 10:19:42 paperless paperless-webui[78907]: warnings.warn(f"No directory at: {root}") Not sure if this is important as I can't see anything not working and the health check also reports everything as OK, but I suspect that whitenoise module actually wants to run in '/usr/local/www/paperless-ngx/' - at least there's a 'static' directory. No idea though how to make this happen. Since whitenoise is a module for serving static files I don't think it's required if one serves those though a proper webserver as intended and PAPERLESS_STATICDIR is unset in paperless.conf. THANK YOU for all the work you put into this port!
Hi, Thank you for testing and the detailed response. The reason I bumped many dependencies - besides that some were technically necessary to fix bugs - is to allow me to test a known combination of packages, at least at the time of update. Otherwise bug reports could get really strange/hard to reproduce with all the possible variations. I'm not planning to backport this one to quarterly, as it's neither a build- nor a security fix. As stated on https://wiki.freebsd.org/Ports/QuarterlyBranch: [the purpose of the quarterly branch is] A user experience (UX), for port and package installations and upgrades, that is more predictable and stable than the UX for the more dynamic main branch (familiarly: latest). In essence, changes are limited to: - **non-feature** updates. Good news is that 2025Q3 is only about six weeks away ;)
(In reply to Michael Gmelin from comment #28) p.s. I don't see the whitenoise error message. Strange.
> I'm not planning to backport this one to quarterly, as it's neither a build- nor a security fix. > Good news is that 2025Q3 is only about six weeks away ;) Full ACK. It's only a personal issue. I got bitten once too often by running 'latest' packages on servers - so although I built all packages myself, I still only run latest on non-critical test-systems/jails... > p.s. I don't see the whitenoise error message. Strange. I always get this message twice in the logs after (re)starting paperless-webui. However, as I already suspected, the message vanishes if 'PAPERLESS_STATICDIR' is set and not commented out in paperless.conf - which should only be enabled if not run behind a webserver/proxy. Essentialy, whitenoise doesn't even need to run if no static files have to be served, so the message is bogus for any standard setup where a real webserver is serving the files in /usr/local/www/paperless-ngx/static.
>[...] I still only run latest on non-critical test-systems/jails... s/latest/quarterly/ Regarding that whitenoise error, I just found out that this seems to be a "bogus warning by design": https://github.com/evansd/whitenoise/issues/215