Created attachment 223004 [details] pyjwt.diff QA: * portlint: OK (looks fine.) * testport: OK (poudriere: 12.2, amd64 tested) * maketest: OK (173 passed, 1 skipped, 1 xfailed)
Build and package info is available at https://gitlab.com/swills/freebsd-ports/pipelines/266092718
Hi Goran, hi Terje, As far as I can say until now, this update also needs an update of devel/py-msal and sysutils/conan. Both give same type of error: py37-msal-1.9.0 depends on package: py37-pyjwt>=1.0.0<2 - not found conan-1.34.0 depends on package: py37-pyjwt>=1.4.0,<2.0.0 - not found Could you investigate into them, please? Another issue: The changelog for www/py-pyjwt says cryptography >= 3 is needed. For now, we do not have cryptography-3.x in the ports, because of some trouble with OpenSSL. Did you try, if cryptography-2.x also works?
I will check depending ports. For cryptography, 3.3 is already in ports: https://www.freshports.org/security/py-cryptography/
While msal is OK with newer JWT (https://github.com/AzureAD/microsoft-authentication-library-for-python/blob/dev/setup.py#L76), conan isn't (https://github.com/conan-io/conan/blob/develop/conans/requirements.txt#L1). I'll talk to upstream about it.
(In reply to Goran Mekić from comment #3) Oops, you are right. I must have looked into one of my older svn repos ;)
Conan upstream ticket: https://github.com/conan-io/conan/issues/8609 What is the best course of action? Having pyjwt1 and pyjwt in ports?
(In reply to Goran Mekić from comment #6) > What is the best course of action? Having pyjwt1 and pyjwt in ports? Hmm, I really don't know because I don't work with any of those ports. If you can't wait to update www/py-pyjwt because you need the www/py-flask-jwt-extended update, it's probably best to have an old port and a current port for www/py-pyjwt. Renaming the old one to something like www/py-pyjwt1 seems plausible to me in this case. If you are not in a hurry, you could wait until conan has updated its dependencies upstream ... Have you already had contact with the maintainer of www/py-pyjwt?
Created attachment 225729 [details] pyjwt1.diff I ran "make -DBATCH test clean" on all ports I changed dependency for. This patch adds pyjwt1 port and upgrades current one to 2.1.0. I am currently running poudriere bulk on those ports on https://pkg.tilda.center/build.html?mastername=FreeBSD%3A13%3Aamd64-local&build=2021-06-11_13h51m10s and as I see LLVM in the build queue, it will take a while.
Sorry, forgot to switch the branch on the build server, so I had to restart the build: https://pkg.tilda.center/build.html?mastername=FreeBSD%3A13%3Aamd64-local&build=2021-06-11_14h10m11s
Poudriere was successfull. All packages were in there except flask-jwt-extended because that one will be updated to use pyjwt 2.x.
What's our next move?
(In reply to Goran Mekić from comment #11) Hi Goran, Sorry for the long delay since your updated patch in June. I had the hope that the conan project in the meantime also adaptations for pyjwt >= v2.0.0 committed. That still seems to be delayed though, see post https://github.com/conan-io/conan/pull/8952#issuecomment-864019633. I would like to avoid creating an additional port www/py-pyjwt1. However, if we can't avoid it, move the current port py-pyjwt to py-pyjwt1: Have you contacted the maintainer to see if he agrees? (For v2.x.x you have registered yourself as maintainer ...). Ok, probably a maintainer timeout after so long? And is there a strategy to check the pyjwt v1.7.1 dependent ports to see if they also build and work on >= v2.0.0? Translated with www.DeepL.com/Translator (free version)
I tried to contact the current maintainer before and never received any reply. For example, this issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242818 As for the version I wanted to keep the current state by introducing version 1 and 2, and keeping existing ports' dependencies to version 1. As I use only few ports that rely on JWT, I'm afraid I wouldn't be much of a tester for ports like conan as I'm afraid it would be easy for me to miss some edge case. I do have time to run few tests, I'm just not confident it's enough.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=e6ec12f6646f71fe84268d21f3a6901191ebf60e commit e6ec12f6646f71fe84268d21f3a6901191ebf60e Author: Goran Mekic <meka@tilda.center> AuthorDate: 2021-10-04 17:07:25 +0000 Commit: Rainer Hurling <rhurlin@FreeBSD.org> CommitDate: 2021-10-04 17:10:05 +0000 www/py-pyjwt: Rename to www/py-pyjwt1 To make it possible to easily import py-pyjwt v2.x move the current port to a versioned directory. Bump consumers after rename of the dependency. PR: 254038 MOVED | 1 + UPDATING | 14 ++++++++++++++ databases/py-python-arango/Makefile | 3 ++- devel/py-apns2/Makefile | 3 ++- devel/py-buildbot/Makefile | 3 ++- devel/py-msal/Makefile | 3 ++- devel/py-pygithub/Makefile | 3 ++- devel/py-qcs-api-client/Makefile | 3 ++- devel/py-twilio/Makefile | 3 ++- net-im/py-matrix-synapse/Makefile | 3 ++- net-mgmt/py-adal/Makefile | 3 ++- net/ceph14/Makefile | 4 ++-- security/py-oauthlib/Makefile | 5 +++-- security/py-social-auth-core/Makefile | 3 ++- sysutils/conan/Makefile | 3 ++- sysutils/py-azure-cli-core/Makefile | 4 ++-- www/Makefile | 2 +- www/py-flask-jwt-extended/Makefile | 3 ++- www/py-mwoauth/Makefile | 3 ++- www/{py-pyjwt => py-pyjwt1}/Makefile | 2 +- www/{py-pyjwt => py-pyjwt1}/distinfo | 0 www/{py-pyjwt => py-pyjwt1}/pkg-descr | 0 www/seahub/Makefile | 3 ++- 23 files changed, 52 insertions(+), 22 deletions(-)
Hi Goran, the first step should be done now, hopefully without any breakage ;) In the next step I will create the new www/py-pyjwt v2.1.0 ... For the third step: Which dependend ports are known to work well with v2.1.0 right now?
Hi Goran, there is probably a bigger problem that I was not aware of before: Both pkg-plist, www/py-pyjwt1 and www/py-pyjwt, are partially identical. And this is for all files under %%PYTHON_SITELIBDIR%%/jwt/... This must be solved before v2.1.0 can be committed.
(In reply to Rainer Hurling from comment #15) To answer working ports: I'm using www/py-flask-jwt-extended that I know works with pyjwt2. As for plist, I'll take a look now and try to figure it out.
(In reply to Rainer Hurling from comment #16) I think the only resolution is to add CONFLICTS in pyjwt 1 and 2. What do you think, Rainer?
(In reply to Goran Mekić from comment #18) To be honest, I think that is a very bad solution. This would mean that only either dependencies from the old or from the new version can be used, but not all dependencies at the same time as before. Maybe there are examples among other BSD derivatives or in the Linux distributions how both versions can be used together? The other way would be to thoroughly check for all dependents if they already cope with v2.1.0 and switch over.
(In reply to Rainer Hurling from comment #19) I don't really like the situation myself, but I checked NetBSD, OpenBSD and Arch Linux and all of them have only one version of pyjwt https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/textproc/py-JWT/index.html http://ports.su/www/py-jwt,python3 https://archlinux.org/packages/community/any/python-pyjwt/ Maybe there's a Linux distro that handles this better, but I can't find it. I'll go through the list of ports over the weekend and try to use them with PyJWT 2.1.0.
(In reply to Goran Mekić from comment #20) > I don't really like the situation myself, but I checked NetBSD, OpenBSD > and Arch Linux and all of them have only one version of pyjwt I was afraid of that :( > Maybe there's a Linux distro that handles this better, but I can't > find it. I'll go through the list of ports over the weekend and try > to use them with PyJWT 2.1.0. Many thanks! The list of dependent ports is at least the ones that got the port revision increment in the last commit. Presumably there are also some more indirectly affected ports? Let me know if I can test something :) (However, in the next few days, including the weekend, I have very little time, there may be delays.)
Unfortunately, testing ports requiring PyJWT is not feasible for me because good number of them is tied to some service where I don't have an account. Just to give an example devel/py-apns2, devel/py-msal, devel/py-twilio, etc. Which tells me that there are only two ways of action: - Give up on PyJWT 2.1 - Have CONFLICT between versions 1 and 2 I would hate for us to decide to give on PyJWT 2 and this is why. We already have ports that don't work with PyJWT 1 (I know of flask-jwt-extended, at least) and giving up on PyJWT would make number of ports smaller. If we decide on CONFLICT then at least all ports will be available. Even if I need two ports which depend on different versions of PyJWT I could have a workaround like using them in different jails or something. All I'm trying to say is that both alternatives are bad solutions, it's just that CONFLICT is lesser evil. What do you think, Rainer?
A commit in branch 2021Q4 references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=127e49fb45063a84e3f57540ef05b842d116a041 commit 127e49fb45063a84e3f57540ef05b842d116a041 Author: Goran Mekic <meka@tilda.center> AuthorDate: 2021-10-04 17:07:25 +0000 Commit: Ashish SHUKLA <ashish@FreeBSD.org> CommitDate: 2021-11-24 11:20:18 +0000 www/py-pyjwt: Rename to www/py-pyjwt1 To make it possible to easily import py-pyjwt v2.x move the current port to a versioned directory. Bump consumers after rename of the dependency. PR: 254038 (cherry picked from commit e6ec12f6646f71fe84268d21f3a6901191ebf60e) MOVED | 1 + UPDATING | 14 ++++++++++++++ databases/py-python-arango/Makefile | 3 ++- devel/py-apns2/Makefile | 3 ++- devel/py-buildbot/Makefile | 3 ++- devel/py-msal/Makefile | 3 ++- devel/py-pygithub/Makefile | 3 ++- devel/py-qcs-api-client/Makefile | 3 ++- devel/py-twilio/Makefile | 3 ++- net-im/py-matrix-synapse/Makefile | 3 ++- net-mgmt/py-adal/Makefile | 3 ++- net/ceph14/Makefile | 4 ++-- security/py-oauthlib/Makefile | 5 +++-- security/py-social-auth-core/Makefile | 3 ++- sysutils/conan/Makefile | 3 ++- sysutils/py-azure-cli-core/Makefile | 4 ++-- www/Makefile | 2 +- www/py-flask-jwt-extended/Makefile | 3 ++- www/py-mwoauth/Makefile | 3 ++- www/{py-pyjwt => py-pyjwt1}/Makefile | 2 +- www/{py-pyjwt => py-pyjwt1}/distinfo | 0 www/{py-pyjwt => py-pyjwt1}/pkg-descr | 0 www/seahub/Makefile | 3 ++- 23 files changed, 52 insertions(+), 22 deletions(-)
Sorry for the poor responsability from my side in the last weeks. I had and have a lot of trouble at work :( And thanks for this commit, which is probably an important next step in the right direction. But shouldn't you have mentioned the maintainer timeout here?
Created attachment 229749 [details] patch with remaining diff for new port py-pyjwt After renaming old port into py-pyjwt1 and adapting the dependencies in commit https://cgit.freebsd.org/ports/commit/?id=127e49fb45063a84e3f57540ef05b842d116a041, the remaining part of the patch is creating the new port py-pyjwt v2.1.0. As mentioned already in comment #18, there is a conflict between the old and the new port: Installing py38-pyjwt-2.1.0... pkg-static: py38-pyjwt-2.1.0 conflicts with py38-pyjwt1-1.7.1 (installs files into the same place). Problematic file: /usr/local/lib/python3.8/site-packages/jwt/__init__.py We need to find an elegant way for both ports to coexist for a transition period. Simply putting a CONFLICTS= in both ports falls short, as then gradually some other ports would use both py-pyjwt1 and py-pyjwt dependencies. The best solution would be to change the installation path for py-pyjwt1 a bit and thus avoid the conflict ;)
What are our options? rename directory for pyjwt to pyjwt2 inside site-packages and patch downstream ports? Or maybe rename pyjwt1? Any other options?
(In reply to Goran Mekić from comment #26) I would prefer to rename the correct places within py-pyjwt1. E.g. that the path instead of lib/python3.x/site-packages/jwt is then lib/python3.x/site-packages/jwt1. Then the future port py-pyjwt v2.x would not need to be bent. But I personally lack the knowledge to estimate how to do this in Python without errors.
That's exactly what I'm afraid of. Practically changing module name requires all downstream ports to have a patch, and that's a lot of work. Not that amount of work is a problem, but finding all the corner cases for all downstream ports. As some of those ports are for integrating 3rd party services, it would be quite hard for someone to cover it all.
*** Bug 260447 has been marked as a duplicate of this bug. ***
It looks like on Oct 11th, Arch switched to pwjwt 2: https://archlinux.org/packages/community/any/python-pyjwt/ https://github.com/archlinux/svntogit-community/commits/packages/python-pyjwt/trunk So they have abandoned pyjwt 1. I think it is an OK course of action to add pyjwt2 as a separate port with conflicts for pyjwt1. If people complain we can direct them back to the project that still depends on pyjwt1. Since ultimately user pressure is probably the best way to get these projects to update.
py-pyjwt version 2.3.0 was added in 1688f5f7ce02ee55dfa137d41f3a6c80a5c04682 so I'm closing this one.
(In reply to Goran Mekić from comment #31) Yes, Sunpoet already committed it yesterday :P