Bug 254038 - www/py-pyjwt: Update to 2.1.0
Summary: www/py-pyjwt: Update to 2.1.0
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Rainer Hurling
URL: https://github.com/jpadilla/pyjwt/blo...
Keywords: buildisok
Depends on:
Blocks: 254039 257369
  Show dependency treegraph
 
Reported: 2021-03-05 15:22 UTC by Goran Mekić
Modified: 2021-11-27 09:35 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (terje)


Attachments
pyjwt.diff (814 bytes, patch)
2021-03-05 15:22 UTC, Goran Mekić
no flags Details | Diff
pyjwt1.diff (12.28 KB, patch)
2021-06-11 11:58 UTC, Goran Mekić
no flags Details | Diff
patch with remaining diff for new port py-pyjwt (1.86 KB, patch)
2021-11-26 20:23 UTC, Rainer Hurling
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Goran Mekić 2021-03-05 15:22:03 UTC
Created attachment 223004 [details]
pyjwt.diff

QA:

  * portlint: OK (looks fine.)
  * testport: OK (poudriere: 12.2, amd64 tested)
  * maketest: OK (173 passed, 1 skipped, 1 xfailed)
Comment 1 Automation User 2021-03-05 15:40:41 UTC
Build and package info is available at https://gitlab.com/swills/freebsd-ports/pipelines/266092718
Comment 2 Rainer Hurling freebsd_committer 2021-03-07 06:45:56 UTC
Hi Goran, hi Terje,

As far as I can say until now, this update also needs an update of devel/py-msal and sysutils/conan. Both give same type of error:

py37-msal-1.9.0 depends on package: py37-pyjwt>=1.0.0<2 - not found
conan-1.34.0 depends on package: py37-pyjwt>=1.4.0,<2.0.0 - not found

Could you investigate into them, please?


Another issue: The changelog for www/py-pyjwt says cryptography >= 3 is needed. For now, we do not have cryptography-3.x in the ports, because of some trouble with OpenSSL. Did you try, if cryptography-2.x also works?
Comment 3 Goran Mekić 2021-03-07 09:18:32 UTC
I will check depending ports. For cryptography, 3.3 is already in ports: https://www.freshports.org/security/py-cryptography/
Comment 4 Goran Mekić 2021-03-07 09:29:53 UTC
While msal is OK with newer JWT (https://github.com/AzureAD/microsoft-authentication-library-for-python/blob/dev/setup.py#L76), conan isn't (https://github.com/conan-io/conan/blob/develop/conans/requirements.txt#L1). I'll talk to upstream about it.
Comment 5 Rainer Hurling freebsd_committer 2021-03-07 09:55:04 UTC
(In reply to Goran Mekić from comment #3)

Oops, you are right. I must have looked into one of my older svn repos ;)
Comment 6 Goran Mekić 2021-03-09 16:07:30 UTC
Conan upstream ticket: https://github.com/conan-io/conan/issues/8609

What is the best course of action? Having pyjwt1 and pyjwt in ports?
Comment 7 Rainer Hurling freebsd_committer 2021-03-09 16:28:11 UTC
(In reply to Goran Mekić from comment #6)

> What is the best course of action? Having pyjwt1 and pyjwt in ports?
Hmm, I really don't know because I don't work with any of those ports.

If you can't wait to update www/py-pyjwt because you need the www/py-flask-jwt-extended update, it's probably best to have an old port and a current port for www/py-pyjwt. Renaming the old one to something like www/py-pyjwt1 seems plausible to me in this case.

If you are not in a hurry, you could wait until conan has updated its dependencies upstream ...

Have you already had contact with the maintainer of www/py-pyjwt?
Comment 8 Goran Mekić 2021-06-11 11:58:19 UTC
Created attachment 225729 [details]
pyjwt1.diff

I ran "make -DBATCH test clean" on all ports I changed dependency for. This patch adds pyjwt1 port and upgrades current one to 2.1.0. I am currently running poudriere bulk on those ports on https://pkg.tilda.center/build.html?mastername=FreeBSD%3A13%3Aamd64-local&build=2021-06-11_13h51m10s and as I see LLVM in the build queue, it will take a while.
Comment 9 Goran Mekić 2021-06-11 12:11:17 UTC
Sorry, forgot to switch the branch on the build server, so I had to restart the build: https://pkg.tilda.center/build.html?mastername=FreeBSD%3A13%3Aamd64-local&build=2021-06-11_14h10m11s
Comment 10 Goran Mekić 2021-06-14 10:15:51 UTC
Poudriere was successfull. All packages were in there except flask-jwt-extended because that one will be updated to use pyjwt 2.x.
Comment 11 Goran Mekić 2021-06-24 11:10:50 UTC
What's our next move?
Comment 12 Rainer Hurling freebsd_committer 2021-09-26 15:30:27 UTC
(In reply to Goran Mekić from comment #11)

Hi Goran,
Sorry for the long delay since your updated patch in June. 

I had the hope that the conan project in the meantime also adaptations for pyjwt >= v2.0.0 committed. That still seems to be delayed though, see post https://github.com/conan-io/conan/pull/8952#issuecomment-864019633. 

I would like to avoid creating an additional port www/py-pyjwt1. However, if we can't avoid it, move the current port py-pyjwt to py-pyjwt1: Have you contacted the maintainer to see if he agrees? (For v2.x.x you have registered yourself as maintainer ...).  Ok, probably a maintainer timeout after so long?

And is there a strategy to check the pyjwt v1.7.1 dependent ports to see if they also build and work on >= v2.0.0?

Translated with www.DeepL.com/Translator (free version)
Comment 13 Goran Mekić 2021-10-01 08:47:18 UTC
I tried to contact the current maintainer before and never received any reply. For example, this issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242818

As for the version I wanted to keep the current state by introducing version 1 and 2, and keeping existing ports' dependencies to version 1. As I use only few ports that rely on JWT, I'm afraid I wouldn't be much of a tester for ports like conan as I'm afraid it would be easy for me to miss some edge case. I do have time to run few tests, I'm just not confident it's enough.
Comment 14 commit-hook freebsd_committer 2021-10-04 17:13:41 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=e6ec12f6646f71fe84268d21f3a6901191ebf60e

commit e6ec12f6646f71fe84268d21f3a6901191ebf60e
Author:     Goran Mekic <meka@tilda.center>
AuthorDate: 2021-10-04 17:07:25 +0000
Commit:     Rainer Hurling <rhurlin@FreeBSD.org>
CommitDate: 2021-10-04 17:10:05 +0000

    www/py-pyjwt: Rename to www/py-pyjwt1

    To make it possible to easily import py-pyjwt v2.x move the current port
    to a versioned directory. Bump consumers after rename of the dependency.

    PR:             254038

 MOVED                                 |  1 +
 UPDATING                              | 14 ++++++++++++++
 databases/py-python-arango/Makefile   |  3 ++-
 devel/py-apns2/Makefile               |  3 ++-
 devel/py-buildbot/Makefile            |  3 ++-
 devel/py-msal/Makefile                |  3 ++-
 devel/py-pygithub/Makefile            |  3 ++-
 devel/py-qcs-api-client/Makefile      |  3 ++-
 devel/py-twilio/Makefile              |  3 ++-
 net-im/py-matrix-synapse/Makefile     |  3 ++-
 net-mgmt/py-adal/Makefile             |  3 ++-
 net/ceph14/Makefile                   |  4 ++--
 security/py-oauthlib/Makefile         |  5 +++--
 security/py-social-auth-core/Makefile |  3 ++-
 sysutils/conan/Makefile               |  3 ++-
 sysutils/py-azure-cli-core/Makefile   |  4 ++--
 www/Makefile                          |  2 +-
 www/py-flask-jwt-extended/Makefile    |  3 ++-
 www/py-mwoauth/Makefile               |  3 ++-
 www/{py-pyjwt => py-pyjwt1}/Makefile  |  2 +-
 www/{py-pyjwt => py-pyjwt1}/distinfo  |  0
 www/{py-pyjwt => py-pyjwt1}/pkg-descr |  0
 www/seahub/Makefile                   |  3 ++-
 23 files changed, 52 insertions(+), 22 deletions(-)
Comment 15 Rainer Hurling freebsd_committer 2021-10-04 17:17:02 UTC
Hi Goran,

the first step should be done now, hopefully without any breakage ;)

In the next step I will create the new www/py-pyjwt v2.1.0 ...

For the third step: Which dependend ports are known to work well with v2.1.0 right now?
Comment 16 Rainer Hurling freebsd_committer 2021-10-06 19:15:14 UTC
Hi Goran,

there is probably a bigger problem that I was not aware of before:

Both pkg-plist, www/py-pyjwt1 and www/py-pyjwt, are partially identical. And this is for all files under %%PYTHON_SITELIBDIR%%/jwt/...

This must be solved before v2.1.0 can be committed.
Comment 17 Goran Mekić 2021-10-06 19:22:49 UTC
(In reply to Rainer Hurling from comment #15)
To answer working ports: I'm using www/py-flask-jwt-extended that I know works with pyjwt2.

As for plist, I'll take a look now and try to figure it out.
Comment 18 Goran Mekić 2021-10-06 20:19:25 UTC
(In reply to Rainer Hurling from comment #16)
I think the only resolution is to add CONFLICTS in pyjwt 1 and 2. What do you think, Rainer?
Comment 19 Rainer Hurling freebsd_committer 2021-10-07 06:15:21 UTC
(In reply to Goran Mekić from comment #18)

To be honest, I think that is a very bad solution. This would mean that only either dependencies from the old or from the new version can be used, but not all dependencies at the same time as before.

Maybe there are examples among other BSD derivatives or in the Linux distributions how both versions can be used together?

The other way would be to thoroughly check for all dependents if they already cope with v2.1.0 and switch over.
Comment 20 Goran Mekić 2021-10-07 08:33:03 UTC
(In reply to Rainer Hurling from comment #19)
I don't really like the situation myself, but I checked NetBSD, OpenBSD and Arch Linux and all of them have only one version of pyjwt

https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/textproc/py-JWT/index.html
http://ports.su/www/py-jwt,python3
https://archlinux.org/packages/community/any/python-pyjwt/

Maybe there's a Linux distro that handles this better, but I can't find it. I'll go through the list of ports over the weekend and try to use them with PyJWT 2.1.0.
Comment 21 Rainer Hurling freebsd_committer 2021-10-07 08:40:22 UTC
(In reply to Goran Mekić from comment #20)
> I don't really like the situation myself, but I checked NetBSD, OpenBSD
> and Arch Linux and all of them have only one version of pyjwt
I was afraid of that :(

> Maybe there's a Linux distro that handles this better, but I can't
> find it. I'll go through the list of ports over the weekend and try
> to use them with PyJWT 2.1.0.
Many thanks! The list of dependent ports is at least the ones that got the port revision increment in the last commit. Presumably there are also some more indirectly affected ports?

Let me know if I can test something :)

(However, in the next few days, including the weekend, I have very little time, there may be delays.)
Comment 22 Goran Mekić 2021-10-09 09:39:28 UTC
Unfortunately, testing ports requiring PyJWT is not feasible for me because good number of them is tied to some service where I don't have an account. Just to give an example devel/py-apns2, devel/py-msal, devel/py-twilio, etc. Which tells me that there are only two ways of action:

- Give up on PyJWT 2.1
- Have CONFLICT between versions 1 and 2

I would hate for us to decide to give on PyJWT 2 and this is why. We already have ports that don't work with PyJWT 1 (I know of flask-jwt-extended, at least) and giving up on PyJWT would make number of ports smaller. If we decide on CONFLICT then at least all ports will be available. Even if I need two ports which depend on different versions of PyJWT I could have a workaround like using them in different jails or something. All I'm trying to say is that both alternatives are bad solutions, it's just that CONFLICT is lesser evil. What do you think, Rainer?
Comment 23 commit-hook freebsd_committer 2021-11-24 11:22:28 UTC
A commit in branch 2021Q4 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=127e49fb45063a84e3f57540ef05b842d116a041

commit 127e49fb45063a84e3f57540ef05b842d116a041
Author:     Goran Mekic <meka@tilda.center>
AuthorDate: 2021-10-04 17:07:25 +0000
Commit:     Ashish SHUKLA <ashish@FreeBSD.org>
CommitDate: 2021-11-24 11:20:18 +0000

    www/py-pyjwt: Rename to www/py-pyjwt1

    To make it possible to easily import py-pyjwt v2.x move the current port
    to a versioned directory. Bump consumers after rename of the dependency.

    PR:             254038
    (cherry picked from commit e6ec12f6646f71fe84268d21f3a6901191ebf60e)

 MOVED                                 |  1 +
 UPDATING                              | 14 ++++++++++++++
 databases/py-python-arango/Makefile   |  3 ++-
 devel/py-apns2/Makefile               |  3 ++-
 devel/py-buildbot/Makefile            |  3 ++-
 devel/py-msal/Makefile                |  3 ++-
 devel/py-pygithub/Makefile            |  3 ++-
 devel/py-qcs-api-client/Makefile      |  3 ++-
 devel/py-twilio/Makefile              |  3 ++-
 net-im/py-matrix-synapse/Makefile     |  3 ++-
 net-mgmt/py-adal/Makefile             |  3 ++-
 net/ceph14/Makefile                   |  4 ++--
 security/py-oauthlib/Makefile         |  5 +++--
 security/py-social-auth-core/Makefile |  3 ++-
 sysutils/conan/Makefile               |  3 ++-
 sysutils/py-azure-cli-core/Makefile   |  4 ++--
 www/Makefile                          |  2 +-
 www/py-flask-jwt-extended/Makefile    |  3 ++-
 www/py-mwoauth/Makefile               |  3 ++-
 www/{py-pyjwt => py-pyjwt1}/Makefile  |  2 +-
 www/{py-pyjwt => py-pyjwt1}/distinfo  |  0
 www/{py-pyjwt => py-pyjwt1}/pkg-descr |  0
 www/seahub/Makefile                   |  3 ++-
 23 files changed, 52 insertions(+), 22 deletions(-)
Comment 24 Rainer Hurling freebsd_committer 2021-11-24 12:17:56 UTC
Sorry for the poor responsability from my side in the last weeks. I had and have a lot of trouble at work :(

And thanks for this commit, which is probably an important next step in the right direction. But shouldn't you have mentioned the maintainer timeout here?
Comment 25 Rainer Hurling freebsd_committer 2021-11-26 20:23:41 UTC
Created attachment 229749 [details]
patch with remaining diff for new port py-pyjwt

After renaming old port into py-pyjwt1 and adapting the dependencies in commit https://cgit.freebsd.org/ports/commit/?id=127e49fb45063a84e3f57540ef05b842d116a041, the remaining part of the patch is creating the new port py-pyjwt v2.1.0.

As mentioned already in comment #18, there is a conflict between the old and the new port:

Installing py38-pyjwt-2.1.0...
pkg-static: py38-pyjwt-2.1.0 conflicts with py38-pyjwt1-1.7.1 (installs files into the same place).  Problematic file: /usr/local/lib/python3.8/site-packages/jwt/__init__.py



We need to find an elegant way for both ports to coexist for a transition period. Simply putting a CONFLICTS= in both ports falls short, as then gradually some other ports would use both py-pyjwt1 and py-pyjwt dependencies.

The best solution would be to change the installation path for py-pyjwt1 a bit and thus avoid the conflict ;)
Comment 26 Goran Mekić 2021-11-26 21:28:06 UTC
What are our options? rename directory for pyjwt to pyjwt2 inside site-packages and patch downstream ports? Or maybe rename pyjwt1? Any other options?
Comment 27 Rainer Hurling freebsd_committer 2021-11-27 06:05:52 UTC
(In reply to Goran Mekić from comment #26)

I would prefer to rename the correct places within py-pyjwt1. E.g. that the path instead of lib/python3.x/site-packages/jwt is then lib/python3.x/site-packages/jwt1. Then the future port py-pyjwt v2.x would not need to be bent.

But I personally lack the knowledge to estimate how to do this in Python without errors.
Comment 28 Goran Mekić 2021-11-27 09:35:24 UTC
That's exactly what I'm afraid of. Practically changing module name requires all downstream ports to have a patch, and that's a lot of work. Not that amount of work is a problem, but finding all the corner cases for all downstream ports. As some of those ports are for integrating 3rd party services, it would be quite hard for someone to cover it all.