Bug 245929
| Summary: | security/py-cryptography: Update to 2.9.2 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Daniel Engberg <diizzy> | ||||||||
| Component: | Individual Port(s) | Assignee: | Kubilay Kocak <koobs> | ||||||||
| Status: | Closed FIXED | ||||||||||
| Severity: | Affects Only Me | CC: | ale_sagra, diizzy, dmitry.wagin, kai, mandree, pi, swills | ||||||||
| Priority: | --- | Flags: | koobs:
maintainer-feedback+
koobs: merge-quarterly- |
||||||||
| Version: | Latest | ||||||||||
| Hardware: | Any | ||||||||||
| OS: | Any | ||||||||||
| URL: | https://github.com/pyca/cryptography/blob/2.9.2/CHANGELOG.rst | ||||||||||
| See Also: |
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238968 https://github.com/pyca/cryptography/pull/5362 |
||||||||||
| Bug Depends on: | 245928 | ||||||||||
| Bug Blocks: | 245930, 250337 | ||||||||||
| Attachments: |
|
||||||||||
I apologize beforehand if this was supposed to go into the other PR. Anyhow, given https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238968#c3 I'll create a new patch later today or adressing that. Created attachment 213873 [details]
Patch for py-cryptography v2
Remove py-asn1crypto dependency, reported by koobs@ in PR 238968
Hi Kubilay, Is there anything else I can do regarding this PR? Best regards, Daniel koobs, ping? @Matthias This (2.8 -> 2.9.*) update requires substantial QA in particular for dependent ports and the versions they support. The changelogs for 2.8 and onward include several backward incompatible changes, and these need to be qualified for dependent ports. Making QA somewhat more challenging is that many python ports do not correctly/precisely declare version-specs in their *_DEPENDS, so a simple 'grep of these' isn't sufficient, compounding the challenge is that many ports don't have their test suites hooked up so while builds may succeed, runtime can remain broken. It's likely that an update to 2.8 first, and then subsequently to 2.9 and onward is the best approach. Where people can help: - If anyone wants to run a bulk run for a 2.8 update that would be handy - If anyone can run through current cryptography dependents to identify any first-order incompatible-with-2.8+ ports, that would be handy. *** Bug 247403 has been marked as a duplicate of this bug. *** Looking at repology the majority of "recently" released distros they're using 2.8+ ( https://repology.org/project/python:cryptography/versions ) so breakage isn't as bad as we think? lwhsu@ suggested on irc that we could copy the current version and keep it as backup if we run into issues as an alternative solution for troublesome ports. 3.0 is out, depends on py-cryptography-vectors 3.0 (In reply to Kubilay Kocak from comment #5) Here's an overview of the actual consumers and the QA results of security/py-cryptography: > Port Status Source Remarks > ------------------------------------- ------ -------------- ------------------------------------------------------- > cad/uranium OK N/A Not defined, but needed by Uranium-8d2bada/UM/Trust.py > databases/py-mycli OK >=1.0.0 setup.py > databases/py-sqlalchemy-utils OK >=0.6 setup.py > devel/py-adb OK N/A Noted as 'cryptography' in setup.py > devel/py-aiortc OK >=2.2 setup.py > devel/py-apns2 OK >=1.7.2 setup.py > devel/py-azure-keyvault OK >=2.1.4 setup.py > devel/py-azure-multiapi-storage OK N/A Noted as 'cryptography' in setup.py > devel/py-azure-storage-common OK N/A Noted as 'cryptography' in setup.py > devel/py-castellan OK >=2.1 requirements.txt > devel/py-castellan1 OK >=2.1 requirements.txt > devel/py-cursive OK !=1.3.0,>=1.0 requirements.txt > devel/py-fabric OK N/A Only referenced in documentation, seems not to be required by code > devel/py-oci OK ==2.8 Pinned requirements aren't checked at runtime, seems to work, should be patched out > devel/py-openstacksdk OK >=2.1 requirements.txt > devel/py-openstacksdk043 OK >=2.1 requirements.txt > devel/py-twisted OK >=2.5 src/twisted/python/_setup.py > dns/py-dns-lexicon OK N/A Noted as 'cryptography' in setup.py > finance/py-stripe OK N/A Not referenced in any file, might be superfluous > misc/py-cinder OK !=2.0,>=1.9 requirements.txt > net-mgmt/py-adal OK >=1.1.0 setup.py > net/py-ripe.atlas.sagan OK N/A Noted as 'cryptography' in setup.py > net/py-transip OK N/A Noted as 'cryptography' in setup.py > net/py-urllib3 OK >=1.3.4 setup.py > news/sabnzbdplus OK >=1.0 INSTALL.txt > security/cowrie OK >=0.9.1 setup.py > security/py-SecretStorage OK N/A Noted as 'cryptography' in setup.py > security/py-acme OK >=1.2.3 setup.py > security/py-asyncssh OK >=2.6.1 setup.py > security/py-authlib OK N/A Noted as 'cryptography' in setup.py > security/py-certbot OK >=1.2.3 setup.py > security/py-dfvfs OK >=2.0.2 setup.cfg > security/py-fido2 OK >=1.5 setup.py > security/py-josepy OK >=0.8 setup.py > security/py-keystone OK !=2.0,>=1.9 requirements.txt > security/py-msoffcrypto-tool OK >=2.3 setup.py > security/py-oauthlib OK N/A Noted as 'cryptography' in setup.py > security/py-openssl OK >=2.3 src/pyOpenSSL.egg-info/requires.txt (in setup.py as >= 2.2.1) > security/py-paramiko OK >=2.5 setup.py > security/py-paramiko1 OK N/A Seems not to be required > security/py-pgpy OK >=2.6 setup.py > security/py-pysaml2 OK >=1.4 setup.cfg > security/py-pysaml24 OK >=1.4 setup.cfg > security/py-python-axolotl OK N/A Noted as 'cryptography' in setup.py > security/py-requests-credssp OK N/A Noted as 'cryptography' in setup.py > security/py-securesystemslib OK >=2.2.2 setup.py > security/py-service_identity OK N/A Noted as 'cryptography' in setup.py > security/py-social-auth-core OK >=1.4 requirements-base.txt > security/py-trustme OK N/A Noted as 'cryptography' in setup.py > security/py-txtorcon OK N/A requirements.txt > sysutils/ansible OK N/A Noted as 'cryptography' in setup.py > sysutils/py-azure-cli OK >=2.3.1,<3.0.0 setup.py > www/buku OK >=1.2.3 setup.py > www/mitmproxy OK >=2.1.4,<2.4 setup.py (patched out) > www/py-aiohttp OK N/A Not referenced in code, listed as TEST_DEPENDS > www/py-azure-storage OK N/A Noted as 'cryptography' in setup.py > www/py-flask-jwt-extended OK >=2.3 setup.py > www/py-pyjwt OK >=1.4 setup.py > www/py-requests_ntlm OK >=1.3 setup.py > x11/xpra OK N/A Noted as 'cryptography' in setup.py A poudriere bulk run (11.3, 11.4, 12.1-RELEASE, 13.0-CURRENT@r363689) against all consumers listed from above gave no fallouts. Results of the testsuite via "make test": 11.3-RELEASE / OpenSSL 1.0.2s: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - py27: 97987 passed, 9101 skipped, 3 warnings in NNN seconds [amd64, i386] - py35, py36, py37, py38: 97987 passed, 9101 skipped, 26 warnings in NNN seconds [amd64, i386] 11.4-RELEASE / OpenSSL 1.0.2u: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - py27: 1 failed, 97986 passed, 9101 skipped, 3 warnings in NNN seconds [amd64, i386] - py35, py36, py37, py38: 1 failed, 97986 passed, 9101 skipped, 26 warnings in NNN seconds [amd64, i386] The test "TestECDSACertificate.test_load_ecdsa_no_named_curve" fails with: > self = <tests.x509.test_x509.TestECDSACertificate object at 0x859017810>, backend = <cryptography.hazmat.backends.openssl.backend.Backend object at 0x8026c3450> > > def test_load_ecdsa_no_named_curve(self, backend): > _skip_curve_unsupported(backend, ec.SECP256R1()) > cert = _load_cert( > os.path.join("x509", "custom", "ec_no_named_curve.pem"), > x509.load_pem_x509_certificate, > backend > ) > with pytest.raises(NotImplementedError): > > cert.public_key() > E Failed: DID NOT RAISE <class 'NotImplementedError'> > > tests/x509/test_x509.py:4133: Failed This is because OpenSSL 1.0.2t got some curve matching parameter backported and the checks assume at least OpenSSL 1.1.0+ for this feature. Upstream of security/py-cryptography recently added a workaround to cope with OpenSSL 1.0.2t/u in that situation. 12.1-RELEASE / OpenSSL 1.1.1d: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - py27: 103849 passed, 3239 skipped, 3 warnings in NNN seconds [amd64] 103847 passed, 3241 skipped, 3 warnings in NNN seconds [i386] - py35, py36, py37, py38 103849 passed, 3239 skipped, 26 warnings in NNN seconds [amd64] 103847 passed, 3241 skipped, 26 warnings in NNN seconds [i386] 13.0-CURRENT@r363689 [amd64] / OpenSSL 1.1.1g: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - py27: 103849 passed, 3239 skipped, 3 warnings in NNN seconds [amd64] 103847 passed, 3241 skipped, 3 warnings in NNN seconds [i386] - py35, py36, py37, py38 103849 passed, 3239 skipped, 26 warnings in NNN seconds [amd64] 103847 passed, 3241 skipped, 26 warnings in NNN seconds [i386] The upgrade path of security/py-cryptography also contains a few backwards incompatible changes as already noted in comment #5. Those changes should cause no problems to its consumers and because there are no bug/security fixes it isn't a MFH candidate: 2.9.x: ~~~~~~ - Support for Python 3.4 has been removed. -> Python 3.4 isn't present in the Ports tree => OK - Support for OpenSSL 1.0.1 has been removed -> FreeBSD 11.3-RELEASE uses OpenSSL 1.0.2s -> FreeBSD 11.4-RELEASE uses OpenSSL 1.0.2u -> FreeBSD 12.1-RELEASE uses OpenSSL 1.1.1d -> FreeBSD 13.0@r361498 uses OpenSSL 1.1.1g => OK - Support for LibreSSL 2.6.x has been removed -> security/libressl is at 3.1.3 in the Ports tree => OK - Reversed the order in which "cryptography.x509.Name.rfc4514_string" -> security/py-trustme is the only consumer that uses the "rfc4514_string" method. -> test suite of security/py-trustme -> OK, 13 passed => OK 2.8.x: ~~~~~~ - No backwards incompatible changes 2.7.x: ~~~~~~ - Removed the "cryptography.hazmat.primitives.mac.MACContext" interface. -> No port uses the "cryptography.hazmat.primitives.mac.MACContext" interface. => OK Given all those facts I'd suggest to land the release 2.9.2 of security/py-cryptography to have a new milestone. (In reply to daniel.engberg.lists from comment #8) The 3.0 release is still somewhat fresh and might break at least sysutils/py-azure-client (see also the related line in comment #9). We should revise the situation in the next quarter and as far I can tell, support for OpenSSL 1.0.2 will be dropped in the 3.1 release of security/py-cryptography so we need to be careful regarding FreeBSD 11.3/11.4 which use OpenSSL 1.0.2. (In reply to daniel.engberg.lists from comment #7) Indeed, for the case if there are problems once the 2.9.2 release is available in the Ports tree it should be not a problem to restore the older release by repo-copying it to security/py-cryptography26 and assign the problematic ports to it. Created attachment 216949 [details]
py-cryptography-2.9.2-with-openssl-102u.fix
Here's an updated patch that is based on Daniel's original patch:
The updated patch adds the workaround for OpenSSL 1.0.2u/t if the package is built for FreeBSD 11.3-STABLE and 11.4-RELEASE/STABLE.
It also keeps the line of the TEST_DEPENDS for security/py-cryptography-vectors unchanged as it's in the current state a good indicator to check the version of that port as well and update it, if required.
With the backported workaround the results of "make test" with FreeBSD 11.4 are:
py27:
97987 passed, 9101 skipped, 3 warnings in 370.64 seconds [amd64, i386]
py35, py36, py37, py38:
97987 passed, 9101 skipped, 26 warnings in 424.51 seconds [amd64, i386]
Please let me know if you have ideas or suggestions. If everyone is fine with the patch then security/py-cryptography should be updated soon.
(In reply to Kai Knoblich from comment #11) P.S.: The attached patch contains at line #65 a commented out line which is a remnant of a testing session. I already removed that line in my local repository. Great news, thanks for all your work on this! @Kai: Any news on progress? (In reply to daniel.engberg.lists from comment #14) Koobs, Kai@ being silent, what needs to happen to get this moving again? Landing this shortly, just confirming MFH intention with Kai A commit references this bug: Author: koobs Date: Fri Dec 4 11:31:24 UTC 2020 New revision: 556973 URL: https://svnweb.freebsd.org/changeset/ports/556973 Log: security/py-cryptography: Update to 2.9.2 [2] - Remove patch-PR4855, upstreamed [1] - Remove asn1crypto, no longer an install_requires (RUN_DEPENDS) [1] - Add workaround for OpenSSL 1.0.2u/t when building for FreeBSD 11.3-STABLE and 11.4-RELEASE/STABLE. [2] Changelog: https://github.com/pyca/cryptography/blob/2.9.2/CHANGELOG.rst HUGE thank you to Kai for running through extensive QA and producing the final changeset. PR: 245929 Submitted by: Daniel <daniel.engberg.lists pyret.net> [1] Submitted by: kai [2] MFH: No (backward incompatiblities, substantial dependents count) Changes: head/security/py-cryptography/Makefile head/security/py-cryptography/distinfo head/security/py-cryptography/files/openssl102u/ head/security/py-cryptography/files/openssl102u/patch-src___cffi__src_openssl_cryptography.py head/security/py-cryptography/files/openssl102u/patch-src_cryptography_hazmat_backends_openssl_backend.py head/security/py-cryptography/files/openssl102u/patch-src_cryptography_hazmat_backends_openssl_ec.py head/security/py-cryptography/files/patch-PR4855 Committed without MFH Thank you Daniel and Kai for the assistance and in particular the extensive requisite QA |
Created attachment 213807 [details] Patch for py-cryptography Update to 2.9.2 Remove patch-PR4855, fix committed upstream Adjust do-test section Tested on FreeBSD 13.0-CURRENT #0 r358620 (AMD64) Poudriere OK 12.1-RELEASE (AMD64) Test results (make test): 2.6.1 ===== 3 failed, 103737 passed, 3236 skipped, 26 warnings in 323.10 seconds ===== 2.9.2 ========== 103849 passed, 3239 skipped, 26 warnings in 350.54 seconds ==========