Bug 266937 - security/py-certbot broken in v 1.31.0,1
Summary: security/py-certbot broken in v 1.31.0,1
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-python (Nobody)
URL:
Keywords:
Depends on:
Blocks: 266989
  Show dependency treegraph
 
Reported: 2022-10-10 10:05 UTC by freebsd
Modified: 2022-10-15 07:42 UTC (History)
20 users (show)

See Also:
bugzilla: maintainer-feedback? (python)


Attachments
Adding latest poudriere build result for possible clues. (82.39 KB, text/plain)
2022-10-10 10:08 UTC, freebsd
no flags Details
Adding latest poudriere build result for possible clues. (41.75 KB, text/plain)
2022-10-10 10:08 UTC, freebsd
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description freebsd 2022-10-10 10:05:21 UTC
# /usr/local/bin/certbot renew
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==1.31.0', 'console_scripts', 'certbot')())
  File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/certbot/main.py", line 6, in <module>
    from certbot._internal import main as internal_main
  File "/usr/local/lib/python3.9/site-packages/certbot/_internal/main.py", line 19, in <module>
    import josepy as jose
  File "/usr/local/lib/python3.9/site-packages/josepy/__init__.py", line 40, in <module>
    from josepy.json_util import (
  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 14, in <module>
    from OpenSSL import crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/crypto.py", line 14, in <module>
    from cryptography import utils, x509
  File "/usr/local/lib/python3.9/site-packages/cryptography/x509/__init__.py", line 6, in <module>
    from cryptography.x509 import certificate_transparency
  File "/usr/local/lib/python3.9/site-packages/cryptography/x509/certificate_transparency.py", line 10, in <module>
    from cryptography.hazmat.bindings._rust import x509 as rust_x509
ImportError: cannot import name 'x509' from 'cryptography.hazmat.bindings._rust' (unknown location)
Comment 1 freebsd 2022-10-10 10:08:11 UTC
Created attachment 237185 [details]
Adding latest poudriere build result for possible clues.

Adding latest poudriere build result for possible clues.
Comment 2 freebsd 2022-10-10 10:08:51 UTC
Created attachment 237186 [details]
Adding latest poudriere build result for possible clues.

Adding latest poudriere build result for possible clues. (apache plugin)
Comment 3 freebsd 2022-10-10 10:13:11 UTC
These are the ports that were upgraded with py-certbot, triggering the error

py39-cryptography upgraded: 3.4.8 -> 38.0.1 
dbus upgraded: 1.14.0,1 -> 1.14.4,1 
py39-requests-toolbelt upgraded: 0.9.1_1 -> 0.10.0 
py39-acme upgraded: 1.30.0,1 -> 1.31.0,1 
ghostscript9-agpl-base upgraded: 9.56.1_4 -> 9.56.1_5 
bash upgraded: 5.2.2 -> 5.2.2_1 
py39-certbot upgraded: 1.30.0,1 -> 1.31.0,1 

Assuming that only the python ports contain a possible culprit.
Comment 4 freebsd 2022-10-10 14:53:23 UTC
I see that py-cryptography was touched Oct 9 (the day before this happened, and the certbot modules were done on Oct 7). Is that the likely issue?
Comment 5 freebsd 2022-10-10 14:55:24 UTC
Sorry for dragging in koobs and sunpoet
Comment 6 freebsd 2022-10-10 14:56:00 UTC
and yasu

But this appears to be a weird interplay of recently updated stuff.
Comment 7 Marius Schamschula 2022-10-10 17:38:51 UTC
It appears to be an issue with py-cryptography.

Under MacPorts I have installed py310-certbot 1.31.0 alongside with py310-cryptography 37.0.2 and I don't see this error.

Both my FreeBSD 13.1 servers show this error.
Comment 8 freebsd 2022-10-10 18:07:48 UTC
I reverted to Friday's builds (py39-certbot-1.30.0,1, py39-cryptography-3.4.8, py39-acme-1.30.0,1) and everything went back to normal.
Comment 9 freebsd 2022-10-10 18:11:34 UTC
Now also referenced in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266946
Comment 10 bagas 2022-10-10 18:26:12 UTC
As I understand it, me need to roll back to the old version of py39-cryptography-3.4.8, right?
Comment 11 Klaus 2022-10-10 21:02:42 UTC
(In reply to bagas from comment #10)
Yes, rolling back py-cryptography does the job.
Comment 12 Konstantin Belousov freebsd_committer freebsd_triage 2022-10-10 21:03:05 UTC
As I understand, the problems come from the following part of the
py-cryptography/Makefile:

post-patch:
        @${RM} -r ${WRKSRC}/src/rust/

?
Comment 13 Guido Falsi freebsd_committer freebsd_triage 2022-10-11 15:21:40 UTC
For information, ansible is affected too in the same way.
Comment 14 Guido Falsi freebsd_committer freebsd_triage 2022-10-11 15:27:18 UTC
(In reply to Konstantin Belousov from comment #12)

The proposed update (now outdated) in bug #254853 contains changes to also build the rust parts.

Looks like the rust parts are not optional anymore.
Comment 15 Guido Falsi freebsd_committer freebsd_triage 2022-10-11 15:55:07 UTC
Since I was needing ansible, I tested applying bug #254853 proposed patch (plus a PORTREVISION bump) and can confirm it works fine and fixes the issue.

I'll add that bug here as a dependency, since it actually fixes the issue.
Comment 16 Ivan Rozhuk 2022-10-11 16:29:34 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254853
contain patch to build latest py-cryptography with rust, it fixes this issue.

py-cryptography 3.4.8 does not with latest libressl.

IMHO we can not continue use old python staff, it requires more and more time to support.
Comment 17 Dima Panov freebsd_committer freebsd_triage 2022-10-11 22:19:36 UTC
(In reply to Ivan Rozhuk from comment #16)

rust bits makes all py-cryptograpgy consumers missed in any qemu builds (for example, aarch64 on amd64 host) due to impossible build rust itself under emulation.

Just my 2ยข. No reason to use libressl or any non-base *ssl. Welcome to dll hell, dude :)
Comment 18 Ivan Rozhuk 2022-10-11 23:34:57 UTC
(In reply to Dima Panov from comment #17)

I do not like rust, imho this is a big mistake to use it in any existing project.
But I use some apps that uses some libs where authors make this mistake and I have no resources to create forks, remove rust crap and backport all features from upstream in realtime.

My position is simple: rust is crap but I have no choice.

You can not forever use old py-cryptograpgy, so if you need something strange (aarch64) then open PR in py-cryptograpgy repo and let them to know about this.

DLL hell on linux/bsd is: openssl/gnutls/nss - too many TLS implementations that required at same time on my desktop.
Comment 19 Guido Falsi freebsd_committer freebsd_triage 2022-10-12 07:09:06 UTC
(In reply to Dima Panov from comment #17)

Not sure I understand the point. At present py-cryptography is broken for everyone. Allowing building rust parts will fix it for many.

I'm sorry I have no alternative solution to propose, if you need the old version you can fork the port.

I'm personally neutral towards rust , but I agree it contributes to a tendency of open source software to grow expensive dependencies (for example everything is now growing dependencies on HTML rendering engines with all the problems that entails). But it looks like the world has already made a choice about this, fighting this choice looks like fighting windmills.
Comment 20 bagas 2022-10-12 07:37:32 UTC
Decided, on new servers I will use security/acme.sh.
On older servers, I will continue to use certbot for now.
Comment 21 Guido Falsi freebsd_committer freebsd_triage 2022-10-12 08:57:00 UTC
(In reply to bagas from comment #20)

While this bug is titled security/py-certbot many other python software is affected and broken at present.

I personally never used py-certbot and always used acme.sh, but I also use ansible a lot and at present it is broken for everyone. This needs a solution soon.

Personally I've already solved it for me by applying the fix from bug #254853 locally.
Comment 22 Konstantin Belousov freebsd_committer freebsd_triage 2022-10-12 11:48:07 UTC
https://lwn.net/Articles/845535/
Comment 23 bagas 2022-10-13 10:12:42 UTC
When will there be an official fix in the ports?
Comment 24 Guido Falsi freebsd_committer freebsd_triage 2022-10-13 15:45:03 UTC
I just discovered the py-cryptography update also causes issues with py-openssl.

https://github.com/pyca/pyopenssl/issues/1143

Looks like py-openssl should be updated to 22.0.0 minimum
Comment 26 Yasuhiro Kimura freebsd_committer freebsd_triage 2022-10-15 07:42:11 UTC
Fixed by reverting update of security/py-cryptography to 38.0.1 with ports ed4bec1e0139.