Bug 254853 - security/py-cryptography: Update to 41.0.3
Summary: security/py-cryptography: Update to 41.0.3
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Li-Wen Hsu
URL: https://www.freshports.org/security/p...
Keywords: needs-qa
: 266989 (view as bug list)
Depends on: 254851 275397
Blocks: 256885 258413 264993 266680 272885 273508
  Show dependency treegraph
 
Reported: 2021-04-07 12:10 UTC by Jeroen Pulles
Modified: 2024-02-19 13:20 UTC (History)
43 users (show)

See Also:
tcberner: maintainer-feedback-
brd: merge-quarterly?


Attachments
patch for 3.4.7 (8.25 KB, patch)
2021-04-07 12:10 UTC, Jeroen Pulles
no flags Details | Diff
Don't build rust modules (3.4.7 specific) (4.32 KB, patch)
2021-06-07 11:17 UTC, Jeroen Pulles
no flags Details | Diff
git diff for security/py-cryptography (14.73 KB, patch)
2022-05-15 16:02 UTC, Bernard Spil
brnrd: maintainer-approval? (sunpoet)
Details | Diff
update to 38.0.1 with rust build (22.56 KB, patch)
2022-09-28 13:19 UTC, Ivan Rozhuk
no flags Details | Diff
update to 38.0.1 with rust build (15.75 KB, patch)
2022-10-09 22:35 UTC, Ivan Rozhuk
no flags Details | Diff
update to 38.0.1 with rust build (22.85 KB, patch)
2022-10-15 00:54 UTC, Ivan Rozhuk
no flags Details | Diff
update to 38.0.1 with rust build (25.42 KB, patch)
2023-01-11 19:59 UTC, Ivan Rozhuk
no flags Details | Diff
update to 38.0.1 with rust build (26.57 KB, patch)
2023-03-12 19:43 UTC, Ivan Rozhuk
no flags Details | Diff
update to 39.0.2 with rust build (28.86 KB, patch)
2023-03-18 20:53 UTC, Ivan Rozhuk
no flags Details | Diff
update to 39.0.2 with rust build (29.66 KB, patch)
2023-03-29 00:42 UTC, Ivan Rozhuk
rozhuk.im: maintainer-approval?
Details | Diff
v0 (26.26 KB, patch)
2023-07-12 14:39 UTC, Mikael Urankar
no flags Details | Diff
v1 (26.26 KB, patch)
2023-07-26 08:28 UTC, Mikael Urankar
no flags Details | Diff
Update py-cryptography{,-vectors} to 41.0.3 (28.04 KB, patch)
2023-08-29 14:38 UTC, Li-Wen Hsu
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jeroen Pulles 2021-04-07 12:10:33 UTC
Created attachment 223896 [details]
patch for 3.4.7

py-cryptography is now using a bit of rust in an extension; 

I've filed a bug for a new port for setuptools-rust, #254851

We've made this to work by USES=cargo, but then making sure that setuptools-rust does the compilation, with the CARGO_BUILD CARGO_INSTALL CARGO_TEST =no
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-04 02:02:55 UTC
Thanks Jeroen, does this depend on bug 254851? If so please add it to the Depends On field, thanks!
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-04 12:41:44 UTC
I have it from cryptography upstream that the 3.4 branch is safe to use without rust (and doesnt require it), as long as we set the environment variable to not use it.

Looks like the patch attached here is the reverse of what was intended?

@Jeroen Are you able to do a plain 3.4.x update here (adding the right env var not to use rust, see setup.py), and then create a separate issue for the future 35.0 branch coming soon that includes the changes you included here? (using rust).
Comment 3 Jeroen Pulles 2021-06-07 11:17:51 UTC
Created attachment 225623 [details]
Don't build rust modules (3.4.7 specific)

You mean setting `CRYPTOGRAPHY_DONT_BUILD_RUST=`. The setup.py still imports setuptools_rust. Under the assumption that it is completely safe to also remove the setuptools_rust import, I've patched setup.py, and not set the environment variable. 

I don't see anything else using CRYPTOGRAPHY_DONT_BUILD_RUST.

The environment variable is documented in https://cryptography.io/en/3.4.7/faq.html. It took me a while to understand that that instruction is gone with the master version of docs.
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-08 00:01:42 UTC
(In reply to Jeroen Pulles from comment #3)

Requesting clarity from upstream on the nature of the import error and the extent to which its related to, or should be conditional on the env var not being present
Comment 5 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-08 00:52:56 UTC
Upstream advises that:

 - patching out the error and passing the env var, if the tests pass, is OK
 - that they'll accept an upstream for conditionalizing the import/error block on the non presence of the env var
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-11 22:23:28 UTC
Minor changes to commit:

 - Use specific *_DEPENDS lines (with <version-specs>) over shared macros 
 - Add comment to patch "setuptools_rust try/except should be conditional on environment variable"
Comment 7 Pavel Timofeev 2022-01-17 19:49:19 UTC
Looking forward for this
Comment 8 Rene Ladan freebsd_committer freebsd_triage 2022-03-07 19:54:58 UTC
Maintainer reset.
Comment 9 Jeroen Pulles 2022-03-10 16:58:45 UTC
The python.mk do-install step should register all files with the `--record` argument. On my installs this fails to register the cryptography/hazmat/bindings/_rust.abi3.so file that is in the build and stage directories. The _rust.abi3.so file is not in the .PLIST.pymodtmp file, and thus isn't installed. 

If I set `PYDISTUTILS_SETUP=${PYSETUP}` in the py-cryptography makefile, I get the same result: no _rust.abi3.so file. 

However, if I manually run `python3.9 setup.py install --record my-file.log` in the work directory, then _rust.abi3.so is found in "my-file.log". 

I don't understand where this goes wrong. 

How can I get this file included in the (auto-generated) plist?
Comment 10 mark burdett 2022-03-23 10:44:45 UTC
Hopefully this port can be updated soon, as py-openssl has been updated and now requires py-cryptography >= 35.0
Comment 11 mark burdett 2022-03-24 20:29:06 UTC
(Never mind that, py-openssl has been downgraded so things are no longer broken, see bug #262750)
Comment 12 Bernard Spil freebsd_committer freebsd_triage 2022-04-30 10:11:09 UTC
From security/py-openssl:

> # We need to keep old py-cryptography and py-penssl for 11.x release
> # due to outdated OpenSSL version in base

No longer seems relevant after 2021-09-31 EoL of 11.x

What does it take to land this in ports?
Comment 13 Bernard Spil freebsd_committer freebsd_triage 2022-05-15 16:02:46 UTC
Created attachment 233937 [details]
git diff for security/py-cryptography

```
security/py-cryptography: Update to 37.0.2

 * Remove Rust dependency
 * Supports LibreSSL 3.2 - 3.5


PR:   254853
Submitted by: Jeroen Pulles
```
Comment 14 Andris Raugulis 2022-09-26 20:11:49 UTC
Any movement on this?
Comment 15 Ivan Rozhuk 2022-09-28 13:19:40 UTC
Created attachment 236907 [details]
update to 38.0.1 with rust build

works for me
Comment 16 Ivan Rozhuk 2022-09-28 13:22:28 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266670 required to build with fresh libressl.
Comment 17 Ivan Rozhuk 2022-10-09 22:17:33 UTC
Latest 38.0.1 from ports does not work for me:
ImportError: cannot import name 'ObjectIdentifier' from 'cryptography.hazmat.bindings._rust' (unknown location)
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/site-packages/homeassistant/__main__.py", line 203, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/homeassistant/__main__.py", line 159, in main
    args = get_arguments()
  File "/usr/local/lib/python3.9/site-packages/homeassistant/__main__.py", line 69, in get_arguments
    from . import config as config_util
  File "/usr/local/lib/python3.9/site-packages/homeassistant/config.py", line 19, in <module>
    from . import auth
  File "/usr/local/lib/python3.9/site-packages/homeassistant/auth/__init__.py", line 10, in <module>
    import jwt
  File "/usr/local/lib/python3.9/site-packages/jwt/__init__.py", line 1, in <module>
    from .api_jwk import PyJWK, PyJWKSet
  File "/usr/local/lib/python3.9/site-packages/jwt/api_jwk.py", line 6, in <module>
    from .algorithms import get_default_algorithms
  File "/usr/local/lib/python3.9/site-packages/jwt/algorithms.py", line 6, in <module>
    from .utils import (
  File "/usr/local/lib/python3.9/site-packages/jwt/utils.py", line 7, in <module>
    from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurve
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/primitives/asymmetric/ec.py", line 11, in <module>
    from cryptography.hazmat._oid import ObjectIdentifier
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/_oid.py", line 7, in <module>
    from cryptography.hazmat.bindings._rust import (
ImportError: cannot import name 'ObjectIdentifier' from 'cryptography.hazmat.bindings._rust' (unknown location)
==============================================================

with mine patch that include rust it work.
Comment 18 Ivan Rozhuk 2022-10-09 22:35:03 UTC
Created attachment 237172 [details]
update to 38.0.1 with rust build

Rebased patch includes rust code build.
Comment 19 Ralf van der Enden 2022-10-12 02:59:11 UTC
I've successfully built the 38.0.1+rust patch. This however also requires py-opensll to be upgraded to 22.0
0. With that combination py-certbot works fine again.
Comment 20 freebsd 2022-10-12 19:48:18 UTC
Hi, I cannot succeed in patching port from attachmnent 237172

It's a long time since I did any programming, and I do not know what should be in the patch file

I etracted the "raw unified" view, but when using patch it asks for the name of the file to pach.

If someone can help...
Comment 21 Ivan Rozhuk 2022-10-12 20:06:43 UTC
Try:
patch -p1 --directory /usr/ports/ --input RAW_UNIFIED.patch
Comment 22 freebsd 2022-10-13 09:03:21 UTC
(In reply to Ivan Rozhuk from comment #21)
Thanks, now duplicity works again, and I have backups
Comment 23 Ivan Rozhuk 2022-10-15 00:54:58 UTC
Created attachment 237314 [details]
update to 38.0.1 with rust build
Comment 24 Ivan Rozhuk 2023-01-11 19:59:09 UTC
Created attachment 239411 [details]
update to 38.0.1 with rust build
Comment 25 László Károlyi 2023-02-05 15:29:40 UTC
I see diffs are added here to build 38.0.1.

Is there anything that blocks this from being rolled out? A lot of broken dependencies because the obsolete version of this package, py39-matrix-synapse being one of them.
Comment 27 Enji Cooper freebsd_committer freebsd_triage 2023-03-06 23:11:51 UTC
Looking at the project's setup.py file, py-setuptools-rust is a hard requirement.
Comment 28 Enji Cooper freebsd_committer freebsd_triage 2023-03-07 08:57:03 UTC
Proposed patch posted to Differential Review: https://reviews.freebsd.org/D38939 .
Comment 29 Ralf van der Enden 2023-03-10 23:04:35 UTC
(In reply to Enji Cooper from comment #28)
The diff from the review builds a package that fails when running certbot. The diff 'update to 38.01 with rust build' updated to 39.0.2 works as expected. I think it has to do with where the tarball if fetched from, since changing to PYPI from GH creates a non-working package.
Comment 30 Enji Cooper freebsd_committer freebsd_triage 2023-03-12 19:32:18 UTC
(In reply to Ralf van der Enden from comment #29)

Hmm... I see. I'll see if I can replicate what you're describing.
Comment 31 Ivan Rozhuk 2023-03-12 19:43:54 UTC
Created attachment 240799 [details]
update to 38.0.1 with rust build
Comment 32 Enji Cooper freebsd_committer freebsd_triage 2023-03-13 09:02:32 UTC
(In reply to Enji Cooper from comment #30)

I incorporated the missing post-install logic and applied the other logic to the diff. Please re-review.
Comment 33 Charlie Li freebsd_committer freebsd_triage 2023-03-13 10:39:57 UTC
Until Rust builds under QEMU_EMULATING, this really can't be updated to something that hard requires Rust. You're talking obliteration of a large swath of the Python ecosystem for those who can't build on native hardware.

The cryptography team only migrated the non-SSL/TLS stuff, like X.509 support, to Rust; the rest is still a wrapper around OpenSSL/LibreSSL. I backported much of what was needed to make cryptography 3.4 work with current OpenSSL/LibreSSL, though not necessarily the API from later cryptography: https://github.com/vishwin/py-cryptography/tree/3.4.x

It is better to provide a separate security/py-cryptography-rust (or similarly named) in the same vein as graphics/librsvg2{,-rust} with an appropriate selector.
Comment 34 Sascha Biberhofer 2023-03-13 19:16:03 UTC
(In reply to Charlie Li from comment #33)

On the other hand, not updating this port puts us in a bind with other projects, which now require newer versions of py-cryptography than the one we're shipping. py-openssl, for example, requires cryptography>=38.0.0 and py-twisted in turn had to be hackfixed/patched to even work with our version of py-openssl anymore. Maintaining an ever growing set of patches doesn't seem like that great of an idea either.

Regardless of whether keep the old version around and place the new one in security/py-cryptography-rust or update this port, I think we should decide on one of these solutions rather soon so that other ports can move ahead.
Comment 35 Enji Cooper freebsd_committer freebsd_triage 2023-03-13 21:51:39 UTC
I tend to agree with Sascha: as of right now all projects reliant on modern copies of py-openssl, py-pycryptography, etc, are stuck. This is blocking modern python support (newer versions of py-pycryptography are guaranteed to be better supported with 3.11+) and this is blocking us from switching over to openssl3.
I am game for conditionalizing support -- but it shouldn't block the upgrades. I'll start a conversation upstream with the cryptography project so setuptools-rust dependent pieces can (maybe) be split off into their own packages/wheels.
Comment 36 Enji Cooper freebsd_committer freebsd_triage 2023-03-13 22:02:20 UTC
(In reply to Enji Cooper from comment #35)

https://github.com/pyca/cryptography/issues/8513
Comment 37 Enji Cooper freebsd_committer freebsd_triage 2023-03-13 22:45:04 UTC
Looking into this, the whole splitting of rust vs non-rust building is an all or nothing change that will affect multiple ports that require x509 support. I don't think it's worth implementing 
If folks are interested in conditionalizing the rust build for their specific build use cases, I could see adding that support, but introducing additional complexity in the port doesn't make a ton of sense since it would require monkey patching the port a lot to make it conditionally build and install hazmat/fernet modules which rely on rust bindings, e.g., x509, etc.
Comment 38 Enji Cooper freebsd_committer freebsd_triage 2023-03-13 22:46:14 UTC
(In reply to Enji Cooper from comment #37)

Incomplete thought...

"I don't think it's worth implementing in FreeBSD ports, since it would introduce a larger longterm maintenance burden."
Comment 39 Charlie Li freebsd_committer freebsd_triage 2023-03-13 23:26:19 UTC
Considering that I did all that backporting so 3.4 could remain operable, with LibreSSL to boot, the maintenance burden is well understood and at least I have been willing to keep it up. Upstream does a good job with their CI, since they test consumers in addition to their own stuff, so keeping that synced makes things easier. I actually just renewed a Let's Encrypt certificate with the latest certbot using this on Python 3.10, so not everything is broken or has a bleak outlook.

Splitting the rust bits off isn't a good idea, agreed, since SSL/TLS libraries always had the functionality that cryptography wrapped/bound, until they decided that C was not good enough for anything but the actual SSL/TLS bits. But I also agree that we do need to make an updated cryptography available, hence suggesting (and has been on my list for time) the graphics/librsvg2{,-rust} approach.
Comment 40 Ivan Rozhuk 2023-03-18 20:53:20 UTC
Created attachment 240959 [details]
update to 39.0.2 with rust build
Comment 41 Ivan Rozhuk 2023-03-29 00:42:09 UTC
Created attachment 241170 [details]
update to 39.0.2 with rust build

rebased
Comment 42 Po-Chuan Hsieh freebsd_committer freebsd_triage 2023-04-09 21:29:59 UTC
*** Bug 266989 has been marked as a duplicate of this bug. ***
Comment 43 Po-Chuan Hsieh freebsd_committer freebsd_triage 2023-04-09 21:30:44 UTC
I'm working on the 40.0.1 update [1]. Please test it. Thanks.

[1] https://people.freebsd.org/~sunpoet/patch/security-py-cryptography.txt
Comment 44 Charlie Li freebsd_committer freebsd_triage 2023-04-09 21:42:57 UTC
This should still be a separate port with a DEFAULT_VERSIONS knob. QEMU_EMULATING users are still affected.
Comment 45 Enji Cooper freebsd_committer freebsd_triage 2023-04-16 02:54:26 UTC
(In reply to Charlie Li from comment #44)

Upstream has made a convincing argument against _not_ doing that. It sounds like the longterm goal is to expand rust use, not reduce it, in the port. I think FreeBSD just needs to bite the bullet and do something to either help by throwing resources against our private builders, or assist upstream with producing regular prebuilt binaries (which should greatly help everyone the community at large).
Comment 46 Enji Cooper freebsd_committer freebsd_triage 2023-04-16 03:37:48 UTC
(In reply to Enji Cooper from comment #45)

To clarify, when I said "upstream" in this comment, I meant rust, not py-cryptography.

If rust doesn't have to be built from scratch each time, it will definitely reduce the cost on the qemu builders.
Comment 47 László Károlyi 2023-04-16 16:51:26 UTC
(In reply to Enji Cooper from comment #46)
If it's any help, the creator of poudriere is working on an update where some packages can be pre-added to an enviroment (such as rust) so they don't build anew.

I had this problem with clamd and a jail I built from the start.

Here's the github issue about it: https://github.com/freebsd/poudriere/issues/822
Comment 48 Enji Cooper freebsd_committer freebsd_triage 2023-04-16 17:06:22 UTC
(In reply to László Károlyi from comment #47)

Thank you for the link László! I'll reach out to bdrewery about that effort -- it would definitely address the need for this port dealing with rust.
Comment 49 Charlie Li freebsd_committer freebsd_triage 2023-04-21 02:44:58 UTC
That poudriere issue is not relevant here, not in the slightest, because it does not address how to avoid building or executing rustc in QEMU_EMULATING, which alone prevents building any port with a Rust component in that environment. You would need something akin to the static ccache option *inserted* into a builder jail on creation to avoid QEMU_EMULATING since packages are built for specific architectures.
Comment 50 László Károlyi 2023-04-21 07:55:10 UTC
It certainly solves the building part, but I didn't know it has problems with executing under QEMU. That's very bad.
Comment 51 epopen 2023-04-24 17:00:01 UTC
Because I have a story, I hope to upgrade the security/py-cryptography too.


I had upgraded security/openssl to 3.1.
and I running certification file update script @ security/py-certbot and got error as follows... 
Renewing Let's Encrypt certificates:
Traceback (most recent call last):
  File "/usr/local/bin/certbot-3.9", line 33, in <module>
    sys.exit(load_entry_point('certbot==2.4.0', 'console_scripts', 'certbot')())
  File "/usr/local/bin/certbot-3.9", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/certbot/main.py", line 6, in <module>
    from certbot._internal import main as internal_main
  File "/usr/local/lib/python3.9/site-packages/certbot/_internal/main.py", line 21, in <module>
    import josepy as jose
  File "/usr/local/lib/python3.9/site-packages/josepy/__init__.py", line 40, in <module>
    from josepy.json_util import (
  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 14, in <module>
    from OpenSSL import crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/crypto.py", line 17, in <module>
    from OpenSSL._util import (
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: /usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so: Undefined symbol "ERR_GET_FUNC"

Errors were reported when renewing Let's Encrypt certificate(s).

After googled, answer is current version of security/py-cryptography too old (3.4.8)

Thanks a lot.
Comment 52 Enji Cooper freebsd_committer freebsd_triage 2023-05-02 01:46:07 UTC
Bug 254853 looks like the issue where rust fails to build with QEMU_EMULATING.
Comment 53 Guido Falsi freebsd_committer freebsd_triage 2023-06-28 07:56:38 UTC
Hi,

I'm adding this information here, instead of filing a new bug report.

security/py-cryptography fails at runtime oh head with OpenSSL 3, most probably version 3.4.8 present in the ports tree does not properly support the new OpenSSL.

The error is:

ImportError: /usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so: Undefined symbol "ERR_GET_FUNC"


I guess getting this library updated is now even more urgent.
Comment 54 Guido Falsi freebsd_committer freebsd_triage 2023-06-28 07:58:28 UTC
(In reply to Guido Falsi from comment #53)

Opps, this information was already present, but I missed it for some reason, sorry for the noise.
Comment 55 Mikael Urankar freebsd_committer freebsd_triage 2023-07-12 14:39:31 UTC
Created attachment 243353 [details]
v0

Update to 41.0.2

It fixes ansible for me
Comment 56 Mikael Urankar freebsd_committer freebsd_triage 2023-07-12 14:41:22 UTC
(In reply to Mikael Urankar from comment #55)
this is based on Po-Chuan Hsieh's patch
Comment 57 Enji Cooper freebsd_committer freebsd_triage 2023-07-12 17:09:50 UTC
Patches which upgrade the port have been available for several months now. With OpenSSL 3 in base for the past few weeks, I don’t understand why continuing to build broken ports is acceptable.
What is blocking the update from being done?
Comment 58 Brad Davis freebsd_committer freebsd_triage 2023-07-20 15:31:27 UTC
I think it is time to go ahead with this and deal with Tier-2 archs later.
Comment 59 Tobias C. Berner freebsd_committer freebsd_triage 2023-07-22 05:50:33 UTC
Moin moin 

I'll take this with my portmgr hat.

mfg Tobias
Comment 60 commit-hook freebsd_committer freebsd_triage 2023-07-22 07:49:18 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=b28a739884e37c25bb57ea5f157252adb63065d3

commit b28a739884e37c25bb57ea5f157252adb63065d3
Author:     Tobias C. Berner <tcberner@FreeBSD.org>
AuthorDate: 2023-07-22 07:38:27 +0000
Commit:     Tobias C. Berner <tcberner@FreeBSD.org>
CommitDate: 2023-07-22 07:46:38 +0000

    framework: convert tree to use USES=pycryptography

    A future commit will update to security/py-cryptography will introduce a
    rust dependency.

    PR:             254853

 cad/uranium/Makefile                       | 3 +--
 comms/py-esptool/Makefile                  | 3 +--
 databases/py-mycli/Makefile                | 3 +--
 databases/py-ormar/Makefile                | 3 +--
 databases/py-sqlalchemy-utils/Makefile     | 2 +-
 devel/oci-cli/Makefile                     | 3 +--
 devel/osc/Makefile                         | 5 ++---
 devel/py-adb/Makefile                      | 5 ++---
 devel/py-aiortc/Makefile                   | 3 +--
 devel/py-azure-identity/Makefile           | 3 +--
 devel/py-azure-keyvault/Makefile           | 3 +--
 devel/py-azure-multiapi-storage/Makefile   | 3 +--
 devel/py-azure-storage-common/Makefile     | 3 +--
 devel/py-castellan/Makefile                | 5 ++---
 devel/py-cursive/Makefile                  | 3 +--
 devel/py-moto/Makefile                     | 3 +--
 devel/py-msal/Makefile                     | 5 ++---
 devel/py-oci/Makefile                      | 3 +--
 devel/py-openstacksdk/Makefile             | 3 +--
 devel/py-pip/Makefile                      | 5 ++---
 devel/py-twisted/Makefile                  | 2 +-
 dns/letsdns/Makefile                       | 5 ++---
 dns/py-dns-crawler/Makefile                | 3 +--
 dns/py-dns-lexicon/Makefile                | 3 +--
 dns/py-dnspython/Makefile                  | 2 +-
 finance/electrum/Makefile                  | 3 +--
 finance/py-ccxt/Makefile                   | 5 ++---
 mail/py-flanker/Makefile                   | 3 +--
 misc/py-apache-beam/Makefile               | 5 ++---
 misc/py-cinder/Makefile                    | 3 +--
 net-im/py-matrix-synapse/Makefile          | 3 +--
 net-mgmt/py-adal/Makefile                  | 5 ++---
 net-p2p/reticulum/Makefile                 | 5 ++---
 net/onionprobe/Makefile                    | 5 ++---
 net/py-cepa/Makefile                       | 5 ++---
 net/py-ripe.atlas.sagan/Makefile           | 5 ++---
 net/py-smbprotocol/Makefile                | 5 ++---
 net/py-transip/Makefile                    | 5 ++---
 net/py-urllib3/Makefile                    | 6 +++---
 net/scapy/Makefile                         | 5 ++---
 news/sabnzbd/Makefile                      | 2 +-
 security/caldera/Makefile                  | 3 +--
 security/cowrie/Makefile                   | 3 +--
 security/py-SecretStorage/Makefile         | 5 ++---
 security/py-acme/Makefile                  | 5 ++---
 security/py-asyncssh/Makefile              | 5 ++---
 security/py-authlib/Makefile               | 4 +---
 security/py-azure-keyvault-keys/Makefile   | 3 +--
 security/py-badkeys/Makefile               | 5 ++---
 security/py-cert-human/Makefile            | 5 ++---
 security/py-certbot/Makefile               | 3 +--
 security/py-dfvfs/Makefile                 | 3 +--
 security/py-fido2/Makefile                 | 1 +
 security/py-josepy/Makefile                | 5 ++---
 security/py-msoffcrypto-tool/Makefile      | 5 ++---
 security/py-noiseprotocol/Makefile         | 4 +---
 security/py-oauthlib/Makefile              | 6 +++---
 security/py-openssl/Makefile               | 5 ++---
 security/py-paramiko/Makefile              | 3 +--
 security/py-pgpy/Makefile                  | 5 ++---
 security/py-plaso/Makefile                 | 3 +--
 security/py-pyhanko-certvalidator/Makefile | 3 +--
 security/py-pyhanko/Makefile               | 3 +--
 security/py-pysaml2/Makefile               | 5 ++---
 security/py-pysaml24/Makefile              | 5 ++---
 security/py-pysaml26/Makefile              | 5 ++---
 security/py-pyspnego/Makefile              | 3 +--
 security/py-python-axolotl/Makefile        | 5 ++---
 security/py-requests-credssp/Makefile      | 5 ++---
 security/py-securesystemslib/Makefile      | 5 ++---
 security/py-service_identity/Makefile      | 3 +--
 security/py-social-auth-core/Makefile      | 5 ++---
 security/py-sshpubkeys/Makefile            | 5 ++---
 security/py-sslyze/Makefile                | 5 ++---
 security/py-stem/Makefile                  | 3 +--
 security/py-trustme/Makefile               | 5 ++---
 security/py-txtorcon/Makefile              | 3 +--
 security/py-webauthn/Makefile              | 3 +--
 security/py-yubikey-manager/Makefile       | 3 +--
 sysutils/datadog-integrations/Makefile     | 9 +++------
 sysutils/py-ansible-core/Makefile          | 5 ++---
 sysutils/py-azure-cli-core/Makefile        | 3 +--
 sysutils/py-azure-cli/Makefile             | 3 +--
 textproc/py-pdfminer.six/Makefile          | 5 ++---
 www/buku/Makefile                          | 3 +--
 www/mitmproxy/Makefile                     | 3 +--
 www/py-autobahn/Makefile                   | 5 ++---
 www/py-azure-storage/Makefile              | 3 +--
 www/py-flask-jwt-extended/Makefile         | 3 +--
 www/py-pyjwt/Makefile                      | 2 +-
 www/py-pyjwt1/Makefile                     | 4 ++--
 www/py-requests_ntlm/Makefile              | 5 ++---
 www/py-scrapy/Makefile                     | 5 ++---
 x11/xpra/Makefile                          | 3 +--
 94 files changed, 141 insertions(+), 229 deletions(-)
Comment 61 commit-hook freebsd_committer freebsd_triage 2023-07-22 07:49:25 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=173ac9651cc9ef0ab952b29e4cffa56d28e8770b

commit 173ac9651cc9ef0ab952b29e4cffa56d28e8770b
Author:     Tobias C. Berner <tcberner@FreeBSD.org>
AuthorDate: 2023-07-22 06:00:10 +0000
Commit:     Tobias C. Berner <tcberner@FreeBSD.org>
CommitDate: 2023-07-22 07:46:37 +0000

    framework: add PYCRYPTOGRAPHY default version

    A future commit will update to security/py-cryptography will introduce a
    rust dependency.

    PR:             254853

 Mk/bsd.default-versions.mk |  8 +++++++-
 UPDATING                   | 12 ++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)
Comment 62 commit-hook freebsd_committer freebsd_triage 2023-07-22 07:49:29 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=7bb64b89d0e5ec8d77b28f8341269ffbad279ebf

commit 7bb64b89d0e5ec8d77b28f8341269ffbad279ebf
Author:     Tobias C. Berner <tcberner@FreeBSD.org>
AuthorDate: 2023-07-22 06:16:46 +0000
Commit:     Tobias C. Berner <tcberner@FreeBSD.org>
CommitDate: 2023-07-22 07:46:38 +0000

    framework: add Mk/Uses/pycryptography

    With the added new DEFAULT_VERSION "PYCRYPTOGRAHY_DEFAULT=rust|legacy"
    it is important to transparently depend on the right one.

    This is the job of this new uses.

    So instead of manually adding dependency lines like

            RUN_DEPENDS=${PYTHON_PKGNAMEPREFIX}cryptography>0:security/py-cryptography@${PY_FLAVOR}

    this should now be

            USES=pycryptography:run

    Supported arguments: <none>,build,run,test
    If no argument is given, it defaults to 'build,run'.

    A future commit will update to security/py-cryptography will introduce a
    rust dependency.

    PR:             254853

 CHANGES                         | 20 ++++++++++++++++++
 Mk/Uses/pycryptography.mk (new) | 46 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+)
Comment 63 commit-hook freebsd_committer freebsd_triage 2023-07-22 07:49:32 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=c380909c8060259c0eb8aa067de08534677924b4

commit c380909c8060259c0eb8aa067de08534677924b4
Author:     Tobias C. Berner <tcberner@FreeBSD.org>
AuthorDate: 2023-07-22 05:51:54 +0000
Commit:     Tobias C. Berner <tcberner@FreeBSD.org>
CommitDate: 2023-07-22 07:46:37 +0000

    security/py-cryptography: copy port to -legacy variant prior to update

    A future commit will update to security/py-cryptography will introduce a
    rust dependency.

    PR:             254853

 security/Makefile                                  |   1 +
 security/py-cryptography-legacy/Makefile (new)     |  51 ++++
 security/py-cryptography-legacy/distinfo (new)     |   3 +
 .../files/patch-libressl (new)                     | 316 +++++++++++++++++++++
 .../files/patch-setup.py (new)                     |  55 ++++
 security/py-cryptography-legacy/pkg-descr (new)    |   7 +
 6 files changed, 433 insertions(+)
Comment 64 Tobias C. Berner freebsd_committer freebsd_triage 2023-07-22 07:51:08 UTC
Moin moin 

I added a new pycryptography uses and a default version PYCRYPTOGRAPHY_DEFAULT=rust|legacy.

@mikael, please update the security/py-cryptography port to the one using rust.


Please let me know if I broke something :)

mfg Tobias
Comment 65 Tobias C. Berner freebsd_committer freebsd_triage 2023-07-22 11:39:32 UTC
(In reply to Charlie Li from comment #33)

I think it would be advisable to switch the legacy-ports upstream to your fork, given the looming incompatibility with base OpenSSL.

Could you do that part, please?


mfg Tobias
Comment 66 Mikael Urankar freebsd_committer freebsd_triage 2023-07-26 08:28:27 UTC
Created attachment 243623 [details]
v1
Comment 67 Guido Falsi freebsd_committer freebsd_triage 2023-07-26 08:58:40 UTC
(In reply to Mikael Urankar from comment #66)

Great work with your patches! Hope they get committed to the tree ASAP.

I am using locally your v0 patch at present. It works but there is an issue with it, you correctly replaced `.include <bsd.port.post.mk>` with `.include <bsd.port.mk>` but you left `.include <bsd.port.pre.mk>` which is wrong and causes the build to fail in poudriere due to failed sanity checks.

Apart from this v0 patch looks fine to me. I'm in the process of applying this new v1 patch.
Comment 68 void 2023-08-02 20:38:35 UTC
(In reply to Mikael Urankar from comment #66)

Hi,

I made the following PR earlier:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272885

Your patch applied cleanly with git apply and py-cryptography built and installed fine. 

This PR was later pointed out to me by markml because I was getting the error "Undefined symbol "ERR_GET_FUNC"" when running certbot. 

After reinstalling py-cryptography with your patch, I recompiled certbot (just from the ports tree, this is on recent -current with a ports tree as of a few hrs ago). certbot builds and installs fine but when I try to run it, it bails with this error at the end: 

  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 43, in _legacy_provider_error
    raise RuntimeError(
RuntimeError: OpenSSL 3.0's legacy provider failed to load. This is a fatal error by default, but cryptography supports running without legacy algorithms by setting the environment variable CRYPTOGRAPHY_OPENSSL_NO_LEGACY. If you did not expect this error, you have likely made a mistake with your OpenSSL configuration.
Comment 69 Mikael Urankar freebsd_committer freebsd_triage 2023-08-03 07:34:36 UTC
(In reply to void from comment #68)
setenv CRYPTOGRAPHY_OPENSSL_NO_LEGACY 1
Comment 70 void 2023-08-03 09:38:45 UTC
(In reply to Mikael Urankar from comment #69)

Hi,

I tried the following in /usr/ports/security/py-certbot

# setenv CRYPTOGRAPHY_OPENSSL_NO_LEGACY 1
# make distclean clean config deinstall reinstall

It builds and installs fine.

Then ran certbot certonly and the error message is now different:

  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 14, in <module>
    from OpenSSL import crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/crypto.py", line 3279, in <module>
    _lib.OpenSSL_add_all_algorithms()
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'
Comment 71 Enji Cooper freebsd_committer freebsd_triage 2023-08-03 12:07:23 UTC
(In reply to void from comment #70)

pyOpenSSL probably needs to be upgraded: https://reviews.freebsd.org/D39037 .

I really wish I had a ports commit bit sometimes (*sigh*)…
Comment 72 Enji Cooper freebsd_committer freebsd_triage 2023-08-03 12:14:48 UTC
(In reply to void from comment #68)
(In reply to Mikael Urankar from comment #69)

FreeBSD should default to disabling the legacy crypto providers and should have a build flag to flip the default around. The legacy crypto providers are considered insecure by upstream (and many in the security community) today; it’s best to default a secure crypto stance (and add something to RELNOTES for folks upgrading to 14.0-RELEASE) instead of doing the opposite. That way folks who are getting a fresh install are defaulting to more secure cryptography.
Comment 73 void 2023-08-03 12:25:41 UTC
(In reply to Enji Cooper from comment #71)

# patch -p0 < py-openssl.patch
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|Index: security/py-openssl/Makefile
|===================================================================
|--- security/py-openssl/Makefile
|+++ security/py-openssl/Makefile
--------------------------
File to patch: Makefile
Patching file Makefile using Plan A...
Hunk #1 failed at 1.
Hunk #2 failed at 19.
Hunk #3 failed at 27.
Hunk #4 failed at 43.
4 out of 4 hunks failed--saving rejects to Makefile.rej
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|Index: security/py-openssl/distinfo
|===================================================================
|--- security/py-openssl/distinfo
|+++ security/py-openssl/distinfo
--------------------------
File to patch: distinfo
Patching file distinfo using Plan A...
Hunk #1 failed at 1.
1 out of 1 hunks failed--saving rejects to distinfo.rej
done
Comment 74 void 2023-08-03 12:30:15 UTC
(In reply to Enji Cooper from comment #72)

Just a thought - do you think later python versions might be a workaround?
Or maybe libressl built as default ssl before then building all other ssl-related packages like certbot?
Comment 75 Enji Cooper freebsd_committer freebsd_triage 2023-08-03 12:56:57 UTC
(In reply to void from comment #73)

Yeah... I need to update the patch (I forgot that conflicts were introduced after I opened the differential revision).
Comment 76 Enji Cooper freebsd_committer freebsd_triage 2023-08-03 12:59:01 UTC
(In reply to void from comment #74)

1. re: later versions of cpython: I don't think so.
2. re: using libressl: yes, that's probably a viable workaround, but given libressl and openssl's shared history (and exposed symbols), my gut says mix-and-matching the libraries might not work as desired.
Comment 77 Brad Davis freebsd_committer freebsd_triage 2023-08-03 18:48:45 UTC
(In reply to Enji Cooper from comment #75)

There is newer py-openssl in bug 266680.
Comment 78 Mark Millard 2023-08-04 05:27:12 UTC
(In reply to Charlie Li from comment #49)

QUOTE
That poudriere issue is not relevant here, not in the slightest, because it does not address how to avoid building or executing rustc in QEMU_EMULATING
END QUOTE

That is an armv6 issue but armv7 has been built via the ampere[123]
build servers that support aarch32/armv7 code exectution for about
a year as of now (2023-August). qemu is not used for building armv7
ports into packages any more for official builds.


https://pkg-status.freebsd.org/?all=1&type=package reports the first ampere
armv7 build as:

default default 130releng-armv7 a51002eeb3d0 32299 29273 294 1791 941 0 stopped:done: Fri, 22 Jul 2022 07:39:45 GMT 66:37:20 ampere3

The first build of armv7 main:

default default main-armv7 p83aeeda2ebb7_s30253da1a 32288 29099  (+27041) 274  (+147) 1948  (+26) 967  (+247) 0 stopped:done: Fri, 29 Jul 2022 21:46:08 GMT 88:40:40 ampere2

The first build of armv7 quarterly:

default quarterly 130releng-armv7 8662e656d870 32236 29152  (+27142) 238  (+140) 1871  (+9) 975  (+237) 0 stopped:done: Fri, 05 Aug 2022 13:04:55 GMT 70:00:06 ampere1

(I've been building armv7 ports that way for longer, but on different
hardware.)

From Warner's list activity, it looks to me like not supporting armv6 may
be a viable alternative. You might want to ask him about such. From what
I see looking at the build history, less than half of the ports manage to
build for armv6 via qemu currently.
Comment 79 void 2023-08-04 05:31:39 UTC
(In reply to Brad Davis from comment #77)

bingo! result is a working certbot :D

TYVM :D
Comment 80 Charlie Li freebsd_committer freebsd_triage 2023-08-04 05:44:57 UTC
(In reply to Mark Millard from comment #78)
Good for you, not everyone else has a suitable real armv7 machine to perform builds, especially when not everyone can use prefetch at all. I personally haven't built any armv6 for years, or even ever. The better way in the long term is to make qemu-user expendable in our cross tooling, particularly when executing compilers or interpreters.
Comment 81 Mark Millard 2023-08-04 06:02:59 UTC
(In reply to Charlie Li from comment #80)

May be I need to provide more context for my note. Looks like
I took to much context from how I got here.

John Baldwin had written on freebsd-arch:

QUOTE
rust isn't supported on armv7 on FreeBSD (and someone would
need to do the work to make that happen)
END QUOTE

and later:

QUOTE
Please see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254853
as an example.  According to that bug at least rustc is not doable
under qemu-user which is how armv7 packages are built.
END QUOTE

I was really just making notes that would avoid the errors of
interpretation that occurred:

A) "rust isn't supported on armv7 on FreeBSD": it is
B) "rustc is not doable under qemu-user which is how armv7 packages are built":
   qemu is not how (official) armv7 packages are built

I was not trying to get into the viability of not having even, say,
an RPi4B to build armv7 packages with via poudriere-devel and an
armv7 poudriere jail. (Not what I normally use these days but I've
done so and am still set up to do so. I've used a Windows Dev Kit
2023 and a HoneyComb as well.)
Comment 82 Michal Meloun freebsd_committer freebsd_triage 2023-08-04 06:51:41 UTC
To be a bit more precise: armv7 can compile rust by itself (with a few patches), including some selected applications. I do this in an infinite loop (as part of my platform stability test), along with a full kde5 build (and a few other ports). Of course, poudriere is not the right tool for 32-bit platforms, so all these ports are built using direct make.
Comment 83 Michal Meloun freebsd_committer freebsd_triage 2023-08-04 07:05:42 UTC
Err, "(with a few fixes)" should be attached to kde5, rust itself doesn't need any out of tree fixes.
Comment 84 Enji Cooper freebsd_committer freebsd_triage 2023-08-04 19:44:33 UTC
I updated the differential revision with the latest version of py-cryptography, which required updating all of the cargo crate pins: https://reviews.freebsd.org/D38939 .
Comment 86 Li-Wen Hsu freebsd_committer freebsd_triage 2023-08-29 14:38:17 UTC
Created attachment 244442 [details]
Update py-cryptography{,-vectors} to 41.0.3
Comment 87 Li-Wen Hsu freebsd_committer freebsd_triage 2023-08-29 14:41:22 UTC
(In reply to Li-Wen Hsu from comment #86)
I'm doing a test build for all the ports have USES=pycryptography with https://bugs.freebsd.org/bugzilla/attachment.cgi?id=244442 . Will commit it with tcberner's approval in comment #64 (I've checked with him.)
Comment 88 commit-hook freebsd_committer freebsd_triage 2023-08-29 17:30:04 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=82c675ccaa87127ca291de1d224aa18cb782149f

commit 82c675ccaa87127ca291de1d224aa18cb782149f
Author:     Li-Wen Hsu <lwhsu@FreeBSD.org>
AuthorDate: 2023-08-29 17:28:54 +0000
Commit:     Li-Wen Hsu <lwhsu@FreeBSD.org>
CommitDate: 2023-08-29 17:28:54 +0000

    security/py-cryptography*: Update to 41.0.3

    This patch combines the efforts from the people invloved in the PR.
    I just do the integrating and testing.

    PR:             254853
    Approved by:    tcberner (portmgr)
    Sponsored by:   The FreeBSD Foundation

 security/py-cryptography-vectors/Makefile          |   7 +-
 security/py-cryptography-vectors/distinfo          |   6 +-
 security/py-cryptography/Makefile                  |  30 +-
 security/py-cryptography/Makefile.crates (new)     |  53 ++++
 security/py-cryptography/distinfo                  | 112 +++++++-
 .../py-cryptography/files/patch-libressl (gone)    | 316 ---------------------
 .../py-cryptography/files/patch-setup.py (gone)    |  55 ----
 7 files changed, 186 insertions(+), 393 deletions(-)
Comment 89 vometia 2023-08-30 04:55:31 UTC
The change from 3.4.8 to 41.0.3 seems to have broken py-dnspython; switching to py-cryptography-legacy (IOW regressing to 3.4.8) seems to have fixed it for now but probably isn't ideal.

Traceback (most recent call last):
  File "/usr/local/bin/spf.py", line 155, in <module>
    import dns.resolver  # http://www.dnspython.org
  File "/usr/local/lib/python3.9/site-packages/dns/resolver.py", line 30, in <module>
    import dns._ddr
  File "/usr/local/lib/python3.9/site-packages/dns/_ddr.py", line 12, in <module>
    import dns.nameserver
  File "/usr/local/lib/python3.9/site-packages/dns/nameserver.py", line 5, in <module>
    import dns.asyncquery
  File "/usr/local/lib/python3.9/site-packages/dns/asyncquery.py", line 32, in <module>
    import dns.quic
  File "/usr/local/lib/python3.9/site-packages/dns/quic/__init__.py", line 4, in <module>
    import aioquic.quic.configuration  # type: ignore
  File "/usr/local/lib/python3.9/site-packages/aioquic/quic/configuration.py", line 6, in <module>
    from ..tls import (
  File "/usr/local/lib/python3.9/site-packages/aioquic/tls.py", line 40, in <module>
    from OpenSSL import crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/crypto.py", line 3279, in <module>
    _lib.OpenSSL_add_all_algorithms()
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'
Comment 90 Ralf van der Enden 2023-08-30 07:24:25 UTC
(In reply to vometia from comment #89)
The problem is that py-openssl needs to be updated as well.

py-cryptography 41.0.3 has a minimum version requirement for py-openssl (version 22.0, but possible higher).
Comment 91 Andrey Pevnev 2023-08-30 13:22:45 UTC
(In reply to Ralf van der Enden from comment #90)
There is a PR to update it, someone just needs to push it thru https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266680
Comment 92 Li-Wen Hsu freebsd_committer freebsd_triage 2023-08-31 13:59:57 UTC
Move two depending rust tickets to see also, so we can close this one and the ones depend on this. We can open or create another ticket for the follow-ups but let's clear the thoughts for now.
Comment 93 Mark Millard 2023-09-01 16:19:12 UTC
I'll have to pursue the details later, but on aarch64 it looks
like kyua tests involving python are broken after updating to
the recent changes:

# /usr/bin/kyua test -k /usr/tests/Kyuafile examples/test_examples.py
examples/test_examples.py:__test_cases_list__  ->  broken: Test program did not exit cleanly  [0.002s]

By contrast, on amd64 it still runs:

# /usr/bin/kyua test -k /usr/tests/Kyuafile examples/test_examples.py
you have mail
examples/test_examples.py:TestExampleSimple::test_get_properties  ->  skipped: comment me to run the test  [0.274s]
examples/test_examples.py:TestExampleSimple::test_one  ->  skipped: comment me to run the test  [0.276s]
examples/test_examples.py:TestExampleSimple::test_parametrize[AF_INET6]  ->  skipped: comment me to run the test  [0.275s]
examples/test_examples.py:TestExampleSimple::test_parametrize[AF_INET]  ->  skipped: comment me to run the test  [0.274s]
examples/test_examples.py:TestExampleSimple::test_parametrize[FAMILY_39]  ->  skipped: comment me to run the test  [0.273s]
examples/test_examples.py:TestExampleSimple::test_syscall_failure  ->  skipped: comment me to run the test  [0.276s]
examples/test_examples.py:TestExampleSimple::test_two  ->  skipped: Required file '/path/file1' not found  [0.001s]
examples/test_examples.py:TestExampleSimple::test_with_cleanup  ->  passed  [0.272s]
examples/test_examples.py:TestExampleSimplest::test_one  ->  skipped: comment me to run the test  [0.270s]
examples/test_examples.py:TestVnetDual1::test_ifstat  ->  skipped: comment me to run the test  [0.271s]
examples/test_examples.py:TestVnetSimple::test_ping  ->  skipped: comment me to run the test  [0.270s]
examples/test_examples.py:TestVnetSimple::test_topology  ->  skipped: comment me to run the test  [0.271s]

Results file id is usr_tests.20230901-160656-320496
Results saved to /usr/home/root/.kyua/store/results.usr_tests.20230901-160656-320496.db

12/12 passed (0 failed)

[an armv7 chroot (on aarch64 system) gets the same kind of result as
aarch64. But armv7 had other issues before the change as well.]

For reference:

# uname -apKU
FreeBSD CA78C-WDK23-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT aarch64 1500000 #13 main-n265027-2f06449d6429-dirty: Fri Aug 25 09:20:31 PDT 2023     root@CA78C-WDK23-ZFS:/usr/obj/BUILDs/main-CA78C-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA78C arm64 aarch64 1500000 1500000

# ~/fbsd-based-on-what-commit.sh -C /usr/ports/
8f7f59bbc250 (HEAD -> main, freebsd/main, freebsd/HEAD) sysutils/bmd-plugin-hookcmd: add new ports.
Author:     Vanilla I. Shu <vanilla@FreeBSD.org>
Commit:     Vanilla I. Shu <vanilla@FreeBSD.org>
CommitDate: 2023-09-01 02:54:14 +0000
branch: main
merge-base: 8f7f59bbc2504672483c7840638c7fc171e1b9f5
merge-base: CommitDate: 2023-09-01 02:54:14 +0000
n632086 (--first-parent --count for merge-base)
Comment 94 Mark Millard 2023-09-02 01:03:18 UTC
(In reply to Mark Millard from comment #93)

I have submitted the aarch64 kyua phython use issue in:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273506

I've initially guessed that that Base System changes would be
involved in a fix but that may be wrong (only a ports issue
then). So my classification could be wrong.

The submittal has a Python backtrace that at its end indicates:

QUOTE
RuntimeError: OpenSSL 3.0's legacy provider failed to load.
END QUOTE
Comment 95 Mark Millard 2023-09-02 07:08:10 UTC
(In reply to Mark Millard from comment #94)

I tracked the aarch64 example failure reprting down to dlerror message
text via use of gdb:

/usr/lib/ossl-modules/legacy.so: Undefined symbol \"MD4_Update\"", '\000' <repeats 449 times>

See:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273506
Comment 96 Mark Millard 2023-09-02 16:30:42 UTC
(In reply to Mark Millard from comment #95)

amd64 did not have:

devel/py-pytest-twisted
devel/py-twisted
net/py-dpkt
security/nist-kat
security/openvpn

installed and so was not doing the same sequence of dlopen
calls.

After adding those, the amd64 run fails like aarch64: It then starts to try
to use .../cryptography/hazmat/bindings/_rust.abi3.so leads to the problem.

[Detaching after fork from child process 98729]

Breakpoint 1.1, dlopen (name=name@entry=0x803239710 "libc.so.7", mode=2) at /usr/main-src/libexec/rtld-elf/rtld.c:3662
3662		return (rtld_dlopen(name, -1, mode));
(gdb) c
Continuing.

Breakpoint 1.1, dlopen (name=name@entry=0x8032c80d0 "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so", mode=2) at /usr/main-src/libexec/rtld-elf/rtld.c:3662
3662		return (rtld_dlopen(name, -1, mode));
(gdb) c
Continuing.

Breakpoint 1.1, dlopen (name=name@entry=0x8032c7440 "/usr/local/lib/python3.9/site-packages/_cffi_backend.cpython-39.so", mode=2) at /usr/main-src/libexec/rtld-elf/rtld.c:3662
3662		return (rtld_dlopen(name, -1, mode));
(gdb) c
Continuing.

Breakpoint 1.1, dlopen (name=name@entry=0x80219d080 "/usr/lib/ossl-modules/legacy.so", mode=2) at /usr/main-src/libexec/rtld-elf/rtld.c:3662
3662		return (rtld_dlopen(name, -1, mode));
(gdb) c
Continuing.
INTERNALERROR> Traceback (most recent call last):
. . .

This appears to be tied to devel/py-twisted and/or devel/py-pytest-twisted .
Comment 97 Mark Millard 2023-09-03 00:54:13 UTC
(In reply to Mark Millard from comment #96)

Looking at the dlopen of /lib/libcrypto.so.30 shows mode=2, i.e.:

mode=(RTLD_NOW|RTLD_LOCAL)

The RTLD_LOCAL means that MD4_Update and the like are not
available for the later dlopen of /usr/lib/ossl-modules/legacy.so
to bind to. 

. . .

The mode value comes from dlfcn_load in:

/usr/main-src/crypto/openssl/crypto/dso/dso_dlfcn.c

static int dlfcn_load(DSO *dso)
{
    void *ptr = NULL;
    /* See applicable comments in dso_dl.c */
    char *filename = DSO_convert_filename(dso, NULL);
    int flags = DLOPEN_FLAG;
    int saveerrno = get_last_sys_error();

    if (filename == NULL) {
        ERR_raise(ERR_LIB_DSO, DSO_R_NO_FILENAME);
        goto err;
    }
# ifdef RTLD_GLOBAL
    if (dso->flags & DSO_FLAG_GLOBAL_SYMBOLS)
        flags |= RTLD_GLOBAL;
# endif
# ifdef _AIX
    if (filename[strlen(filename) - 1] == ')')
        flags |= RTLD_MEMBER;
# endif
    ptr = dlopen(filename, flags);
. . .

So far I've not found anything that would lead to

dso->flags & DSO_FLAG_GLOBAL_SYMBOLS

being non-zero.

Thus the failures would result: /usr/lib/ossl-modules/legacy.so use
is designed for RTLD_GLOBAL use for /lib/libcrypto.so.30 .
Comment 98 Mark Millard 2023-09-03 01:53:42 UTC
The following crude patch leads to kyua's python use working
for examples/test_examples.py with devel/py-pytest-twisted and
devel/py-twisted and the like installed:

# git -C /usr/main-src/ diff crypto/openssl/crypto/init.c
diff --git a/crypto/openssl/crypto/init.c b/crypto/openssl/crypto/init.c
index cacf637c89f8..77f693fcfa05 100644
--- a/crypto/openssl/crypto/init.c
+++ b/crypto/openssl/crypto/init.c
@@ -159,7 +159,7 @@ DEFINE_RUN_ONCE_STATIC(ossl_init_load_crypto_nodelete)
         if (!err_shelve_state(&err))
             return 0;
 
-        dso = DSO_dsobyaddr(&base_inited, DSO_FLAG_NO_UNLOAD_ON_FREE);
+        dso = DSO_dsobyaddr(&base_inited, DSO_FLAG_NO_UNLOAD_ON_FREE|DSO_FLAG_GLOBAL_SYMBOLS);
         /*
          * In case of No!, it is uncertain our exit()-handlers can still be
          * called. After dlclose() the whole library might have been unloaded

(The above note is also in my separate bugzilla.)

For reference:

# /usr/bin/kyua test -k /usr/tests/Kyuafile examples/test_examples.py
examples/test_examples.py:TestExampleSimple::test_get_properties  ->  skipped: comment me to run the test  [0.582s]
examples/test_examples.py:TestExampleSimple::test_one  ->  skipped: comment me to run the test  [0.589s]
examples/test_examples.py:TestExampleSimple::test_parametrize[AF_INET6]  ->  skipped: comment me to run the test  [0.582s]
examples/test_examples.py:TestExampleSimple::test_parametrize[AF_INET]  ->  skipped: comment me to run the test  [0.595s]
examples/test_examples.py:TestExampleSimple::test_parametrize[FAMILY_39]  ->  skipped: comment me to run the test  [0.589s]
examples/test_examples.py:TestExampleSimple::test_syscall_failure  ->  skipped: comment me to run the test  [0.590s]
examples/test_examples.py:TestExampleSimple::test_two  ->  skipped: Required file '/path/file1' not found  [0.001s]
examples/test_examples.py:TestExampleSimple::test_with_cleanup  ->  passed  [0.583s]
examples/test_examples.py:TestExampleSimplest::test_one  ->  skipped: comment me to run the test  [0.583s]
examples/test_examples.py:TestVnetDual1::test_ifstat  ->  skipped: comment me to run the test  [0.593s]
examples/test_examples.py:TestVnetSimple::test_ping  ->  skipped: comment me to run the test  [0.589s]
examples/test_examples.py:TestVnetSimple::test_topology  ->  skipped: comment me to run the test  [0.588s]

Results file id is usr_tests.20230903-014520-255324
Results saved to /usr/home/root/.kyua/store/results.usr_tests.20230903-014520-255324.db

12/12 passed (0 failed)
Comment 99 László Károlyi 2023-09-03 20:33:59 UTC
(In reply to vometia from comment #89)
It's py39-openssl that's broken (see your error message), and basically everything that requires it.

That includes dnspython, certbot, synapse (matrix server), and much more.

Until that gets fixed, everything remains broken.
Comment 100 Mark Millard 2023-09-03 21:03:51 UTC
(In reply to László Károlyi from comment #99)

After my system openssl patch in comment #98 it works
just like before the recent changes.

/usr/src/crypto/openssl/crypto/init.c is preventing
/lib/libcrypto.so.30 from publishing its definitions
so that /usr/lib/ossl-modules/legacy.so can put them
to use.

/lib/libcrypto.so.30 needs to be dlopen'd via:

mode=(RTLD_NOW|RTLD_GLOBAL)

instead of:

mode=(RTLD_NOW|RTLD_LOCAL)

In crypto/openssl/crypto/init.c terms this means that
"|DSO_FLAG_GLOBAL_SYMBOLS" needs to be used --but it is
no currently:

dso = DSO_dsobyaddr(&base_inited,
          DSO_FLAG_NO_UNLOAD_ON_FREE|DSO_FLAG_GLOBAL_SYMBOLS);

For use of the system libraries, it is a system bug.
Comment 101 David Horn 2023-09-05 13:04:43 UTC
@Mark Millard:  Looks like your patch to base openssl 3 legacy provider also fixes the root cause of my py-cryptography legacy provider issue.
@All:
Hopefully this patch is under review, and can be committed to stable/14 (and CURRENT) soon.

My py-cryptography stable/14 issue and tests with patch: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273505#c17

I will be glad to re-test all of this once the patch lands.

--Thanks!
Comment 102 Pierre Pronchery 2023-09-14 17:54:15 UTC
(In reply to Mark Millard from comment #100)
On Linux (as per Debian 12) legacy.so is linked to libcrypto.so. In FreeBSD I chose not to, since this situation seems to be supported by OpenSSL 3, and legacy.so from the security/openssl30 port did not get linked to libcrypto.so.

It was also easier to build legacy.so for FreeBSD this way, when dealing with the build order: libcrypto.so would need to be built before legacy.so.
Comment 103 Enji Cooper freebsd_committer freebsd_triage 2023-09-14 18:18:48 UTC
Pierre: do we know why Linux does that and FreeBSD doesn’t?
Comment 104 Li-Wen Hsu freebsd_committer freebsd_triage 2023-09-14 22:08:57 UTC
Reopen for legacy.so loading issue.
Comment 105 Pierre Pronchery 2023-09-15 17:36:22 UTC
(In reply to Enji Cooper from comment #103)
Revisiting this issue, it could be that I was wrong. (Or the port was updated since I last compared?)

Regardless I have updated the patch in bug #273506 as a new candidate for a fix. (Also as review D41879)
Comment 106 commit-hook freebsd_committer freebsd_triage 2023-09-16 15:56:11 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=1a18383a52bc373e316d224cef1298debf6f7e25

commit 1a18383a52bc373e316d224cef1298debf6f7e25
Author:     Pierre Pronchery <pierre@freebsdfoundation.org>
AuthorDate: 2023-09-15 15:14:16 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-09-16 15:55:12 +0000

    libcrypto: link engines and the legacy provider to libcrypto

    OpenSSL's legacy provider module and engines need to link to
    libcrypto.so, as it provides some of the actual implementations of
    legacy routines.

    This is a little tricky due to build order issues.  Introduce a small
    hack (LIBCRYPTO_WITHOUT_SUBDIRS) that builds libcrypto.so in its usual
    early phase without any OpenSSL provider modules or engines.  This is
    intended to restore the test suite; a future change should remove the
    hack and replace it with a better approach.

    PR:             254853, 273528
    Discussed with: Folks at EuroBSDCon in Coimbra
    Sponsored by:   The FreeBSD Foundation

 Makefile.inc1                                | 2 +-
 secure/lib/libcrypto/Makefile                | 2 ++
 secure/lib/libcrypto/engines/Makefile.inc    | 2 ++
 secure/lib/libcrypto/modules/legacy/Makefile | 1 +
 4 files changed, 6 insertions(+), 1 deletion(-)
Comment 107 David Horn 2023-09-17 11:09:54 UTC
I tested the commit from Comment 106 on -CURRENT/15, and it resolves the OpenSSL 3 legacy loading issue for me.  

FreeBSD freebsd14 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n265364-9080190b970c: Sat Sep 16 13:42:11 EDT 2023     root@freebsd14:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64

`make test` on py-cryptography, py-fido2, and py-yubikey-manager all succeed without errors (And as desired, this removes the warnings about setting CRYPTOGRAPHY_OPENSSL_NO_LEGACY since the legacy provider from OpenSSL 3 now works as desired)

I will watch for the MFC to stable/14 and re-test all of my fail cases again at that time.
Comment 108 commit-hook freebsd_committer freebsd_triage 2023-09-19 15:03:20 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=fd4b13acd90bdc107a072660d91baa163474ebef

commit fd4b13acd90bdc107a072660d91baa163474ebef
Author:     Pierre Pronchery <pierre@freebsdfoundation.org>
AuthorDate: 2023-09-15 15:14:16 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-09-19 15:01:55 +0000

    libcrypto: link engines and the legacy provider to libcrypto

    OpenSSL's legacy provider module and engines need to link to
    libcrypto.so, as it provides some of the actual implementations of
    legacy routines.

    This is a little tricky due to build order issues.  Introduce a small
    hack (LIBCRYPTO_WITHOUT_SUBDIRS) that builds libcrypto.so in its usual
    early phase without any OpenSSL provider modules or engines.  This is
    intended to restore the test suite; a future change should remove the
    hack and replace it with a better approach.

    PR:             254853, 273528
    Discussed with: Folks at EuroBSDCon in Coimbra
    Sponsored by:   The FreeBSD Foundation

    (cherry picked from commit 1a18383a52bc373e316d224cef1298debf6f7e25)

 Makefile.inc1                                | 2 +-
 secure/lib/libcrypto/Makefile                | 2 ++
 secure/lib/libcrypto/engines/Makefile.inc    | 2 ++
 secure/lib/libcrypto/modules/legacy/Makefile | 1 +
 4 files changed, 6 insertions(+), 1 deletion(-)
Comment 109 commit-hook freebsd_committer freebsd_triage 2023-09-19 17:19:52 UTC
A commit in branch releng/14.0 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=b7312b90289e7244c124da9845c0a3011ba8d936

commit b7312b90289e7244c124da9845c0a3011ba8d936
Author:     Pierre Pronchery <pierre@freebsdfoundation.org>
AuthorDate: 2023-09-15 15:14:16 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-09-19 17:19:17 +0000

    libcrypto: link engines and the legacy provider to libcrypto

    OpenSSL's legacy provider module and engines need to link to
    libcrypto.so, as it provides some of the actual implementations of
    legacy routines.

    This is a little tricky due to build order issues.  Introduce a small
    hack (LIBCRYPTO_WITHOUT_SUBDIRS) that builds libcrypto.so in its usual
    early phase without any OpenSSL provider modules or engines.  This is
    intended to restore the test suite; a future change should remove the
    hack and replace it with a better approach.

    PR:             254853, 273528
    Discussed with: Folks at EuroBSDCon in Coimbra
    Sponsored by:   The FreeBSD Foundation

    (cherry picked from commit 1a18383a52bc373e316d224cef1298debf6f7e25)
    (cherry picked from commit fd4b13acd90bdc107a072660d91baa163474ebef)

    Approved by:    re (gjb)

 Makefile.inc1                                | 2 +-
 secure/lib/libcrypto/Makefile                | 2 ++
 secure/lib/libcrypto/engines/Makefile.inc    | 2 ++
 secure/lib/libcrypto/modules/legacy/Makefile | 1 +
 4 files changed, 6 insertions(+), 1 deletion(-)
Comment 110 David Horn 2023-09-20 01:43:16 UTC
I retested on stable/14 with the py-cryptography, py-fido2, py-yubikey-manager.  

With latest commit openssl and py-cryptography looks good from my standpoint. Ports work, `make test` succeeds for all of the cases where it previously complained about openssl legacy provider support missing.

commit fd4b13acd90bdc107a072660d91baa163474ebef
FreeBSD freebsd14 14.0-STABLE FreeBSD 14.0-STABLE #2 stable/14-n265155-fd4b13acd90b: Tue Sep 19 18:53:18 EDT 2023     root@freebsd14:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64

I will also test aarch64 later.
Comment 111 Ivan Rozhuk 2023-09-20 11:49:04 UTC
Make file missed:
MAKE_ENV=	OPENSSL_DIR="${OPENSSLBASE}"
without this line it links with OpenSSL from base instead of libressl.

# ldd /usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so:
	libssl.so.111 => /usr/lib/libssl.so.111 (0x80147a000)
	libcrypto.so.111 => /lib/libcrypto.so.111 (0x8020b3000)
	libthr.so.3 => /lib/libthr.so.3 (0x801514000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x801542000)
	libc.so.7 => /lib/libc.so.7 (0x801082000)
Comment 112 Enji Cooper freebsd_committer freebsd_triage 2023-09-21 04:59:04 UTC
(In reply to Ivan Rozhuk from comment #111)

Could you please open up a new issue? It would be nice to close this issue and fix followup issues instead of keeping this open indefinitely.
Comment 113 David Horn 2023-09-29 11:53:29 UTC
(In reply to David Horn from comment #110)
arm64/aarch64 looks good now as well on 14-BETA3 (has legacy OpenSSL 3 provider fix in base) after latest package builders completed for aarch64. 

All of my failcases have been resolved.

Thanks!
Comment 114 commit-hook freebsd_committer freebsd_triage 2023-11-03 07:26:26 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=5202fbf4e8e4dd1793c24780d2c17ee5e0610a05

commit 5202fbf4e8e4dd1793c24780d2c17ee5e0610a05
Author:     Ben Shertenlieb <b.shertenlieb@yahoo.com>
AuthorDate: 2023-11-03 07:24:55 +0000
Commit:     Li-Wen Hsu <lwhsu@FreeBSD.org>
CommitDate: 2023-11-03 07:24:55 +0000

    multimedia/plexmediaserver-plexpass: Update to 1.32.8.7639

    New release from Plex.tv

      Changelog:

       https://forums.plex.tv/t/plex-media-server/30447/607

        NEW:
         (Web) Updated to 4.116.1

        FIXES:
         (Library) “Episode sorting” and “Album sorting”
          preferences had meaningless “Library default”
          global library setting.
         (Library) The default season sort order in TV
          libraries may not work as intended.
         (LiveTV) Transcoded video would fail to play in
          some circumstances.
         (Metadata) Items could lose metadata if a refresh
          request failed due to a network failure.
         (Player) Controlling a player didn’t work in some
          cases.
         (Sharing) Sharing a single episode was not playable
          by sharee.
         (Transcoder) HW transcoding failed on GeminiLake CPUs
          when running on linux.
         (Transcoder) Videos HW encoded on the shield would
          display a black screen on some devices.
         (Updater) Windows Updater would hang when run with
          PMSService.

    PR:             254853

 multimedia/plexmediaserver-plexpass/Makefile | 4 ++--
 multimedia/plexmediaserver-plexpass/distinfo | 6 +++---
 2 files changed, 5 insertions(+), 5 deletions(-)
Comment 115 Dmitry 2023-11-05 13:21:17 UTC

ImportError: /usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so: Undefined symbol "EVP_default_properties_is_fips_enabled"

All ports have been rebuilt - everything is updated - the latest FreeBSD 13.2

# certbot
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==2.7.4', 'console_scripts', 'certbot')())
  File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/certbot/main.py", line 6, in <module>
    from certbot._internal import main as internal_main
  File "/usr/local/lib/python3.9/site-packages/certbot/_internal/main.py", line 21, in <module>
    import josepy as jose
  File "/usr/local/lib/python3.9/site-packages/josepy/__init__.py", line 40, in <module>
    from josepy.json_util import (
  File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 14, in <module>
    from OpenSSL import crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import SSL, crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/SSL.py", line 9, in <module>
    from OpenSSL._util import (
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 15, in <module>
    from cryptography.exceptions import InternalError
  File "/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py", line 9, in <module>
    from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
ImportError: /usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so: Undefined symbol "EVP_default_properties_is_fips_enabled"
Comment 116 Mark Millard 2023-11-05 18:43:58 UTC
(In reply to Dmitry from comment #115)

security/py-cryptography already has the following update committed
on 2023-Sep-22:

security/py-cryptography: Update to 43.0.4

The update to 43.0.1 was back 2023-Aug-29 so it has been in place
for a while.

I'm unsure if a new bugzilla is needed at this point instead of
continuing to grow this one for issues that might require additional
changes. It may be that this one should be closed now.



It is also not clear to me if you built your own ports, used a
package from 2023Q4 (quarterly), or used a package from main
(latest). Wherever the submittal goes, you probably need to also
indicate such context related to the certbot command that you
used that ended up reporting EVP_default_properties_is_fips_enabled
as missing.
Comment 117 Ivan Rozhuk 2023-11-05 19:43:21 UTC
(In reply to Mark Millard from comment #116)

https://pypi.org/project/cryptography/
https://github.com/pyca/cryptography/tags
41.0.5

I see no 43.0+ in wild, where source for it?
Comment 118 Dmitry 2023-11-05 20:34:42 UTC
(In reply to Mark Millard from comment #116)
py39-cryptography-41.0.5,1

this is the only one available in the ports - all ports have been updated to the latest ones - FreeBSD version 13.2

where is 43.0.4 available?
Comment 119 Mark Millard 2023-11-05 22:37:47 UTC
(In reply to Dmitry from comment #118)

Dumb digit position swapping (spanning several characters) on my part
from what I can tell.

So copy/paste from the update after the 41.0.3 commit instead:

security/py-cryptography: Update to 41.0.5
- Update WWW

From:

https://cgit.freebsd.org/ports/commit/security/py-cryptography?id=469b9cacad27

Sorry for the misleading text.
Comment 120 Mark Millard 2023-11-05 22:46:09 UTC
(In reply to Mark Millard from comment #119)

Actually, I previously copied/pasted the bad reference
that has a "3 instead of 1" typo:

https://cgit.freebsd.org/ports/commit/security/py-cryptography?id=5935d168accf

reports itself as:

security/py-cryptography: Update to 43.0.4

instead of as an update to 41.0.4 .

Still, sorry for the misleading text.
Comment 121 Dmitry 2023-11-06 09:49:10 UTC
(In reply to Mark Millard from comment #120)

on the contrary - if you downgrade the version, it works
  
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273770#c8