LibreSSL 2.7.0 was released on 21 March in security/libressl-devel and introduced support for many OpenSSL 1.0.2 and 1.1 APIs. However, this has broken building of this port when DEFAULT_VERSIONS contains ssl=libressl-devel. Upstream is actively working on a fix. This PR serves to coordinate downstream patching efforts before upstream releases their fix.
Created attachment 191799 [details] svn diff for security/py-cryptography ``` security/py-cryptography: Fix build with LibreSSL 2.7 From: http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/security/py-cryptography/patches/ Author: sthen@OpenBSD.org PR: 226906 See also: https://github.com/pyca/cryptography/pull/4169 ``` QA: https://brnrd.eu/poudriere/data/111libre-default/2018-03-22_21h36m44s/logs/py27-cryptography-2.1.4.log
(In reply to Bernard Spil from comment #1) Happy for this to land if it passes QA (incl make test (test suite)) Are the build failures upstream (see pull request) anything to worry about, or is it just related to their master (where this is a backport/custom patch to the current ports version) ?
It looks like upstream's CI stem from their master, but I don't think this is relevant. The bigger issue found in brnrd@'s follow-up pull request is that their CI is misconfigured [1], and while anyone can edit those configuration files and include them in pull requests, they have no effect unless one of their committers makes the edit (mere commits and merges don't count): > (Reminder that modifications to the jenkinsfile will not be honored for non-committers) [1] Example of python 3.6 run: https://travis-ci.org/pyca/cryptography/jobs/358012204
Created attachment 192331 [details] Output of make test Not seeing issues when running make test > 90132 passed, 7403 skipped in 419.70 seconds
I was made aware of this, due to LibreSSL 2.7.0 being default now.
(In reply to Kubilay Kocak from comment #2) Patch from OpenBSD only worked for LibreSSL 2.7, no longer for other versions. I've just sent a new pull-request to pyca/crypography with a version that retains compatibility with other libcrypto/ssl providers. https://github.com/pyca/cryptography/pull/4210
*** Bug 227852 has been marked as a duplicate of this bug. ***
Created attachment 192933 [details] svn diff for security/py-cryptography This patch is the same as in https://github.com/pyca/cryptography/pull/4210
I've run successful builds with Python 2.7 and 3.6 flavors on - 10.4 amd64 - 10.4 i386 - 11.1 amd64 - 11.1 i386 Using the following libcrypto providers - base - security/libressl - (skipped security/libressl-devel) - security/openssl - security/openssl-devel - security/openssl-master (1.1.1-pre5) Build logs can be found on https://keg.brnrd.eu/ and should be self-explanatory. Look for logs 2018-04-30 around 17:00 (all logs there are UTC) Upstream is running a verification build https://github.com/pyca/cryptography/pull/4211 for the patch that is attached.
Comment on attachment 192933 [details] svn diff for security/py-cryptography Approved by: koobs (maintainer), Please add comments with upstream issue/commit references to any new patches. @Bernard, commit when you're comfortable/confident on QA
Patch probably incomplete: install: security/py-openssl run: python import OpenSSL get: >>> import OpenSSL Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/OpenSSL/__init__.py", line 8, in <module> from OpenSSL import crypto, SSL File "/usr/local/lib/python2.7/site-packages/OpenSSL/crypto.py", line 16, in <module> from OpenSSL._util import ( File "/usr/local/lib/python2.7/site-packages/OpenSSL/_util.py", line 6, in <module> from cryptography.hazmat.bindings.openssl.binding import Binding File "/usr/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 13, in <module> from cryptography.hazmat.bindings._openssl import ffi, lib ImportError: /usr/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/_openssl.so: Undefined symbol "DTLS_method"
As the upstream fix seems to take longer, it would be good if the patch could be committed to the port to have a working system with the new libressl version.
(In reply to Peter Putzer from comment #12) As a quick workaround for anyone waiting on a permanent solution, grab the attached diff and drop it into a file at patch-issue4210 in /usr/ports or wherever your ports tree is located. Then just run `mkdir security/py-cryptography/files && patch -p0 < patch-issue4210 && rm security/py-cryptography/files/patch-issue4210.orig`.
WOW, you saved my Day. Works with the patch, thank you very much.
What are the problems with just committing the patch if it works?
Is there any problem with committing this patch? I can confirm it fixes my own package builds (with python26 and libressl set as defaults) -- and it seems most of the interesting package I want to build seem to depend on py-cryptography.
(In reply to Matthew Seaman from comment #16) s/python26/python36/, dammit.
Comment 2 is clear on what is required: confirmation of QA. Specifically, this entails (but is not limited to): - confirming no regressions for all (other) values of ssl= - Ruling out comment 11 as a symptom of the patch (just replacing libressl with openssl, without recompiling dependents may be the cause). Additionally, there are indications that an update to the patch is required (which will come from the PR Bernard is working on upstream, if that is the case). And finally to clarify, the issue is not blocked on upstream accepting or merging the PR.
Currently in Bernards capable hands
It is build OK, but it does not work. See comment #11 for test case.
(In reply to rozhuk.im from comment #20) That is a bit in line with the feedback from upstream. There are errors on DTLS_ variables as well. It takes me some time to cycle back to this problem.
would it hurt to go back to 2.6.4 for security/libressl and leave 2.7 in security/libressl-devel for a bit longer, until those things are working?
Created attachment 193350 [details] svn diff for security/py-cryptography Updated patch that resolves all 'implicit declaration' warnings at build and passes most tests during `make test`. Upstreamed in issue 4210 with additional changes for master, I guess upstream will merge and run another test in issue 4211.
Created attachment 193351 [details] Output of make test Updated output of make test > 27 failed, 90118 passed, 7390 skipped
Any progress with: https://github.com/pyca/cryptography/pull/4234 ?
The patch seems stable (running in my various jails for month now, no problems). It would be helpful to get into ports.
I'm not sure if this is on-topic, but with the patched py-cryptography, the script py-certbot (for Let's Encrypt ssl certificates) runs into the following error: Undefined symbol "DTLS_method" Traceback (most recent call last): File "/usr/local/bin/certbot", line 11, in <module> load_entry_point('certbot==0.24.0', 'console_scripts', 'certbot')() File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2693, in load_entry_point return ep.load() File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in load return self.resolve() File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2330, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "/usr/local/lib/python3.6/site-packages/certbot/main.py", line 10, in <module> import josepy as jose File "/usr/local/lib/python3.6/site-packages/josepy/__init__.py", line 44, in <module> from josepy.interfaces import JSONDeSerializable File "/usr/local/lib/python3.6/site-packages/josepy/interfaces.py", line 8, in <module> from josepy import errors, util File "/usr/local/lib/python3.6/site-packages/josepy/util.py", line 4, in <module> import OpenSSL File "/usr/local/lib/python3.6/site-packages/OpenSSL/__init__.py", line 8, in <module> from OpenSSL import crypto, SSL File "/usr/local/lib/python3.6/site-packages/OpenSSL/crypto.py", line 16, in <module> from OpenSSL._util import ( File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 6, in <module> from cryptography.hazmat.bindings.openssl.binding import Binding File "/usr/local/lib/python3.6/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 13, in <module> from cryptography.hazmat.bindings._openssl import ffi, lib ImportError: /usr/local/lib/python3.6/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so: Undefined symbol "DTLS_method"
(In reply to Jan Siero from comment #27) Afaik LibreSSL removed DTLS support.
Heads up: upstream finally merged in libressl support. https://github.com/pyca/cryptography/pull/4270
*** Bug 228651 has been marked as a duplicate of this bug. ***
https://github.com/pyca/cryptography/pull/4210 closed (in favour of): https://github.com/pyca/cryptography/pull/4270
Created attachment 193887 [details] 2.2.2 with upstream libressl support It's way past my bedtime, but here's a patch to the ports tree incorporating upstream's changes to support libressl. This is a simple diff between their master branch and version 2.2.2 on just the relevant src/_cffi_src/openssl directory. The delta between version 2.1.4 and upstream's libressl support was too great, actually refusing to build. Passes testport, but needs functional testing, as this is an unofficial backport.
security/py-certbot works with the upstream changes on top of 2.2.2. Tested on an armv6 system for a change.
Hi koobs, I believe this update calls for an exp-run? I can create the PR for that, but will need your blessing on the current patch.
Created attachment 193965 [details] svn diff for security/py-cryptography Using the patch I just added: LibreSSL + 2.7.15: 93566 passed, 7404 skipped, 51 warnings LibreSSL + 3.6.5: 93566 passed, 7404 skipped, 51 warnings base r1.0.2k + 2.7.15: 97623 passed, 3347 skipped, 51 warnings vishwin's patch: 93566 passed, 7404 skipped, 57
(In reply to Bernard Spil from comment #34) Updates to this port don't require an exp-run. However, if the commit(s) to fix this issue require updates to libressl, then perhaps so, though that doesn't appear to be the case per attachment 193965 [details]
Comment on attachment 193965 [details] svn diff for security/py-cryptography You have/had implicit approval for this change (fix) given the complexity of the background work to get it sorted. You now have explicit approval to handle resolution, with any commit (subsequent to QA of your satisfaction)
Request for exp-run with patch from att 193965
As I didn't hear anything about exp-run feedback I took the liberty to set the exp-run flag. Also setting merge-quarterly, as we're currently stuck with 2018Q3 not building for our purposes. @Bernard Maybe you can give some update? Thanks!
python@ said an exp-run was not needed.
(In reply to Antoine Brodin from comment #40) Thanks, so what keeps us from committing and MFHing this to 2018Q3?
Comment 36 (koobs) stated an exp-run was not *required* Comment 37 (koobs) provided approval "with any commit (subsequent to *QA of your satisfaction*) Comment 38 (brnrd) requested an exp-run Accordingly: - Restore assignee to correct last assignee (brnrd) - Cancel exp-run request (for now, to reset) @Bernard / Michael, *If* either of you feel an exp-run is necessary, please explicitly re-request it (assigning to portmgr@ as well) Otherwise, the change is approved (for either of you) pending your own QA satisfaction. Please assign yourselves (take issue) as necessary.
*** Bug 229680 has been marked as a duplicate of this bug. ***
py-cryptography 2.3 has been released with support for LibreSSL, can we update now?
The new version obviates the SSL patches, so update title. Currently running testport on the new version with all DEFAULT_VERSIONS settings to verify successful builds; will upload patch when done.
Created attachment 195386 [details] py-cryptography-2.3 No ssl patches, as this release fully supports both major implementations. Passes testport with all DEFAULT_VERSIONS settings, and py-certbot works fine as a runtime test.
(In reply to Michael Gmelin from comment #41) The question is whether an upgrade from 2.1.x to 2.2/2.3 would be eligible for MFH. It's probably not a trivial blanket build fix, since the minor version changes. So do the Python port experts here think a merge-back would be fine after this lands in head?
(In reply to Charlie Li from comment #46) Thank-you for the patch, which works nicely on FreeBSD 11.2-STABLE r336359M & libressl 2.7.4 :)
*** Bug 230277 has been marked as a duplicate of this bug. ***
*** Bug 229840 has been marked as a duplicate of this bug. ***
QA'ing now. Fails to build against current openssl-devel (see bug 229223)
2.3 also contains a security fix (with CVE) [1] VuXML addition pending [1] https://github.com/pyca/cryptography/pull/4360
A commit references this bug: Author: koobs Date: Mon Aug 6 03:23:24 UTC 2018 New revision: 476487 URL: https://svnweb.freebsd.org/changeset/ports/476487 Log: security/py-cryptography: Add tag forgery vulnerability PR: 226906 Changes: head/security/vuxml/vuln.xml
A commit references this bug: Author: koobs Date: Mon Aug 6 03:25:21 UTC 2018 New revision: 476488 URL: https://svnweb.freebsd.org/changeset/ports/476488 Log: security/py-cryptography: Update to 2.3 This version update fixes builds with libressl >= 2.7. Builds with openssl-devel are broken due to an underlying lang/python* port issue [1]. Changelog: https://github.com/pyca/cryptography/blob/2.3/CHANGELOG.rst [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229223 PR: 226906 Submitted by: Charli Li <ml+freebsd vishwin info> Reported by: many MFH: 2018Q3 (fixes package build, security fix) Security: 9e2d0dcf-9926-11e8-a92d-0050562a4d7b Changes: head/security/py-cryptography/Makefile head/security/py-cryptography/distinfo
A commit references this bug: Author: koobs Date: Wed Aug 22 02:15:06 UTC 2018 New revision: 477764 URL: https://svnweb.freebsd.org/changeset/ports/477764 Log: MFH: r476488 security/py-cryptography: Update to 2.3 This version update fixes builds with libressl >= 2.7. Builds with openssl-devel are broken due to an underlying lang/python* port issue [1]. Changelog: https://github.com/pyca/cryptography/blob/2.3/CHANGELOG.rst [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229223 PR: 226906 Submitted by: Charli Li <ml+freebsd vishwin info> Reported by: many Security: 9e2d0dcf-9926-11e8-a92d-0050562a4d7b Approved by: ports-secteam (miwi) Changes: _U branches/2018Q3/ branches/2018Q3/security/py-cryptography/Makefile branches/2018Q3/security/py-cryptography/distinfo