Created attachment 231643 [details] devel/py-twisted: Update to 22.1.0 This patch updates devel/py-twisted to 22.1.0, which includes a security fix, see [1]. The update should be fairly unspectacular from a ports perspective as it's mostly a version bump, but testing it is a bit difficult as the ports testsuite doesn't pass (which is a rather traditional problem at this point). I've also moved py-Hamcrest to the test dependencies, as it's mentioned as an exclusive test dependency in setup.cfg and appears to occur only in test-related functions in the source itself. As such: * portlint: OK * testport: OK (poudriere: 130amd64) * do-test: "OK" (FAILED (skips=1829, failures=8, errors=11, successes=10240)) The testsuite failures mostly center around issues with directly executing the git command (despite being installed an in PATH) and a missing python dependency that's exclusively used for a few tests and not currently packaged in ports and appear to be fixable, but I currently lack the time to dive into the problem. To provide some more QA, using the port still allows the testsuites of net-im/py-matrix-synapse and www/treq to pass without issues. I've also used the resulting package for running synapse in production without encountering any problems. I'm currently attempting to build the other consumers of this port and will report on the results of that once that's done, but my testbox is quite slow, at least as soon as rust and llvm need to be built. I'll also try and write a suitable vuxml entry and add it to this PR. Cheers, Sascha [1] https://github.com/twisted/twisted/security/advisories/GHSA-92x2-jw7w-xvvx
Created attachment 231645 [details] vuln-2022.xml diff for py-twisted I've basically converted the security advisory to a vuln.xml entry, I hope this is fine.
Thank you for the report and patches. - Given this will need to be MFH'd, with alternative options being a backport to 21.x instead (for merging) updating head to 22.1.x, could you peruse the changelog between current port version and 22.1.0 looking for changes that might provide backward compat issues. In particular, of the main issues will be consuming ports that pin their twisted version specs to <22.1, or dont ping, but don't support later versions for whatever reason. This class of issue will result in builds succeeding, but with run time failures. An assessment of the version specs (in upstream sources that are used in either in setup.py or in external files like requirements files) for consuming ports to identify problematic ports will go a long way.
22.1.0 also added python 3.10 support [1] (not previously supported), so this is a good opportunity to fix the version spec of the port to match. [1] "Python 3.10 is now a supported platform"
(In reply to Kubilay Kocak from comment #2) I've done poudriere testbuilds of all consumers with the exception of multimedia/syncplay, which pulls in rust and llvm and my build system simply didn't have the RAM to make that work. All of them build just fine (which is rather unsurprising). I then ran all the available testsuites with the following results: www/py-treq: PASS www/py-autobahn: PASS databases/py-txredisapi: PASS net-im/py-matrix-synapse: PASS The following ports fail, but the failures appear unrelated to twisted, I think. I've added the twisted version they depend on/pin: net/irrd: FAIL, test only dep, requirements.txt twisted==21.7.0 devel/py-pytest: FAIL, test only dep, no info devel/py-txaio: FAIL, 'twisted>=12.1.0' devel/py-buildbot: FAIL, twisted_ver = ">= 17.9.0" www/py-spyne: FAIL, no info www/py-txrequests: FAIL, 'twisted>=9.0.0' The following ports depend on twisted in some way but provide no testsuite: security/cowrie: NOOP, setup.py: 'twisted>=17.1.0', requirements.txt: 'twisted==20.3.0' security/py-txtorcon: NOOP, Twisted[tls]>=15.5.0 mail/py-alot: NOOP, 'twisted>=18.4.0', sysutils/py-python-consul2: NOOP, 'twisted' net/py-txamqp: NOOP, 'Twisted' net/py-magic-wormhole: NOOP, "twisted[tls] >= 17.5.0" net/py-tofu: NOOP, no info (404 timeout) net/kippo: NOOP, no info (no setup.cfg/setup.py/whatever, last release 2014) net/py-matrix-synapse-ldap3: NOOP, Twisted>=15.1.0 net/py-msrplib: NOOP, no info (source archive 404?) multimedia/syncplay: NOOP, twisted[tls]>=16.4.0 devel/py-epsilon: NOOP, twisted[tls] >= 13.2.0 devel/py-xcaplib: NOOP, no info (source archive unavailable) devel/py-Automat: NOOP, "Twisted>=16.1.1" devel/py-testoob: NOOP, no info finance/py-python-obelisk: NOOP, 'twisted' www/py-nevow: NOOP, MINIMUM_TWISTED_VERSION = "13.0" net-im/py-punjab: NOOP, no info net-im/py-unmessage: NOOP, 'Twisted[tls]>=16.6.0', net-p2p/deluge-cli: NOOP, 'twisted[tls]>=17.1', net-p2p/py-vertex: NOOP, 'Twisted>=13.1.0' net-mgmt/py-prometheus-client: NOOP, 'twisted' databases/py-carbon: NOOP, 'Twisted' audio/py-python-mpd2: NOOP, 'Twisted' As far as backporting the change is concerned: From a quick glance the security "fix" appears to be a simple removal of the affected parts of the code, as they were marked as deprecated, see [1]. Because of this I'm not sure the impact of backporting this is going to be any less than merging the new version, but if that works better then I could prepare such a patch for the quarterly port. Please let me know if there's any other way I can help with this. Cheers, Sascha [1] https://github.com/twisted/twisted/pull/1683
(In reply to Sascha Biberhofer from comment #4) Great job QA'ing and reporting on those consumer dependencies Sascha, confidence++ in this change. Happy to exempt multimedia/syncplay in this case, but for future use, you should be able to trivially use packages for dependencies with poudriere now (I do), which precludes building all the things in most cases. Just make sure you don't run any custom (non-default) OPTIONS in those poudriere environments, so the options match those in the official repo's, which will maximise the chance binary packages are selected for dependency use. If there's no open questions or other blockers (please confirm), I think we're good to commit and merge this, particularly given the thorough reverse dependents version-spec assessment and detail provided in comment 4 We'll still want to cap USES:python to 3.10 (see comment 3), ify ou can take care of that
Created attachment 231704 [details] devel/py-twisted: Update to 22.1.0, limit to python 3.10 I've updated the patch to limit supported versions from 3.7 to 3.10. I had to take a quick peek into Mk/uses/python.mk to find out how to do that, didn't even know we had that ability. :-) I'll also take a look at using poudriere with packages in the future, which would speed this kind of testing up by an order of magnitude. I did check whether I could cheat and fetch llvm from pkg, but the version shipped in the latest package repo as of right now appears to be a minor release behind the one currently in ports, so that wouldn't have helped, I think. I might look into updating my buildsetup if these testing sprees continue. ;-) From my point of view, I think we can move ahead and merge this, unless there are additional concerns that need to be addressed. There's also another security release for twisted that's already been announced for next week, so we may need to repeat this little dance in a few days time. This time I'll try and get the QA done right away. :D
Comment on attachment 231704 [details] devel/py-twisted: Update to 22.1.0, limit to python 3.10 Reviewed by: koobs (python, maintainer) Approved by: koobs (python, maintainer) MFH: 2022Q1
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=78783e7e45213fa7bb27f58e35858e13c780aeba commit 78783e7e45213fa7bb27f58e35858e13c780aeba Author: Sascha Biberhofer <ports@skyforge.at> AuthorDate: 2022-02-13 09:05:02 +0000 Commit: Kai Knoblich <kai@FreeBSD.org> CommitDate: 2022-02-13 09:14:32 +0000 security/vuxml: Document devel/py-twisted vulnerabilities PR: 261791 security/vuxml/vuln-2022.xml | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=642a3adc710e0b214ea0bea0e9fb42f9b8323d47 commit 642a3adc710e0b214ea0bea0e9fb42f9b8323d47 Author: Sascha Biberhofer <ports@skyforge.at> AuthorDate: 2022-02-13 09:09:46 +0000 Commit: Kai Knoblich <kai@FreeBSD.org> CommitDate: 2022-02-13 09:14:32 +0000 devel/py-twisted: Update to 22.1.0, limit to python 3.10 [1] * Support for Python 3.10 was added with this release. [1] Changelog: https://github.com/twisted/twisted/releases/tag/twisted-22.1.0 PR: 261791 Reviewed by: koobs (python, maintainer) Approved by: koobs (python, maintainer) MFH: 2022Q1 Security: 24049967-88ec-11ec-88f5-901b0e934d69 devel/py-twisted/Makefile | 8 ++++---- devel/py-twisted/distinfo | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-)
A commit in branch 2022Q1 references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=4e83d2bcc005fe0bbb852b2acce978ffb04affba commit 4e83d2bcc005fe0bbb852b2acce978ffb04affba Author: Sascha Biberhofer <ports@skyforge.at> AuthorDate: 2022-02-13 09:09:46 +0000 Commit: Kai Knoblich <kai@FreeBSD.org> CommitDate: 2022-02-13 09:17:09 +0000 devel/py-twisted: Update to 22.1.0, limit to python 3.10 [1] * Support for Python 3.10 was added with this release. [1] Changelog: https://github.com/twisted/twisted/releases/tag/twisted-22.1.0 PR: 261791 Reviewed by: koobs (python, maintainer) Approved by: koobs (python, maintainer) MFH: 2022Q1 Security: 24049967-88ec-11ec-88f5-901b0e934d69 (cherry picked from commit 642a3adc710e0b214ea0bea0e9fb42f9b8323d47) devel/py-twisted/Makefile | 8 ++++---- devel/py-twisted/distinfo | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-)
Committed to main branch and merged into the 2022Q1 branch! Thank you, Sascha, for the patch and the QA. Of course thanks you, Kubilay, for the review and feedback.