Bug 270358
| Summary: | devel/py-setuptools: update to 75.3.0 | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Wen Heping <wen> | ||||||||||||||||||||||||||
| Component: | Individual Port(s) | Assignee: | Charlie Li <vishwin> | ||||||||||||||||||||||||||
| Status: | Open --- | ||||||||||||||||||||||||||||
| Severity: | Affects Some People | CC: | agh, alster, asomers, chris, cy, dch, djohnson, dpetrov67, dvl, eduardo, fabian, fcharlier, franco, freebsd-ports, freebsd, george, gert, henrichhartzer, jlduran, johntdarrah, kirill, knan-bfo, ko, mfisher911, ml, n.perraut, ngie, ohartmann, ports-security, rcm, rhurlin, sunpoet, takefu, tomas, vishwin, void, vvd, wen, wenheping2000 | ||||||||||||||||||||||||||
| Priority: | --- | Flags: | sunpoet:
maintainer-feedback+
antoine: exp-run+ |
||||||||||||||||||||||||||
| Version: | Latest | ||||||||||||||||||||||||||||
| Hardware: | Any | ||||||||||||||||||||||||||||
| OS: | Any | ||||||||||||||||||||||||||||
| URL: | https://reviews.freebsd.org/D46236 | ||||||||||||||||||||||||||||
| See Also: | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280819 | ||||||||||||||||||||||||||||
| Bug Depends on: | 270510, 273722, 280661, 281083, 270243, 280646 | ||||||||||||||||||||||||||||
| Bug Blocks: | 271673, 274407, 275003, 275005, 275877, 277552, 281973 | ||||||||||||||||||||||||||||
| Attachments: |
|
||||||||||||||||||||||||||||
Note: we can't do this until USE_PYTHON=distutils is switched to devel/py-setuptools58. Exp-run looks fine (0 new failure compared to setuptools 63.1.0) (In reply to Charlie Li from comment #1) Hope this switch soon, some ports update require setuptools >65.0. (In reply to Wen Heping from comment #3) I have ports update pending for newer setuptools. It'll be great if you could commit it tomorrow. Otherwise I'd like to do so. Thanks. Please don't yet. I have it running locally as well, but only for those under PEP-517 and not the distutils (setup.py) method. The exp-run showing no build failures is a mere coincidence. The diff as it stands right now will not work for Python 3.12 at all. Additionally, the commented TEST_DEPENDS et al need to remain, since they will become usable once setuptools is removed as a RUN_DEPENDS unconditionally (addressed separately). Hmm, I don't quite follow you regarding the PY_SETUPTOOLS and py-setuptools58. We definitely would have: - some port use USE_PYTHON=pep517 with newer setuptools - many other ports still use USE_PYTHON=distutils If you want to use py-setuptools58 (or anything except py-setuptools) for the latter, how can these ports coexist? What's your plan to handle the conflict between py-setuptools and py-setuptools58? Take textproc/codespell 2.2.4 for example. from pyproject.toml (https://github.com/codespell-project/codespell/blob/v2.2.4/pyproject.toml#L66-L67): [build-system] requires = ["setuptools>=64", "setuptools_scm[toml]>=6.2"] What is your solution for not updating py-setuptools? Direct invocation of setup.py is explicitly not supported upstream after setuptools 58, so USE_PYTHON=distutils will use devel/py-setuptools58. Long technical read, but: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html setuptools is to be removed and forbidden to be specified as a RUN_DEPENDS. It is not necessary after deprecating pkg_resources and replacing its functionality with importlib, causes precisely the conflicts you describe and even wrong dependency trees. wheel demonstrates a wrong dependency tree, since setuptools has no wheel functionality of its own and thus relies on it for bdist_wheel, etc. Since Python 3.12 no longer includes distutils, the current setuptools build procedure will not work and will have to use the PEP-517 procedure to build itself, which includes having wheel available beforehand. I know all of them already but it does not answer my question at the end of comment #6. I agree that setuptools should be removed from runtime dependency, but it still lives as a build dependency. I'm talking about the real world. Not everyone builds packages in a clean environment, e.g. poudriere or jail. Users could build/run things directly on a FreeBSD box. Let's say, a users needs both ports: - port A uses USE_PYTHON=pep517 with newer setuptools ==> port A and py-setuptools (its build dependency) - port B uses USE_PYTHON=distutils ==> port B and py-setuptoosl58 (its build dependency) And your solution just does not work because py-setuptools conflicts with py-setuptoosl58. Even if "the exp-run showing no build failures is a mere coincidence", it means we could use setuptools 67.6.0 now. (In reply to Po-Chuan Hsieh from comment #8) It's unfortunate that they have to conflict, particularly in not clean environments, but this is the upstream directive. Remember that much of Python's "real world" revolves around virtual environments (and pre-built wheels when available), not "bare metal". When upstream have been pleading everyone for years to stop directly executing setup.py, going as far as setting a last supported version for it, it's time to take heed even if "bare metal" conflicts arise. Conflicts like this suck big time but the ecosystem has to move forward. What is further unsustainable is continuing to use latest setuptools on USE_PYTHON=distutils and running exp-runs on every update, knowing full well that this method is explicitly unsupported upstream and will eventually break and go away. This port will get updated to 67.6.0 or whatever release it may be once this is settled, but only after USE_PYTHON=distutils is rightfully banished to setuptools 58. py39-setuptools-63.1.0 is vulnerable: py39-setuptools -- denial of service vulnerability CVE: CVE-2022-40897 WWW: https://vuxml.FreeBSD.org/freebsd/1b38aec4-4149-4c7d-851c-3c4de3a1fbd0.html Created attachment 241522 [details]
devel/py-setuptools update to 67.6.1
Looks like this won't be applied, but if it works for some environments, maybe the patch can save someone 5 minutes.
Version 67.7.2 was released just a few hours ago: https://pypi.org/project/setuptools/ Created attachment 241717 [details]
Patch to update to 67.7.2
In the spirit of the previous attachment, here's one that updates to 67.7.2. It compiles, but I have not done any real testing of the result yet.
Update must build under PEP-517; the current method does not work on Python 3.12. This also does not get updated until after setuptools is both removed as RUN_DEPENDS and USE_PYTHON=distutils switched to setuptools58. *** Bug 271364 has been marked as a duplicate of this bug. *** Because of py39-setuptools-63.1.0 is vulnerable: py39-setuptools -- denial of service vulnerability CVE: CVE-2022-40897 WWW: https://vuxml.FreeBSD.org/freebsd/1b38aec4-4149-4c7d-851c-3c4de3a1fbd0.html priority has to be set to: affects many I'd appreciate if we could have an version in the ports for Python3.9 that has no vulnerabilities. That should have priority over problems in Python3.12. (In reply to p5B2EA84B3 from comment #16) No. The logistical problem affects every version of Python. There are still way too many packages that use distutils/setup.py, which is explicitly not supported past setuptools 58. This setup is already delicate enough and any updates have to be done in a specific order. Is there any way for us minions watching this to monitor progress? (In reply to Dan Langille from comment #18) The big task currently is bug 270510, primarily to minimise the blast radius in rebuilding. Most packages have not needed setuptools to run for quite some time. Created attachment 242931 [details]
py-setuptools-68.0.0.patch
A patch I am using locally for some minor Python Port development. It is provided here for interested users; I think it meets the current plans for Python and Setuptools in the Ports tree, if not, am happy to work on it.
Comment on attachment 242931 [details] py-setuptools-68.0.0.patch The only BUILD_DEPENDS and RUN_DEPENDS should be wheel. build and installer are already included as part of USE_PYTHON=pep517. plist is autogenerated from RECORD so USE_PYTHON=autoplist needs to remain. You will also need to rebase on the latest tree revision and remove everything under files/; nothing in there is/will be relevant anymore (easy_install is no longer installed). Additionally, the test suite can be exposed once bug 270510 is completed. Created attachment 242941 [details]
py-setuptools-68.0.0.patch
(In reply to Charlie Li from comment #21) OK, I think this round of feedback has been addressed with the updated patch, thanks. > Additionally, the test suite can be exposed once bug 270510 is completed. I will post a comment in that thread, thank you. Created attachment 242943 [details]
py-setuptools-68.0.0.patch
Restore ${PYDISTUTILS_SETUP}
Comment on attachment 242943 [details]
py-setuptools-68.0.0.patch
${PYDISTUTILS_SETUP} is not used with USE_PYTHON=pep517. In any case, this is incorrect since this would not work with the upcoming Python 3.12 which removed distutils entirely.
Comment on attachment 242941 [details]
py-setuptools-68.0.0.patch
Restore previous path removing ${PYDISTUTILS_SETUP}
Comment on attachment 242943 [details]
py-setuptools-68.0.0.patch
Remove patch with unnecessary re-addition of ${PYDISTUTILS_SETUP}
This bug gets increasingly annoying. It has the potential for a rant, but I’ ll ask questions instead: 1. Why is the importance of this bug set to “Affects Only Me“? This is obviously wrong. It has to be changed to “Affects Many People ”. 2. Bug 270510 is a blocker. There importance is correctly set to “Affects Many People “. There are 2 lists of ports to work at, each three months old. The latest comment is from 2023-03-30 09:40:18 UTC. There is no indication visible that there has been done any work since then. 3. Have port maintainers of the ports on the lists of Bug 270510 been informed that they should “remove setuptools from RUN_DEPENDS”? 4. Do we need to create new PRs for each port of the Bug 270510 lists? 5. As “the list” is three months old, is there a need for an actualized list? 6. Do we need here a supervision of the FreeBSD bugmasters for coordinating the progress as progress is not visible except for someone trying to provide a patch but that won't solve the Bug 270510 lists? (In reply to p5B2EA84B3 from comment #28) All of these bugs are ports framework-level bugs, as the dependencies involved are not specified in individual ports, but may have some transience; it's trickier than it looks. Lack of comments et al does not in any way imply lack of activity. That exp-run list is still current as the failure modes largely have not changed. There is nothing you can do here. (In reply to Charlie Li from comment #29) As transparency lacks and words like "tricky" may hide a bigger mess, please show the place where reviews are discussed and state of progress can be looked at. I doubt that this issue is handled in an appropriate professional way. Can we do better, please? A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=9a80a6ee5e8ebf5c279b3d7a9e7e8b7dbabbaf0a commit 9a80a6ee5e8ebf5c279b3d7a9e7e8b7dbabbaf0a Author: Charlie Li <vishwin@FreeBSD.org> AuthorDate: 2023-07-18 00:16:21 +0000 Commit: Charlie Li <vishwin@FreeBSD.org> CommitDate: 2023-07-18 00:53:09 +0000 devel/py-setuptools: convert individual consumers to ${PY_SETUPTOOLS} Currently a no-op, but in the future outputs the correct setuptools port depending on whether USES_PYTHON=distutils is specified. With hat: python PR: 270510, 270358 archivers/py-blosc2/Makefile | 2 +- archivers/py-python-rpm-packaging/Makefile | 2 +- archivers/py-sozipfile/Makefile | 2 +- archivers/py-zopfli/Makefile | 2 +- astro/py-metpy/Makefile | 2 +- astro/py-spacetrack/Makefile | 2 +- audio/py-gtts/Makefile | 2 +- benchmarks/uica/Makefile | 2 +- biology/py-resdk/Makefile | 2 +- databases/py-aiomysql/Makefile | 2 +- databases/py-asyncmy/Makefile | 2 +- databases/py-fastparquet/Makefile | 2 +- databases/py-geoalchemy2/Makefile | 2 +- databases/py-mysqlclient/Makefile | 2 +- databases/py-psycopg-c/Makefile | 2 +- databases/py-psycopg-pool/Makefile | 2 +- databases/py-psycopg/Makefile | 2 +- databases/py-sqlalchemy20/Makefile | 2 +- databases/py-tiledb/Makefile | 2 +- databases/rrdtool/Makefile | 2 +- deskutils/py-send2trash/Makefile | 2 +- deskutils/recoll/Makefile | 2 +- deskutils/zim/Makefile | 2 +- devel/ipython/Makefile | 2 +- devel/libgusb/Makefile | 2 +- devel/mercurial/Makefile | 2 +- devel/nanopb/Makefile | 2 +- devel/py-QtPy/Makefile | 2 +- devel/py-aiosignal/Makefile | 2 +- devel/py-ansi/Makefile | 2 +- devel/py-anyio/Makefile | 2 +- devel/py-apptools/Makefile | 2 +- devel/py-aspectlib/Makefile | 2 +- devel/py-astroid/Makefile | 2 +- devel/py-attrs-strict/Makefile | 2 +- devel/py-autopage/Makefile | 2 +- devel/py-avro/Makefile | 2 +- devel/py-b2sdk/Makefile | 2 +- devel/py-backports.cached-property/Makefile | 2 +- devel/py-backports.entry-points-selectable/Makefile | 2 +- devel/py-blinker/Makefile | 2 +- devel/py-bytecode/Makefile | 2 +- devel/py-cachetools/Makefile | 2 +- devel/py-calver/Makefile | 2 +- devel/py-canonicaljson/Makefile | 2 +- devel/py-case/Makefile | 2 +- devel/py-cmd2/Makefile | 2 +- devel/py-configobj/Makefile | 2 +- devel/py-configupdater/Makefile | 2 +- devel/py-ctypesgen/Makefile | 2 +- devel/py-curtsies/Makefile | 2 +- devel/py-custodian/Makefile | 2 +- devel/py-cwcwidth/Makefile | 2 +- devel/py-cython-test-exception-raiser/Makefile | 2 +- devel/py-d2to1/Makefile | 2 +- devel/py-dask/Makefile | 2 +- devel/py-datasette/Makefile | 2 +- devel/py-ddtrace/Makefile | 2 +- devel/py-debugpy/Makefile | 2 +- devel/py-deepmerge/Makefile | 2 +- devel/py-distributed/Makefile | 2 +- devel/py-editdistance/Makefile | 2 +- devel/py-envier/Makefile | 2 +- devel/py-envisage/Makefile | 4 ++-- devel/py-epsilon/Makefile | 2 +- devel/py-ewah-bool-utils/Makefile | 2 +- devel/py-executing/Makefile | 2 +- devel/py-fasteners/Makefile | 2 +- devel/py-fixtures/Makefile | 2 +- devel/py-flake8-black/Makefile | 2 +- devel/py-flake8-bugbear/Makefile | 2 +- devel/py-flake8-comprehensions/Makefile | 2 +- devel/py-flake8_simplify/Makefile | 2 +- devel/py-google-crc32c/Makefile | 2 +- devel/py-hunter/Makefile | 2 +- devel/py-importlib-metadata/Makefile | 2 +- devel/py-inflect/Makefile | 2 +- devel/py-jaraco.classes/Makefile | 2 +- devel/py-jaraco.collections/Makefile | 2 +- devel/py-jaraco.context/Makefile | 2 +- devel/py-jaraco.env/Makefile | 2 +- devel/py-jaraco.functools/Makefile | 2 +- devel/py-jaraco.itertools/Makefile | 2 +- devel/py-jaraco.logging/Makefile | 2 +- devel/py-jaraco.text/Makefile | 2 +- devel/py-jaraco.timing/Makefile | 2 +- devel/py-jaraco.util/Makefile | 2 +- devel/py-jupyter-server-mathjax/Makefile | 2 +- devel/py-klepto/Makefile | 2 +- devel/py-libcst/Makefile | 2 +- devel/py-libtmux/Makefile | 2 +- devel/py-llfuse/Makefile | 2 +- devel/py-monkeytype/Makefile | 2 +- devel/py-moto/Makefile | 2 +- devel/py-msgpack/Makefile | 2 +- devel/py-mypy-protobuf/Makefile | 2 +- devel/py-mypy/Makefile | 2 +- devel/py-mystic/Makefile | 2 +- devel/py-opentelemetry-api/Makefile | 2 +- devel/py-opentelemetry-sdk/Makefile | 2 +- devel/py-parameterized/Makefile | 2 +- devel/py-parver/Makefile | 2 +- devel/py-path/Makefile | 2 +- devel/py-pathos/Makefile | 2 +- devel/py-pdoc/Makefile | 2 +- devel/py-pep621/Makefile | 2 +- devel/py-pint-pandas/Makefile | 2 +- devel/py-pint/Makefile | 2 +- devel/py-pip-requirements-parser/Makefile | 2 +- devel/py-pip-run/Makefile | 2 +- devel/py-pip-tools/Makefile | 4 ++-- devel/py-pipenv/Makefile | 4 ++-- devel/py-pluggy/Makefile | 2 +- devel/py-pooch/Makefile | 2 +- devel/py-pox/Makefile | 2 +- devel/py-ppft/Makefile | 2 +- devel/py-prance/Makefile | 2 +- devel/py-py-partiql-parser/Makefile | 2 +- devel/py-pyee/Makefile | 2 +- devel/py-pyface/Makefile | 2 +- devel/py-pyina/Makefile | 2 +- devel/py-pyls-black/Makefile | 2 +- devel/py-pyperf/Makefile | 2 +- devel/py-pyproject-metadata/Makefile | 2 +- devel/py-pyscaffold/Makefile | 2 +- devel/py-pytest-checkdocs/Makefile | 2 +- devel/py-pytest-enabler/Makefile | 2 +- devel/py-pytest-helpers-namespace/Makefile | 2 +- devel/py-pytest-html/Makefile | 2 +- devel/py-pytest-localserver/Makefile | 2 +- devel/py-pytest-mpl/Makefile | 2 +- devel/py-pytest-plus/Makefile | 2 +- devel/py-pytest-services/Makefile | 2 +- devel/py-pytest-subtests/Makefile | 2 +- devel/py-pytest-sugar/Makefile | 2 +- devel/py-pytest/Makefile | 2 +- devel/py-pythonfinder/Makefile | 2 +- devel/py-pytweening/Makefile | 2 +- devel/py-pytz-deprecation-shim/Makefile | 2 +- devel/py-pyvisa-py/Makefile | 2 +- devel/py-pyvisa/Makefile | 2 +- devel/py-pyyaml-include/Makefile | 2 +- devel/py-represent/Makefile | 2 +- devel/py-requirementslib/Makefile | 2 +- devel/py-resolvelib1/Makefile | 2 +- devel/py-retype/Makefile | 2 +- devel/py-rope/Makefile | 2 +- devel/py-rstr/Makefile | 2 +- devel/py-scikit-base/Makefile | 2 +- devel/py-semver/Makefile | 2 +- devel/py-setuptools-declarative-requirements/Makefile | 2 +- devel/py-setuptools-pkg/Makefile | 2 +- devel/py-setuptools-rust/Makefile | 4 ++-- devel/py-shtab/Makefile | 2 +- devel/py-simpleeval/Makefile | 2 +- devel/py-subprocess-tee/Makefile | 2 +- devel/py-tempora/Makefile | 2 +- devel/py-time-machine/Makefile | 2 +- devel/py-trove-classifiers/Makefile | 2 +- devel/py-twine/Makefile | 2 +- devel/py-txi2p-tahoe/Makefile | 2 +- devel/py-typeguard/Makefile | 2 +- devel/py-validate-pyproject/Makefile | 2 +- devel/py-versioneer/Makefile | 2 +- devel/py-versioningit/Makefile | 2 +- devel/py-vistir/Makefile | 2 +- devel/py-watermark/Makefile | 2 +- devel/py-yamllint/Makefile | 2 +- devel/py-zarr/Makefile | 2 +- devel/py-zipp/Makefile | 2 +- devel/pylint/Makefile | 2 +- devel/pytype/Makefile | 2 +- finance/py-technical/Makefile | 2 +- graphics/py-PyOpenGL-accelerate/Makefile | 2 +- graphics/py-fotokilof/Makefile | 2 +- graphics/py-pillow/Makefile | 2 +- graphics/py-pygeos/Makefile | 2 +- graphics/py-pyproj/Makefile | 2 +- graphics/py-rasterstats/Makefile | 2 +- graphics/py-traitsui/Makefile | 2 +- graphics/py-webcolors/Makefile | 2 +- math/e-antic/Makefile | 2 +- math/py-autograd/Makefile | 2 +- math/py-cvxopt/Makefile | 2 +- math/py-fsph/Makefile | 2 +- math/py-jacobi/Makefile | 2 +- math/py-jax/Makefile | 2 +- math/py-ml-dtypes/Makefile | 2 +- math/py-numba-stats/Makefile | 2 +- math/py-openTSNE/Makefile | 2 +- math/py-pandas/Makefile | 2 +- math/py-permutation/Makefile | 2 +- math/py-py-find-1st/Makefile | 2 +- math/py-pygsl/Makefile | 2 +- math/py-pytensor/Makefile | 2 +- math/py-resample/Makefile | 2 +- math/py-rpy2/Makefile | 2 +- math/py-scikit-fem/Makefile | 2 +- misc/libcomps/Makefile | 2 +- misc/py-crudini/Makefile | 2 +- misc/py-openai/Makefile | 2 +- misc/py-palettable/Makefile | 2 +- misc/py-spdx-tools/Makefile | 2 +- misc/py-tqdm/Makefile | 2 +- misc/py-wandb/Makefile | 2 +- multimedia/py-pymediainfo/Makefile | 2 +- multimedia/py-pysubs2/Makefile | 2 +- multimedia/streamlink/Makefile | 2 +- net-mgmt/crmsh/Makefile | 4 ++-- net-mgmt/net-snmp/Makefile | 4 ++-- net/onionprobe/Makefile | 2 +- net/py-matrix-synapse-ldap3/Makefile | 2 +- net/py-pyroute2.minimal/Makefile | 2 +- net/py-pyroute2/Makefile | 2 +- news/py-sabctools/Makefile | 2 +- print/py-freetype-py/Makefile | 2 +- print/py-python-ly/Makefile | 2 +- print/py-ufolib2/Makefile | 2 +- print/py-uharfbuzz/Makefile | 2 +- science/py-asdf-standard/Makefile | 2 +- science/py-asdf-transform-schemas/Makefile | 2 +- science/py-asdf-unit-schemas/Makefile | 2 +- science/py-asdf/Makefile | 2 +- science/py-pymatgen-analysis-alloys/Makefile | 2 +- science/py-segregation/Makefile | 2 +- science/py-signac/Makefile | 2 +- science/py-spaghetti/Makefile | 2 +- science/smoldyn/Makefile | 2 +- science/thermofun/Makefile | 2 +- security/gpgme/Makefile | 4 ++-- security/libpwquality/Makefile | 2 +- security/py-certomancer/Makefile | 2 +- security/py-httpx-auth/Makefile | 2 +- security/py-pyhanko-certvalidator/Makefile | 2 +- security/py-pyhanko/Makefile | 2 +- security/py-pyspnego/Makefile | 2 +- sysutils/datadog-integrations/Makefile | 2 +- sysutils/py-ansible-compat/Makefile | 2 +- sysutils/py-ansible-core/Makefile | 2 +- sysutils/py-ansible-lint/Makefile | 2 +- sysutils/py-distro/Makefile | 2 +- sysutils/py-hcloud/Makefile | 2 +- sysutils/py-liquidctl/Makefile | 2 +- sysutils/py-ploy/Makefile | 2 +- sysutils/py-ploy_ezjail/Makefile | 2 +- sysutils/py-psutil/Makefile | 2 +- sysutils/syslog-ng/Makefile | 2 +- textproc/codespell/Makefile | 2 +- textproc/py-asciinema/Makefile | 2 +- textproc/py-ini2toml/Makefile | 2 +- textproc/py-jq/Makefile | 2 +- textproc/py-linkify-it-py/Makefile | 2 +- textproc/py-markdown/Makefile | 2 +- textproc/py-markups/Makefile | 2 +- textproc/py-mistune/Makefile | 2 +- textproc/py-mistune2/Makefile | 2 +- textproc/py-nbsphinx/Makefile | 2 +- textproc/py-normality/Makefile | 2 +- textproc/py-ocrmypdf/Makefile | 2 +- textproc/py-openstep-plist/Makefile | 2 +- textproc/py-parse-type/Makefile | 2 +- textproc/py-parse/Makefile | 2 +- textproc/py-petl/Makefile | 2 +- textproc/py-pyaml/Makefile | 2 +- textproc/py-pygments/Makefile | 2 +- textproc/py-python-markdown-math/Makefile | 2 +- textproc/py-sphinx-autoapi/Makefile | 2 +- textproc/py-sphinxcontrib-towncrier/Makefile | 2 +- textproc/py-uc-micro-py/Makefile | 2 +- textproc/py-unicodeitplus/Makefile | 2 +- textproc/translate-toolkit/Makefile | 2 +- textproc/xml2rfc/Makefile | 2 +- www/onionbalance/Makefile | 2 +- www/py-aiohttp/Makefile | 2 +- www/py-aioquic/Makefile | 2 +- www/py-branca/Makefile | 2 +- www/py-cheroot/Makefile | 2 +- www/py-cherrypy/Makefile | 2 +- www/py-cssutils/Makefile | 2 +- www/py-dj41-django-cors-headers/Makefile | 2 +- www/py-dj41-django-rich/Makefile | 2 +- www/py-django-bleach/Makefile | 2 +- www/py-django-braces/Makefile | 2 +- www/py-django-cors-headers/Makefile | 2 +- www/py-django-crispy-forms/Makefile | 2 +- www/py-django-haystack/Makefile | 2 +- www/py-django-rich/Makefile | 2 +- www/py-feedgenerator/Makefile | 2 +- www/py-flask-migrate/Makefile | 2 +- www/py-flask-socketio/Makefile | 2 +- www/py-flask-wtf/Makefile | 2 +- www/py-flask/Makefile | 2 +- www/py-folium/Makefile | 2 +- www/py-hyperlink/Makefile | 2 +- www/py-lektor/Makefile | 2 +- www/py-pastedeploy/Makefile | 2 +- www/py-pyjwt/Makefile | 2 +- www/py-pylsqpack/Makefile | 2 +- www/py-scgi/Makefile | 2 +- www/py-tornado/Makefile | 2 +- www/py-waitress/Makefile | 2 +- www/py-werkzeug/Makefile | 2 +- www/py-woob/Makefile | 2 +- www/radicale/Makefile | 2 +- x11-fonts/py-afdko/Makefile | 2 +- x11-fonts/py-cffsubr/Makefile | 2 +- x11-fonts/py-glyphsLib/Makefile | 2 +- x11-fonts/py-opentype-sanitizer/Makefile | 2 +- x11-fonts/py-ufonormalizer/Makefile | 2 +- 309 files changed, 316 insertions(+), 316 deletions(-) (In reply to commit-hook from comment #31) Thank you for this. I am now able to clear monitoring alerts which are over 95 days old. The current version is 68.1.2 according to PyPI: https://pypi.org/project/setuptools/ . I noticed this because I was looking at what we were shipping in relation to what is available with FreeBSD and I was surprised by the fact that the copy provided by FreeBSD is over a year old at this point. I echo the need for transparency, given that there're CVEs out for devel/py-setuptools currently: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-40897#:~:text=Python%20Packaging%20Authority%20(PyPA)%20setuptools,ReDoS)%20in%20package_index.py. . What is being done to rectify any issues present with the port(s) and update them to non-vulnerable versions? A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=121d8af16a7efd4f4ec25c5851f9976b396192c0 commit 121d8af16a7efd4f4ec25c5851f9976b396192c0 Author: Dan Langille <dvl@FreeBSD.org> AuthorDate: 2023-10-11 12:09:02 +0000 Commit: Dan Langille <dvl@FreeBSD.org> CommitDate: 2023-10-11 12:09:48 +0000 devel/py-setuptools: Update to 68.0.0 PR: 270358 devel/py-setuptools/Makefile | 3 +-- devel/py-setuptools/distinfo | 6 +++--- .../files/patch-setuptools_package__index.py (gone) | 11 ----------- 3 files changed, 4 insertions(+), 16 deletions(-) Im sorry, I committed this without realizing. Reverting. A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=b1f7e31dea5e702afc08d2cd7e993f36ecf824cc commit b1f7e31dea5e702afc08d2cd7e993f36ecf824cc Author: Dan Langille <dvl@FreeBSD.org> AuthorDate: 2023-10-11 12:16:09 +0000 Commit: Dan Langille <dvl@FreeBSD.org> CommitDate: 2023-10-11 12:17:08 +0000 devel/py-setuptools: revert upgrade to 68.0.0 This reverts commit 121d8af16a7efd4f4ec25c5851f9976b396192c0. PR: 270358 devel/py-setuptools/Makefile | 3 ++- devel/py-setuptools/distinfo | 6 +++--- .../files/patch-setuptools_package__index.py (new) | 11 +++++++++++ 3 files changed, 16 insertions(+), 4 deletions(-) Created attachment 245574 [details] devel/py-setuptools: Update to 68.2.2 devel/py-setuptools: Update to 68.2.2 https://setuptools.pypa.io/en/stable/history.html#v68-2-2 (In reply to p5B2EA84B3 from comment #30) I failed miserably at making any movement or progress with Python on FreeBSD, I have next to no idea on what is going on, or how to proceed, attempting to pick up the pieces is a waste of everyone's time. Ports remain outdated, FreeBSD's Python platform lags as a result. Formal, and informal members of the community have leveled legitimate criticism on the progress in this space, see comment #28, comment 30, and comment 33; true to community values, silence is the response. https://wiki.freebsd.org/Python/setuptools Specifically on this bug, bug 270510 and switching all USE_PYTHON=distutils to use devel/py-setuptools58 must be completed, in that order, before this can be committed. Any help is welcome. Life, other obligations and fires get in the way, and their effects are amplified during rather complicated processes like this. (In reply to Charlie Li from comment #39) Please feel free to reach out to me at ngie@ if you think I could help with this. $work directly benefits from the update, as well as others, so I wouldn't mind doing the some of the heavy lifting (if it hasn't already been done) and passing off the work to someone else in the ports space (usually my ports patch submissions stall in the handoff/review phase since I only have a src-committers bit). BTW, I've installed py39-setuptools-68.2.2 using the attached patch locally. No issues so far. sysutils/py-ansible-core, which needs this version builds and installs correctly. Added sysutils/py-ansible-core PR to dependency list. Added PR/275005 to depends. Created attachment 246650 [details] Update to 69.0.2 Update to 69.0.2. > poudriere-testport 12.4-{amd64,i386}-release: OK > poudriere-testport 13.2-{amd64,i386}-release: OK > poudriere-testport 14.0-{amd64,i386}-release: OK > poudriere-testport main-{amd64,i386}: OK In case anyone missed it, there is fabulous piece of writing in the latest FreeBSD Journal, by one Charlie Li, PEP 517: Python Packaging’s New World Order. The article, rather eloquently, explains the context behind this Bug Report: https://freebsdfoundation.org/our-work/journal/browser-based-edition/pep-517-python-packagings-new-world-order/ Thanks, Charlie. Created attachment 247216 [details] devel/py-setuptools: Update to 69.0.3 Update to 69.0.3: https://github.com/pypa/setuptools/compare/v63.1.0...v69.0.3 I've applied this patch to my up and coming tree (main + patches like this one, including making python 3.10 default). There have been on build issues so far. Is depending on PR/270510 necessary? (In reply to Cy Schubert from comment #46) It is necessary. Further, there will be an additional dependency to switch the USE_PYTHON=distutils consumers to devel/py-setuptools58 due to the publicised deprecation and incompatibility for directly executing setup.py. Only one setuptools can be present in the same environment at a time. In the end, this specific order is to limit churn by only performing one mass PORTREVISION bump (bug 270510). Created attachment 248496 [details] devel/py-setuptools: Update to 69.1.0 devel/py-setuptools: Update to 69.1.0 Update to 69.1.0: https://github.com/pypa/setuptools/compare/v63.1.0...v69.1.0 - Make use of PEP517 for installation - Use bsd.port.options.mk/bsd.port.mk inclusion pair over legacy. - Use ${DISTVERSION} over ${PORTVERSION} - Port Makefile housekeeping - Drop obsolete patches poudriere-testport (Python-3.11): > 13.2-{amd64,i386}-release: OK > 14.0-{amd64,i386}-release: OK > main-{amd64,i386}: OK Created attachment 248818 [details] devel/py-setuptools: Update to 69.1.1 Update to 69.1.1: https://github.com/pypa/setuptools/compare/v63.1.0...v69.1.1 - Drop obsolete patches - Port Makefile housekeeping - Switch to PEP517 build - Use ${DISTVERSION} over ${PORTVERSION} - Use bsd.port.options.mk/bsd.port.mk inclusion pair over legacy. poudriere-testport: > 13.2-{amd64,i386}-release: OK > 14.0-{amd64,i386}-release: OK > main-{amd64,i386}: OK What's the status of this? Reason: I have a number ansible commits (in a branch) waiting in the wings pending this update. Once this is pushed can I push the batch of commits. (In reply to Cy Schubert from comment #50) At least bug 270510 must be completed. At least all USE_PYTHON=distutils cases need to move off this port before proceeding. There is no way around this without causing far too much unnecessary churn. (In reply to Charlie Li from comment #51) Shall we create devel/py-setuptools69 for temporarily use ? https://data.safetycli.com/vulnerabilities/CVE-2022-40897/52495/ is dated Dec 23, 2022. Affecting all versions < 65.5.1. Given that in ports we have $ pkg search -x setuptools py27-setuptools44-44.1.1_1 Python packages installer py310-setuptools-63.1.0_1 Python packages installer py310-setuptools58-58.5.3_3 Python packages installer py311-setuptools-63.1.0_1 Python packages installer py311-setuptools58-58.5.3_3 Python packages installer py38-setuptools-63.1.0_1 Python packages installer py38-setuptools58-58.5.3_3 Python packages installer py39-setuptools-63.1.0_1 Python packages installer why aren't any of these in vuln.xml.xz ? (In reply to void from comment #53) Does there have to be a reason? Please add them. :) Created attachment 249587 [details] devel/py-setuptools: Update to 69.2.0 Update to 69.2.0: https://github.com/pypa/setuptools/compare/v63.1.0...v69.2.0 [TESTPORT] poudriere-testport devel/py-setuptools: [TESTPORT] main-amd64-current: PASS [TESTPORT] main-i386-current: PASS [TESTPORT] 13_2-amd64-release: PASS [TESTPORT] 13_2-i386-release: PASS [TESTPORT] 13_3-amd64-release: PASS [TESTPORT] 13_3-i386-release: PASS [TESTPORT] 14_0-amd64-release: PASS [TESTPORT] 14_0-i386-release: PASS This is now additionally blocked on a necessary upstream commit to the current devel/py-pyproject_hooks, as this will not build otherwise, PR and patch forthcoming. 71.0.0 now allows for de-vendoring its own dependencies, and in fact now prefers them already installed in the environment. Work will now include making sure those declared dependencies are not circular. 72.0.0 removed setup.py test, will probably need corresponding python.mk changes. Working diff posted as review D46236. To test, you will need to apply all dependent PRs before this one, especially bug 280646. net/py-suds is a casualty of the setuptools.command.test removal/deprecation stubbing:
* Getting build dependencies for wheel...
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 380, in <module>
main()
File "/usr/local/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 364, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 141, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
self.run_setup()
File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 497, in run_setup
super().run_setup(setup_script=setup_script)
File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "<string>", line 509, in <module>
ImportError: cannot import name 'normalize_path' from 'setuptools.command.test' (/usr/local/lib/python3.11/site-packages/setuptools/command/test.py)
ERROR Backend subprocess exited when trying to invoke get_requires_for_build_wheel
*** Error code 1
Stop.
make: stopped making "build" in /usr/ports/net/py-suds
This package does not have a pyproject.toml and is thus not compliant with PEP-517, but even reverting the port to USE_PYTHON=distutils results in the same error.
72.2.0 released. Current version also vulnerable to CVE-2024-6345, fixed in 70.0.0 and later. https://nvd.nist.gov/vuln/detail/CVE-2024-6345 https://huntr.com/bounties/d6362117-ad57-4e83-951f-b8141c6e7ca5 Also not in vulnxml. distutils removed legacy msvc compiler modules, which made it into setuptools 74.0.0. Broke some consumers that happen to reference them, notably devel/gobject-introspection and math/py-numpy. I just ran into an issue with some Python scripts for a development environment requiring py-setuptools > 64. How is this proceeding? *** Bug 282353 has been marked as a duplicate of this bug. *** (In reply to Po-Chuan Hsieh from comment #4) Agreed. Our ansible ports are considerably out of date. Updating them requires a new py-setuptools. 75.3.0 available on phab. Latest upstream version is 75.5.0, which drops Python 3.8 support and requires devel/py-packaging 24.2, which I trust sunpoet@ will get to soon. |
Created attachment 241004 [details] Update to 67.6.0 - Update to 67.6.0