===> Fetching all distfiles required by py38-pyparsing-3.0.6 for building ===> Extracting for py38-pyparsing-3.0.6 => SHA256 Checksum OK for pyparsing-3.0.6.tar.gz. ===> Patching for py38-pyparsing-3.0.6 ===> py38-pyparsing-3.0.6 depends on package: py38-setuptools>0 - found ===> py38-pyparsing-3.0.6 depends on file: /usr/local/bin/python3.8 - found ===> py38-pyparsing-3.0.6 depends on file: /usr/local/bin/ccache - found ===> Configuring for py38-pyparsing-3.0.6 Traceback (most recent call last): File "<string>", line 1, in <module> File "setup.py", line 23, in <module> setup( # Distribution meta-data File "/usr/local/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup return distutils.core.setup(**attrs) File "/usr/local/lib/python3.8/distutils/core.py", line 108, in setup _setup_distribution = dist = klass(attrs) File "/usr/local/lib/python3.8/site-packages/setuptools/dist.py", line 455, in __init__ _Distribution.__init__(self, { File "/usr/local/lib/python3.8/distutils/dist.py", line 292, in __init__ self.finalize_options() File "/usr/local/lib/python3.8/site-packages/setuptools/dist.py", line 800, in finalize_options for ep in sorted(eps, key=by_order): File "/usr/local/lib/python3.8/site-packages/setuptools/dist.py", line 799, in <lambda> eps = map(lambda e: e.load(), pkg_resources.iter_entry_points(group)) File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in load self.require(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2472, in require items = working_set.resolve(reqs, env, installer, extras=self.extras) File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 772, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'packaging>=20.0' distribution was not found and is required by the application *** Error code 1
Trying to install py-packaging with portmaster will fail, since py-pyparsing is listed as a runtime requirement for py-packaging (so portmaster tries to run the py-pyparsing build before the install step from py-packaging). I was able to proceed by running 'make install clean' within /usr/ports/devel/py-packaging, then within /usr/ports/devel/py-pyparsing, then continuing with the rest of the portmaster step I was completing.
Probably this is not py-pyparsing error. Same error was with other python ports until I run "pkg autoremove", it remove: py38-Babel: 2.9.1 py38-Jinja2: 3.0.1 py38-beaker: 1.11.0 py38-cython: 0.29.24 py38-docutils: 0.17.1,1 py38-html5lib: 1.0.1 py38-lxml: 4.6.4 py38-mako: 1.0.14_1 py38-markupsafe: 2.0.1 py38-ply: 3.11 py38-pyaes: 1.6.1 py38-pyparsing: 3.0.4 py38-pytest-runner: 5.3.1 py38-setuptools_scm: 6.3.2 py38-tomli: 1.2.2 py38-webencodings: 0.5.1 py38-wheel: 0.36.2 Somehow py38-packaging was not installed, but py38-setuptools_scm requires this. Removing py38-setuptools_scm - fix issue for me.
Upstream declares: ./work-py39/setuptools_scm-6.3.2/pyproject.toml: "packaging>=20.0" But devel/pysetuptools_scm declares packaging as a RUN_DEPENDS, but should be a BUILD_DEPENDS. Workaround until resolved: install devel/py-packaging manually
(In reply to Kubilay Kocak from comment #3) Clarifying, upstream declares: [build-system] requires=..., "packaging>=20.0",...
The problem occurs in environments where devel/py-setuptools_scm is installed and devel/py-packaging is also updated together with other Python ports during an upgrade run via portmaster. The old version of devel/py-packaging will be removed at some point and the new version will be installed too late which will then leads to the issue described in comment #2. This is because devel/py-packaging is required for run time [1] by devel/py-setuptools_scm and the latter one is very interlocked with devel/py-setuptools. It's executed together almost every time a Python port is built and will fail if a dependency is missing during run time. The issue can be reproduced via: - Install/Build devel/py-pyparsing 3.0.4 - Install/Build devel/py-packaging 21.2_1 - Install/Build devel/py-setuptools_scm - Apply patches to update devel/py-packaging to 21.3 and devel/py-pyparsing to 3.0.6 - Try to upgrade the ports with portmaster The whole problem will most likely recur the next time devel/py-packaging is updated. Probably a long-term solution would be if portmaster treats ports like devel/py-setuptools* and their dependencies with higher priority over other Ports during an upgrade run. (In reply to Kubilay Kocak from comment #3) I'll add devel/py-packaging to BUILD_DEPENDS as it is in "pyproject.toml" and you have already mentioned it although it's not the solution to the problem reported with devel/py-pyparsing as described above. -- [1] https://github.com/pypa/setuptools_scm/blob/v6.3.2/setup.cfg#L30
I can confirm that py-setuptools_scm installed prevents building of other Python ports, if py-packaging or py-pyparsing are not available. But I do not see any issue when updating these ports with portmaster. I have tested all permutations of port names in the following command: # portmaster devel/py-pyparsing devel/py-packaging devel/py-setuptools_scm And in all cases the re-installation succeeded (as would have a port upgrade): ===>>> The following actions were performed: Re-installation of py38-pyparsing-3.0.6 Re-installation of py38-packaging-21.3 Re-installation of py38-setuptools_scm-6.3.2 The dependency checks performed by portmaster are somewhat different from those performed by a plain make, since portmaster plans all updates before starting the first one. But the updates are then performed in the same way as by a plain "make all deinstall install" in the port directory. Portmaster considers RUN_DEPENDS to actually mean that, i.e. a dependency that has to be provided to run the program built by the port. This is different from a plain "make", since that performs builds and installations of RUN_TIME dependencies before starting the "stage" phase. But this difference is irrelevant for this PR. A solution to this issue might be to treat a missing py-pyparsing or py-packaging in the same way as if py-setuptools_scm was not installed. The huge majority of Python ports does not depend on py-setuptools_scm - specifically py-packaging and py-pyparsing do not depend on it. (Of more than 7000 devel/py-* ports less than 70 directly depend on py-setuptools_scm.) But if py-setuptools_scm *is* installed, a missing py-pyparsing oder py-packages does suddenly become a configure dependency, since py-setuptools_scm is (unnecessarily) invoked, but fails due to the missing run-time dependency. Thus, a missing py-pyparsing or py-packaging should not cause py-setuptools_scm to fail in such a way, that setup.py is stopped. For ports that do not directly depend on py-setuptools_scm, such a failure should be ignored.
*** Bug 259993 has been marked as a duplicate of this bug. ***
Created attachment 229940 [details] py-setuptools_scm-workaround-for-py-packaging-v1.patch Hm, devel/py-setuptools_scm has a fallback (via "try ... except") in [1] when devel/py-packaging is absent. It's also the only location where devel/py-packaging is (optionally) required at the moment. However, this code is not executed because devel/py-setuptools, which checks the dependencies upon invocation and terminates as soon as these are not met, which leads then to the errors in comment #0. Attached is a workaround which should remedy the issue by patching out devel/py-packaging from "setup.cfg" but keep it in RUN_DEPENDS. It also adds devel/py-packaging to BUILD_DEPENDS as defined in pyproject.toml although it's not strictly required to build the port. I'll leave this patch here for a few days for general testing and feedback. QA: ~~~ - poudriere -> OK (12.2-RELEASE amd64 against all consumers) - portmaster -> OK (using the scenario outlined in comment #5) - "make test" -> OK -- [1] https://github.com/pypa/setuptools_scm/blob/v6.3.2/src/setuptools_scm/_version_cls.py
(In reply to Kai Knoblich from comment #8) I have tested the patch with multiple pre-conditions and build tools, and it fixes all issues caused by either or both of devel/py-pyparsing and devel/py-packaging missing when devel/py-setuptools_scm is installed. Please commit this patch to fix the issues reported in this PR and in PR 259993.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=ad7f115cddd949d6ffd41bd9889328c06f8906e4 commit ad7f115cddd949d6ffd41bd9889328c06f8906e4 Author: Kai Knoblich <kai@FreeBSD.org> AuthorDate: 2021-12-09 18:49:49 +0000 Commit: Kai Knoblich <kai@FreeBSD.org> CommitDate: 2021-12-09 18:49:49 +0000 devel/py-setuptools_scm: Turn py-packaging into a soft dependency * Since the 6.3.0 release, devel/py-packaging has been defined in the "setup.cfg" as an unconditional dependency. However this leads to problems if py-packaging is uninstalled but py-setuptools_scm persists, then the missing port can no longer be built. [1] Although py-setuptools_scm has a fallback to devel/py-setuptools when py-packaging is absent this fallback code is not executed. This is because py-setuptools, which checks the dependencies of py-setuptools_scm upon invocation and terminates as soon as these are not met. Fix this issue by converting py-packaging into an optional dependency for py-setuptools_scm so that the fallback code is really used if py-packaging is absent. But keep the latter one in the RUN_DEPENDS in order to continue to reflect the actual dependencies. * Also add py-packaging to BUILD_DEPENDS as defined in "pyproject.toml" although it's not strictly required to build the port. [2] * Bump PORTREVISION due package change. PR: 259981 Reported by: many [1], koobs [2] Tested by: se, myself devel/py-setuptools_scm/Makefile | 4 +++- devel/py-setuptools_scm/files/patch-setup.cfg (new) | 19 +++++++++++++++++++ 2 files changed, 22 insertions(+), 1 deletion(-)
I'll leave this bug open as reference for a few more days.
Beautiful. Python dynamic, conditional, at run-time or try/except dependency checks and code path changes are notorious for creating these situations, which are of the class 'non-deterministic dependency sets' and resolving them at the root cause upstream is absolutely the correct resolution.
Good to close Kai, or remaining things to resolve?
(In reply to Kubilay Kocak from comment #14) Thanks for the info, Kubilay. It's good to close so I'm closing this now.