Created attachment 174236 [details] Fix PYTHON_VERSION propagation. In the following (not so corner) case, the propagation of PYTHON_VERSION through DEPENDS_ARGS will result in dependency bugs: PYTHON_VERSION?= python${_PYTHON_VERSION} DEPENDS_ARGS+= PYTHON_VERSION=${PYTHON_VERSION} So, on a submake through the depends target and Mk/Scripts/do-depends.sh PYTHON_VERSION will be set. Assume to python3.4. If a dependency specifies: USES= python:2 A warning is shown and a version is looked for that matches the port's desire. In this case _PYTHON_VERSION (with leading underscore) will be set to 2.7. However, since "Command line variables" are imported into the Makefile's variables it will be set already, so the downgraded version is not respected. How to Repeat: 0. Add DEFAULT_VERSIONS+=python=3.4 to /etc/make.conf 1. Pre-build devel/py-setuptools and ensure package lives in correct place for USE_PACKAGE_DEPENDS to work. 2. Uninstall setuptools 3. make -C ${PORTSDIR}/editors/libreoffice -DUSE_PACKAGE_DEPENDS clean depends Expected result: Full build of libreoffice dependencies Actual result: ===> serf-1.3.8_1 depends on file: /usr/local/bin/scons - not found ===> install_depends "/usr/ports/devel/scons" "full-package" "" /!\ WARNING /!\ needs Python 2.7 at most. But a port depending on this one specified 3.4 ===> NOTICE: The scons port currently does not have a maintainer. As a result, it is more likely to have unresolved issues, not be up-to-date, or even be removed in the future. To volunteer to maintain this port, please create an issue at: https://bugs.freebsd.org/bugzilla More information about port maintainership is available at: https://www.freebsd.org/doc/en/articles/contributing/ports-contributing.html#maintain-port ===> License MIT accepted by the user ===> scons-2.5.0 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by scons-2.5.0 for building ===> Extracting for scons-2.5.0 => SHA256 Checksum OK for scons-2.5.0.tar.gz. ===> Patching for scons-2.5.0 ===> Applying FreeBSD patches for scons-2.5.0 ===> scons-2.5.0 depends on package: py27-setuptools27>0 - not found ===> install_depends "/usr/ports/devel/py-setuptools27" "full-package" "PYTHON_VERSION=python3.4" ===> Installing existing package /var/packages/devstation/All/py34-setuptools34-23.1.0.txz [pkg.earth.jail] Installing py34-setuptools34-23.1.0... [pkg.earth.jail] Extracting py34-setuptools34-23.1.0: 100% Message from py34-setuptools34-23.1.0: ******************************************************************* Only /usr/local/bin/easy_install-3.4 script has been installed since Python 3.4 is not the default Python version. ******************************************************************* ===> scons-2.5.0 depends on package: py27-setuptools27>0 - not found *** Error code 1 Note: The lines with ===> install_depends is shown by an inserted debug output in Mk/Scripts/do-depends.sh: diff --git a/Mk/Scripts/do-depends.sh b/Mk/Scripts/do-depends.sh index 399a999..74534d8 100644 --- a/Mk/Scripts/do-depends.sh +++ b/Mk/Scripts/do-depends.sh @@ -170,6 +170,7 @@ for _line in ${dp_RAWDEPENDS} ; do fi # Now actually install the dependencies + echo "===> install_depends \"${origin}\" \"${target}\" \"${depends_args}\"" install_depends "${origin}" "${target}" "${depends_args}" # Recheck if the installed dependency validates the pattern except for /nonex [ "${fct}" = "false" ] || ${fct} "${pattern}" This shows: a) The warning message about downgrading the package version b) The unchanged PYTHON_VERSION being propagated. The patch fixes the issue. Whether keeping this PYTHON_VERSION around later screws up _PYTHON_FEATURE_CONCURRENT, is currently beyond my scope. P.S. While researching Mk/Scripts/do-depends.sh it also showed two cases where depends_args is overwritten instead of appended to before install_depends is called. I'm currently assuming that's intentional.
So, to be clear, the case is: Grandparent or older ancestor: USES=python Parent: USES=python:2 *_DEPENDS+= ${PYTHON_PKGNAMEPREFIX}-foo>0:cat/portname Kid: USES=python Kid will use the wrong version. Parent will complain about missing dependency. Libreoffice isn't the only case, just the one I stumbled upon it.
The patch looks right. I'm updating my local checkout and reviewing recent changes and then will take it.
It also seems OK already... ~/svn/ports/devel/py-bison # make -V DEPENDS_ARGS PYTHON_VERSION=python2.7 ~/svn/ports/devel/py3-babel # make -V DEPENDS_ARGS PYTHON_VERSION=python3.5 Note: PYTHON_VERSION - The chosen Python interpreter including the version, e.g. python2.7, python3.3, etc. This allows the user to override the currently chosen default version and to install the port for a specific Python version. It must not be set by a port. PYTHON_VERSION is supposed to already include 'python', so the change is not correct.
"A warning is shown and a version is looked for that matches the port's desire. In this case _PYTHON_VERSION (with leading underscore) will be set to 2.7. However, since "Command line variables" are imported into the Makefile's variables it will be set already, so the downgraded version is not respected." This sounds like the actual issue. I'm trying to understand the exact case in the example to test further.
Example to recreate: /etc/make.conf: DEFAULT_VERSIONS+=python=3.4 ``` ~/svn/ports/www/serf # make -DUSE_PACKAGE_DEPENDS depends DEPENDS_ARGS=PYTHON_VERSION=python3.4 ===> serf-1.3.9_1 depends on file: /usr/local/sbin/pkg - found ===> serf-1.3.9_1 depends on shared library: libapr-1.so - found (/usr/local/lib/libapr-1.so) ===> serf-1.3.9_1 depends on file: /usr/local/bin/scons - not found /root/svn/ports/devel/scons -- install -- PYTHON_VERSION=python3.4 ===> scons-2.5.0 depends on package: py27-setuptools27>0 - not found /root/svn/ports/devel/py-setuptools27 -- install -- PYTHON_VERSION=python3.4 /!\ WARNING /!\ You have security/openssl installed but do not have DEFAULT_VERSIONS+=ssl=openssl set in your make.conf ===> License PSFL accepted by the user ===> py34-setuptools34-28.1.0 depends on file: /usr/local/sbin/pkg - found => setuptools-28.1.0.tar.gz doesn't seem to exist in /mnt/distfiles/python. => Attempting to fetch https://pypi.python.org/packages/source/s/setuptools/setuptools-28.1.0.tar.gz ``` I don't think this is easily fixable. Here's the problem: editors/libreoffice sets DEPENDS_ARGS=PYTHON_VERSION=python3.4 It builds www/serf with make *argument* (to override, intended) PYTHON_VERSION=python3.4. This PYTHON_VERSION make override propagates all the way down the chain, overriding the USES=python:2 in devel/scons. It tries to set PYTHON_VERSION=python2.7, but the make argument forces it back to python3.4. DEPENDS_ARGS is an interesting, and very old, hack that only really works on direct dependencies. Once it gets into a 2nd-order dependency then it can cause issues like in this bug.
A commit references this bug: Author: bdrewery Date: Mon Oct 31 20:54:50 UTC 2016 New revision: 425026 URL: https://svnweb.freebsd.org/changeset/ports/425026 Log: Don't truncate DEPENDS_ARGS here. This code seems unused though since bsd.port.mk has its own handling for clean-depends. PR: 212273 With hat: portmgr Changes: head/Mk/Scripts/do-depends.sh
Any chance that python@ commits the real fix?
Reject by lack of communication from python@.