As transpired in the discussion of Bug #209204, Python bits have their own way for system-admins to specify the desired Python version. Though the existing method is perfectly fine in itself, it is wrong for it to ignore the DEFAULT_VERSIONS method used by all (?) other "ecosystems". As I type this, Bugzilla also helpfully lists other potentially-related problems: Bug #207809 and Bug #207937. It would seem, DEFAULT_VERSIONS _may sometimes_ play a role in determining python-version, but, as Bug #209204 shows, not _always_ -- and _never_ officially. This needs fixing -- DEFAULT_VERSIONS ought to become the "source of truth" for Python-version as it already is (and is documented to be) for other things.
Hi, I will propose that reordering priorities of those variables -- ${LOCALBASE}/bin/python and DEFAULT_VERSION. The state quo: if ${LOCALBASE}/bin/python exists: priority: PYTHON_VERSION > ${LOCALBASE}/bin/python > DEFAULT_VERSIONS else: priority: PYTHON_VERSION > DEFAULT_VERSIONS I think make `DEFAULT_VERSIONS` override `${LOCALBASE}/bin/python` will be better: priority: PYTHON_VERSION > DEFAULT_VERSIONS > ${LOCALBASE}/bin/python The PYTHON_VERSION still has the highest priority. This variable is handy for installing same ports with different python version at the same time. For example, I need the `database/py-sqlite3` installed for both py27 and py35. I just simply issue make with `PYTHON_VERSION`: make install clean PYTHON_VERSION=2.7 && make install clean PYTHON_VERSION=3.5 So, preserving the flexibility to override default is necessary; PYTHON_VERSION do it well currently. We just need to switch the order of 'local python' and DEFAULT_VERSION.
*** Bug 209204 has been marked as a duplicate of this bug. ***
Created attachment 171562 [details] Depreciate PYTHON_VERSION, dump up PYTHON_DEFAULT priority See bug #209204, comment #8 for details.
Created attachment 171564 [details] propose patch: keep PYTHON_VERSION The PYTHON_VERSION will be handy for install non-conclict python package (e.g. py27-sqlite3 + py34-sqlite3). Please consider to keep it! I make a patch following the rule in comment #1 Here describe the testing flow for this patch: 1. prepare a new jail 2. check the current priority of `${LOCALBASE}/bin/python` $ pkg install python # should create a symlink `/usr/local/bin/python` $ cd /usr/ports/databases/py-sqlite3 $ make -V PYTHON_VERSION DEFAULT_VERSIONS=python=3.5 python2.7 # the symlink win now 3. apply this patch 4. check that DEFAULT_VERSIONS got higher priority $ cd /usr/ports/databases/py-sqlite3 $ make -V PYTHON_VERSION python2.7 $ make -V PYTHON_VERSION DEFAULT_VERSIONS=python=3.5 python3.5 5. one more check for priority of symlink `/usr/local/bin/python` $ pkg delete python $ pkg install python3 # we will get python3.4 installed $ cd /usr/local/bin $ ln -s python3 python $ ls -l python lrwxr-xr-x 1 root wheel 7 Jun 19 14:16 python@ -> python3 $ cd /usr/ports/databases/py-sqlite3 $ make -V PYTHON_VERSION DEFAULT_VERSIONS=python=3.5 python3.5 $ make -V PYTHON_VERSION python3.4 # ya! symlink auto-detect correct!
@Mikhail, can you please provide a few example cases that show: - The port that is affected - The intended/expected result - The actual result - The result with any workarounds claimed to fix the issue (if there are any) Currently, and as far as I know/believe, the existing logic is intended as there are cases where the user-specified DEFAULT_VERSION of python desired, may *not* be appropriate/correct for the port in question, based on the versions of Python that port is declared to support (whether those declared are correct or not). This is further complicated in dependency cases where some ports depend on only one or more Python versions. A shorter way of saying the above is that the user literally cannot know, be expected to know, nor state the exact and only version of Python they want to be used in any particular case. Obvious examples are: - User specifies default version 3.x, port only supports 2.x - User specifies default version 2.x, port only supports 3.x - User specifies default version x.y, port explicitly does not work with x.y (say, for example, older 3.x versions) Having said that, there *are* ports in the tree right now that incorrectly, incompletely or inaccurately declare those versions. These all need to be fixed, with the documentation (at least in Uses/python) improved to make it clearer how to do it, and do it correctly. Examples are: Ports that specify USES=python:2 (or 3). This declaration means *only* that the port depends on the python2 or python3 *symlink* (and thus port). These should instead be USES=python:X.Y[+] or [-]X.Y, with an additional *_DEPENDS on the respective lang/python{2,3} symlink port. Ports that specify X.Y+ arbitrarily, as almost all Python packages support only up to the latest Python X.Y, and often break with future/newer ones until they are tested/verified. Additionally, our USES syntax is unable to specify SOME version specifications that are perfectly normal (and common) for Python packages. There is an open task documented here: https://wiki.freebsd.org/Python#Goals
(In reply to Kubilay Kocak from comment #5) > - The port that is affected Hmm... every python ports will be affected if the symlink `/usr/local/bin/python` (not `/usr/local/pythonX`) exists. This line (https://svnweb.freebsd.org/ports/head/Mk/Uses/python.mk?view=markup&pathrev=414168#l276) make this behavior. So if user install `lang/python` with python27 first, THEN, the DEFAULT_VERSION become useless. e.g $ cd /to/py-sqlite3 # choose any py2/3 compatable ports $ make -V PYTHON_VERSION DEFAULT_VERSIONS=python=3.5 python3.5 # expected result $ pkg install lang/python ... $ make -V PYTHON_VERSION DEFAULT_VERSIONS=python=3.5 python2.7 # OMG So i think the point is not the python package required...
Ha. I got the history: https://svnweb.freebsd.org/ports/head/Mk/Uses/python.mk?r1=196699&r2=196700&pathrev=414168#l282. When this line was written, the python 3k did not born yet... Time to fix it to adapt to py2 & py3? Any policy propose?
(In reply to Iblis Lin from comment #7) We can see the original design make the priority chain as follow: `PYTHON_DEFAULT_VERSION` > symlink And... as time evolves, `PYTHON_DEFAULT_VERSION` was deprecated. Then, i guess someone accidentally make the priority reverse.
(In reply to Iblis Lin from comment #8) Finally, i found this commit introduce this strange behavior described in commen #6 https://svnweb.freebsd.org/ports/head/Mk/Uses/python.mk?r1=328833&r2=329164&pathrev=414168#l220
(In reply to Kubilay Kocak from comment #5) If I understand this PR correctly (in the context of bug #209204) the issue is that given the Ports Framework's DEFAULT_VERSIONS the behavior for Python is NOT consistent with user expectations. Namely, users expect DEFAULT_VERSIONS=python=x to behave the same as PYTHON_VERSION=x, however this is not the case. (As an aside, it does not appear that PYTHON_VERSION is documented outside of Mk/Uses/python.mk.) To elaborate, I believe the expected behavior is: 1) if no ports are installed then DEFAULT_VERSIONS behaves the same as PYTHON_DEFAULT_VERSION (this, it does). 2) if python is already installed then DEFAULT_VERSIONS behaves the same as PYTHON_VERSION (this, it does NOT). Both patches achieve this goal.
While I understand there is some confusion in this matter I don't see that adjusting default versions will do anything more than cause more trouble. Changing the default python version to build one port will only lead to breaking concurrent installs for other python versions. Setting DEFAULT_VERSIONS is used to determine naming and linking behavior. When a python port which uses concurrent installs bin/script then bin/script gets renamed to bin/script-2.7 and if PYTHON_VERSION equals DEFAULT_VERSION bin/script is added as a link to bin/script-2.7 By installing a port using the default of DEFAULT_VERSION=python=2.7 then when you install it again using DEFAULT_VERSIONS=python=3.5 it will conflict with the existing install by also trying to install bin/script There are three ways a python port fails (apart from FreeBSD specific patching), a port doesn't support py3 or a port requires py3 and the default version prevents it's dependencies from building with the matching python version. The third failure is a port runs a script using python as the command, not python${UNIQUE_SUFFIX} in which case the port should be adjusted by using shebangfix or patching. Fixing the first means making changes to the code to support py3. While ports are being duplicated to force a py3 dependent port, they are only making conflicts with existing installs and making temporary changes that will later need to be undone. I expect the true fix for the ports that require py3 is adding support for VARIANTS to the ports infrastructure. So DEFAULT_VERSION should be set once in /etc/make.conf and left that way, the same default for every port built. PYTHON_VERSION can then be used to build a second version of a port for a non-default python version. Using `make PYTHON_VERSION=3.5 install` fails when dependent ports need to be built as they don't inherit the make variables from the initial port. Meaning `make PYTHON_VERSION=3.5 install` will build the current port using py3.5 but if a dependency needs to be built it will use DEFAULT_VERSION. The better way to install a port using a non-default python version is to use setenv PYTHON_VERSION 3.5;make install or for bash export PYTHON_VERSION=3.5;make install By setting an environment variable you are setting a variable that is also visible to every dependent port. This way if py27-dpendent is already installed then py35-dependent will be built as a dependency. As well as the above manual method, to test and install multiple python ports I use poudriere, I setup different builds for each python version. I always leave DEFAULT_VERSION=python=2.7 in each build so that the packages can be installed together. For each build I set PYTHON_VERSION to the desired version.
As a submitter of bug 207809, 207937 (and not mentioned 708202), I think that the bug reports I have sent are not related to this bug report, if this bug report is about the order of PYTHON_VERSION and DEFAULT_VERSIONS. The ports I have sent the patches were unable to build with python 3.x, but not treated so in the Makefiles. The submitted patches are to make them build whether the default python is python 2.x or python 3.x. For example, japanese/mozc-server (dealt with in bug 207937) has python scripts run at build time. They run with python 2.x but not with 3.x. Instead of patching the scripts run with both python 2.x and 3.x, which seemed difficult for me, I changed the port build depend on python2 and made scripts call python2 instead of python, which can be ether python 2.x or 3.x. In this case, depending on python2 seems to be correct. The bug 207809 (devel/pygobject3-common) addresses the problem that it always depend on graphics/py-cairo which is for python 2.x. For python 3.x, there is graphics/py3-cairo. The patch in bug 208202 changed textproc/ibus depend on python 2.x, because one of the ports it depends works only with python 2.x. When setting the default python version to 3.x, there are some ports that fail to build. I tried to fix a few of these. These ports could not be built not because of the priority of PYTHON_VERSION and DEFAULT_VERSIONS, but because they forgot to limit the dependency on python 2.x only, or they forgot to look the case of python 3.x. So, I say again that I think the bug 207809 and 207937 are not related to this bug report.
Hopefully I understand this PR correctly (I have haven't, please comment with an example and I'll reopen). Changing the priority order from: PYTHON_VERSION > ${LOCALBASE}/bin/python > DEFAULT_VERSIONS to PYTHON_VERSION > DEFAULT_VERSIONS > ${LOCALBASE}/bin/python will not work if: a) the user installs the metaport lang/python2 or lang/python3, then b) changes DEFAULT_VERSIONS, and c) the port depends on a python metaport. For example, lang/python3 is installed referencing python3.4 but the user (or the framework) changes DEFAULT_VERSIONS to python3.5. In this situation any port depending on a metaport (i.e. USES=python:2 or USES=python:3) will think it is depending on a version it is not (i.e. python3 is python3.4 but the port will think it is python3.5).
Assign to resolver/closer.