Bug 252277
| Summary: | cmake find_package(python3...) fails to get default python version | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Shane <FreeBSD> | ||||||||||
| Component: | Individual Port(s) | Assignee: | freebsd-kde (group) <kde> | ||||||||||
| Status: | Closed FIXED | ||||||||||||
| Severity: | Affects Some People | CC: | adridg, lbartoletti, linimon, tingox+freebsd | ||||||||||
| Priority: | --- | ||||||||||||
| Version: | Latest | ||||||||||||
| Hardware: | Any | ||||||||||||
| OS: | Any | ||||||||||||
| Attachments: |
|
||||||||||||
|
Description
Shane
2020-12-30 00:01:12 UTC
So, is this a bug against cmake, or a bug against the ports that use cmake? It is not clear to me from the description. Against cmake, any port that calls find_package(python3 ...) will get the wrong version. At least when more than one python version is installed. The old method using find_package(PythonInterp ...) still works as expected. The newer method find_package(Python3 ...) does not find the default version, it always finds the newest python version. This differs from the previous methods behaviour. This new method was added to cmake v3.12 and has not yet been adopted by every port using cmake and python. This is still a problem with cmake 3.19.6:
root@kg-core2# pkg info cmake* python3*
cmake-3.19.6
python37-3.7.10
python38-3.8.8
trying to build and install libSavitar:
---> Installing '/usr/ports/devel/libsavitar'
===> Deinstalling for libSavitar
===> libSavitar not installed, skipping
===> Installing for libSavitar-4.5.0_5
===> Checking if libSavitar is already installed
===> Registering installation for libSavitar-4.5.0_5
pkg-static: Unable to access file /usr/ports/devel/libsavitar/work/stage/usr/local/lib/python3.7/site-packages/Savitar.so:No such file or directory
*** Error code 1
Stop.
make[1]: stopped in /usr/ports/devel/libsavitar
*** Error code 1
Stop.
make: stopped in /usr/ports/devel/libsavitar
** Command failed (exit code 1): env MAKE_JOBS_NUMBER_LIMIT=12 make reinstall
---> Restoring the old version
---> Installing '/var/tmp/pkg_replace.dDeaiZ/libSavitar-4.5.0_3/libSavitar-4.5.0_3.txz'
Installing libSavitar-4.5.0_3...
Extracting libSavitar-4.5.0_3: 100%
---> Cleaning the preserved shared libraries
** Fix the problem and try again.
---> ** [5/5] - 0 done, 4 ignored, 0 skipped, 1 failed
---> Listing the results (!:failed)
! libSavitar-4.5.0_3 (install error)
and if I check the relevant directory in work:
root@kg-core2# l /usr/ports/devel/libsavitar/work/stage/usr/local/lib
./ cmake/ libSavitar.so.0@
../ debug/ libSavitar.so.0.1.1*
X11/ libSavitar.so@ python3.8/
root@kg-core2# l /usr/ports/devel/libsavitar/work/stage/usr/local/lib/python3.8/
./ ../ site-packages/
root@kg-core2# l /usr/ports/devel/libsavitar/work/stage/usr/local/lib/python3.8/site-packages/
./ ../ Savitar.so*
This on
root@kg-core2# freebsd-version -ku
11.4-RELEASE-p5
11.4-RELEASE-p5
ports tree updated yesterday (March 21st, 2021).
Assigning to kde@ as the maintainers of CMake (effectively, to me). It'd be really useful to have a couple of sample CMakeLists.txt attached to this PR that demonstrate "this works, this doesn't" so I don't have to come up with complete examples while also debugging the actual CMake module code. Created attachment 224394 [details]
simple cmake file for testing
The simplest setup is a CMakeLists.txt that only has the two python searches. Multiple python versions need to be installed.
run `cmake .` in the same dir
The old style find package finds py36
-- Found PythonInterp: /usr/local/bin/python3.6 (found suitable version "3.6.12", minimum required is "3.6")
-- Found PythonLibs: /usr/local/lib/libpython3.6m.so (found suitable version "3.6.12", minimum required is "3.6")
The new variation finds py39
-- Found Python3: /usr/local/bin/python3.9 (found suitable version "3.9.0", minimum required is "3.6") found components: Interpreter Development Development.Module Development.Embed
They should find the same default py version.
Created attachment 224396 [details]
test port makefile
Actually it would be better from a port, to use the ports env
Make a new port dir (misc/cmakepytest) and put the CmakeLists.txt in files as files/CMakeLists.txt.in
With this I see three versions --
-- Found PythonInterp: /usr/local/bin/python3.6 (found suitable version "3.6.12", minimum required is "3.6")
-- Found PythonLibs: /usr/local/lib/libpython3.7m.so (found suitable version "3.7.9", minimum required is "3.6")
-- Found Python3: /usr/local/bin/python3.9 (found suitable version "3.9.0", minimum required is "3.6") found components: Interpreter Development Development.Module Development.Embed
Created attachment 224735 [details]
Makefile using EXACT
Created attachment 224736 [details]
CMakeLists using EXACT
Indeed, this method will search for the highest version available on the system. Normally, we can use the Python3_ROOT_DIR hints, but unless I misuse it, it doesn't work properly. cmake -DPython3_ROOT_DIR=/usr/local/lib/python3.8 should select py3.8, but cmake continue to take the highest version available (eg. 3.9 on my system). Another way to do this, which works well, is to use EXACT. example: find_package(Python3 3.6 EXACT REQUIRED COMPONENTS Interpreter Development) One possible workaround for the ports would be to patch as I did in Makefile_EXACT and CMakeLists_EXACT.txt Where is this DEFAULT_VERSIONS set? How are you testing it? The (old-style) PythonInterp runs `python3` and/or `python` to find whatever version that points to, and pretends that that is the preferred version. There's no specific reference to FreeBSD's default versions. If you have a program named `python3` in your $PATH, then that will be the version of Python3 that is found-by-default. As an illustration: ``` [adridg@beastie /tmp/pr-252277]$ ln -s `which python3.7` ~/bin/python3 [adridg@beastie /tmp/pr-252277]$ which python3 /home/adridg/bin/python3 ``` Here, the example CMakeLists.txt will find Python 3.7. Re-jig the link to point to Python 3.9 and then that is what is found. This works for **me** because ~/bin is in my $PATH. In many installations, the *python3* metaport installs a link to some default Python version, and that gets picked up. If there is **no** `python3` in $PATH, then you'll find the highest-versioned Python3 that is installed. [[ it **is** quite true, though, that finding PythonInterp and finding Python3 behave very differently, because the latter doesn't run `python3` for versioning. ]] To match the FindInterp behavior, to some extent, set Python3_FIND_UNVERSIONED_NAMES to the value "FIRST". I'm going to patch that into the next CMake update. A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=731a0cba8469ce33d4406f735afac00f6914e158 commit 731a0cba8469ce33d4406f735afac00f6914e158 Author: Adriaan de Groot <adridg@FreeBSD.org> AuthorDate: 2022-04-10 15:34:55 +0000 Commit: Adriaan de Groot <adridg@FreeBSD.org> CommitDate: 2022-04-10 15:34:55 +0000 devel/cmake: prefer python3 over versioned python3.x Depending on which mechanism is used to find Python (Python3 in particular, since there's only one Python2 version left), either python3 is found -- installed by the python3 metaport, and presumed to be the "preferred / default Python3 version" -- of a versioned python3. Asking for, say, Python 3.6 will get you whatever python3 points at -- assuming it is newer than 3.6 -- or, with the other mechanism, the newest python (e.g. highest installed version). Those are not necessarily the same. Switch the new mechanism to preferring python3; there's a knob in CMake itself for that, we just need to set it in the FreeBSD config-file. PR: 252277 devel/cmake/Makefile | 1 + devel/cmake/files/patch-Modules_Platform_FreeBSD.cmake (new) | 12 ++++++++++++ 2 files changed, 13 insertions(+) After upgrading CMake to this new PORTREVISION, behaves as expected (except it doesn't look in PATH overall, so my ~/bin/python3 link doesn't work; chasing that is more work than this is worth). |
