After bug 227427 lands this port will end up BROKEN. Can you help? libboost_python.so is now libboost_python27.so libboost_python3.so is now libboost_python36.so $ fetch -qo- 'https://reviews.freebsd.org/D15030?download=true' | patch -Efsp0 -d /usr/ports $ make all deinstall install clean -C/usr/ports/devel/boost-jam $ make all deinstall install clean -C/usr/ports/devel/boost-libs $ make all deinstall install clean -C/usr/ports/devel/boost-python-libs $ make -C/usr/ports/net/ceph [...] CMake Error at /usr/local/share/cmake/Modules/FindBoost.cmake:1960 (message): Unable to find the requested Boost libraries. Boost version: 1.67.0 Boost include path: /usr/local/include Could not find the following Boost libraries: boost_python [...] build log: https://ptpb.pw/8qfO regressed by: https://github.com/boostorg/python/commit/d4d41d94aecc
More ports are affected this time. $ make -C/usr/ports/math/kig [...] -- Could NOT find BoostPython (missing: BoostPython_LIBRARIES BoostPython_INCLUDE_DIRS) [...] ===> Checking for items in pkg-plist which are not in STAGEDIR Error: Missing: share/katepart5/syntax/python-kig.xml [...] https://ptpb.pw/RMyh
(In reply to Jan Beich from comment #1) Hi Jan, I think that the fallout stems from Cmake not understandig that the library is not called libboost_python any longer, so perhaps the fix should go into the Cmake module that tries to find and there account for the python-versioning.
(In reply to Willem Jan Withagen from comment #2) The orignal code in Ceph is like: list(APPEND BOOST_COMPONENTS python) ...... find_package(Boost 1.66 COMPONENTS ${BOOST_COMPONENTS} REQUIRED) And then the CMAKE modules do their stuff. Asking for a different python version there fixes the issue when there is that version library. So it is fixable from Ceph, but I strongly think it needs to be fixed in the actual CMake code that tries to find the libboost_python{PY_VER}.
There is a CMake 3.11 update in exp-run, and I hear from tcberner@ that there is a hefty rewrite in FindBoost (again) that specifically tackles Python. So I'll check if that rewrite is in 3.11.0 release. If it is, this PR may go away automagically.
CMake 3.11.0 doesn't help. net/ceph: https://ptpb.pw/dFRs math/kig: https://ptpb.pw/UUwO
(In reply to Jan Beich from comment #5) Is there a ENV during CMake-ing that will tell the wanted Python version? That will allow me to request either 2.7 or 3.x during CMake Which will atleast work when libboost_python27.so is requested.
Created attachment 192545 [details] workaround (In reply to Willem Jan Withagen from comment #6) > Is there a ENV during CMake-ing that will tell the wanted Python version? Like this? ;)
(In reply to Jan Beich from comment #7) So to make sure: `CMAKE_ARGS+= -DBOOST_PYTHON_SUFFIX:STRING=${PYTHON_SUFFIX}` gets automagically added to the Cmake args if USES=python is set. In which case I'll fix the Cmake build to differentiate on this.
If -DBOOST_PYTHON_SUFFIX=27 is passed "find_package(Boost COMPONENTS python)" will look not only for libboost_python.so but also libboost_python27.so. Ideally, CMake should try Python_ADDITIONAL_VERSIONS for Boost then fall back to common suffixes. Requiring ugly Boost.Python ifdefs in consumers is non-starter unless a specifc Boost version is explicitly requested i.e., "find_package(Boost 1.67 ...)". And consumers cannot jump on the latest Boost version without alienating downstream[1]. [1] https://repology.org/metapackage/boost/versions (In reply to Willem Jan Withagen from comment #8) > In which case I'll fix the Cmake build to differentiate on this. No need. With the patch net/ceph makes past "configure" stage until it hits C++ bustage (a la ports r467364).
(In reply to Jan Beich from comment #9) Oke, I understand the solution, but the Ceph Cmake code is: list(APPEND BOOST_COMPONENTS python) ...... find_package(Boost 1.66 COMPONENTS ${BOOST_COMPONENTS} REQUIRED) Which is there because of all kinds of reasons, mainly having to do with different distro and versions of distros holding boost libs with a too low version. So from you remark I conclude that this code WILL require handholding. Even after things are patched? I do expect to see some python version in Ceph in the near future, because they are running into 2.7 <> 3.x issues as well. The other compilation errors are a "problem" as I have very little time at the moment. Trying to get $WORK projects finished before I have a little bit of R&R.
I see jbeich@ committing lots of workarounds for Boost::Python 1.67 .. how much of that would just go away if we fix the CMake module? Also, how much goes away by itself with CMake 3.11.0? There's a specific commit in there FindBoost: Add support for Boost 1.67 with Python version suffixes Which, if it doesn't work, needs fixing on our (my, CMake) end.
(In reply to Adriaan de Groot from comment #11) If the fix in the Cmake stuff (either FreeBSD specific or not) allows for this call find_package(Boost 1.66 COMPONENTS ${BOOST_COMPONENTS} REQUIRED) to do "the right thing" (tm), then No changes to the ceph-source are needed. As things go now I'll have to add a selector on FreeBSD to append the python-version. No biggie, but again specific code for a specific problem. That said, to has started with a discussion on ceph-dev to determine what to do with 2.7 and how fast to fase it out, and make all 3.x code. That could/would leave FreeBSD users that are mainly on 2.7 in the cold. Which should not be a major problem from opperational point, since you do not want to do much other on the ceph storage nodes. It could however conflict on compute nodes where scripting depends on 2.7 and 2.7 compatible classes. So I'm driven here by what Ceph does in the code, and by FreeBSD packages in what is possible and desirable. And that in the negative time I have available ATM.
(In reply to Adriaan de Groot from comment #11) > how much of that would just go away if we fix the CMake module? None. My ${PY_BOOST_LIB} fixes are for non-CMake ports or CMake ports with their own way to find Boost.Python. > how much goes away by itself with CMake 3.11.0? None. CMake added "pythonXY" (versioned) support as Boost component. If a project doesn't switch from "python" to "pythonXY" CMake won't find libboost_pythonXY installed by Boost 1.67. If "pythonXY" is used then Boost < 1.67 users are out of luck. Obviously, downstream (e.g., FreeBSD Ports) maybe unhappy with whatever Python version was requested, often preferring a more recent one.
(In reply to Jan Beich from comment #13) > If "pythonXY" is used then Boost < 1.67 users are out of luck. Err, mainly FreeBSD Ports users. According to FindBoost.cmake some distributions (Debian/Ubuntu, Gentoo, RPMs) implemented their own versioning *before* Boost 1.67.
A commit references this bug: Author: jbeich Date: Wed Apr 18 13:58:16 UTC 2018 New revision: 467712 URL: https://svnweb.freebsd.org/changeset/ports/467712 Log: devel/cmake: allow consumers to qualify boost_python version PR: 227428 Exp-run by: antoine Changes: head/Mk/Uses/python.mk head/devel/cmake/Makefile head/devel/cmake/files/patch-Modules_FindBoost.cmake
Boost 1.67 has landed. Try commenting out -DBOOST_PYTHON_SUFFIX in Mk/Uses/python.mk then build devel/libarea in order to better understand CMake 3.11.0 shortcomings.
(In reply to Jan Beich from comment #16) extracts from the Cpeh-python discussion: Mimic - after a "best effort" to migrate, some Python code still runs on Python 2, but Shaman has a Python 3 build flavor and teuthology is capable of testing in Python 3-only environment. Nautilus - all remaining packaged Python code has been migrated and explicitly runs (and is tested) on Python 3. Python 2 versions of our code do not need to be provided or packaged, because all target platforms support Python 3. There might still be some Python 2 scripts in the git repo, but these are either special cases or for developer use only, and are not packaged. (These are the 2 upcoming releases) And also: > I'm not objecting to making the leap (it's better than trying to > support 2 and 3 at the same time!), but let's make sure we understand > consequences for users. Meaning that Ceph never intended to have both a 2.7 and 3.x python version. And it is currently 2.7 for most of the consumer facing items. Some of the test parts are actually already 3.x compatible, and or running on 3.x
(In reply to Willem Jan Withagen from comment #17) Yesterday I had some time to work a bit on this, and with boost 1.67 and CMake 3.11.0_2 I was able to build and test correctly. This was using a default environment based on python 2.7, even though Ceph starts to move to python3. (I have both installed) So a challenge might be to have a poudriere env that will be able to have both python 2.7 AND python3? As I have the feeling that this is not really what the ports team is aiming for?
(In reply to Willem Jan Withagen from comment #17) Hi Jan, There is an update to the luminous tree: 12.2.5. So I ran the current net/ceph port through poudriere after fixing the version and distinfo. That makes it compile without much problems. - virtenv is indeed not needed. - boost_pythons complains with an error: --- Error: /usr/local/bin/ceph-mgr is linked to /usr/local/lib/libboost_python.so.1.66.0 from devel/boost-python-libs but it is not declared as a dependency Warning: you need LIB_DEPENDS+=libboost_python.so:devel/boost-python-libs --- Which also shows the "main problem". it is still linked against Boost 1.66. That even though I portsnapped to update the complete ports tree for the poudriere env. Is this expected?
(In reply to Willem Jan Withagen from comment #19) > Error: /usr/local/bin/ceph-mgr is linked to /usr/local/lib/libboost_python.so.1.66.0 from devel/boost-python-libs but it is not declared as a dependency > Warning: you need LIB_DEPENDS+=libboost_python.so:devel/boost-python-libs Did you use an old ports/ tree? https://lists.freebsd.org/pipermail/svn-ports-head/2018-March/170962.html https://svnweb.freebsd.org/changeset/ports/463782 > Which also shows the "main problem". it is still linked against Boost 1.66. HEAD has Boost 1.67.0 2018Q2 has Boost 1.66.0 2017Q4 and 2018Q1 has Boost 1.65.1 2017Q3 has Boost 1.64.0 ... If "pkg -vv | fgrep url" returns /quarterly at the end then you're on 2018Q2.
(In reply to Jan Beich from comment #20) I update the ports tree with poudriere ports -p local12 -u So where would I find the tree/database to run pkg against? Or does that always run against the pkg.conf in /usr/local/etc?
(In reply to Willem Jan Withagen from comment #21) Must have been a pilot error, since I reloaded ports into the jail. And now poudriere just builds fine. And I'll submit an upgrade. Note that I can compile ceph/HEAD without your patch, so I'll test your boost-patch to see what the compiler complains about, and use that to submit a PR to Ceph if needed.
A commit references this bug: Author: adridg Date: Fri Apr 5 11:46:46 UTC 2019 New revision: 497948 URL: https://svnweb.freebsd.org/changeset/ports/497948 Log: Update CMake to latest release, 3.14.1 Changelog: https://cmake.org/cmake/help/v3.14/release/3.14.html Local patches: fixes for Boost, Python, and OpenMP Affected ports: - opencpn, hugin needed help in finding wx - kadu is over-enthusiastic in finding non-existent X11 components - xlife++ does weird things in parsing help output from cmake Thanks antoine@ for multiple exp-runs. Thanks tcberner@ for much prep-work. PR: 236534 223678 227428 Changes: head/astro/opencpn/files/patch-CMakeLists.txt head/devel/cmake/Makefile head/devel/cmake/distinfo head/devel/cmake/files/patch-Modules_FindBoost.cmake head/devel/cmake/files/patch-Modules_FindLua.cmake head/devel/cmake/files/patch-Modules_FindOpenMP.cmake head/devel/cmake/files/patch-git_ff06b35 head/devel/cmake/pkg-plist head/devel/cmake-doc/Makefile head/devel/cmake-doc/pkg-plist head/devel/cmake-gui/Makefile head/graphics/hugin/files/patch-sf_f6f56eb9 head/math/xlife++/files/patch-cmake_xlifepp__user__management.cmake head/polish/kadu/files/patch-kadu-core_CMakeLists.txt head/polish/kadu/files/patch-plugins_idle_CMakeLists.txt head/polish/kadu/files/patch-plugins_pcspeaker_CMakeLists.txt head/polish/kadu/files/patch-plugins_screenshot_CMakeLists.txt