Bug 245835

Summary: math/kig python-kig.xml fails install when chosen python flavor is 3.7 and DEBUG is set
Product: Ports & Packages Reporter: Jaeden Amero <freebsd>
Component: Individual Port(s)Assignee: freebsd-kde (Team) <kde>
Status: Closed FIXED    
Severity: Affects Only Me CC: adridg, tcberner
Priority: --- Flags: adridg: maintainer-feedback?
Version: Latest   
Hardware: Any   
OS: Any   
Description Flags
Output from svn info
CMakeError.log showing build test failure
Shell script to reproduce the issue with poudriere none

Description Jaeden Amero 2020-04-22 21:03:52 UTC
When building with python 3.7 as the chosen flavor of python, pkg-static fails to find python-kig.xml during poudriere build.

Steps to reproduce:
1. Set desired python flavor to 3.7 via make.conf
    DEFAULT_VERSIONS=python=3.7 python2=2.7 python3=3.7
2. Build math/kig (make)
3. Attempt to install (make install)
4. Observe error message about missing python-kig.xml, which is listed in math/kig/pkg-plist
    # make install
===>  Installing for kig-19.12.3_1
===>  Checking if kig is already installed
===>   Registering installation for kig-19.12.3_1
pkg-static: Unable to access file /usr/obj/usr/ports/math/kig/work/stage/usr/local/share/katepart5/syntax/python-kig.xml:No such file or directory
*** Error code 74

make[1]: stopped in /usr/ports/math/kig
*** Error code 1

make: stopped in /usr/ports/math/kig
Comment 1 Jaeden Amero 2020-04-22 21:08:41 UTC
Created attachment 213704 [details]
Output from svn info

Output from svn info attached.

Reproduced with r527897.
Comment 2 Jaeden Amero 2020-04-24 07:40:13 UTC
Output from make includes the following:

-- Looking for C++ include boost/shared_ptr.hpp
-- Looking for C++ include boost/shared_ptr.hpp - found
-- Performing Test BoostPython_FromCMake
-- Performing Test BoostPython_FromCMake - Failed
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "1.6.3")
-- Could NOT find BoostPython (missing: BoostPython_LIBRARIES BoostPython_INCLUDE_DIRS)

Without BoostPython, kig will not attempt to run the CMake install rule (in scripting/CMakeLists.txt) which copies python-kig.xml to the location where the pkg-plist expects.
Comment 3 Tobias C. Berner freebsd_committer 2020-04-25 18:28:39 UTC
(In reply to freebsd from comment #2)
Moin moin 

What python version got picked by cmake?

mfg Tobias
Comment 4 Tobias C. Berner freebsd_committer 2020-04-25 18:30:19 UTC
I have no issue building it against py37.
Comment 5 Jaeden Amero 2020-04-26 07:58:36 UTC
Created attachment 213796 [details]
CMakeError.log showing build test failure
Comment 6 Jaeden Amero 2020-04-26 08:18:58 UTC
The FindBoostPython.cmake script appears to be having trouble detecting Boost Python.

Does this line:
    find_package(Boost QUIET COMPONENTS python)
need to be changed to this?
    find_package(Boost QUIET COMPONENTS python3.7)

If I do that, I see that "Boost_PYTHON3.7_LIBRARY" is set to "/usr/local/lib/libboost_python37.so" and "Boost_PYTHON3.7_FOUND" is set to "ON". Without that change, the variables "Boost_PYTHON_LIBRARY" and "Boost_PYTHON_FOUND" are not set (i.e. blank when printed with manually-added CMake debug prints) after the "find_package(Boost QUIET COMPONENTS python)" line.

However, even changing find_package() to look for the Boost python3.7 component doesn't seem sufficient. It looks like Boost Python, for Python 3, has changed APIs. Specifically, the build test that CMake is doing to see if the detected BoostPython works is failing with:
ld: error: undefined symbol: boost::python::detail::init_module(char const*, void (*)())

This is because that function has changed to the following, for Python 3,:
PyObject* boost:python::detail::init_module(PyModuleDef&, void(*)());

You can see that based on the version of Python being targeted, Boost will pick either the old init_module API or the new one. (https://www.boost.org/doc/libs/1_72_0/boost/python/module_init.hpp)

    #  if PY_VERSION_HEX >= 0x03000000

    BOOST_PYTHON_DECL PyObject* init_module(PyModuleDef&, void(*)());


    BOOST_PYTHON_DECL PyObject* init_module(char const* name, void(*)());


There are two ways kig's CMake detects BoostPython. One is via using CMake and then verifying with a build, the other is via pkg-config. Do you know if your successful build used the CMake-detected BoostPython or if it went the pkg-config detection route? (Although the ports Makefile modifies the CMake detection route so I suspect CMake detection is the intended-to-work option.)

The output from "cmake --trace .." running in a "cmake_build2" folder inside the kig working directory (/usr/obj/usr/ports/math/kig/work/kig-20.04.0 for this test) is available at https://patater.com/cmake-trace.log (it's 3.2 MB, over the attachment size limit).

I've attached the CMake error log which shows the build failure CMake encounters when testing if it can use BoostPython.

Another strange thing I've noticed:
The test with CMake to see if BoostPython is usable has the following linker option for libraries:
-Wl,-rpath,/usr/local/lib /usr/local/lib/libboost_python37.so /usr/local/lib/libpython2.7.so

Why would it try to use libpython2.7.so? That doesn't seem right to me.

I'm not sure where to draw the line between fixing this problem with ports (e.g. ${REINPLACE_CMD}) vs fixing it upstream. I'm happy to raise this upstream if the problem is legitimate.

Thanks for the help.
Comment 7 Jaeden Amero 2020-05-14 20:12:24 UTC
Created attachment 214501 [details]
Shell script to reproduce the issue with poudriere

I've attached a shell script to reproduce the bug. The key was setting the option "DEBUG". I haven't worked out yet if it's kig or one of its dependencies that fails when "DEBUG" is set.
Comment 8 Adriaan de Groot freebsd_committer 2021-05-31 22:28:26 UTC
I can't reproduce this -- not with the settings from the script, either. Now, we're two CMake minor versions later, two kig major versions later, and several boost versions as well .. so it's not really comparable any more, I don't think.

I'd like to put this in "NEEDSINFO" state (it *has* been a year since this was opened; sorry for the very. slow. response.)
Comment 9 Jaeden Amero 2021-06-27 11:58:52 UTC
Thanks for having a look, Adriaan.

I had another go at building this too. I couldn't reproduce it either. Happy to close.
Comment 10 Adriaan de Groot freebsd_committer 2021-06-27 12:06:59 UTC
Calling it fixed (although "overcome by events" or "magically went away" might apply, too).