Bug 263341
| Summary: | math/Imath builds but does not install | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | D'Arcy J.M. Cain <darcy> | ||||||||||
| Component: | Individual Port(s) | Assignee: | Matthias Andree <mandree> | ||||||||||
| Status: | Closed FIXED | ||||||||||||
| Severity: | Affects Only Me | CC: | joel, seneca | ||||||||||
| Priority: | --- | Flags: | mandree:
maintainer-feedback+
mandree: merge-quarterly+ |
||||||||||
| Version: | Latest | ||||||||||||
| Hardware: | Any | ||||||||||||
| OS: | Any | ||||||||||||
| Bug Depends on: | 262109 | ||||||||||||
| Bug Blocks: | |||||||||||||
| Attachments: |
|
||||||||||||
|
Description
D'Arcy J.M. Cain
2022-04-16 14:03:56 UTC
D'Arcy, thanks for the report. However, I cannot reproduce it. I need more details on your build, environment, settings, /etc/make.conf, options, logs, how exactly are you building, ... are you sure there are no previous errors reported that get missed in a restarted build, or when a "make install" is run in spite of a failing "make"? Also be sure that your Python installation and the configured default Python is consistent. You cannot build Imath on a system that does not have pyXXX-numpy for your default Python. Try to update the entire ports tree, rebuild and reinstall all ports that Imath depends on, and then unset NOCLEANDEPENDS 2>/dev/null || : sed -i '' -e '/NOCLEANDEPENDS/d' /etc/make.conf make clean make DEVELOPER=yes test make DEVELOPER=yes check-plist make DEVELOPER=yes package If that reproduces the crash, please provide all information and reopen the bug. Alternatively, you can deselect the PYTHON option, then clean and build. NOCLEANDEPENDS is not in my environment or in my make.conf. Checked and py39-numpy is installed. make DEVELOPER=yes test passed with zero errors. make DEVELOPER=yes check-plist failed: ====> Checking for pkg-plist issues (check-plist) ===> Parsing plist ===> Checking for items in STAGEDIR missing from pkg-plist Error: Orphaned: local/%%PYTHON_SITELIBDIR%%/imath.so Error: Orphaned: local/%%PYTHON_SITELIBDIR%%/imathnumpy.so ===> Checking for items in pkg-plist which are not in STAGEDIR Error: Missing: %%PYTHON_SITELIBDIR%%/imath.so Error: Missing: %%PYTHON_SITELIBDIR%%/imathnumpy.so ===> Error: Plist issues found. *** Error code 1 Now we're getting closer. The port uses whatever would be the port system's default Python, which currently is 3.8, but your system uses 3.9. No FLAVORS currently. Are you setting Python 3.9 as your system's default (through make.conf or otherwise through DEFAULT_VERSIONS)? D'Arcy, please help me help you. I have changed my Python default to 3.9 in make.conf (DEFAULT_VERSIONS+=python3=3.9 python=3.9) then rebuilt/reinstalled python and python3 then successfully built math/Imath with PYTHON option enabled. Can you please show me your /etc/make.conf and any environment options you set to influence the build? Can you also check your Imath configure and build stages for warnings? If you cannot make heads or tails of them, "make clean" and "make |& tee build.log", then xz build.log and attach the build.log.xz. I suspect something is hosed about parallel Python versions, or the default configuration is not set properly, and if cmake messes up the Python detection on your system, it does not on mine, so I cannot find the bug or a workaround. TIA. I should have mentioned that I would also like to see these files after your failed "make": work/.build/CMakeFiles/CMakeOutput.log work/.build/CMakeFiles/CMakeError.log (or is it CMakeErrors.log?) - this may be missing work/.build/CMakeCache.txt It may take a few days before I can debug more. Created attachment 233265 [details]
The make.conf from the problem system.
Created attachment 233267 [details]
As requested.
Created attachment 233268 [details]
Also as requested
Could not find any Errors log.
I have run out of ideas for now, I see no differences in the files you posted and mine if I use your make.conf with minimal adaptations to my file paths and with permitting a MAKE_JOBS_NUMBER. can you please do: cd /usr/ports/math/Imath make clean make pretty-print-config # <- keep the output make DEVELOPER=yes check-plist package &| tee make.log then please attache the make.log (feel free to xz or gzip it) and paste the output from make pretty-print-config here (it's fine if you put it in the attachment's comment). Apologies for a typo, make that one line (pipe THEN ampersand): make clean make DEVELOPER=yes check-plist package |& tee make.log Created attachment 233271 [details]
Log from test run.
Thanks for the make.log. There is something fishy about the Python sitelib path in your output. The "Will install to:" is a relative path starting with local/lib/... (for me it is lib/...) and then you get the double /usr/local/local/... on installation of the two Python .so modules -- for some reason I cannot currently explain. I would suggest you go find out where that comes from. Is /usr/local or /usr/local/local a symlink or null mount on your computer? Here are relevant snippets from the log: -- Found Python: /usr/local/bin/python3.9 (found suitable exact version "3.9.12") found components: Interpreter Development Development.Module Development.Embed -- Found Python3: /usr/bin/python3 (found suitable exact version "3.9.12") found components: Interpreter Development Development.Module Development.Embed -- Found Python 3.9.12 -- Found Python3 libraries: 39 -- Will install to: local/lib/python3.9/site-packages ... -- Installing: /usr/obj/usr/ports/math/Imath/work/stage/usr/local/lib/libPyImath_Python3_9-3_1.so.29 -- Set runtime path of "/usr/obj/usr/ports/math/Imath/work/stage/usr/local/lib/libPyImath_Python3_9-3_1.so.29.4.0" to "/usr/local/lib" -- Installing: /usr/obj/usr/ports/math/Imath/work/stage/usr/local/lib/libPyImath_Python3_9-3_1.so -- Installing: /usr/obj/usr/ports/math/Imath/work/stage/usr/local/local/lib/python3.9/site-packages/imath.so -- Set runtime path of "/usr/obj/usr/ports/math/Imath/work/stage/usr/local/local/lib/python3.9/site-packages/imath.so" to "/usr/local/lib" -- Installing: /usr/obj/usr/ports/math/Imath/work/stage/usr/local/local/lib/python3.9/site-packages/imathnumpy.so -- Set runtime path of "/usr/obj/usr/ports/math/Imath/work/stage/usr/local/local/lib/python3.9/site-packages/imathnumpy.so" to "/usr/local/lib" /usr/bin/strip /usr/obj/usr/ports/math/Imath/work/stage/usr/local/lib/libImath-3_1.so I just noticed something odd. Most Python scripts are in pkg-plist as "%%PYTHON_SITELIBDIR%%/xxx.py" but this package has these two as "%%PYTHON%%%%PYTHON_SITELIBDIR%%/xxx.py". Tried this change but it still fails. diff --git a/math/Imath/pkg-plist b/math/Imath/pkg-plist index 04ff9e1bd567..baf6b6847da8 100644 --- a/math/Imath/pkg-plist +++ b/math/Imath/pkg-plist @@ -87,6 +87,6 @@ libdata/pkgconfig/Imath.pc %%PYTHON%%lib/libPyImath_Python%%PYVER%%-%%MAJORVER%%.so %%PYTHON%%lib/libPyImath_Python%%PYVER%%-%%MAJORVER%%.so.%%VER%% %%PYTHON%%lib/libPyImath_Python%%PYVER%%-%%MAJORVER%%.so.%%VER%%.%%MINVER%%.%%PLVER%% -%%PYTHON%%%%PYTHON_SITELIBDIR%%/imath.so -%%PYTHON%%%%PYTHON_SITELIBDIR%%/imathnumpy.so +%%PYTHON_SITELIBDIR%%/imath.so +%%PYTHON_SITELIBDIR%%/imathnumpy.so %%PYTHON%%libdata/pkgconfig/PyImath.pc But still, are these lines as weird as they look to me? Hi D'Arcy, what you have observed is in working shape for me, and the explanation is: - the %%PYTHON%% goes back to the OPTIONS_DEFINE=PYTHON/OPTIONS_SUB=yes stuff, and expands in the pkg-plist to either empty (if the PYTHON option is enabled) or @comment (if the PYTHON option is disabled), - and only the %%PYTHON_SITELIBDIR%% is what goes between the prefix and the remainder of the pkg-plist line. Anything about my /usr/local/local... finding in your logs? (In reply to Matthias Andree from comment #16) I'm not D'Arcy, but I have access to that one particular computer with the failing builds. There's some weirdness in there, specifically the presence of Python symlinks. That computer normally has /usr/bin/python and /usr/bin/python3 as symlinks to /usr/local/bin/python3.9. Remove the symlinks and Imath builds whilst the python scripts that have their roots in antiquity fail. It looks like I should be able to explicitly provide the full path to the real python interpreter, but I'm not all that familiar with the details of how best to prod cmake into doing what I want. I had found quite a few extra "local"s in the generated cmake_install.cmake files for the Python modules (e.g. PyImath_Python3_SITEARCH_REL was stated to be 'local/lib/python3.9/site-packages' in CMakeCache.txt). These path names came to be because the "python" found was one of the /usr/bin symlinks and that base directory of "/usr/bin" when combined with the returned site-packages of "/usr/local/lib/python3.9/site-packages" resulted in cmake computing an incorrect site-packages path. reopening fur further investigation based on new information Seneca,
thanks for the concrete hint to reproducing the issue, namely:
ln -sfn /usr/local/bin/python3 /usr/bin/python3
ln -sfn /usr/local/bin/python3 /usr/bin/python
allowed me to reproduce the issue.
I will now fix this by setting Python_EXECUTABLE to ${PYTHON_CMD}, which fixes the issue for me.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=d443494a71b73dc543cc0eddec69a01d6280ce02 commit d443494a71b73dc543cc0eddec69a01d6280ce02 Author: Matthias Andree <mandree@FreeBSD.org> AuthorDate: 2022-07-12 19:06:08 +0000 Commit: Matthias Andree <mandree@FreeBSD.org> CommitDate: 2022-07-12 19:20:13 +0000 math/Imath: ensure PYTHON build with /usr/bin/python[3] links When /usr/bin/python[3] symlinks are present (which is nonstandard), the FindPython logic inside cmake will grab it over $PREFIX/bin/python[3], but still identify (by default) /usr/local/lib/... as the sitelib path, but goof up when calculating the relative path in case of /usr/bin vs /usr/local/bin mismatch. Fix: Provide ${PYTHON_CMD} through the cmake Python_EXECUTABLE variable to avoid any ambiguities and pin the right executable. I believe it should be safe to NOT bump PORTREVISION, but I will fetch far and err on the safe side and bump it, because that is easier than researching all possible paths how and where Python paths might find their way into the resulting package. The detailed report by Seneca helped identify and solve this issue, so: Thanks to: Seneca Cunningham <seneca@vybenetworks.com> PR: 263341 Reported by: D'Arcy J.M. Cain <darcy@druid.net> Reported by: Seneca Cunningham <seneca@vybenetworks.com> MFH: 2022Q3 math/Imath/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) A commit in branch 2022Q3 references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=50aed9b03a2ccb85c7a5f45866d914e97f2ef42d commit 50aed9b03a2ccb85c7a5f45866d914e97f2ef42d Author: Matthias Andree <mandree@FreeBSD.org> AuthorDate: 2022-07-12 19:06:08 +0000 Commit: Matthias Andree <mandree@FreeBSD.org> CommitDate: 2022-07-12 19:20:52 +0000 math/Imath: ensure PYTHON build with /usr/bin/python[3] links When /usr/bin/python[3] symlinks are present (which is nonstandard), the FindPython logic inside cmake will grab it over $PREFIX/bin/python[3], but still identify (by default) /usr/local/lib/... as the sitelib path, but goof up when calculating the relative path in case of /usr/bin vs /usr/local/bin mismatch. Fix: Provide ${PYTHON_CMD} through the cmake Python_EXECUTABLE variable to avoid any ambiguities and pin the right executable. I believe it should be safe to NOT bump PORTREVISION, but I will fetch far and err on the safe side and bump it, because that is easier than researching all possible paths how and where Python paths might find their way into the resulting package. The detailed report by Seneca helped identify and solve this issue, so: Thanks to: Seneca Cunningham <seneca@vybenetworks.com> PR: 263341 Reported by: D'Arcy J.M. Cain <darcy@druid.net> Reported by: Seneca Cunningham <seneca@vybenetworks.com> MFH: 2022Q3 (cherry picked from commit d443494a71b73dc543cc0eddec69a01d6280ce02) math/Imath/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) Fixed as of 3.1.5_1. Yeah, I had the same problem after I rebuilt python without debug support so that a couple of other ports would build. I rebuilt py-numpy and the Imath installed. fyi |
