Created attachment 227773 [details] Stub out multiarch detection in lang/python3X ports for clang 13 During an exp-run for llvm 13 (see bug 258209), it turned out that lang/python3([6-9]|10) do not build with clang 13.0.0 [1][2][3][4][5]: LD_LIBRARY_PATH=/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14 CC='cc' LDSHARED='cc -shared -lpthread -L/usr/local/lib -fstack-protector-strong ' OPT='-DNDEBUG' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py -q build Failed to import the site module Traceback (most recent call last): File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/site.py", line 553, in <module> main() File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/site.py", line 539, in main known_paths = addusersitepackages(known_paths) File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/site.py", line 282, in addusersitepackages user_site = getusersitepackages() File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/site.py", line 258, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/site.py", line 248, in getuserbase USER_BASE = get_config_var('userbase') File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/sysconfig.py", line 601, in get_config_var return get_config_vars().get(name) File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/sysconfig.py", line 550, in get_config_vars _init_posix(_CONFIG_VARS) File "/wrkdirs/usr/ports/lang/python36/work/Python-3.6.14/Lib/sysconfig.py", line 421, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata_m_freebsd14_x86_64-unknown-freebsd14' *** Error code 1 The reason this breaks with clang >= 13, is that it now started supporting the -print-multiarch option, but in its output it adds a major.minor version number, for example x86_64-unknown-freebsd14.0. The dot confuses Python, as it appends the multiarch name to the sysconfig module name, and then can't load the resulting _sysconfigdata__freebsd14_x86_64-unknown-freebsd14.0.py file: ModuleNotFoundError: No module named '_sysconfigdata__freebsd14_x86_64-unknown-freebsd14' Since we do not support multiarch, and the configure script has no way to disable the multiarch check, I would like to propose to just stub it out during post-patch, as per the attached diff. [1] http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/python36-3.6.14.log [2] http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/python37-3.7.11.log [3] http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/python38-3.8.11.log [4] http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/python39-3.9.7.log [5] http://gohan04.nyi.freebsd.org/data/mainamd64PR258209-default/2021-09-05_20h27m09s/logs/errors/python310-3.10.0.r1.log
Comment on attachment 227773 [details] Stub out multiarch detection in lang/python3X ports for clang 13 >commit f7c6323844a874743c0d8a0400a41fbc7af197db >Author: Dimitry Andric <dim@FreeBSD.org> >Date: Wed Sep 8 23:59:36 2021 +0200 > > lang/python3([6-9]|10): disable detection of multiarch > > It breaks with clang >= 13, which adds a major.minor version number in > -print-multiarch output, and the dot confuses Python: > > ModuleNotFoundError: No module named '_sysconfigdata__freebsd14_x86_64-unknown-freebsd14' > > Since we do not support multiarch, and the configure script has no way > to disable the multiarch check, stub it out during post-patch. > >diff --git a/lang/python310/Makefile b/lang/python310/Makefile >index 3dbf8d5d004e..64aba656d340 100644 >--- a/lang/python310/Makefile >+++ b/lang/python310/Makefile >@@ -124,6 +124,10 @@ post-patch: > # which introduces hidden dependency and breaks build > @${REINPLACE_CMD} -e 's|uuid/uuid.h|ignore_&|' ${WRKSRC}/configure > @${REINPLACE_CMD} -e '/find_library_file/ s|uuid|ignore_&|' ${WRKSRC}/setup.py >+# disable detection of multiarch as it breaks with clang >= 13, which adds a >+# major.minor version number in -print-multiarch output, confusing Python >+ @${REINPLACE_CMD} -e 's|^MULTIARCH=.*|MULTIARCH=|' ${WRKSRC}/configure \ >+ ${WRKSRC}/configure.ac > # Apply DISABLED_EXTENSIONS > @${ECHO_CMD} '*disabled*' > ${WRKSRC}/Modules/Setup.local > . for _module in ${DISABLED_EXTENSIONS} >diff --git a/lang/python36/Makefile b/lang/python36/Makefile >index 636f8d198350..b52d72bc3e06 100644 >--- a/lang/python36/Makefile >+++ b/lang/python36/Makefile >@@ -121,6 +121,12 @@ DISABLED_EXTENSIONS+= nis > PLIST_SUB+= NO_NIS="" > .endif > >+post-patch: >+# disable detection of multiarch as it breaks with clang >= 13, which adds a >+# major.minor version number in -print-multiarch output, confusing Python >+ @${REINPLACE_CMD} -e 's|^MULTIARCH=.*|MULTIARCH=|' ${WRKSRC}/configure \ >+ ${WRKSRC}/configure.ac >+ > post-install: > .if ! ${PORT_OPTIONS:MDEBUG} > ${RM} ${STAGEDIR}${PREFIX}/lib/libpython3.so # Upstream Issue: https://bugs.python.org/issue17975 >diff --git a/lang/python37/Makefile b/lang/python37/Makefile >index 8909af0dd7ab..4db00e66e9d2 100644 >--- a/lang/python37/Makefile >+++ b/lang/python37/Makefile >@@ -111,6 +111,10 @@ post-patch: > # which introduces hidden dependency and breaks build > @${REINPLACE_CMD} -e 's|uuid/uuid.h|ignore_&|' ${WRKSRC}/configure > @${REINPLACE_CMD} -e '/find_library_file/ s|uuid|ignore_&|' ${WRKSRC}/setup.py >+# disable detection of multiarch as it breaks with clang >= 13, which adds a >+# major.minor version number in -print-multiarch output, confusing Python >+ @${REINPLACE_CMD} -e 's|^MULTIARCH=.*|MULTIARCH=|' ${WRKSRC}/configure \ >+ ${WRKSRC}/configure.ac > # Apply DISABLED_EXTENSIONS > @${ECHO_CMD} '*disabled*' > ${WRKSRC}/Modules/Setup.local > . for _module in ${DISABLED_EXTENSIONS} >diff --git a/lang/python38/Makefile b/lang/python38/Makefile >index 68359a8579a8..f3dcec0757fa 100644 >--- a/lang/python38/Makefile >+++ b/lang/python38/Makefile >@@ -115,6 +115,10 @@ post-patch: > # which introduces hidden dependency and breaks build > @${REINPLACE_CMD} -e 's|uuid/uuid.h|ignore_&|' ${WRKSRC}/configure > @${REINPLACE_CMD} -e '/find_library_file/ s|uuid|ignore_&|' ${WRKSRC}/setup.py >+# disable detection of multiarch as it breaks with clang >= 13, which adds a >+# major.minor version number in -print-multiarch output, confusing Python >+ @${REINPLACE_CMD} -e 's|^MULTIARCH=.*|MULTIARCH=|' ${WRKSRC}/configure \ >+ ${WRKSRC}/configure.ac > # Apply DISABLED_EXTENSIONS > @${ECHO_CMD} '*disabled*' > ${WRKSRC}/Modules/Setup.local > . for _module in ${DISABLED_EXTENSIONS} >diff --git a/lang/python39/Makefile b/lang/python39/Makefile >index 4a0d31ba17f2..2d052cf3741d 100644 >--- a/lang/python39/Makefile >+++ b/lang/python39/Makefile >@@ -116,6 +116,10 @@ post-patch: > # which introduces hidden dependency and breaks build > @${REINPLACE_CMD} -e 's|uuid/uuid.h|ignore_&|' ${WRKSRC}/configure > @${REINPLACE_CMD} -e '/find_library_file/ s|uuid|ignore_&|' ${WRKSRC}/setup.py >+# disable detection of multiarch as it breaks with clang >= 13, which adds a >+# major.minor version number in -print-multiarch output, confusing Python >+ @${REINPLACE_CMD} -e 's|^MULTIARCH=.*|MULTIARCH=|' ${WRKSRC}/configure \ >+ ${WRKSRC}/configure.ac > # Apply DISABLED_EXTENSIONS > @${ECHO_CMD} '*disabled*' > ${WRKSRC}/Modules/Setup.local > . for _module in ${DISABLED_EXTENSIONS}
Thank you for the report and patch Dim. Would an upstream --disable-multiarch or similar configure option or anything else be useful? Also, can you confirm whether python27 is affected as well?
(In reply to Kubilay Kocak from comment #2) > Would an upstream --disable-multiarch or similar configure option or anything else be useful? Probably; as jrtc27@ remarked yesterday already: <@jrtc27> and surprised there's no configure option for it... having it always inferred seems like a bad idea <@jrtc27> would expect a normal --with-multiarch=(auto|triple) and --without-multiarch set of options > Also, can you confirm whether python27 is affected as well? It has exactly the same configure.ac/configure parts, but doesn't actually seem to *do* anything with the detected MULTIARCH variable, so it works out of the box with clang 13. But maybe we can also patch it similarly, for the sake of consistency. (And not bother submitting it upstream, as that no longer exists ;)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=6bef09666460fb9f9052550854fa69f5d5e4a0e8 commit 6bef09666460fb9f9052550854fa69f5d5e4a0e8 Author: Dimitry Andric <dim@FreeBSD.org> AuthorDate: 2021-09-27 17:39:05 +0000 Commit: Dimitry Andric <dim@FreeBSD.org> CommitDate: 2021-09-27 17:56:59 +0000 lang/python3([6-9]|10): disable detection of multiarch It breaks with clang >= 13, which adds a major.minor version number in -print-multiarch output, and the dot confuses Python: ModuleNotFoundError: No module named '_sysconfigdata__freebsd14_x86_64-unknown-freebsd14' Since we do not support multiarch, and the configure script has no way to disable the multiarch check, stub it out during post-patch. PR: 258377 Approved by: maintainer timeout (2 weeks) MFH: 2021Q3 lang/python310/Makefile | 4 ++++ lang/python36/Makefile | 6 ++++++ lang/python37/Makefile | 4 ++++ lang/python38/Makefile | 4 ++++ lang/python39/Makefile | 4 ++++ 5 files changed, 22 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=4af982e6bea019a33ae85c6c92a3042fadccb739 commit 4af982e6bea019a33ae85c6c92a3042fadccb739 Author: Dimitry Andric <dim@FreeBSD.org> AuthorDate: 2021-11-26 16:02:16 +0000 Commit: Dimitry Andric <dim@FreeBSD.org> CommitDate: 2021-11-26 18:48:57 +0000 lang/python3(6|7|8|9|10|11): work around unwanted multiarch detection again In ports 6bef09666460 we patched out python's erroneous multiarch detection using sed, but upstream changed the surrounding parts in the configure script to defeat the regex we were using. Update the regex to cope with the change. PR: 259896, 258377 MFH: 2021Q4 Approved by: maintainer timeout (9 days) lang/python310/Makefile | 3 +-- lang/python311/Makefile | 3 +-- lang/python36/Makefile | 3 +-- lang/python37/Makefile | 3 +-- lang/python38/Makefile | 3 +-- lang/python39/Makefile | 3 +-- 6 files changed, 6 insertions(+), 12 deletions(-)
Please file bugs upstream against CPython any time you find yourself patching something to support your platform. Every patch is technical debt. https://bugs.python.org/issue46263 didn't need to happen so late.
(In reply to gpshead from comment #6) We are usually very good with upstreaming. Thank you for the heads up