During a poudriere build the system fails when building devel/py38-pyside2-5.15.2 The build log is attached.
Created attachment 224813 [details] gzipped poudriere log file
I'm experiencing the same problem with the same 'Illegal instruction' error during build using synth on 11-STABLE.
This happens when I try to build the port manually. The specific instructions leading to the problem are: /usr/local/bin/cmake -E env LD_LIBRARY_PATH=/usr/ports/devel/pyside2/work-py38/.build/libpyside:/usr/local/lib /usr/local/bin/python3.8 -vvd /usr/ports/devel/pyside2/work-py38/pyside-setup-opensource-src-5.15.2/sources/pyside2/PySide2/Qt3DRender/../support/generate_pyi.py Qt3DRender --sys-path /usr/ports/devel/pyside2/work-py38/.build The -vv option in the line above produces the following output before failing: import 'PySide2.support.signature.lib.tool' # <_frozen_importlib_external.SourceFileLoader object at 0x8009b8af0> # trying /usr/ports/devel/pyside2/work-py38/.build/PySide2/Qt3DRender.cpython-38.so Illegal instruction
The problem is that files/patch-src_core_configure.json in graphics/qt5-3d turns on the avx2 instructions on cpus that do not have these instructions. Removing the patch file allows pyside2 to be built on these machines, such as my penryn.
I can confirm that the the patch proposed by crahman works for me also. My machine holds two INTEL X5650 XEON processors that do not support this avx2 functionality. So the root of this problem is not pyside2, but graphics/qt5-3d. In case it is helpful, here is an exerpt of the system startup log, that shows the processor capabilities: May 25 08:25:12 arwen kernel: FreeBSD 12.2-RELEASE-p6 GENERIC amd64 May 25 08:25:12 arwen kernel: FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2) May 25 08:25:12 arwen kernel: VT(vga): resolution 640x480 May 25 08:25:12 arwen kernel: CPU: Intel(R) Xeon(R) CPU X5650 @ 2.67GHz (2660.06-MHz K8-class CPU) May 25 08:25:12 arwen kernel: Origin="GenuineIntel" Id=0x206c2 Family=0x6 Model=0x2c Stepping=2 May 25 08:25:12 arwen kernel: Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,F XSR,SSE,SSE2,SS,HTT,TM,PBE> May 25 08:25:12 arwen kernel: Features2=0x29ee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,P OPCNT,AESNI> May 25 08:25:12 arwen kernel: AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> May 25 08:25:12 arwen kernel: AMD Features2=0x1<LAHF> May 25 08:25:12 arwen kernel: VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID May 25 08:25:12 arwen kernel: TSC: P-state invariant, performance statistics May 25 08:25:12 arwen kernel: real memory = 94491377664 (90114 MB)
Refer to bug#256152, "graphics/qt5-3d: fails to work on processors with CPU < haswell (anything without AVX2)".
Could you please share a copy of the content of /usr/local/lib/qt5/mkspecs/qmodule.pri from qt5-qmake (devel/qt5-qmake)? If it mentions avx2, that might be the source of the problem.
Hello Steve, here it is, indeed it seems to mention avx2: QT_CPU_FEATURES.x86_64 = mmx sse sse2 QT.global_private.enabled_features = sse2 alloca avx2 dlopen network posix_fallocate reduce_ex ports reduce_relocations relocatable sql system-zlib testlib xml QT.global_private.disabled_features = alloca_h alloca_malloc_h android-style-assets private_te sts dbus dbus-linked gc_binaries gui intelcet libudev release_tools stack-protector-strong wid gets zstd PKG_CONFIG_EXECUTABLE = pkgconf QMAKE_LIBS_LIBDL = QT_COORD_TYPE = double QMAKE_LIBS_ZLIB = -lz CONFIG -= precompile_header CONFIG += sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx5 12er avx512ifma avx512pf avx512vbmi avx512vl compile_examples f16c largefile rdrnd rdseed shan i x86SimdAlways QT_BUILD_PARTS += libs tools QT_HOST_CFLAGS_DBUS += ~
(In reply to Frans van der Veer from comment #8) Ok, thanks for confirming. Was your qmake built on your host that does not support AVX2 or on another host that does support AVX2? If it was built on a host that does not support AVX2 then I think it shouldn't mention avx2 and that might be the source of the issue. In the mean time, you may be able to simply edit it not to mention avx2 and other things that CPU does not support and avoid the issue.
(In reply to Steve Wills from comment #9) In this case is was built on the same host, but within a poudriere jail. That is what you will regard as being the same host also I presume. Come to think of that, the contents of the file is from the host filing system, not from the jail filing system.
(In reply to Frans van der Veer from comment #10) Yes, that's all the same host really. Can you please share a complete copy of the poudriere log for devel/qt5-qmake?
Created attachment 225275 [details] Poudriere log file for qt5-qmake-5.15.2 Steve, here it is.
(In reply to Frans van der Veer from comment #12) Thanks. Pretty strange to me that log contains: Checking for AVX2 instructions... yes I wonder why?
(In reply to Steve Wills from comment #13) Hi Steve, I also wonder why it reports UNAME_v=FreeBSD 12.2-RELEASE-p3 UNAME_r=12.2-RELEASE-p3 while I did a jail updates upto current p6. Just checked that once again: # poudriere jail -u -j amd64 [00:00:00] Upgrading using ftp /etc/resolv.conf -> /usr/local/poudriere/jails/amd64/etc/resolv.conf Looking up update.FreeBSD.org mirrors... 2 mirrors found. Fetching metadata signature for 12.2-RELEASE from update1.freebsd.org... done. Fetching metadata index... done. Inspecting system... done. Preparing to download files... done. No updates needed to update system to 12.2-RELEASE-p6. 12.2-RELEASE-p6 [00:01:45] Recording filesystem state for clean...cannot create snapshot 'zroot/poudriere/jails/amd64@clean': dataset already exists done Even the latest builds of today still report p3 in the log. But OK, I presume that is quite another issue. What I also find strange about this situation is that a poudriere build should to my opinion produce CPU architecture independent packages so you can distribute then anywhere. So making these decisions at package compile time seems odd to me. Regards, Frans
(In reply to Frans van der Veer from comment #14) > Even the latest builds of today still report p3 in the log. But OK, I presume that is quite another issue. I think this may be due to the updates, where some cause kernel changes and others don't, and is expected. > What I also find strange about this situation is that a poudriere build should to my opinion produce CPU architecture independent packages so you can distribute then anywhere. I don't really understand what you mean here. Compiled binaries always have a target CPU architecture they run on. Even so-called "fat-binaries" or multi-architecture binaries have a list of architectures they support and don't run "anywhere". Regardless, as far as this issue goes, it's not clear to me what qmake means here when it says "Checking for AVX2 instructions". I almost think it's just checking if the compiler supports them, not if the host CPU supports them. And then there's the question of if they should be used on the target CPU, which could actually be completely different. I will have to have someone who knows qmake better chime in on these questions I'm afraid.
(In reply to Frans van der Veer from comment #14) If possible, could you share a copy of the config.log from the WRKSRC directory after running make build on devel/qt5-qmake on your host that doesn't support avx2?
Created attachment 225284 [details] qt5-qmake build config.log file Hi Steve, I don't know how to retrieve this from the poudriere jail, since I seem only to be able to make poudriere save a wrkdir on build failure. So this is taken outside poudriere build from the (updated) hosts ports tree itself as it was filed under: /usr/ports/devel/qt5-qmake/work/qtbase-everywhere-src-5.15.2 I hope this helps.
(In reply to Frans van der Veer from comment #17) That's fine, thanks. It's running: > clang++ -c -pipe -mavx2 -O2 -w -fPIC -DNO_ATTRIBUTE -DQT_COMPILER_SUPPORTS_AVX2 -I. -I/usr/local/include -I/usr/ports/devel/qt5-qmake/work/qtbase-everywhere-src-5.15.2/mkspecs/freebsd-clang -o main.o main.cpp without issue. Maybe qmodule.pri is meant to indicate what the compiler supports, not what the host or target support? It's quite confusing.
Created attachment 225295 [details] patch for qmake (In reply to Frans van der Veer from comment #17) Ok, I did some more digging and I think the found the real source of the issue. Can you test rebuilding your qmake with this and then rebuilding graphics/qt5-3d and then also devel/pyside2 and share the contents of the /usr/local/lib/qt5/mkspecs/qmodule.pri from the new qmake?
(In reply to Steve Wills from comment #19) Oh, I should have mentioned, also remove graphics/qt5-3d/files/patch-src_core_configure.json before rebuilding graphics/qt5-3d.
I've done the tests you requested - I added the patch file in comment#19 to qt5-qmake/files and recompiled it, and then built qt5-3d and py38-pyside2. This is all on a machine without avx2 instructions. qmodule.pri was unchanged by the qt5-qmake patch: QT_CPU_FEATURES.x86_64 = mmx sse sse2 QT.global_private.enabled_features = sse2 alloca avx2 dlopen network posix_fallocate reduce_exports reduce_relocations relocatable sql system-zlib testlib xml QT.global_private.disabled_features = alloca_h alloca_malloc_h android-style-assets private_tests dbus dbus-linked gc_binaries gui intelcet libudev release_tools stack-protector-strong widgets zstd PKG_CONFIG_EXECUTABLE = pkgconf QMAKE_LIBS_LIBDL = QT_COORD_TYPE = double QMAKE_LIBS_ZLIB = -lz CONFIG -= precompile_header CONFIG += sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx512er avx512ifma avx512pf avx512vbmi avx512vl compile_examples f16c largefile rdrnd rdseed shani x86SimdAlways QT_BUILD_PARTS += libs tools QT_HOST_CFLAGS_DBUS += The patch from comment#19 did not change the avx2 issue in qt5-3d. With or without the patch, if qt5-3d/files/patch-src_core_configure.json is present, qt5-3d is compiled with -mavx2, and matrix4x4_avx2.cpp is compiled, and the configure summary shows: Use SSE2 instructions .................. yes Use AVX2 instructions .................. yes and pyside2 fails to build. With or without the patch, if qt5-3d/files/patch-src_core_configure.json is not present, qt5-ed is compiled without -mavx2, matrix4x4.avx2.cpp is compiled, and the configure summary shows: Use SSE2 instructions .................. yes Use AVX2 instructions .................. no and pyside2 builds successfully. The matrix4x4_avx2_cpp contents are conditionalized with an #ifdef QT_COMPILER_SUPPORTS_AVX2, if that's of any use.
(In reply to crahman from comment #21) > qmodule.pri was unchanged by the qt5-qmake patch: Good! It should be. > With or without the patch, if qt5-3d/files/patch-src_core_configure.json is not present, qt5-ed is compiled without -mavx2, matrix4x4.avx2.cpp is compiled, and the configure summary shows: > > Use SSE2 instructions .................. yes > Use AVX2 instructions .................. no > > and pyside2 builds successfully. Good! The patch to qt5-3d was actually not quite right, but it allowed it to build qt5-3d with avx2 enabled. Here we've removed the patch to qt5-3d and instead added this patch to qmake which fixes the qt5-3d issue in the avx2 case and also doesn't break things in the non-avx2 case. I've tested the avx2 case and everything is fine, but I couldn't test the without avx2 case fully. You just tested the without avx2 case and confirmed that it all works. So, I believe removing the patch to qt5-3d and adding this patch to qmake is the correct solution. It allows building everything properly both without avx2 and with avx2. Just to be on the safe side, I'd like some input from KDE Team, but this seems right to me.
Created attachment 227351 [details] patch-qmake_revert_248155_qt5-3d Steve, this may be due to me still learning git but I had issues with your patch applying unless I included --directory=devel and applying from the devel directory. This is probably user error on my end. But to make things easier for folks like me: git apply with this patch from just being in /usr/ports without additional switches to git. I created this patch including reverting 248155 by removing the offending file. I applied these patches from a ports tree pulled about an hour prior, with everything in the tree that can build allowed to updated including qgis in the blocking notice. All this does is include your patch to qmake and delete the qt5-3d patch file for AVX. qt5-qmake and qt5-3d rebuilt, qt5-3d no longer shows using AVX in configure stage. py38-pyside2 builds fine following patch qgis runs without recompilation (though it was freshly updated, and failed prior to the patch with the newer version) without illegal instructions, and so is fixed PR 257402. It the qmake fix helped folks with AVX on qt5-3d then this should work for them too, but I can only test the non-AVX case. patch produced by git diff --staged, and confirmed to apply to new test branch in my local tree. Maybe this will be useful to folks, but this gets me a little closer to trying to migrate to Releng 13.0 if there is something wrong with the format of my patch let me know, I want to learn how to do this properly.
It appears that this problem resurfaced again. My "weekly" portsupdate and poudriere build tells the build of py38-pyside2-5.15.2_2 fails. This is the last part of the build log file: Qt3DRender --sys-path /wrkdirs/usr/ports/devel/pyside2/work-py38/.build /usr/local/lib/python3.8/site-packages/shiboken2 FAILED: PySide2/Qt3DRender/CMakeFiles/Qt3DRender_pyi /wrkdirs/usr/ports/devel/pyside2/work-py38/.build/PySide2/Qt3DRender/CMakeFiles/Qt3DRender_pyi cd /wrkdirs/usr/ports/devel/pyside2/work-py38/.build/PySide2/Qt3DRender && /usr/local/bin/cmake -E env LD_LIBRARY_PATH=/wrkdirs/usr/ports/devel/pyside2/work-py38/.build/libpyside:/usr/local/lib /usr/local/bin/python3.8 /wrkdirs/usr/ports/devel/pyside2/work-py38/pyside-setup-opensource-src-5.15.2/sources/pyside2/PySide2/Qt3DRender/../support/generate_pyi.py Qt3DRender --sys-path /wrkdirs/usr/ports/devel/pyside2/work-py38/.build /usr/local/lib/python3.8/site-packages/shiboken2 Illegal instruction ninja: build stopped: subcommand failed. *** Error code 1 Stop. make: stopped in /usr/ports/devel/pyside2 =>> Cleaning up wrkdir ===> Cleaning for py38-pyside2-5.15.2_2 build of devel/pyside2 | py38-pyside2-5.15.2_2 ended at Thu Oct 28 07:11:33 CEST 2021 build time: 00:11:16 !!! build failure encountered !!!
(In reply to Frans van der Veer from comment #24) not surprising, its never been patched in ports. This is one of a few I watch like a hawk. But if you apply my patch, or Steve's and revert the qt3d patch he references instead things will work. I just re-confirmed py38-pyside2 still builds without the illegal instruction error linking to the forced AVX instructions that end up in qt5-3d without patching qmake and reverting the change in qt5-3d that forced it on even on <Haswell machines without AVX.
Hi, any updates here ?
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=183d079fd681df544b67cf2f03c7d1d5aa7853fc commit 183d079fd681df544b67cf2f03c7d1d5aa7853fc Author: Dima Panov <fluffy@FreeBSD.org> AuthorDate: 2021-12-19 14:04:58 +0000 Commit: Dima Panov <fluffy@FreeBSD.org> CommitDate: 2021-12-19 14:04:58 +0000 graphics/qt5-q3d: convert AVX2 detection hack to OPTIONS (+) Force qt5-qmake to pass CXXFLAGS to cpu features test. Define AVX OPTION only amd64/i386 arches due to cpu used. Off by default to keep out-of-box behavior and allow to build qt5-q3d and its consumes on any old processors. Necromages, go forward! :) PR: 255753 With hat: desktop Sponsored by: Netzkommune GmbH devel/qt5-qmake/Makefile | 2 +- devel/qt5-qmake/files/patch-config.tests_arch_arch.pro (new) | 6 ++++++ graphics/qt5-3d/Makefile | 8 +++++++- ...src_core_configure.json => extrapatch-src_core_configure.json} | 0 4 files changed, 14 insertions(+), 2 deletions(-)
Presumed fixed by comment #27