Bug 255753 - devel/pyside2: py38-pyside2-5.15.2 poudriere build fails
Summary: devel/pyside2: py38-pyside2-5.15.2 poudriere build fails
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-kde (Team)
URL:
Keywords:
Depends on:
Blocks: 257402
  Show dependency treegraph
 
Reported: 2021-05-10 16:11 UTC by Frans van der Veer
Modified: 2021-08-21 04:59 UTC (History)
6 users (show)

See Also:
bugzilla: maintainer-feedback? (kde)


Attachments
gzipped poudriere log file (108.75 KB, application/gzip)
2021-05-10 16:14 UTC, Frans van der Veer
no flags Details
Poudriere log file for qt5-qmake-5.15.2 (188.22 KB, text/plain)
2021-05-26 11:31 UTC, Frans van der Veer
no flags Details
qt5-qmake build config.log file (71.23 KB, text/plain)
2021-05-26 16:20 UTC, Frans van der Veer
no flags Details
patch for qmake (454 bytes, patch)
2021-05-26 21:52 UTC, Steve Wills
no flags Details | Diff
patch-qmake_revert_248155_qt5-3d (1.25 KB, patch)
2021-08-21 03:53 UTC, alt2600
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Frans van der Veer 2021-05-10 16:11:51 UTC
During a poudriere build the system fails when building devel/py38-pyside2-5.15.2

The build log is attached.
Comment 1 Frans van der Veer 2021-05-10 16:14:09 UTC
Created attachment 224813 [details]
gzipped poudriere log file
Comment 2 crahman 2021-05-23 09:18:03 UTC
I'm experiencing the same problem with the same 'Illegal instruction' error during build using synth on 11-STABLE.
Comment 3 crahman 2021-05-24 07:24:26 UTC
This happens when I try to build the port manually.  The specific instructions leading to the problem are:

/usr/local/bin/cmake -E env LD_LIBRARY_PATH=/usr/ports/devel/pyside2/work-py38/.build/libpyside:/usr/local/lib /usr/local/bin/python3.8 -vvd /usr/ports/devel/pyside2/work-py38/pyside-setup-opensource-src-5.15.2/sources/pyside2/PySide2/Qt3DRender/../support/generate_pyi.py Qt3DRender --sys-path /usr/ports/devel/pyside2/work-py38/.build

The -vv option in the line above produces the following output before failing:

import 'PySide2.support.signature.lib.tool' # <_frozen_importlib_external.SourceFileLoader object at 0x8009b8af0>
# trying /usr/ports/devel/pyside2/work-py38/.build/PySide2/Qt3DRender.cpython-38.so
Illegal instruction
Comment 4 crahman 2021-05-25 03:28:15 UTC
The problem is that files/patch-src_core_configure.json in graphics/qt5-3d turns on the avx2 instructions on cpus that do not have these instructions.

Removing the patch file allows pyside2 to be built on these machines, such as my penryn.
Comment 5 Frans van der Veer 2021-05-25 14:51:12 UTC
I can confirm that the the patch proposed by crahman works for me also. My machine holds two INTEL X5650 XEON processors that do not support  this avx2 functionality.

So the root of this problem is not pyside2, but graphics/qt5-3d.

In case it is helpful, here is an exerpt of the system startup log, that shows the processor capabilities:

May 25 08:25:12 arwen kernel: FreeBSD 12.2-RELEASE-p6 GENERIC amd64
May 25 08:25:12 arwen kernel: FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
May 25 08:25:12 arwen kernel: VT(vga): resolution 640x480
May 25 08:25:12 arwen kernel: CPU: Intel(R) Xeon(R) CPU           X5650  @ 2.67GHz (2660.06-MHz K8-class CPU)
May 25 08:25:12 arwen kernel:   Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2
May 25 08:25:12 arwen kernel:   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,F
XSR,SSE,SSE2,SS,HTT,TM,PBE>
May 25 08:25:12 arwen kernel:   Features2=0x29ee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,P
OPCNT,AESNI>
May 25 08:25:12 arwen kernel:   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
May 25 08:25:12 arwen kernel:   AMD Features2=0x1<LAHF>
May 25 08:25:12 arwen kernel:   VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
May 25 08:25:12 arwen kernel:   TSC: P-state invariant, performance statistics
May 25 08:25:12 arwen kernel: real memory  = 94491377664 (90114 MB)
Comment 6 crahman 2021-05-25 16:13:28 UTC
Refer to bug#256152, "graphics/qt5-3d: fails to work on processors with CPU < haswell (anything without AVX2)".
Comment 7 Steve Wills freebsd_committer 2021-05-25 21:21:26 UTC
Could you please share a copy of the content of /usr/local/lib/qt5/mkspecs/qmodule.pri from qt5-qmake (devel/qt5-qmake)? If it mentions avx2, that might be the source of the problem.
Comment 8 Frans van der Veer 2021-05-25 21:42:11 UTC
Hello Steve,

here it is, indeed it seems to mention avx2:

QT_CPU_FEATURES.x86_64 = mmx sse sse2
QT.global_private.enabled_features = sse2 alloca avx2 dlopen network posix_fallocate reduce_ex
ports reduce_relocations relocatable sql system-zlib testlib xml
QT.global_private.disabled_features = alloca_h alloca_malloc_h android-style-assets private_te
sts dbus dbus-linked gc_binaries gui intelcet libudev release_tools stack-protector-strong wid
gets zstd
PKG_CONFIG_EXECUTABLE = pkgconf
QMAKE_LIBS_LIBDL =
QT_COORD_TYPE = double
QMAKE_LIBS_ZLIB = -lz
CONFIG -= precompile_header
CONFIG += sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx5
12er avx512ifma avx512pf avx512vbmi avx512vl compile_examples f16c largefile rdrnd rdseed shan
i x86SimdAlways
QT_BUILD_PARTS += libs tools
QT_HOST_CFLAGS_DBUS +=
~
Comment 9 Steve Wills freebsd_committer 2021-05-25 21:45:45 UTC
(In reply to Frans van der Veer from comment #8)
Ok, thanks for confirming. Was your qmake built on your host that does not support AVX2 or on another host that does support AVX2? If it was built on a host that does not support AVX2 then I think it shouldn't mention avx2 and that might be the source of the issue. In the mean time, you may be able to simply edit it not to mention avx2 and other things that CPU does not support and avoid the issue.
Comment 10 Frans van der Veer 2021-05-25 21:54:43 UTC
(In reply to Steve Wills from comment #9)
In this case is was built on the same host, but within a poudriere jail. That is what you will regard as being the same host also I presume.

Come to think of that, the contents of the file is from the host filing system, not from the jail filing system.
Comment 11 Steve Wills freebsd_committer 2021-05-26 11:04:51 UTC
(In reply to Frans van der Veer from comment #10)
Yes, that's all the same host really. Can you please share a complete copy of the poudriere log for devel/qt5-qmake?
Comment 12 Frans van der Veer 2021-05-26 11:31:33 UTC
Created attachment 225275 [details]
Poudriere log file for qt5-qmake-5.15.2

Steve,

here it is.
Comment 13 Steve Wills freebsd_committer 2021-05-26 11:45:47 UTC
(In reply to Frans van der Veer from comment #12)
Thanks. Pretty strange to me that log contains:

Checking for AVX2 instructions... yes

I wonder why?
Comment 14 Frans van der Veer 2021-05-26 12:02:35 UTC
(In reply to Steve Wills from comment #13)
Hi Steve,

I also wonder why it reports 

UNAME_v=FreeBSD 12.2-RELEASE-p3
UNAME_r=12.2-RELEASE-p3

while I did a jail updates upto current p6. Just checked that once again:


# poudriere jail -u -j amd64
[00:00:00] Upgrading using ftp
/etc/resolv.conf -> /usr/local/poudriere/jails/amd64/etc/resolv.conf
Looking up update.FreeBSD.org mirrors... 2 mirrors found.
Fetching metadata signature for 12.2-RELEASE from update1.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

No updates needed to update system to 12.2-RELEASE-p6.
12.2-RELEASE-p6
[00:01:45] Recording filesystem state for clean...cannot create snapshot 'zroot/poudriere/jails/amd64@clean': dataset already exists
 done

Even the latest builds of today still report p3 in the log. But OK, I presume that is quite another issue.


What I also find strange about this situation is that a poudriere build should to my opinion produce CPU architecture independent packages so you can distribute then anywhere. So making these decisions at package compile time seems odd to me. 

Regards,

Frans
Comment 15 Steve Wills freebsd_committer 2021-05-26 13:02:25 UTC
(In reply to Frans van der Veer from comment #14)

> Even the latest builds of today still report p3 in the log. But OK, I presume that is quite another issue.

I think this may be due to the updates, where some cause kernel changes and others don't, and is expected.

> What I also find strange about this situation is that a poudriere build should to my opinion produce CPU architecture independent packages so you can distribute then anywhere.

I don't really understand what you mean here. Compiled binaries always have a target CPU architecture they run on. Even so-called "fat-binaries" or multi-architecture binaries have a list of architectures they support and don't run "anywhere".

Regardless, as far as this issue goes, it's not clear to me what qmake means here when it says "Checking for AVX2 instructions". I almost think it's just checking if the compiler supports them, not if the host CPU supports them. And then there's the question of if they should be used on the target CPU, which could actually be completely different. I will have to have someone who knows qmake better chime in on these questions I'm afraid.
Comment 16 Steve Wills freebsd_committer 2021-05-26 15:27:03 UTC
(In reply to Frans van der Veer from comment #14)
If possible, could you share a copy of the config.log from the WRKSRC directory after running make build on devel/qt5-qmake on your host that doesn't support avx2?
Comment 17 Frans van der Veer 2021-05-26 16:20:57 UTC
Created attachment 225284 [details]
qt5-qmake build config.log file

Hi Steve,

I don't know how to retrieve this from the poudriere jail, since I seem only to be able to make poudriere  save a wrkdir on build failure.  

So this is taken outside poudriere build from the (updated) hosts ports tree itself as it was filed under:

/usr/ports/devel/qt5-qmake/work/qtbase-everywhere-src-5.15.2

I hope this helps.
Comment 18 Steve Wills freebsd_committer 2021-05-26 16:31:27 UTC
(In reply to Frans van der Veer from comment #17)
That's fine, thanks. It's running:

> clang++ -c -pipe -mavx2 -O2 -w -fPIC -DNO_ATTRIBUTE -DQT_COMPILER_SUPPORTS_AVX2 -I. -I/usr/local/include -I/usr/ports/devel/qt5-qmake/work/qtbase-everywhere-src-5.15.2/mkspecs/freebsd-clang -o main.o main.cpp

without issue. Maybe qmodule.pri is meant to indicate what the compiler supports, not what the host or target support? It's quite confusing.
Comment 19 Steve Wills freebsd_committer 2021-05-26 21:52:28 UTC
Created attachment 225295 [details]
patch for qmake

(In reply to Frans van der Veer from comment #17)
Ok, I did some more digging and I think the found the real source of the issue. Can you test rebuilding your qmake with this and then rebuilding graphics/qt5-3d and then also devel/pyside2 and share the contents of the /usr/local/lib/qt5/mkspecs/qmodule.pri from the new qmake?
Comment 20 Steve Wills freebsd_committer 2021-05-26 21:55:31 UTC
(In reply to Steve Wills from comment #19)
Oh, I should have mentioned, also remove graphics/qt5-3d/files/patch-src_core_configure.json before rebuilding graphics/qt5-3d.
Comment 21 crahman 2021-05-27 07:41:04 UTC
I've done the tests you requested - I added the patch file in comment#19 to qt5-qmake/files and recompiled it, and then built qt5-3d and py38-pyside2.

This is all on a machine without avx2 instructions.

qmodule.pri was unchanged by the qt5-qmake patch:

QT_CPU_FEATURES.x86_64 = mmx sse sse2
QT.global_private.enabled_features = sse2 alloca avx2 dlopen network posix_fallocate reduce_exports reduce_relocations relocatable sql system-zlib testlib xml
QT.global_private.disabled_features = alloca_h alloca_malloc_h android-style-assets private_tests dbus dbus-linked gc_binaries gui intelcet libudev release_tools stack-protector-strong widgets zstd
PKG_CONFIG_EXECUTABLE = pkgconf
QMAKE_LIBS_LIBDL = 
QT_COORD_TYPE = double
QMAKE_LIBS_ZLIB = -lz
CONFIG -= precompile_header
CONFIG += sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx512er avx512ifma avx512pf avx512vbmi avx512vl compile_examples f16c largefile rdrnd rdseed shani x86SimdAlways
QT_BUILD_PARTS += libs tools
QT_HOST_CFLAGS_DBUS += 

The patch from comment#19 did not change the avx2 issue in qt5-3d.  With or without the patch, if qt5-3d/files/patch-src_core_configure.json is present, qt5-3d is compiled with -mavx2, and matrix4x4_avx2.cpp is compiled, and the configure summary shows:

  Use SSE2 instructions .................. yes
  Use AVX2 instructions .................. yes

and pyside2 fails to build.

With or without the patch, if qt5-3d/files/patch-src_core_configure.json is not present, qt5-ed is compiled without -mavx2, matrix4x4.avx2.cpp is compiled, and the configure summary shows:

  Use SSE2 instructions .................. yes
  Use AVX2 instructions .................. no

and pyside2 builds successfully.

The matrix4x4_avx2_cpp contents are conditionalized with an #ifdef QT_COMPILER_SUPPORTS_AVX2, if that's of any use.
Comment 22 Steve Wills freebsd_committer 2021-05-27 12:43:03 UTC
(In reply to crahman from comment #21)

> qmodule.pri was unchanged by the qt5-qmake patch:

Good! It should be.

> With or without the patch, if qt5-3d/files/patch-src_core_configure.json is not present, qt5-ed is compiled without -mavx2, matrix4x4.avx2.cpp is compiled, and the configure summary shows:
> 
>   Use SSE2 instructions .................. yes
>   Use AVX2 instructions .................. no
> 
> and pyside2 builds successfully.

Good!

The patch to qt5-3d was actually not quite right, but it allowed it to build qt5-3d with avx2 enabled.

Here we've removed the patch to qt5-3d and instead added this patch to qmake which fixes the qt5-3d issue in the avx2 case and also doesn't break things in the non-avx2 case.

I've tested the avx2 case and everything is fine, but I couldn't test the without avx2 case fully. You just tested the without avx2 case and confirmed that it all works.

So, I believe removing the patch to qt5-3d and adding this patch to qmake is the correct solution. It allows building everything properly both without avx2 and with avx2.

Just to be on the safe side, I'd like some input from KDE Team, but this seems right to me.
Comment 23 alt2600 2021-08-21 03:53:38 UTC
Created attachment 227351 [details]
patch-qmake_revert_248155_qt5-3d

Steve, this may be due to me still learning git but I had issues with your patch applying unless I included --directory=devel and applying from the devel directory. This is probably user error on my end. But to make things easier for folks like me: git apply with this patch from just being in /usr/ports without additional switches to git. I created this patch including reverting 248155 by removing the offending file. I applied these patches from a ports tree pulled about an hour prior, with everything in the tree that can build allowed to updated including qgis in the blocking notice. All this does is include your patch to qmake and delete the qt5-3d patch file for AVX. qt5-qmake and qt5-3d rebuilt, qt5-3d no longer shows using AVX in configure stage.

py38-pyside2 builds fine following patch
qgis runs without recompilation (though it was freshly updated, and failed prior to the patch with the newer version) without illegal instructions, and so is fixed PR 257402. It the qmake fix helped folks with AVX on qt5-3d then this should work for them too, but I can only test the non-AVX case.

patch produced by git diff --staged, and confirmed to apply to new test branch in my local tree. Maybe this will be useful to folks, but this gets me a little closer to trying to migrate to Releng 13.0

if there is something wrong with the format of my patch let me know, I want to learn how to do this properly.