Bug 274284 - science/py-scipy hangs during configure phase
Summary: science/py-scipy hangs during configure phase
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-python (Nobody)
URL:
Keywords:
: 274285 274286 (view as bug list)
Depends on:
Blocks:
 
Reported: 2023-10-05 20:43 UTC by Marius Schamschula
Modified: 2023-10-13 13:36 UTC (History)
7 users (show)

See Also:
vishwin: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marius Schamschula 2023-10-05 20:43:52 UTC
While re-building py39-scipy 1.11.1 against openblas 0.3.24,1 using Poudriere under FreebSD 13.2-p4 the configure process hangs after reaching

creating /tmp/tmps3hd_khy/tmp
creating /tmp/tmps3hd_khy/tmp/tmps3hd_khy
INFO: compile options: '-c'
INFO: cc: /tmp/tmps3hd_khy/source.c
INFO: cc /tmp/tmps3hd_khy/tmp/tmps3hd_khy/source.o -L/usr/local/lib -L/usr/local/lib/gcc12/gcc/x86_64-portbld-freebsd13.2/12.2.0/../../../ -lopenblas -lgfortran -o /tmp/tmps3hd_khy/a.out
INFO:   FOUND:
INFO:     libraries = ['openblas', 'gfortran', 'openblas', 'gfortran']
INFO:     library_dirs = ['/usr/local/lib', '/usr/local/lib/gcc12/gcc/x86_64-portbld-freebsd13.2/12.2.0/../../../']
INFO:     language = c
INFO:     define_macros = [('HAVE_CBLAS', None)]
INFO: 
INFO:   FOUND:
INFO:     libraries = ['openblas', 'gfortran', 'openblas', 'gfortran']
INFO:     library_dirs = ['/usr/local/lib', '/usr/local/lib/gcc12/gcc/x86_64-portbld-freebsd13.2/12.2.0/../../../']
INFO:     language = c
INFO:     define_macros = [('HAVE_CBLAS', None)]
INFO:
Comment 1 Vladimir Druzenko freebsd_committer freebsd_triage 2023-10-05 23:56:14 UTC
*** Bug 274285 has been marked as a duplicate of this bug. ***
Comment 2 Vladimir Druzenko freebsd_committer freebsd_triage 2023-10-05 23:56:26 UTC
*** Bug 274286 has been marked as a duplicate of this bug. ***
Comment 3 Konstantin Belousov freebsd_committer freebsd_triage 2023-10-06 01:18:28 UTC
I see python process tightly looping doing nothing but calling sched_yield(),
the call comes from libomp.so.1.  Most likely cause is the following function:
void __kmp_infinite_loop(void) {
  static int done = FALSE;

  while (!done) {
    KMP_YIELD(TRUE);
  }
}
I do not know anything about OpenMP runtime to understand more, and binaries
do not have symbols.

BTW, it seems that Julia build also hangs with the similar syndrome.
Comment 4 Kirill 2023-10-06 10:17:14 UTC
May be problem in math/openblas 0.3.24, I tried to build with previous version and it worked.
Comment 5 Charlie Li freebsd_committer freebsd_triage 2023-10-09 07:15:53 UTC
Building using the distutils method has been deprecated and finally removed with the 1.11.2; the new/current method is PEP-517 with the meson-python build backend. I have had a few scipy versions built successfully using PEP-517, including with the current openblas, but have not had a chance to verify runtime or test suite yet, particularly with mathematical correctness (since a couple build variables had to be removed).
Comment 6 Charlie Li freebsd_committer freebsd_triage 2023-10-10 05:09:46 UTC
Please give review D42144 a try.
Comment 7 François Charlier 2023-10-10 08:35:01 UTC
Build works again using Poudriere under FreeBSD 12.4-p5

Log for failed build before applying review D42144: https://pkg.ploup.net/data/FreeBSD:12:amd64-default/2023-10-09_22h21m27s/logs/errors/py39-scipy-1.11.1.log

Logs for successful build after applying review D42144: https://pkg.ploup.net/data/FreeBSD:12:amd64-default/2023-10-10_08h20m28s/logs/py39-scipy-1.11.3.log
Comment 8 John W. O'Brien 2023-10-10 22:18:09 UTC
(In reply to Charlie Li from comment #6)

I can report poudriere build results with the review D42144 patch on 13.2R amd64 as follows.

py38: skipped due to numpy (ignored)
py39: failed during build phase [0]
py310 (default): success
py311: failed during build phase [1]

[0] Excerpt:

===>   py39-scipy-1.11.3 depends on package: py39-pybind11>=2.10.4 - not found
===>   Installing existing package /packages/All/py39-pybind11-2.11.1.pkg
[pkg.saltant.net] Installing py39-pybind11-2.11.1...
[pkg.saltant.net] Extracting py39-pybind11-2.11.1: .......... done
===>   py39-scipy-1.11.3 depends on package: py39-pybind11>=2.10.4 - found
[...]
pybind11-config found: NO need ['>=2.10.4']
Run-time dependency pybind11 found: NO (tried pkgconfig and config-tool)

../scipy/meson.build:84:15: ERROR: Dependency "pybind11" not found, tried pkgconfig and config-tool

[1] Excerpt:

===>   py311-scipy-1.11.3 depends on package: py311-pybind11>=2.10.4 - not found
===>   Installing existing package /packages/All/py311-pybind11-2.11.1.pkg
[pkg.saltant.net] Installing py311-pybind11-2.11.1...
[pkg.saltant.net] Extracting py311-pybind11-2.11.1: .......... done
===>   py311-scipy-1.11.3 depends on package: py311-pybind11>=2.10.4 - found
[...]
pybind11-config found: NO need ['>=2.10.4']
Run-time dependency pybind11 found: NO (tried pkgconfig and config-tool)

../scipy/meson.build:84:15: ERROR: Dependency "pybind11" not found, tried pkgconfig and config-tool
Comment 9 Charlie Li freebsd_committer freebsd_triage 2023-10-10 23:03:48 UTC
Nice catch on the devel/py-pybind11 USE_PYTHON=concurrent case, updated.

I'm more concerned about making sure that we have a baseline of correct runtime, considering that a couple build flags had to be removed and that the test suite is unable to run at the moment.
Comment 10 Michael Glaus 2023-10-13 11:29:13 UTC
I tried review D42144 and the build itself worked fine, but i got a error during stage. The binary /usr/local/bin/unzip can not be found.

make: exec(/usr/local/bin/unzip) failed (No such file or directory)

In the review the line "unzip>0:archivers/unzip" is changed to 	"unzip:archivers/unzip". If i use the line from before, no error is generated

I assume this error only occurs if DOCS is enabled.
Comment 11 Charlie Li freebsd_committer freebsd_triage 2023-10-13 13:36:29 UTC
(In reply to Michael Glaus from comment #10)
updated to remove archivers/unzip entirely and use the base system ${UNZIP_NATIVE_CMD}