Bug 263939

Comment 1 Bjoern A. Zeeb 2022-05-12 11:23:47 UTC

Add See also to the update which seems in progress for a while now but wasn't committed.  I wonder if that would improve the situation?

Comment 2 Bjoern A. Zeeb 2022-05-12 11:49:49 UTC

Trying to build the update to py38-numpy-1.22.3,1 runs into the same error:

...
Processing numpy/random/bit_generator.pyx
Processing numpy/random/_pcg64.pyx
Processing numpy/random/mtrand.pyx
INFO: blas_opt_info:
INFO: blas_armpl_info:
INFO: customize UnixCCompiler
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 461, in <module>
    setup_package()
  File "setup.py", line 453, in setup_package
    setup(**metadata)
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/core.py", line 135, in setup
    config = configuration()
  File "setup.py", line 131, in configuration
    config.add_subpackage('numpy')
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/misc_util.py", line 1054, in add_subpackage
    config_list = self.get_subpackage(subpackage_name, subpackage_path,
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/misc_util.py", line 1020, in get_subpackage
    config = self._get_configuration_from_setup_py(
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/misc_util.py", line 962, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "numpy/setup.py", line 9, in configuration
    config.add_subpackage('core')
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/misc_util.py", line 1054, in add_subpackage
    config_list = self.get_subpackage(subpackage_name, subpackage_path,
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/misc_util.py", line 1020, in get_subpackage
    config = self._get_configuration_from_setup_py(
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/misc_util.py", line 962, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "numpy/core/setup.py", line 813, in configuration
    blas_info = get_info('blas_opt', 0)
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 587, in get_info
    return cl().get_info(notfound_action)
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 847, in get_info
    self.calc_info()
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 2079, in calc_info
    if self._calc_info(blas):
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 2065, in _calc_info
    return getattr(self, '_calc_info_{}'.format(name))()
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 1981, in _calc_info_armpl
    info = get_info('blas_armpl')
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 587, in get_info
    return cl().get_info(notfound_action)
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 847, in get_info
    self.calc_info()
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 1339, in calc_info
    info = self.check_libs2(lib_dirs, armpl_libs)
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 1003, in check_libs2
    exts = self.library_extensions()
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 962, in library_extensions
    c = customized_ccompiler()
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/system_info.py", line 216, in customized_ccompiler
    global_compiler = _customized_ccompiler()
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/__init__.py", line 50, in customized_ccompiler
    c.customize('')
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/ccompiler.py", line 89, in <lambda>
    m = lambda self, *args, **kw: func(self, *args, **kw)
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/ccompiler.py", line 539, in CCompiler_customize
    self.compile([fn], output_dir=tmpdir,
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/ccompiler.py", line 89, in <lambda>
    m = lambda self, *args, **kw: func(self, *args, **kw)
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/ccompiler.py", line 272, in CCompiler_compile
    jobs = get_num_build_jobs()
  File "/var/tmp/work-20220224/usr/ports/math/py-numpy/work-py38/numpy-1.22.3/numpy/distutils/misc_util.py", line 93, in get_num_build_jobs
    cpu_count = len(os.sched_getaffinity(0))
OSError: [Errno 0] Error
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/math/py-numpy

Comment 3 Bjoern A. Zeeb 2022-05-12 13:22:22 UTC

Renaming this as it seems the actual problem is in python itself (also removing the see-also).

On amd64 with python 3.8 installed from pkg I get:

% python3.8
Python 3.8.13 (default, Apr 28 2022, 01:23:51) 
[Clang 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a on freebsd14
Type "help", "copyright", "credits" or "license" for more information.
>>> import os;
>>> os.sched_getaffinity(0);
{0, 1, 2, 3, 4, 5, 6, 7}
>>> quit();
%

On arm64 where I am building I get:

# python3.8
Python 3.8.13 (default, May 11 2022, 17:57:27) 
[Clang 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a on freebsd14
Type "help", "copyright", "credits" or "license" for more information.
>>> import os;
>>> os.sched_getaffinity(0);
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 0] Error
>>> quit();
#


I can see a difference on an older installation where the problem was that the function wasn't present yet to my reading (again here on arm64 but probably in general);  the change is probably driven by @kib's changes to sched.h in src last year so it became supported?

# python3.8
Python 3.8.10 (default, Jul 10 2021, 10:18:40) 
[Clang 12.0.1 (git@github.com:llvm/llvm-project.git llvmorg-12.0.1-rc2-0-ge7dac on freebsd14
Type "help", "copyright", "credits" or "license" for more information.
>>> import os;
>>> os.sched_getaffinity(0);
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'sched_getaffinity'
>>> quit();
#

Comment 4 Antoine Brodin 2022-05-12 13:56:45 UTC

Could this be related to recent 
 https://cgit.FreeBSD.org/src/commit/?id=f35093f8d6d8155ab2e56c11ee03d474688b16a2
?
numpy started to fail on main-amd64 and main-i386 this night, it was successful during the previous build

Comment 5 Bjoern A. Zeeb 2022-05-12 14:05:16 UTC

Thanks Antoine; made it less arm64 specific then too..  I haven't even made it to commit emails yet :(

Comment 6 commit-hook 2022-05-12 17:07:23 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3e11d3f61ae2b2b8a8ffeff4ef32067d95c065a1

commit 3e11d3f61ae2b2b8a8ffeff4ef32067d95c065a1
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2022-05-12 17:06:13 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2022-05-12 17:06:13 +0000

    Fixed the value returned by sched_getaffinity().

    On success gnu libc sched_getaffinity() should return 0, unlike underlying
    Linux syscall which returns the size of CPU mask copied to user.

    PR:             263939
    MFC after:      2 weeks

 lib/libc/gen/sched_getaffinity.c | 4 ----
 1 file changed, 4 deletions(-)

Comment 7 Dmitry Chagin 2022-05-12 20:47:46 UTC

dchagin@mordor:~ % python3.8
Python 3.8.13 (default, Apr  5 2022, 02:04:31) 
[Clang 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a on freebsd14
Type "help", "copyright", "credits" or "license" for more information.
>>> import os;
>>> 
>>> os.sched_getaffinity(0);
{0, 1, 2, 3}
>>> len(os.sched_getaffinity(0));
4
>>>

fixed, thanks. close please

Comment 8 Bjoern A. Zeeb 2022-05-13 11:46:03 UTC

Thank you Dmitry for looking into it and fixing this so quickly!
Much appreciated.

Comment 9 commit-hook 2022-06-17 19:42:13 UTC

A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=46be698d0fb0bfeb96914df5426d3a317314df67

commit 46be698d0fb0bfeb96914df5426d3a317314df67
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2022-05-12 17:06:13 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2022-06-17 19:35:16 +0000

    Fixed the value returned by sched_getaffinity().

    On success gnu libc sched_getaffinity() should return 0, unlike underlying
    Linux syscall which returns the size of CPU mask copied to user.

    PR:             263939
    MFC after:      2 weeks

    (cherry picked from commit 3e11d3f61ae2b2b8a8ffeff4ef32067d95c065a1)

 lib/libc/gen/sched_getaffinity.c | 4 ----
 1 file changed, 4 deletions(-)