Created attachment 229258 [details] Update to 1.21.3 - Update to 1.21.3 - Build with python-3.10 - Update TEST_DEPENDS numpy-1.21.3 require pytest>=6.2.4 while we have pytest-4.* in portstree, I tested with my local pytest-6.2.5, result is : 26 failed, 14703 passed, 881 skipped, 1253 deselected, 19 xfailed, 3 xpassed, 1 warning in 254.69s (0:04:14) Since there are 26 fails, I think we should improve the port, so I do not ask for exp-run here.
(In reply to Wen Heping from comment #0) Could you include the failures as an attachment here
Created attachment 229285 [details] The test log of numpy-1.21.4 with pytest-6.2.5 The test log of numpy-1.21.4 with pytest-6.2.5
Created attachment 229286 [details] Update to 1.21.4 Update to 1.21.4
We're up to 1.22.0 now. Hopefully that fixes the build fallout on -CURRENT amd64 and i386
(In reply to Charlie Li from comment #4) In what sense we are up to 1.22? I still see py39-numpy-1.20.3,1 in local poudruere repo. Keeping numpy up to date would be highly appreciated.
(In reply to Konstantin Belousov from comment #5) Upstream is now at 1.22.0 [1] [1] https://pypi.org/project/numpy/#history
Notable release note entries: The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. [1] Distutils forces strict floating point model on clang [1] [1] https://numpy.org/doc/stable/release/1.22.0-notes.html#compatibility-notes
Created attachment 230615 [details] Update to 1.22.0 Update to 1.22.0, but now there are 26 fails with local pytest-6.2.5 :(
Still fails on -CURRENT main-n251990-c7f93ee5cc2 :-\ but going to see what 1b90dfa5d2b before attempting a dive into at least setup.py, considering all of libc++ is getting rebuilt. Also interesting that we have to null out the configure step (for now I guess) =======================<phase: build >============================ ===== env: DEVELOPER_MODE=yes STRICT_DEPENDS=yes USER=nobody UID=65534 GID=65534 ===> Building for py39-numpy-1.22.0,1 Running from numpy source directory. Processing numpy/random/_bounded_integers.pxd.in Processing numpy/random/_pcg64.pyx Processing numpy/random/_bounded_integers.pyx.in Processing numpy/random/mtrand.pyx Processing numpy/random/bit_generator.pyx Processing numpy/random/_common.pyx Processing numpy/random/_generator.pyx Processing numpy/random/_mt19937.pyx Processing numpy/random/_philox.pyx Processing numpy/random/_sfc64.pyx Cythonizing sources INFO: blas_opt_info: INFO: blas_armpl_info: INFO: customize UnixCCompiler Traceback (most recent call last): File "<string>", line 1, in <module> File "setup.py", line 450, in <module> setup_package() File "setup.py", line 442, in setup_package setup(**metadata) File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/core.py", line 135, in setup config = configuration() File "setup.py", line 120, in configuration config.add_subpackage('numpy') File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/misc_util.py", line 1054, in add_subpackage config_list = self.get_subpackage(subpackage_name, subpackage_path, File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/misc_util.py", line 1020, in get_subpackage config = self._get_configuration_from_setup_py( File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/misc_util.py", line 962, in _get_configuration_from_setup_py config = setup_module.configuration(*args) File "numpy/setup.py", line 9, in configuration config.add_subpackage('core') File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/misc_util.py", line 1054, in add_subpackage config_list = self.get_subpackage(subpackage_name, subpackage_path, File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/misc_util.py", line 1020, in get_subpackage config = self._get_configuration_from_setup_py( File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/misc_util.py", line 962, in _get_configuration_from_setup_py config = setup_module.configuration(*args) File "numpy/core/setup.py", line 806, in configuration blas_info = get_info('blas_opt', 0) File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 587, in get_info return cl().get_info(notfound_action) File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 847, in get_info self.calc_info() File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 2079, in calc_info if self._calc_info(blas): File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 2065, in _calc_info return getattr(self, '_calc_info_{}'.format(name))() File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 1981, in _calc_info_armpl info = get_info('blas_armpl') File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 587, in get_info return cl().get_info(notfound_action) File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 847, in get_info self.calc_info() File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 1339, in calc_info info = self.check_libs2(lib_dirs, armpl_libs) File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 1003, in check_libs2 exts = self.library_extensions() File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 962, in library_extensions c = customized_ccompiler() File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/system_info.py", line 216, in customized_ccompiler global_compiler = _customized_ccompiler() File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/__init__.py", line 50, in customized_ccompiler c.customize('') File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/ccompiler.py", line 89, in <lambda> m = lambda self, *args, **kw: func(self, *args, **kw) File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/ccompiler.py", line 540, in CCompiler_customize self.compile([fn], output_dir=tmpdir, File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/ccompiler.py", line 89, in <lambda> m = lambda self, *args, **kw: func(self, *args, **kw) File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/ccompiler.py", line 272, in CCompiler_compile jobs = get_num_build_jobs() File "/wrkdirs/overlays/overlay/math/py-numpy/work-py39/numpy-1.22.0/numpy/distutils/misc_util.py", line 93, in get_num_build_jobs cpu_count = len(os.sched_getaffinity(0)) OSError: [Errno 34] Result too large *** Error code 1 Stop. make: stopped in /overlays/overlay/math/py-numpy
(In reply to Wen Heping from comment #8) What's the test failure delta between 1.22.0 and current port version?
(In reply to Charlie Li from comment #9) Note: numpy-1.20.3,1 (current port) is failing in configure with the same error: https://portsfallout.com/port/15948/
The patch committed in ports 507c189b2876 (which also applies cleanly to 1.22.0) is no longer needed after base d9cacbf4b010.
(In reply to Kubilay Kocak from comment #10) ==================================================== short test summary info ==================================================== FAILED numpy/core/tests/test_cpu_dispatcher.py::test_dispatcher - AssertionError: FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64-inf-inf-npy_cabs-absolute] - RuntimeWarning: invali... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64-inf--inf-npy_cabs-absolute] - RuntimeWarning: inval... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64--inf-inf-npy_cabs-absolute] - RuntimeWarning: inval... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64--inf--inf-npy_cabs-absolute] - RuntimeWarning: inva... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128-inf-inf-npy_cabs-absolute] - RuntimeWarning: inval... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128-inf--inf-npy_cabs-absolute] - RuntimeWarning: inva... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128--inf-inf-npy_cabs-absolute] - RuntimeWarning: inva... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128--inf--inf-npy_cabs-absolute] - RuntimeWarning: inv... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256-inf-inf-npy_cabs-absolute] - RuntimeWarning: inval... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256-inf--inf-npy_cabs-absolute] - RuntimeWarning: inva... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256--inf-inf-npy_cabs-absolute] - RuntimeWarning: inva... FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256--inf--inf-npy_cabs-absolute] - RuntimeWarning: inv... FAILED numpy/core/tests/test_scalarmath.py::TestBaseMath::test_blocked - RuntimeWarning: invalid value encountered in reciprocal FAILED numpy/core/tests/test_ufunc.py::TestUfuncGenericLoops::test_unary_PyUFunc_O_O_method_full[reciprocal] - AssertionError:... FAILED numpy/core/tests/test_umath.py::TestPower::test_power_float - RuntimeWarning: invalid value encountered in reciprocal FAILED numpy/core/tests/test_umath.py::TestAVXUfuncs::test_avx_based_ufunc - RuntimeWarning: invalid value encountered in reci... FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex64--1] - RuntimeWarning: invalid value... FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex64-1] - RuntimeWarning: invalid value ... FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex128--1] - RuntimeWarning: invalid valu... FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex128-1] - RuntimeWarning: invalid value... FAILED numpy/linalg/tests/test_linalg.py::TestNormDouble::test_vector_return_type - RuntimeWarning: invalid value encountered ... FAILED numpy/linalg/tests/test_linalg.py::TestNormDouble::test_axis - RuntimeWarning: invalid value encountered in reciprocal FAILED numpy/linalg/tests/test_linalg.py::TestNormSingle::test_vector_return_type - RuntimeWarning: invalid value encountered ... FAILED numpy/linalg/tests/test_linalg.py::TestNormInt64::test_vector_return_type - RuntimeWarning: invalid value encountered i... FAILED numpy/linalg/tests/test_linalg.py::TestNormInt64::test_axis - RuntimeWarning: invalid value encountered in reciprocal 26 failed, 17037 passed, 1074 skipped, 1293 deselected, 19 xfailed, 5 xpassed, 1 warning in 349.05s (0:05:49)
we may still want to keep the older numpy around for Python 3.7, which will receive security fixes until 2023-06-27 according to https://www.python.org/downloads/
Created attachment 232098 [details] Update to 1.22.2 Update to 1.22.2
numpy-1.22.2, with mu local pytest-6.2.5 installed, the `make test` result: ==================================== short test summary info ===================================== FAILED numpy/core/tests/test_cpu_dispatcher.py::test_dispatcher - AssertionError: FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64-inf-inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64-inf--inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64--inf-inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex64--inf--inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128-inf-inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128-inf--inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128--inf-inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex128--inf--inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256-inf-inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256-inf--inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256--inf-inf-npy_cabs-absolute] FAILED numpy/core/tests/test_multiarray.py::test_npymath_complex[complex256--inf--inf-npy_cabs-absolute] FAILED numpy/core/tests/test_scalarmath.py::TestBaseMath::test_blocked - RuntimeWarning: invali... FAILED numpy/core/tests/test_ufunc.py::TestUfuncGenericLoops::test_unary_PyUFunc_O_O_method_full[reciprocal] FAILED numpy/core/tests/test_umath.py::TestPower::test_power_float - RuntimeWarning: invalid va... FAILED numpy/core/tests/test_umath.py::TestAVXUfuncs::test_avx_based_ufunc - RuntimeWarning: in... FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex64--1] FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex64-1] FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex128--1] FAILED numpy/core/tests/test_umath_complex.py::TestSpecialComplexAVX::test_array[complex128-1] FAILED numpy/linalg/tests/test_linalg.py::TestNormDouble::test_vector_return_type - RuntimeWarn... FAILED numpy/linalg/tests/test_linalg.py::TestNormDouble::test_axis - RuntimeWarning: invalid v... FAILED numpy/linalg/tests/test_linalg.py::TestNormSingle::test_vector_return_type - RuntimeWarn... FAILED numpy/linalg/tests/test_linalg.py::TestNormInt64::test_vector_return_type - RuntimeWarni... FAILED numpy/linalg/tests/test_linalg.py::TestNormInt64::test_axis - RuntimeWarning: invalid va... 26 failed, 17048 passed, 1074 skipped, 1293 deselected, 19 xfailed, 5 xpassed, 1 warning in 313.90
Remark: version 1.20.3 also encounters failures during the tests. Some of them are caused by missing 128 bits float support: see PR 238129.
Created attachment 232104 [details] Original patch upgrading to 1.22.2 + FOPT BTW, I just rebased your initial patch with an old (uncommitted) oneliner which used to give better results: it just adds FOPT=${FCFLAGS} to MAKE_ENV. But now, I'm unable to run `make test': it produces an error in py38-pluggy and py38-pytest, but I do not understand why…
(In reply to Thierry Thomas from comment #18) Numpy-1.22.x test require pytest-6 while we have pytest-4 in portstree. We have to install pytest-6 manually and then make test.
Comment on attachment 232104 [details] Original patch upgrading to 1.22.2 + FOPT Thanks! Also note that devel/py-iniconfig is required: TEST_DEPENDS+= ${PYTHON_PKGNAMEPREFIX}iniconfig>0:devel/py-iniconfig@${PY_FLAVOR} But unfortunately my proposed patch can be obsoleted: setting FOPT=${FCFLAGS} does not provide any improvement with this version: 30 failed, 17319 passed, 799 skipped, 1293 deselected, 19 xfailed, 5 xpassed, 1 warning in 158.42s (0:02:38) with or without it.
Created attachment 233138 [details] Update to 1.22.3+FOPT Update to 1.22.3+FOPT
Note: 1.22.1 is a maintenance/bugfix release: https://numpy.org/devdocs/release/1.22.1-notes.html 1.22.2 is a bugfix and security release: https://numpy.org/devdocs/release/1.22.2-notes.html 1.22.3 is a bugfix and maintenance release: https://numpy.org/devdocs/release/1.22.3-notes.html @Wen If you're already taking care of this, please self-assign and let us know whats needed to land these updates as soon as possible. If not, and someone else would like to champion this issues resolution, self-assign and we can support you getting it done.
Note also, numpy's cython dependency (for 1.22.3) is "Cython>=0.29.24,<3.0", which is already satisfied by lang/cython (0.29.28) However, the latest numpy release is now 1.22.4, which bumped the cython requirement version to "Cython>=0.29.30,<3.0", via https://github.com/numpy/numpy/commit/14be6174c21961470fc749317164668fdbe46ae2 via https://github.com/numpy/numpy/pull/21514 NumPy 1.22.4 changelog: https://numpy.org/doc/stable/release/1.22.4-notes.html also a bugfix and maintenance (MFH) release. Removing blocking on bug 264110 for now (add to See Also)
(In reply to Kubilay Kocak from comment #22) I shall open a new PR to update numpy to 1.22.4 and ask for a exp-run
(In reply to wen from comment #24) Could you attach the 1.22.4 update here, so we can keep the history intact, thanks Since we're going for a 1.22.4 update and there's security and bugfix releases involved, this depends on cython 0.29.30 being merged via bug 264110 so depend on that too.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264231
1.22.4 was committed but this issue (nor MFH) was referenced, via ports ff9098f7093e ^Triage: Assign to committer resolving. - Pending VuXML entry - Pending 264110 merge to quarterly. - Pending MFH
@Wen Dependent cython update has been MFH'd via bug 264110 We're still pending VuXML entries and now a merge of ports ff9098f7093e
(In reply to Kubilay Kocak from comment #28) The merge into 2022Q2 done. Sorry for document the VuXML entries, I could not connect to github these days :( wen
(In reply to wen from comment #29) I understand Wen, but we can't leave known vulnerable versions not documented as such. For whoever takes care of ports updates, the responsibility is with them to either take care of the required changes, or obtain help/support from others to do so. Otherwise too many things end up falling through the cracks. Also, for future updates, could make sure we tag PR: numbers on commits *and* merges so these changes are tagged on their relevent issues? It's really, really hard to figure out what's happened so far, and whats left to do if anything, without them. @Ports-Secteam Can you help with vuxml entries? ^Triage: - py-numpy merged to 2022Q2 in ports 03a6fffb8ef8 (and followup checksum fix in ports c7535f6bc2ae)
PS: Thank you for taking care of the numpy update
(In reply to Kubilay Kocak from comment #30) Now vuxml done.
Is OK to close this PR ?
(In reply to Wen Heping from comment #33) Anything left to do?