Bug 252648 - Intel oneAPI tbb - Migrating from devel/tbb to devel/onetbb (main PR)
Summary: Intel oneAPI tbb - Migrating from devel/tbb to devel/onetbb (main PR)
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Ganael LAPLANCHE
URL:
Keywords:
Depends on: 252651 252686 252691 252765 252788 252789 252790 252649 252653 252683 252684 252685 252687 252688 252690 252693 252694 252695 252696 252766 252767 252785 252786 252868 252870 252892
Blocks:
  Show dependency treegraph
 
Reported: 2021-01-13 20:34 UTC by Ganael LAPLANCHE
Modified: 2021-02-19 15:15 UTC (History)
14 users (show)

See Also:


Attachments
Patch to existing devel/tbb (1.07 KB, patch)
2021-01-13 20:45 UTC, Ganael LAPLANCHE
no flags Details | Diff
New devel/onetbb port (9.47 KB, text/plain)
2021-01-13 20:45 UTC, Ganael LAPLANCHE
no flags Details
New devel/onetbb port (v2) (10.20 KB, patch)
2021-01-18 15:15 UTC, Ganael LAPLANCHE
no flags Details | Diff
New devel/onetbb port (v2) (10.20 KB, text/plain)
2021-01-18 15:16 UTC, Ganael LAPLANCHE
no flags Details
New devel/onetbb port (v3) (10.07 KB, text/plain)
2021-01-20 18:36 UTC, Ganael LAPLANCHE
no flags Details
/tmp/conformance_blocked_rangeNd-c82975.cpp.bz2 (610.07 KB, application/x-bzip)
2021-01-21 17:14 UTC, Thierry Thomas
no flags Details
/tmp/conformance_blocked_rangeNd-c82975.sh (2.67 KB, application/x-shellscript)
2021-01-21 17:15 UTC, Thierry Thomas
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ganael LAPLANCHE freebsd_committer 2021-01-13 20:34:21 UTC
Intel oneAPI tbb (formerly known as Intel tbb) 2021.1 has been released and has deprecated several interfaces over tbb 2020, see:

 https://software.intel.com/content/www/us/en/develop/articles/intel-oneapi-threading-building-blocks-release-notes.html

and:

  https://software.intel.com/content/www/us/en/develop/articles/tbb-revamp.html

As a consequence, updating devel/tbb in-place is not possible because most of our dependent ports break.

Instead, we will introduce a new port called devel/onetbb (the new name of the library) that will hold new versions of the library. Find attached to that PR expected patches for devel/tbb and devel/onetbb.

The old port, devel/tbb will be kept for a certain amount of time and removed in the future.

We will have to migrate and patch each dependent port before committing the changes alltogether. That PR will help synchronize the migration by attaching a blocking PR for each failing port. 

See also this thread on ports@:

https://lists.freebsd.org/pipermail/freebsd-ports/2021-January/120010.html

Stay tuned!
Comment 1 Ganael LAPLANCHE freebsd_committer 2021-01-13 20:45:27 UTC
Created attachment 221542 [details]
Patch to existing devel/tbb
Comment 2 Ganael LAPLANCHE freebsd_committer 2021-01-13 20:45:57 UTC
Created attachment 221543 [details]
New devel/onetbb port
Comment 3 Yuri Victorovich freebsd_committer 2021-01-17 19:44:11 UTC
The upstream needs to fix this.
Comment 4 Yuri Victorovich freebsd_committer 2021-01-17 19:45:46 UTC
Sorry, wrong bug#. -)
Comment 5 Shane 2021-01-18 09:29:39 UTC
On my system I have devel/hwloc installed (there is also devel/hwloc2) so when I build from ports tree I get extra libs built -

===> Checking for items in STAGEDIR missing from pkg-plist
Error: Orphaned: lib/libtbbbind.so
Error: Orphaned: lib/libtbbbind.so.3
Error: Orphaned: lib/libtbbbind.so.3.1
Comment 6 Ganael LAPLANCHE freebsd_committer 2021-01-18 15:13:45 UTC
Hello Shane,

Good catch, thanks! The port has picked up legacy hwloc and built additional libtbbbind.

I've updated the patch to depend on newer devel/hwloc2 and disabled detection of devel/hwloc.

Can you check again please ?
Comment 7 Ganael LAPLANCHE freebsd_committer 2021-01-18 15:15:03 UTC
Created attachment 221708 [details]
New devel/onetbb port (v2)
Comment 8 Ganael LAPLANCHE freebsd_committer 2021-01-18 15:16:18 UTC
Created attachment 221709 [details]
New devel/onetbb port (v2)
Comment 9 Shane 2021-01-19 02:24:53 UTC
The build is fine, but I would change the lib name for the hwloc. If hwloc is installed the lib matches for dependencies but the configure fails.

Both ports have libhwloc.so, while hwloc has libhwloc.so.5 and hwloc2 has libhwloc.so.15
Comment 10 Ganael LAPLANCHE freebsd_committer 2021-01-19 11:18:26 UTC
Right, I'll fix that, thanks!
Comment 11 Greg V 2021-01-20 13:25:33 UTC
Please remove ONLY_FOR_ARCHS, :noninja, and MAKE_JOBS_UNSAFE.

And add CMAKE_ARGS+=-DTBB_STRICT:BOOL=FALSE to disable Werror, because there's e.g. an unused argument warning for '-MD' on aarch64.

With these changes, builds fine on aarch64.
Comment 12 Ganael LAPLANCHE freebsd_committer 2021-01-20 18:36:48 UTC
Created attachment 221772 [details]
New devel/onetbb port (v3)

Patch updated, thanks!
Comment 13 Thierry Thomas freebsd_committer 2021-01-21 17:11:13 UTC
To handle PR 252870, I tried to build devel/onetbb and it failed on -CURRENT (clang 11):

[97/308] /usr/local/libexec/ccache/c++  -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test/.. -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Wall -Wextra -Wshadow -Wcast-qual -Woverloaded-virtual -Wnon-virtual-dtor -Wno-parentheses -pthread -std=c++11 -MD -MT test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o -MF test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o.d -o test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o -c /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test/conformance/conformance_blocked_rangeNd.cpp
FAILED: test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o 
/usr/local/libexec/ccache/c++  -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test/.. -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Wall -Wextra -Wshadow -Wcast-qual -Woverloaded-virtual -Wnon-virtual-dtor -Wno-parentheses -pthread -std=c++11 -MD -MT test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o -MF test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o.d -o test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o -c /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test/conformance/conformance_blocked_rangeNd.cpp
Assertion failed: (isa<X>(Val) && "cast<Ty>() argument of incompatible type!"), function cast, file /usr/src/contrib/llvm-project/llvm/include/llvm/Support/Casting.h, line 269.
PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /usr/bin/c++ -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Wall -Wextra -Wshadow -Wcast-qual -Woverloaded-virtual -Wnon-virtual-dtor -Wno-parentheses -pthread -std=c++11 -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test/.. -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test -I/usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include -c -o test/CMakeFiles/conformance_blocked_rangeNd.dir/conformance/conformance_blocked_rangeNd.cpp.o /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/test/conformance/conformance_blocked_rangeNd.cpp 
1.      /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include/oneapi/tbb/blocked_rangeNd.h:69:62: current parser token ':'
2.      /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include/oneapi/tbb/blocked_rangeNd.h:34:1: parsing namespace 'tbb'
3.      /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include/oneapi/tbb/blocked_rangeNd.h:35:1: parsing namespace 'tbb::detail'
4.      /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include/oneapi/tbb/blocked_rangeNd.h:36:1: parsing namespace 'tbb::detail::d1'
5.      /usr/ports/devel/onetbb/work/oneTBB-2021.1.1/src/tbb/../../include/oneapi/tbb/blocked_rangeNd.h:55:1: parsing struct/union/class body 'tbb::detail::d1::blocked_rangeNd_impl<Value, N, detail::index_sequence<Is...>>'
#0 0x00000000041d495e PrintStackTrace /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:564:13
#1 0x00000000041d2bb5 RunSignalHandlers /usr/src/contrib/llvm-project/llvm/lib/Support/Signals.cpp:69:18
#2 0x000000000417209e HandleCrash /usr/src/contrib/llvm-project/llvm/lib/Support/CrashRecoveryContext.cpp:77:5
#3 0x0000000004172221 CrashRecoverySignalHandler /usr/src/contrib/llvm-project/llvm/lib/Support/CrashRecoveryContext.cpp:0:51
#4 0x0000000805690bd0 handle_signal /usr/src/lib/libthr/thread/thr_sig.c:0:3
c++: error: clang frontend command failed due to signal (use -v to see invocation)
FreeBSD clang version 11.0.0 (git@github.com:llvm/llvm-project.git llvmorg-11.0.0-0-g176249bd673)
Target: x86_64-unknown-freebsd13.0
Thread model: posix
InstalledDir: /usr/bin
c++: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
c++: note: diagnostic msg: /tmp/conformance_blocked_rangeNd-c82975.cpp
c++: note: diagnostic msg: /tmp/conformance_blocked_rangeNd-c82975.sh
c++: note: diagnostic msg:
Comment 14 Thierry Thomas freebsd_committer 2021-01-21 17:14:37 UTC
Created attachment 221792 [details]
/tmp/conformance_blocked_rangeNd-c82975.cpp.bz2

Compressed file /tmp/conformance_blocked_rangeNd-c82975.cpp as requested by the error message
Comment 15 Thierry Thomas freebsd_committer 2021-01-21 17:15:45 UTC
Created attachment 221793 [details]
/tmp/conformance_blocked_rangeNd-c82975.sh

File /tmp/conformance_blocked_rangeNd-c82975.sh as requested by the error message.
Comment 16 Ganael LAPLANCHE freebsd_committer 2021-01-25 18:22:09 UTC
Switching to PR #252892 for that problem. I could reproduce it and have re-attached build logs. Thanks again Thierry!
Comment 17 commit-hook freebsd_committer 2021-02-19 15:15:00 UTC
A commit references this bug:

Author: martymac
Date: Fri Feb 19 15:14:54 UTC 2021
New revision: 566074
URL: https://svnweb.freebsd.org/changeset/ports/566074

Log:
  Introduce devel/onetbb and mark devel/tbb as legacy

  Intel oneAPI tbb 2021.1 (onetbb) has been released[1][2] and has deprecated
  several interfaces over tbb 2020, breaking most dependent ports.

  Old tbb 2020 will be kept for a certain time to allow transition but will be
  removed in a near future as it CONFLICTS with devel/onetbb. New ports should
  now use devel/onetbb instead of devel/tbb.

  We tried to move a maximum number of dependent ports to devel/onetbb (or
  disable dependency when not possible), but some of them still remain stuck
  to devel/tbb. Remaining ones have not been identified as major dependencies
  themselves and will be fixed as soon as updates are available from upstream.

  PR:		252648, 252688 [3], 252683 [4], 252651 [5], 252690 [3], 252693 [3],
  	252695 [3], 252696 [3], 252786 [3], 252649, 252868 [6], 252870 [5],
  	252684 [7], 252785 [7]
  Approved by:	yuri [3], jwb [4], thierry [5], FreeBSD@Shaneware.biz [6],
  	maintainer timeout [7]

  [1] https://software.intel.com/content/www/us/en/develop/articles/intel-oneapi-threading-building-blocks-release-notes.html
  [2] https://software.intel.com/content/www/us/en/develop/articles/tbb-revamp.html

Changes:
  head/archivers/par2cmdline-tbb/Makefile
  head/biology/bowtie2/Makefile
  head/cad/opencascade/Makefile
  head/devel/Makefile
  head/devel/onetbb/
  head/devel/onetbb/Makefile
  head/devel/onetbb/distinfo
  head/devel/onetbb/files/
  head/devel/onetbb/files/onetbb.pc.in
  head/devel/onetbb/files/patch-src-tbbbind-CMakeLists.txt
  head/devel/onetbb/files/patch-test-common-memory_usage.h
  head/devel/onetbb/files/patch-test-tbbmalloc-test_malloc_compliance.cpp
  head/devel/onetbb/pkg-descr
  head/devel/onetbb/pkg-plist
  head/devel/tbb/Makefile
  head/devel/tbb/pkg-descr
  head/graphics/blender/Makefile
  head/graphics/blender/Makefile.options
  head/graphics/blender/files/patch-intern_cycles_device_device__cpu.cpp
  head/graphics/blender/files/patch-intern_cycles_util_util__task.cpp
  head/graphics/blender/files/patch-intern_cycles_util_util__task.h
  head/graphics/blender/files/patch-intern_cycles_util_util__tbb.h
  head/graphics/blender/files/patch-source_blender_blenlib_BLI__index__range.hh
  head/graphics/blender/files/patch-source_blender_blenlib_BLI__task.h
  head/graphics/blender/files/patch-source_blender_blenlib_intern_task__pool.cc
  head/graphics/embree/Makefile
  head/graphics/oidn/Makefile
  head/graphics/opencv/Makefile
  head/graphics/openimageio/Makefile
  head/math/dune-common/Makefile
  head/math/dune-geometry/Makefile
  head/math/dune-grid/Makefile
  head/math/dune-pdelab/Makefile
  head/math/dune-uggrid/Makefile
  head/math/openturns/Makefile
  head/math/saga/Makefile
  head/math/suitesparse/Makefile
  head/misc/ngraph/Makefile