Created attachment 230412 [details] Tensorflow 1.15.5 Port Tensorflow 1.15.5 C libraries Adding to science as py-tensorflow is in science, so it seems best to keep everything together.
Created attachment 230413 [details] Updated tensorflow 1.15.5 port Removes commented out cruft.
Created attachment 230420 [details] libtensorflow1 patch Update patch to add CONFLICTS_INSTALL=science/libtensorflow2
Thank you for the new port Paul
Created attachment 230480 [details] A slightly modified patchset to fix portlint issues
Hi, Paul, First of all, thanks a lot for taking this! I have made some minor tweaks to your patchset to make portlint happy. When testing the port we did discovered some issue. It seems that the port would attempt to download during build stage: %--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%-- =========================================================================== =======================<phase: build >============================ ===> Building for libtensorflow1-1.15.5 Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Options provided by the client: Inherited 'common' options: --isatty=0 --terminal_columns=80 INFO: Reading rc options for 'build' from /usr/local/etc/bazel/bazel.bazelrc: 'build' options: --strategy=Javac=standalone --host_javabase=@local_jdk//:jdk --action_env=PATH=/bin:/usr/bin/:/usr/local/bin INFO: Reading rc options for 'build' from /wrkdirs/usr/ports/science/libtensorflow1/work/tensorflow-1.15.5/.bazelrc: 'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr/local --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --copt=-w --config=v1 INFO: Reading rc options for 'build' from /wrkdirs/usr/ports/science/libtensorflow1/work/tensorflow-1.15.5/.tf_configure.bazelrc: 'build' options: --action_env PYTHON_BIN_PATH=/usr/local/bin/python3.8 --action_env PYTHON_LIB_PATH=/usr/local/lib/python3.8/site-packages --python_path=/usr/local/bin/python3.8 --define=PREFIX=/usr/local --action_env TF_CONFIGURE_IOS=0 INFO: Found applicable config definition build:v1 in file /wrkdirs/usr/ports/science/libtensorflow1/work/tensorflow-1.15.5/.bazelrc: --define=tf_api_version=1 --action_env=TF2_BEHAVIOR=0 INFO: Call stack for the definition of repository 'io_bazel_rules_closure' which is a http_archive (rule definition at /wrkdirs/usr/ports/science/libtensorflow1/work/bazel_out/bef36fd36dd6cc1e25be0c1e2af46c35/external/bazel_tools/tools/build_defs/repo/http.bzl:292:16): - /wrkdirs/usr/ports/science/libtensorflow1/work/tensorflow-1.15.5/WORKSPACE:5:1 WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: storage.googleapis.com WARNING: Download from https://github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: github.com ERROR: An error occurred during the fetch of repository 'io_bazel_rules_closure': java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz, https://github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz] to /wrkdirs/usr/ports/science/libtensorflow1/work/bazel_out/bef36fd36dd6cc1e25be0c1e2af46c35/external/io_bazel_rules_closure/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz: Unknown host: github.com ERROR: no such package '@io_bazel_rules_closure//closure': java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz, https://github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz] to /wrkdirs/usr/ports/science/libtensorflow1/work/bazel_out/bef36fd36dd6cc1e25be0c1e2af46c35/external/io_bazel_rules_closure/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz: Unknown host: github.com ERROR: no such package '@io_bazel_rules_closure//closure': java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz, https://github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz] to /wrkdirs/usr/ports/science/libtensorflow1/work/bazel_out/bef36fd36dd6cc1e25be0c1e2af46c35/external/io_bazel_rules_closure/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz: Unknown host: github.com INFO: Elapsed time: 4.913s INFO: 0 processes. *** Error code 1 Stop. make: stopped in /usr/ports/science/libtensorflow1 %--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%--%-- The port build infrastructure does not allow downloading during build and requires all artifacts be downloaded by the port framework for security (this ensures that all source artifacts are validated by the maintainer) and to make sure that the build is reproducible (so we have all artifacts, potentially mirrored, instead of relying on the single source site). Could you please make changes to the port so that it downloads the artifacts and place them where blaze expects them?
Created attachment 234271 [details] Added some github fetches If it helps, I've added some GH_TUPLE lines to fetch the Github dependencies.
Created attachment 234529 [details] libtensorflow1 patch Updated patch. Linted, tested on 13.0 and 12.3 using poudriere for build.
I'll take it.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=67d31a51c68c5dd076bf87ceabfbfa6c5bc1374f commit 67d31a51c68c5dd076bf87ceabfbfa6c5bc1374f Author: Paul Armstrong <freebsd@otoh.org> AuthorDate: 2022-05-27 15:55:23 +0000 Commit: Chris Rees <crees@FreeBSD.org> CommitDate: 2022-06-10 19:04:30 +0000 science/libtensorflow1: Add new port C API for TensorFlow, an open source platform for machine learning TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML-powered applications. The TensorFlow C API is typically a requirement of TensorFlow APIs in other languages such as Go and Rust. This is version 1 of the API, which is deprecated but still widely used. PR: ports/260694 Improved by: delphij science/Makefile | 1 + science/libtensorflow1/Makefile (new) | 200 +++++++++++++++++++++ science/libtensorflow1/distinfo (new) | 83 +++++++++ .../files/extra-patch-third_party_repo.bzl (new) | 25 +++ science/libtensorflow1/files/patch-.bazelrc (new) | 52 ++++++ .../libtensorflow1/files/patch-configure.py (new) | 11 ++ science/libtensorflow1/pkg-descr (new) | 9 + science/libtensorflow1/pkg-plist (new) | 16 ++ 8 files changed, 397 insertions(+)
New port added, with minor changes. Thanks!
Are you sure this port isn't going to break in weird ways? You have both https://cgit.freebsd.org/ports/tree/science/libtensorflow1/Makefile?id=67d31a51c68c5dd076bf87ceabfbfa6c5bc1374f#n91 and https://cgit.freebsd.org/ports/tree/science/libtensorflow1/Makefile?id=67d31a51c68c5dd076bf87ceabfbfa6c5bc1374f#n131 defined. On top of that you also have CPUTYPE which is used globally by the framework. In general we avoid native as it will cause software that doesn't have runtime detection to crash on machines missing instructions (such as if the buildbox have avx2 or whatever and target client doesn't). Having a quick look it's probably best if you remove all hand optimization and add a note to use CPUTYPE for best optimization? Best regards, Daniel
Thanks Daniel. Clearly, "-march=native" needs to go (and there's no real point in suppressing warnings either, so I'll yank that too). I'm inclined to leave AVX/AVX2 support as a selectable item with the default being to enable AVX because it makes such a huge difference to Tensorflow. Also, AVX is over a decade old so when I was writing this it seemed like a performance benefit that would be usable by the vast majority of users and the rare cases where it wasn't supported could build it themselves. Thoughts?
(In reply to Paul Armstrong from comment #12) To my knowledge we always target the base subset of instructions (at the expensive of performance in some cases) by default to ensure best compatibility overall. I have no strong opinion about leaving AVX and friends in there but from what I can tell it's a bit redundant as you might as well make use of CPUTYPE if you want to use ports or build packages that are optimized for a specific platform? https://cgit.freebsd.org/src/tree/share/examples/etc/make.conf#n25 Unless CPUTYPE doesn't work I think we should avoid trying to work around the framework. Chris, Xin opinions?
Normally I'd suggest flavours for this, but given the burden of the port on the build systems already I don't think triplicating it is a good idea! I agree with taking native out, and I'd have all of the AVX extensions in by default. It's not as though FreeBSD/Xbox is still a thing ;) The vast majority of people using this will be using a post-2011 processor and unlikely a cheap one, but if people ask about it we could add a -noavx flavour.
There are still very new processors released without AVX support and while not being speed daemons they're necessarily unusable slow either. That being said, I'd not very familiar with tensorflow but it seems to be a thing even on ARM with offloading (?) so it might be worth taking into consideration. https://ark.intel.com/content/www/us/en/ark/products/199288/intel-pentium-gold-g6400-processor-4m-cache-4-00-ghz.html for example Are there any obvious downsides solely relying on CPUTYPE?
This is only for amd64. Alright, perhaps leave it on by default, have a -noavx FLAVOUR and a pkg-message advising users to install that instead if it doesn't work, does that sound like a plan? Chris
(of course CPUTYPE is a build time thing, so for package building not much use)
My point being that there might be users relying on offload support rather than the CPU itself (CUDA, OpenCL etc). We could try enabling AVX by default and see if there are bugs reports of it "being broken". Although personally I'd rather see we target baseline and add a message that you may recieve noticable performance increasement by recompiling with AVX* support or just tell people to set and use CPUTYPE if they concerned about performance. CPUTYPE is also imported for ports globally (try setting it and build lets say audio/exhale) ;-)
TensorFlow builds the Libtensorflow x86_64 binaries that it distributes for Linux and MacOS with AVX enabled -- see bazel config in the TF GitHub repo. The AVX default, along iwith other CPU feature defaults, was a deliberate decision by the TF team and I think it'd made a lot of sense for the default FreeBSD builds to be consistent TF's defaults.
I agree with William, and let's also remember that people will notice brokenness far more easily than slowness. The last thing we need is for FreeBSD's tensorflow to be much slower than Linux's :) Paul, I would suggest then as actions: - Remove the native - Set AVX as default to 'on' - Remove warning suppression I'm happy to do these but you might prefer to. Let me know which :)
OK, I've cleaned out "-march=native" and the warning suppression. I like the idea of flavors, as that gives those using CPUs which don't support AVX an easier path to quickly install while not holding back the majority of users. Here's my working branch Makefile https://github.com/psa/libtensorflow1-freebsd-port/blob/p0/Makefile A testbuild on my poudriere server for science/libtensorflow1-freebsd-port@all is running overnight. If it's successful I'll upload a patch here.
Created attachment 234622 [details] flavors and -march=native cleanup
I'm getting pkg-fallout notices for trying to download bazelbuild/rules_closure - /wrkdirs/usr/ports/science/libtensorflow1/work/tensorflow-1.15.5/WORKSPACE:5:1 WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz failed: class java.net.ConnectException Can't assign requested address (connect failed) WARNING: Download from https://github.com/bazelbuild/rules_closure/archive/308b05b2419edb5c8ee0471b67a40403df940149.tar.gz failed: class java.net.ConnectException Can't assign requested address (connect failed) ERROR: An error occurred during the fetch of repository 'io_bazel_rules_closure': java.io.IOException: Error downloading It's in the GH_TUPLES and I don't see this problem on my personal poudriere instance. Any ideas?
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=34ece039879999c9eb41b4d9484ca51451b30fdf commit 34ece039879999c9eb41b4d9484ca51451b30fdf Author: Paul Armstrong <freebsd@otoh.org> AuthorDate: 2022-06-12 08:20:23 +0000 Commit: Chris Rees <crees@FreeBSD.org> CommitDate: 2022-06-12 08:28:25 +0000 science/libtensorflow1: add flavour without AVX Not all processors bought these days support AVX, for example Pentium Gold. People would most likely wish to have a working package in this case. While here, remove march=native. PR: ports/260694 Discussed with: diizzy Discussed with: William Miur <wamuir@gmail.com> science/libtensorflow1/Makefile | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
Committed. Thanks! Just note the way I've done CONFLICTS- the default flavour doesn't have a -default PKGNAMESUFFIX.
Hi. I am trying to compile it with NOAVX option in a FreeBSD 12.3 jail. But, it comes to the following error: ===> Building for libtensorflow1-1.15.5_1 INFO: Options provided by the client: Inherited 'common' options: --isatty=1 --terminal_columns=80 INFO: Reading rc options for 'build' from /usr/local/etc/bazel/bazel.bazelrc: 'build' options: --strategy=Javac=standalone --java_runtime_version=11 --tool_ java_runtime_version=11 --action_env=PATH=/bin:/usr/bin/:/usr/local/bin INFO: Reading rc options for 'build' from /usr/ports/science/libtensorflow1/work -default/tensorflow-1.15.5/.bazelrc: 'build' options: --apple_platform_type=macos --define framework_shared_object= true --define open_source_build=true --define=use_fast_cpp_protos=true --define= allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --de fine=grpc_no_ares=true --define=PREFIX=/usr/local --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --copt=-w --config=v1 INFO: Reading rc options for 'build' from /usr/ports/science/libtensorflow1/work -default/tensorflow-1.15.5/.tf_configure.bazelrc: 'build' options: --action_env PYTHON_BIN_PATH=/usr/local/bin/python3.8 --actio n_env PYTHON_LIB_PATH=/usr/local/lib/python3.8/site-packages --python_path=/usr/ local/bin/python3.8 --define=PREFIX=/usr/local --action_env TF_CONFIGURE_IOS=0 INFO: Found applicable config definition build:v1 in file /usr/ports/science/lib tensorflow1/work-default/tensorflow-1.15.5/.bazelrc: --define=tf_api_version=1 - -action_env=TF2_BEHAVIOR=0 ERROR: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_docker/container/image.bzl:550:31: invalid escape sequence: \s. Use '\\' to insert '\'. ERROR: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_docker/container/image.bzl:550:40: invalid escape sequence: \c. Use '\\' to insert '\'. INFO: Repository bazel_skylib instantiated at: /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/WORKSPACE:23:21: in <toplevel> /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_closure/closure/repositories.bzl:81:21: in closure_repositories /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_closure/closure/repositories.bzl:203:17: in bazel_skylib Repository rule http_archive defined at: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel> ERROR: error loading package '': at /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/third_party/toolchains/preconfig/generate/workspace.bzl:6:5: at /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_docker/container/container.bzl:18:6: compilation of module 'container/image.bzl' failed INFO: Elapsed time: 2.348s INFO: 0 processes. *** Error code 1 Stop. make[1]: stopped in /usr/ports/science/libtensorflow1 *** Error code 1 Stop. make: stopped in /usr/ports/science/libtensorflow1
Have you done a make clean and tried again?
Hi. Yes, I did it. But, I have done it again. Here is the whole process: root@Emby2:/usr/ports/science/libtensorflow1 # make clean ===> Cleaning for libtensorflow1-1.15.5_1 root@Emby2:/usr/ports/science/libtensorflow1 # make config root@Emby2:/usr/ports/science/libtensorflow1 # make ===> License APACHE20 THIRD_PARTY_TF_C_LICENSES accepted by the user ===> libtensorflow1-1.15.5_1 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by libtensorflow1-1.15.5_1 for building ===> Extracting for libtensorflow1-1.15.5_1 => SHA256 Checksum OK for tensorflow-tensorflow-v1.15.5_GH0.tar.gz. => SHA256 Checksum OK for bazelbuild-rules_closure-308b05b2_GH0.tar.gz. => SHA256 Checksum OK for intel-mkl-dnn-v0.20.6_GH0.tar.gz. => SHA256 Checksum OK for intel-mkl-dnn-v1.0-pc2_GH0.tar.gz. => SHA256 Checksum OK for abseil-abseil-cpp-43ef2148c0936ebf7cb4be6b19927a9d9d145b8f_GH0.tar.gz. => SHA256 Checksum OK for rvagg-rpi-newer-crosstools-eb68350c5c8ec1663b7fe52c742ac4271e3217c5_GH0.tar .gz. => SHA256 Checksum OK for hfp-libxsmm-1.11_GH0.tar.gz. => SHA256 Checksum OK for google-re2-506cfa4bffd060c06ec338ce50ea3468daa6c814_GH0.tar.gz. => SHA256 Checksum OK for googleapis-google-cloud-cpp-v0.10.0_GH0.tar.gz. => SHA256 Checksum OK for googleapis-googleapis-f81082ea1e2f85c43649bee26e0d9871d4b41cdb_GH0.tar.gz. => SHA256 Checksum OK for google-gemmlowp-12fed0cd7cfcd9e169bf1925bc3a7a58725fdcc3_GH0.tar.gz. => SHA256 Checksum OK for google-farmhash-816a4ae622e964763ca0862d9dbd19324a1eaf45_GH0.tar.gz. => SHA256 Checksum OK for glennrp-libpng-v1.6.37_GH0.tar.gz. => SHA256 Checksum OK for abseil-abseil-py-pypi-v0.7.1_GH0.tar.gz. => SHA256 Checksum OK for protocolbuffers-protobuf-310ba5ee72661c081129eb878c1bbcec936b20f0_GH0.tar.g z. => SHA256 Checksum OK for google-nsync-1.22.0_GH0.tar.gz. => SHA256 Checksum OK for google-googletest-b6cd405286ed8635ece71c72f118e659f4ade3fb_GH0.tar.gz. => SHA256 Checksum OK for gflags-gflags-v2.2.1_GH0.tar.gz. => SHA256 Checksum OK for grpc-grpc-4566c2a29ebec0835643b972eb99f4306c4234a3_GH0.tar.gz. => SHA256 Checksum OK for nanopb-nanopb-f8ac463766281625ad710900479130c7fcb4d63b_GH0.tar.gz. => SHA256 Checksum OK for antirez-linenoise-c894b9e59f02203dbe4e2be657572cf88c4230c3_GH0.tar.gz. => SHA256 Checksum OK for llvm-mirror-llvm-7a7e03f906aada0cf4b749b51213fe5784eeff84_GH0.tar.gz. => SHA256 Checksum OK for LMDB-lmdb-LMDB_0.9.22_GH0.tar.gz. => SHA256 Checksum OK for open-source-parsers-jsoncpp-1.8.4_GH0.tar.gz. => SHA256 Checksum OK for google-boringssl-7f634429a04abc48e2eb041c81c5235816c96514_GH0.tar.gz. => SHA256 Checksum OK for google-snappy-1.1.7_GH0.tar.gz. => SHA256 Checksum OK for nvidia-nccl-0ceaec9cee96ae7658aa45686853286651f36384_GH0.tar.gz. => SHA256 Checksum OK for edenhill-librdkafka-v0.11.5_GH0.tar.gz. => SHA256 Checksum OK for google-pprof-c0fb62ec88c411cc91194465e54db2632845b650_GH0.tar.gz. => SHA256 Checksum OK for NVlabs-cub-1.8.0_GH0.tar.gz. => SHA256 Checksum OK for ROCmSoftwarePlatform-rocPRIM-4a33d328f8352df1654271939da66914f2334424_GH0.t ar.gz. => SHA256 Checksum OK for cython-cython-0.28.4_GH0.tar.gz. => SHA256 Checksum OK for intel-ARM_NEON_2_x86_SSE-1200fe90bb174a6224a525ee60148671a786a71f_GH0.tar.g z. => SHA256 Checksum OK for google-double-conversion-3992066a95b823efc8ccc1baf82a1cfc73f6e9b8_GH0.tar.g z. => SHA256 Checksum OK for bazelbuild-rules_android-v0.1.1_GH0.tar.gz. => SHA256 Checksum OK for 01org-tbb-2019_U1_GH0.tar.gz. => SHA256 Checksum OK for NervanaSystems-ngraph-v0.11.0_GH0.tar.gz. => SHA256 Checksum OK for nlohmann-json-v3.4.0_GH0.tar.gz. => SHA256 Checksum OK for NervanaSystems-ngraph-tf-v0.9.0_GH0.tar.gz. => SHA256 Checksum OK for pybind-pybind11-v2.3.0_GH0.tar.gz. => SHA256 Checksum OK for GrahamDumpleton-wrapt-1.11.1_GH0.tar.gz. ===> Patching for libtensorflow1-1.15.5_1 ===> Applying extra patch /usr/ports/science/libtensorflow1/files/extra-patch-third_party_repo.bzl ===> Applying FreeBSD patches for libtensorflow1-1.15.5_1 from /usr/ports/science/libtensorflow1/files /usr/bin/sed -i.bak -e 's,%%PYTHON_CMD%%,/usr/local/bin/python3.8,' -e 's,%%LOCALBASE%%,/usr/local,' /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/.bazelrc ===> libtensorflow1-1.15.5_1 depends on executable: bash - found ===> libtensorflow1-1.15.5_1 depends on executable: bazel - found ===> libtensorflow1-1.15.5_1 depends on executable: git - found ===> libtensorflow1-1.15.5_1 depends on package: gmake>=4.3 - found ===> libtensorflow1-1.15.5_1 depends on file: /usr/local/bin/python3.8 - found ===> Configuring for libtensorflow1-1.15.5_1 ===> FreeBSD 10 autotools fix applied to /usr/ports/science/libtensorflow1/work-default/farmhash-816a4ae622e964763ca0862d9dbd19324a1eaf45/configure ===> FreeBSD 10 autotools fix applied to /usr/ports/science/libtensorflow1/work-default/farmhash-816a4ae622e964763ca0862d9dbd19324a1eaf45/m4/libtool.m4 You have bazel 5.1.0 installed. Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details. --config=mkl # Build with MKL support. --config=monolithic # Config for mostly static monolithic build. --config=gdr # Build with GDR support. --config=verbs # Build with libverbs support. --config=ngraph # Build with Intel nGraph support. --config=numa # Build with NUMA support. --config=dynamic_kernels # (Experimental) Build kernels into separate shared objects. --config=v2 # Build TensorFlow 2.x instead of 1.x. Preconfigured Bazel build configs to DISABLE default on features: --config=noaws # Disable AWS S3 filesystem support. --config=nogcp # Disable GCP support. --config=nohdfs # Disable HDFS support. --config=noignite # Disable Apache Ignite support. --config=nokafka # Disable Apache Kafka support. --config=nonccl # Disable NVIDIA NCCL support. Configuration finished ===> Building for libtensorflow1-1.15.5_1 Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Options provided by the client: Inherited 'common' options: --isatty=1 --terminal_columns=108 INFO: Reading rc options for 'build' from /usr/local/etc/bazel/bazel.bazelrc: 'build' options: --strategy=Javac=standalone --java_runtime_version=11 --tool_java_runtime_version=11 --action_env=PATH=/bin:/usr/bin/:/usr/local/bin INFO: Reading rc options for 'build' from /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/.bazelrc: 'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr/local --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --copt=-w --config=v1 INFO: Reading rc options for 'build' from /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/.tf_configure.bazelrc: 'build' options: --action_env PYTHON_BIN_PATH=/usr/local/bin/python3.8 --action_env PYTHON_LIB_PATH=/usr/local/lib/python3.8/site-packages --python_path=/usr/local/bin/python3.8 --define=PREFIX=/usr/local --action_env TF_CONFIGURE_IOS=0 INFO: Found applicable config definition build:v1 in file /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/.bazelrc: --define=tf_api_version=1 --action_env=TF2_BEHAVIOR=0 DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1556410077 -0400" DEBUG: Repository io_bazel_rules_docker instantiated at: /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/WORKSPACE:35:30: in <toplevel> /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/bazel_toolchains/repositories/repositories.bzl:37:23: in repositories Repository rule git_repository defined at: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel> ERROR: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_docker/container/image.bzl:550:31: invalid escape sequence: \s. Use '\\' to insert '\'. ERROR: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_docker/container/image.bzl:550:40: invalid escape sequence: \c. Use '\\' to insert '\'. INFO: Repository bazel_skylib instantiated at: /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/WORKSPACE:23:21: in <toplevel> /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_closure/closure/repositories.bzl:81:21: in closure_repositories /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_closure/closure/repositories.bzl:203:17: in bazel_skylib Repository rule http_archive defined at: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel> ERROR: error loading package '': at /usr/ports/science/libtensorflow1/work-default/tensorflow-1.15.5/third_party/toolchains/preconfig/generate/workspace.bzl:6:5: at /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_docker/container/container.bzl:18:6: compilation of module 'container/image.bzl' failed INFO: Elapsed time: 24.880s INFO: 0 processes. *** Error code 1 Stop. make[1]: stopped in /usr/ports/science/libtensorflow1 *** Error code 1 Stop. make: stopped in /usr/ports/science/libtensorflow1
Hi. I found the problem and solved it. I had bazel 5.1.0 as stated in the compiling log. This was making it fail: You have bazel 5.1.0 installed. I uninstalled it and I installed bazel 0.29, instead: bazel029-0.29.0_2 Fast and correct build system And it compiled perfectly :) root@Emby2:/usr/ports/science/libtensorflow1 # pkg info libtensorflow1-1.15.5_1 libtensorflow1-1.15.5_1 Name : libtensorflow1 Version : 1.15.5_1 Installed on : Sun Jun 12 18:38:05 2022 UTC Origin : science/libtensorflow1 Architecture : FreeBSD:12:amd64 Prefix : /usr/local Categories : science Licenses : APACHE20, THIRD_PARTY_TF_C_LICENSES Maintainer : freebsd-ports@otoh.org WWW : https://www.tensorflow.org/install/lang_c Comment : C API for TensorFlow, an open source platform for machine learning Options : AVX : off AVX2 : off CUDA : off NOAVX : on OPENCL_SYCL : off ROCM : off XLA : off Shared Libs provided: libtensorflow.so.1 libtensorflow_framework.so.1 Annotations : FreeBSD_version: 1203000 flavor : default Flat size : 135MiB Description : TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML-powered applications. The TensorFlow C API is typically a requirement of TensorFlow APIs in other languages such as Go and Rust. This is version 1 of the API, which is deprecated but still widely used. WWW: https://www.tensorflow.org/install/lang_c Thus, I think it is necessary to make a change in the build dependences check, so only bazel version 0.29 is permitted and do not try to build it.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=7f1928fc68273b24685ec3baf09896382a47fa4b commit 7f1928fc68273b24685ec3baf09896382a47fa4b Author: Chris Rees <crees@FreeBSD.org> AuthorDate: 2022-06-12 19:55:48 +0000 Commit: Chris Rees <crees@FreeBSD.org> CommitDate: 2022-06-12 20:01:13 +0000 science/libtensorflow1: disallow newer Bazel Does not build with newer Bazel: Repository rule git_repository defined at: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel> ERROR: /usr/ports/science/libtensorflow1/work-default/bazel_out/98e92f9393bc83ca8c28c921fe93f245/external/io_bazel_rules_docker/container/image.bzl:550:31: invalid escape sequence: \s. Use '\\' to insert '\'. PR: ports/260694 Discovered by: Dario <darioml@gmail.com> science/libtensorflow1/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Brilliant, thanks. By the way, pkg install libtensorflow1-noavx should work soon, which will save you a bit of CPU time!
Thanks gents, much appreciated! Looks like this was also true of the port build cluster. I'm surprised my personal poudriere instance was doing the right thing but that wasn't.
Created attachment 234725 [details] libtensorflow1-1.5.5_2 After some digging, I a probable solution is to add --distdir= to the build. I also found yet another source of dependencies which need to be downloaded and added them to Makefile.local As updating GH_TUPLE in Makefile is a PITA, it's been separated into Makefile.gh_tuple. Allows for "make generate-gh-tuples > Makefile.gh_tuple"
Comment on attachment 234725 [details] libtensorflow1-1.5.5_2 Does this build currently, or are you still getting failures? If it builds fine, looks like something to do on the next update- also we don't bump revision for build fixes.
It builds on my local server (as has every patch I've submitted). There's something different about the port build cluster and my local machine which means that I can't replicate the failures the port build cluster has.
Created attachment 234806 [details] Update GH_TUPLE/distfiles and add distdir to bazel flags Resubmit of previous patch without bumping PORTREVISION