Bug 240578

Summary: science/py-tensorflow: Fix dependencies, Mark Un'BROKEN
Product: Ports & Packages Reporter: Tino Engel <tino.engel>
Component: Individual Port(s)Assignee: Yuri Victorovich <yuri>
Status: Closed FIXED    
Severity: Affects Many People CC: aehlig, amzo1337, swills, w.schwarzenfeld, yuri
Priority: --- Keywords: needs-patch, needs-qa
Version: LatestFlags: w.schwarzenfeld: maintainer-feedback? (amzo1337)
koobs: merge-quarterly?
Hardware: Any   
OS: Any   
URL: https://svnweb.freebsd.org/ports?view=revision&revision=511540
Attachments:
Description Flags
Makefile patch
none
Here is the patch to fix the fetching of build dependencies amzo1337: maintainer-approval+

Description Tino Engel 2019-09-14 09:57:38 UTC
py-tensorflow compiles simply fine with 'make -DTRYBROKEN'. The formerly missing build dependencies are avbailable online.
Comment 1 Tino Engel 2019-09-14 10:03:22 UTC
Created attachment 207485 [details]
Makefile patch

removed BROKEN line
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-14 10:44:54 UTC
The port was marked BROKEN for attempting to fetch/satisfy dependencies at build time. Dependencies cannot be fetched during build time, they must be either depended on via ports, or made available at fetch time
Comment 3 Anthony Donnelly 2019-09-14 12:46:10 UTC
I don't get what changed upstream as the checksums for the tensorflow remained the same. So something changed but I can't see what. I could've just updated the Makefile.Sites but I haven't had time and wanted to double check and find out where the change is.
Comment 4 Anthony Donnelly 2019-09-14 12:51:04 UTC
I don't have access to a FreeBSD machine as Uni has started back up, but this follow commit I've just made will fix the build of fetching dependencies. It's just a matter of make makesum to update them.

https://github.com/Amzo/FreeBSD-Tensorflow/commit/e8f1e4139328d631c187b17755116cc72ad9b05f#diff-e96a27e917ac096135191dc5af3d9cbf
Comment 5 Anthony Donnelly 2019-09-14 14:53:52 UTC
Did some digging and bazel was upgraded in port tree on September the third. This is what broke it as bazel rules changed up stream. The above patch to add the new rules to Makefile will fix the build again. 

It'll probably break again on the next release of bazel. Wish there were multiple version of bazel available in the ports or alternative a USES case for using bazel. I really dislike bazel as a build system as building certain projects require specific versions of bazel.
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-15 03:27:40 UTC
CC committer of bazel update (ports r510821) and the devel/bazel maintainer

We need to find a way to make these fetch-during-build issues resilient to bazel/other dependency updates, otherwise it makes porting software that use these build mechanics untenable in the long-term.

@Anthony Could you please include the patch mentioned in comment 4 as an attachment, and don't forget to use the maintainer-approval attachment flag (set to +) to signify maintainer submitted/approved patches

Thanks!
Comment 7 Klaus Aehlig 2019-09-15 06:33:45 UTC
> We need to find a way to make these fetch-during-build issues resilient to bazel/other dependency updates

Concerning change of dependencies by updates of bazel: bazel uses most of the embedded rules to build itself; so, in order to make bazel packagable, the bazel source archive contains a directory derived/distdir with all the source archives that otherwise would have to be fetched from the network. The precise definition can be found at https://github.com/bazelbuild/bazel/blob/6118004e0db99820f1b6eb15d7901987c1f5481f/WORKSPACE#L116 What we could do, is to make it part of the install target of devel/bazel to install that directory, e.g., under ${DATADIR}/distfiles so that science/py-tensorflow could add 'build --distdir=%%LOCALBASE%%/share/bazel/distfiles' to files/bazelrc (the --distdir option is cummulative) and would only have to care about the dependencies coming from the tensorflow sources themselves.

However, that would only solve part of the problem, as bazel is still developping fast with one release per month. As of bazel 1.0.0 (comming soon, and I already have a draft port for it, see https://github.com/bazelbuild/bazel/issues/8573) at least incompatible changes should only come with every major release, currently expected to come every 3 months (bazel 1.0.0 itself will also bring _a lot_ of incompatible changes). I'm willing to maintain ports for older versions of bazel, but we probably should have a guideline which old versions to keep, as adding a new bazel port every quarter might clutter the ports tree to much unless we have a good policy on when to remove the ports for the older versions.
Comment 8 Tino Engel 2019-09-18 08:37:06 UTC
@Kubilay
I do not get why it is marked broken, dependencies go through normal checksum check. I assume that requires that they are currently being fetched during make fetch. (I can test at some time if it helps)
Comment 9 Anthony Donnelly 2019-09-18 09:05:52 UTC
Seems to be that a dependency which was already provided by the system, bazel rules_cc is no longer part of bazel, so tensorflow is trying to fetch it automatically. This could be due to the recent upgrade of bazel in ports. It's just a matter of fixing the master sites and distfiles. I fixed the issue on git, I just need to generate a patch when I get time to set up a virtual machine. As Uni has just started up, I'm currently on Windows so haven't been able to generate the patch. If I get time today, I'll create a virtual machine and create the patch to fix the issue.
Comment 10 Anthony Donnelly 2019-09-20 12:24:07 UTC
Created attachment 207649 [details]
Here is the patch to fix the fetching of build dependencies

This patch fixes the issues of trying to fetch files during build.
Comment 11 Yuri Victorovich freebsd_committer freebsd_triage 2019-10-06 06:48:11 UTC
Committed, thanks!
Comment 12 commit-hook freebsd_committer freebsd_triage 2019-10-06 06:48:15 UTC
A commit references this bug:

Author: yuri
Date: Sun Oct  6 06:47:53 UTC 2019
New revision: 513871
URL: https://svnweb.freebsd.org/changeset/ports/513871

Log:
  science/py-tensorflow: Unbreak by fixing dependencies

  PR:		240578
  Submitted by:	amzo1337@gmail.com (maintainer)

Changes:
  head/science/py-tensorflow/Makefile
  head/science/py-tensorflow/Makefile.MASTER_SITES
  head/science/py-tensorflow/distinfo