Bug 210476 - [NEW PORT] math/cadabra2: Computer algebra system for solving field theory problems
Summary: [NEW PORT] math/cadabra2: Computer algebra system for solving field theory pr...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Kurt Jaeger
URL:
Keywords: feature, patch
Depends on:
Blocks:
 
Reported: 2016-06-22 17:39 UTC by Yuri Victorovich
Modified: 2016-07-17 19:24 UTC (History)
3 users (show)

See Also:
koobs: maintainer-feedback+


Attachments
shar archive (3.59 KB, text/plain)
2016-06-22 17:39 UTC, Yuri Victorovich
no flags Details
shar archive (3.72 KB, text/plain)
2016-06-22 20:21 UTC, Yuri Victorovich
no flags Details
shar archive (6.68 KB, text/plain)
2016-06-24 02:34 UTC, Yuri Victorovich
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2016-06-22 17:39:45 UTC
Created attachment 171688 [details]
shar archive
Comment 1 Yuri Victorovich freebsd_committer freebsd_triage 2016-06-22 20:21:51 UTC
Created attachment 171692 [details]
shar archive

Added runtime dependencies.
Comment 2 Yuri Victorovich freebsd_committer freebsd_triage 2016-06-24 02:34:53 UTC
Created attachment 171730 [details]
shar archive

* Added GUI option (currently broken in runtime)
* Unbundled jsoncpp
* Added CONFLICTS/BROKEN statements
Comment 3 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-10 06:20:17 UTC
This is the new port, should be no need for the maintainer-feedback flag?
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2016-07-10 06:26:59 UTC
Correct, except in this case it was set for requesting QA confirmation
Comment 5 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-10 06:46:54 UTC
Passes poudriere on 10.3 amd64.
Comment 6 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-10 07:31:42 UTC
testbuilds are fine, port is marked BROKEN on 9.x
Comment 7 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-10 07:43:23 UTC
Committed, thanks!
Comment 8 commit-hook freebsd_committer freebsd_triage 2016-07-10 07:43:24 UTC
A commit references this bug:

Author: pi
Date: Sun Jul 10 07:42:56 UTC 2016
New revision: 418304
URL: https://svnweb.freebsd.org/changeset/ports/418304

Log:
  New port: math/cadabra2

  Cadabra is a symbolic computer algebra system (CAS) designed
  specifically for the solution of problems encountered in field
  theory. It has extensive functionality for tensor computer algebra,
  tensor polynomial simplification including multi-term symmetries,
  fermions and anti-commuting variables, Clifford algebras and Fierz
  transformations, component computations, implicit coordinate
  dependence, multiple index types and many more. The input format
  is a subset of TeX.

  WWW: http://cadabra.science/

  PR:		210476
  Submitted by:	Yuri Victorovich <yuri@rawbw.com>

Changes:
  head/math/Makefile
  head/math/cadabra2/
  head/math/cadabra2/Makefile
  head/math/cadabra2/distinfo
  head/math/cadabra2/files/
  head/math/cadabra2/files/patch-CMakeLists.txt
  head/math/cadabra2/files/patch-client__server_CMakeLists.txt
  head/math/cadabra2/pkg-descr
  head/math/cadabra2/pkg-plist
Comment 9 John Marino freebsd_committer freebsd_triage 2016-07-17 01:36:55 UTC
This port caused poudriere to break on DragonFly during the scanning phase (specifically with Mk/Uses/compiler.mk)

I believe the error is here:
BUILD_DEPENDS+=	clang36:lang/clang36
RUN_DEPENDS+=	clang36:lang/clang36
CC=		clang36
CXX=		clang++36

Why are full paths not being used for CC and CXX definitions?
e.g. CC=${LOCALBASE}/bin/clang36

I don't believe CC should rely on PATH definitions.  Kurt, thoughts?
Comment 10 John Marino freebsd_committer freebsd_triage 2016-07-17 01:48:54 UTC
okay, even doing that didn't resolve the issues (which are just warnings, not breakage):

sh: /usr/local/bin/clang36: not found
make: "/usr/ports/Mk/Uses/compiler.mk" line 74: warning: "/usr/local/bin/clang36 --version" returned non-zero status
make: "/usr/ports/Mk/Uses/compiler.mk" line 132: warning: "/usr/local/bin/clang++36 -### /dev/null 2>&1" returned non-zero status
WARNING (math/cadabra2): sh: /usr/local/bin/clang36: not found
WARNING (math/cadabra2): make: "/usr/ports/Mk/Uses/compiler.mk" line 74: warning: "/usr/local/bin/clang36 --version" returned non-zero status
WARNING (math/cadabra2): make: "/usr/ports/Mk/Uses/compiler.mk" line 132: warning: "/usr/local/bin/clang++36 -### /dev/null 2>&1" returned non-zero status

the DF compiler.mk differs from FreeBSD's, but this could still be happening.
The compiler is not issued in a clean poudriere environment when compiler.mk is executed.
Comment 11 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 01:55:23 UTC
There should be USES=clang:36 that should do the compiler change correctly on all OS. It shouldn't be the business of individual ports to enable the special versions of clang compiler.
Comment 12 John Marino freebsd_committer freebsd_triage 2016-07-17 01:57:26 UTC
i'll do a local patch to the port and see if poudriere likes that better.
Comment 13 John Marino freebsd_committer freebsd_triage 2016-07-17 02:00:36 UTC
oh, you meant that USES=clang would be nice to have.  It doesn't currently exist.
Comment 14 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 02:15:11 UTC
Yes, it would be nice to have.
Comment 15 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 02:34:09 UTC
(In reply to John Marino from comment #9)
> I don't believe CC should rely on PATH definitions.  Kurt, thoughts?

I have no objections to any changes as long as it works. I will not have the time to setup a DragonFly test environment for the next few weeks.
Comment 16 John Marino freebsd_committer freebsd_triage 2016-07-17 02:37:21 UTC
Kurt,
It's not a "DragonFly" problem.
Those warnings should be happening on FreeBSD too.

I only mention DragonFly because that's what I detected it on.

The problem is CC is overridden, and this seems to break in compiler.mk which uses shell commands like "!= ${CC} --version"
Comment 17 John Marino freebsd_committer freebsd_triage 2016-07-17 03:31:02 UTC
here's proof it's a bug on FreeBSD too:

https://lists.freebsd.org/pipermail/freebsd-ports/2016-July/104034.html

The OP came to conclusion that it was his fault with CCACHE, but I don't think that's the case.  It's the same exact error I'm seeing on poudriere.

You should be able to reproduce this without any issue.
Comment 18 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 03:47:10 UTC
I saw these warnings before I submitted the port. But since they are only warnings, and didn't seem to indicate a problem in the port itself I didn't report them. Poudriere builds finish successfully.
Comment 19 John Marino freebsd_committer freebsd_triage 2016-07-17 03:55:47 UTC
I'm not really following that logic.
A new port should not emit warnings, period.
It's also the *only* port that emits warnings during a poudriere scan.
The forum post indicates that it actually broke a Synth scan, so that even a step higher.

Other ports (although not many) specify an exact version of clang, so perhaps they can be looked at to see how they do it.

As a side note, a port that requires a specific version of a specific compiler exclusively is a bit suspicious.  Was this requirement even documented so it's understood (and agreed)?

I'll wait for Kurt to look this over.
I had to comment out the 4 clang36 lines to kill the warnings.  Maybe Kurt can do something similar and mark the port BROKEN while a better solution is sought.  I don't imagine he would have committed the port if he knew these warnings were being emitted.  (there is no mention of them on this PR either)
Comment 20 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 04:04:18 UTC
They require the version in their documentation about the supported compilers. Earlier compilers break.

I saw some intermittent warnings in poudriere before, that is why I didn't emphasize this one. Poudriere regularly misbehaves for me, for example it fails to start, leaves mounts, consumes local disk, etc. It is not a flawless program. I should have investigated it.
Comment 21 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 04:38:53 UTC
This happens with every port that uses custom version of clang when it also has USES=compiler.

For example, you can reproduce this on devel/tesla if you add compiler:c++11-lang to USES.

So this is in no way specific to math/cadabra2, except that it happened to be the first port to expose the problem.
Comment 22 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 08:44:29 UTC
I do testbuilds with this:

poudriere testport -j $i -o $port -n > ~/logs/${name}-${stmp}.txt

and if the error is not sent to stderr, I take it as "OK".

From the discussion I understand that this should be a problem report about poudriere, not about this port ?
Comment 23 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 08:52:00 UTC
IMO, this is a bug in Uses/compiler.mk.
Comment 24 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 10:41:05 UTC
I'm testbuilding right now, with the CC/CXX lines in the Makefile commented out.
Comment 25 John Marino freebsd_committer freebsd_triage 2016-07-17 12:35:41 UTC
Aren't CC and CXX normally changed in the environment?

e.g. CONFIGURE_ENV+= CC=${LOCALBASE}/bin/clang36 ?

(or CONFIGURE_ARGS)
Comment 26 John Marino freebsd_committer freebsd_triage 2016-07-17 12:49:32 UTC
(In reply to Kurt Jaeger from comment #22)
"From the discussion I understand that this should be a problem report about poudriere, not about this port ?"

No, it's definitely the port.  It's the only port that causes the warning.  You wouldn't see it necessarily by running poudriere on one port and it obviously still builds.  You should see it during the scan phase (tee the poudriere process)

As the previous post suggests, I think the port is wrong in how it changes the compiler selection (setting CC directly instead of the environment or arguments).  Yuri is suggesting that compiler.mk is wrong.  This is why I suggested that he look at all the other ports that change compilers to see how they did it.

I don't think compiler.mk can be changed.  the scan happens without loading any dependencies.  Scanning for features can't possibly happen if the compiler is not in place.  I have my own beef with that though (the implementation), but that is out of scope here.
Comment 27 John Marino freebsd_committer freebsd_triage 2016-07-17 12:55:52 UTC
"This happens with every port that uses custom version of clang when it also has USES=compiler."

Aren't these conflicting actions?
USES=compiler is for selecting an appropriate compiler via USES.

Why is it needed if you are overridding CC and C++ manually?
Comment 28 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 15:53:13 UTC
(In reply to John Marino from comment #27)
The testbuild failed on 10, because compiler:c++11-lang provides clang 3.4 on 10.x, but cadabra2 needs at least 3.6.

I'll test with CONFIGURE_ENV as suggested next.
Comment 29 John Marino freebsd_committer freebsd_triage 2016-07-17 15:57:32 UTC
Kurt, I think you misunderstood.

I wasn't saying to comment out the lines to fix the port.  I was saying to comment out the lines and mark it broken.  The presence of the lines together with USES=compiler is the source of the problem, so I was saying how to provisionally stop the warnings.

The idea was to mark it broken and then figure out how to fix it properly.  If you can't fix it quickly, I'd still recommend doing that.  The port is in a bad state right now.
Comment 30 John Marino freebsd_committer freebsd_triage 2016-07-17 15:59:46 UTC
(In reply to Kurt Jaeger from comment #28)
The testbuild failed on 10, because compiler:c++11-lang provides clang 3.4 on 10.x, "

I would actually love to know why clang-34 is still brought up.  I changed this to clang38 on dragonfly ports.  I think this could be an oversight.  gnustep is also at clang34 and I don't know why.  It works fine with clang38.
Comment 31 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 17:39:10 UTC
(In reply to John Marino from comment #25)

With CONFIGURE_ENV setting CC and CXX, it fails to build on 10.x because cmake does not detect the proper compiler and uses clang instead of clang36.
Comment 32 commit-hook freebsd_committer freebsd_triage 2016-07-17 17:41:06 UTC
A commit references this bug:

Author: pi
Date: Sun Jul 17 17:40:05 UTC 2016
New revision: 418673
URL: https://svnweb.freebsd.org/changeset/ports/418673

Log:
  math/cadabra2: mark BROKEN on 10

  PR:		210476
  Reported by:	marino, mat

Changes:
  head/math/cadabra2/Makefile
Comment 33 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 18:31:35 UTC
I don't really know what the solution might be for math/cadabra2. Any port can run into this problem any time, this is just a matter of time.

compiler.mk should take all options, like required features, suggested compiler version (gcc/clang + version requirement), and work with these consistently. USE_GCC should also be a part of compiler.mk.

Fragmented and inconsistent implementation of compiler support in ports is the root cause of this problem.
Comment 34 John Marino freebsd_committer freebsd_triage 2016-07-17 18:51:35 UTC
Multiple people have said not to use USES=compiler with CC definition.  

You need to explain why you did this, specifically: Why is USES=compiler necessary?  Implicit in your comment above is that this is the correct thing to do, yet no other port does it.  

You've been accusing bugs everywhere else, but the obvious explanation is the bug is contained in this port. Even if compiler.mk should support this (and you need to make a case for that) you need to have a workaround for now.
Comment 35 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 18:54:20 UTC
The problem is not the USES=compiler part, but the fact that one can not easily tell cmake in that port to use the correct clang36 if asked.

So the problem is: How can we get cmake to use the proper clang36 ?
Comment 36 John Marino freebsd_committer freebsd_triage 2016-07-17 18:57:27 UTC
Kurt, As Mat said, you can't have USES=compiler and CC defined together.
You can have USES=compiler alone.  You can have CC definition alone.  You can't do both together.

the cmake issue is apparently solved with CC definition at least.
Did you try removing USES=compiler?  My question remains: why is this even set?  what is USES=compiler set for if the CC is overridden too?
Comment 37 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 19:09:32 UTC
Another test: with compiler:c++14-lang it builds on 12a, 11a, 10.3i.
Comment 38 Yuri Victorovich freebsd_committer freebsd_triage 2016-07-17 19:21:26 UTC
Kurt, if USES=compiler:c++14-lang with removed CC=clang passes builds, could you just re-enable the port with these options?
Comment 39 commit-hook freebsd_committer freebsd_triage 2016-07-17 19:24:14 UTC
A commit references this bug:

Author: pi
Date: Sun Jul 17 19:23:46 UTC 2016
New revision: 418676
URL: https://svnweb.freebsd.org/changeset/ports/418676

Log:
  math/cadabra2: builds on 10.x with compiler:c++14-lang

  PR:		210476
  Approved by:	Yuri Victorovich <yuri@rawbw.com> (maintainer)

Changes:
  head/math/cadabra2/Makefile
Comment 40 Kurt Jaeger freebsd_committer freebsd_triage 2016-07-17 19:24:40 UTC
Thanks all!