Bug 224847 - graphics/mesa-libs: make configure (not config) crashes
Summary: graphics/mesa-libs: make configure (not config) crashes
Status: Closed Unable to Reproduce
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-x11 (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-02 21:12 UTC by Bertram Scharpf
Modified: 2018-05-07 17:29 UTC (History)
5 users (show)

See Also:
bugzilla: maintainer-feedback? (x11)
jbeich: maintainer-feedback? (brooks)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bertram Scharpf 2018-01-02 21:12:52 UTC

    
Comment 1 Bertram Scharpf 2018-01-02 21:41:55 UTC
    # cd /usr/ports/graphics/mesa-libs
    # make clean configure
    ...
    configure: error: Could not find llvm shared libraries:
                    Please make sure you have built llvm with the --enable-shared option
                    and that your llvm libraries are installed in /usr/local/llvm50/lib
                    If you have installed your llvm libraries to a different directory you
                    can use the --with-llvm-prefix= configure flag to specify this directory.
                    NOTE: Mesa is attempting to use llvm shared libraries by default.
                    If you do not want to build with llvm shared libraries and instead want to
                    use llvm static libraries then add --disable-llvm-shared-libs to your configure
                    invocation and rebuild.

The error is raised in ./configure, line 28381, function detect_old_buggy_llvm().

The library name is built using the LLVM version number.

    # llvm-config50 --version
    5.0.0git-f35466a8f

But:

    # ls /usr/local/llvm50/lib/libLLVM-5.0.0*.so
    /usr/local/llvm50/lib/libLLVM-5.0.0.so

As you can see, I'm using Git to update my ports tree. The short commit hash
in LLVM's version number is the Git commit when I built devel/llvm50 what
isn't even returned by "make package-name".

The real problem is that the whole Makefile system is so complicated that it took me several
days to find out what happens just to this point. I'm discouraged and won't search any further.

This problem affects x11/xorg and print/texlive-full as far as I know. Probably some
more essential packages.

Ports is a pain.
Comment 2 Jan Beich freebsd_committer freebsd_triage 2018-01-02 23:00:05 UTC
(In reply to software from comment #1)
> $ llvm-config50 --version
> 5.0.0git-f35466a8f

Brooks, did devel/llvm* ever embed git commit of the ports tree?
Comment 3 Jan Beich freebsd_committer freebsd_triage 2018-01-02 23:32:35 UTC
Not enough details to reproduce. Maybe compare with successful build logs:

10.3 i386  - http://beefy5.nyi.freebsd.org/data/latest-per-pkg/mesa-libs/
10.3 amd64 - http://beefy6.nyi.freebsd.org/data/latest-per-pkg/mesa-libs/
11.1 i386  - http://beefy10.nyi.freebsd.org/data/latest-per-pkg/mesa-libs/
11.1 amd64 - http://beefy9.nyi.freebsd.org/data/latest-per-pkg/mesa-libs/
12.0 i386  - http://beefy11.nyi.freebsd.org/data/latest-per-pkg/mesa-libs/
12.0 amd64 - http://beefy12.nyi.freebsd.org/data/latest-per-pkg/mesa-libs/

(In reply to software from comment #1)
>    configure: error: Could not find llvm shared libraries:

Not sure how it can happen after
https://svnweb.freebsd.org/ports/head/graphics/mesa-dri/Makefile.targets?r1=456530&r2=456529&pathrev=456530

(In reply to software from comment #1)
> The real problem is that the whole Makefile system is so complicated that it took me several
> days to find out what happens just to this point. I'm discouraged and won't search any further.

Please, either use prebuilt packages or poudriere/synth to build ports. portmaster/portupgrade and non-assisted builds (simply calling "make") use an environment that maybe dirty with local modifications, partial upgrades, etc. Debugging dirty environments is up to the affected user i.e., requires time to investigate and/or bisect.
Comment 4 Bertram Scharpf 2018-01-03 03:20:58 UTC
You do not need any details to reproduce it. I found it by remembering
that this problem had occurred here several years ago. As far
as I know, at that time I solved it by a workaround. The ports collection
did not manage to fix this bug up to this day!

The cause for the problem is that LLVM builds a version number
from the Git repository it is held in. Now, in the FreeBSD ports
collection, it gets copied there without a Git history. Still the
configuration searches for a Git repo, and it finds the one in /usr/ports.

I fixed the problem by adding a line in /etc/make.conf:

    CONFIGURE_ENV+=GIT_CEILING_DIRECTORIES=/usr/ports

For the meaning of the environment variable see man 1 git.

The line should be put somewhere into Mk/bsd.port.mk; of course it
would be better to throw away the whole ports system together with portupgrade, poudriere et al. and to rewrite it from scratch.
Comment 5 Val Packett 2018-01-03 12:43:01 UTC
(In reply to software from comment #4)

Oh yeah the LLVM git version thing…

I'm working on the Meson build for Mesa on FreeBSD, had to add a workaround:
https://lists.freedesktop.org/archives/mesa-dev/2017-December/180940.html

Thanks for finding the GIT_CEILING_DIRECTORIES variable, this is very useful :) +1 to adding it to bsd.port.mk.

So for the real fix, either LLVM shouldn't add git revisions to the version number, or LLVM consumers should expect them. Currently, most of them only expect SVN revisions.
Comment 6 Niclas Zeising freebsd_committer freebsd_triage 2018-01-04 20:47:16 UTC
Hi!
This looks like a problem with llvm50 rather than mesa, when building ports from a git tree.
Comment 7 Daniel Eischen freebsd_committer freebsd_triage 2018-01-05 01:56:52 UTC
I'm a little confused about the GIT repo thing.  I have the same exact problem, llvm50 fails in the same exact way, using poudriere.  I don't have any GIT repos, I am using the 2018Q1 branch, and the poudriere ports tree is checked out using svn.

This is the tail of the poudriere llvm50 log:

...
-- Installing: /wrkdirs/usr/ports/devel/llvm50/work/stage/usr/local/llvm50/bin/llvm-readobj
Creating llvm-readelf
-- Installing: /wrkdirs/usr/ports/devel/llvm50/work/stage/usr/local/llvm50/bin/llvm-rtdyld
Creating libLLVM-5.0.0.so
Creating libLLVM.so
-- Installing: /wrkdirs/usr/ports/devel/llvm50/work/stage/usr/local/llvm50/lib/libLLVM-5.0.so
CMake Error at tools/llvm-shlib/cmake_install.cmake:52 (file):
  file INSTALL cannot copy file
  "/wrkdirs/usr/ports/devel/llvm50/work/.build/lib/libLLVM-5.0.so" to
  "/wrkdirs/usr/ports/devel/llvm50/work/stage/usr/local/llvm50/lib/libLLVM-5.0.so".
Call Stack (most recent call first):
  tools/cmake_install.cmake:79 (include)
  cmake_install.cmake:61 (include)


FAILED: CMakeFiles/install/strip.util 
cd /wrkdirs/usr/ports/devel/llvm50/work/.build && /usr/local/bin/cmake -DCMAKE_INSTALL_DO_STRIP=1 -P cmake_install.cmake
ninja: build stopped: subcommand failed.
*** Error code 1

Stop.
make: stopped in /usr/ports/devel/llvm50
Comment 8 Niclas Zeising freebsd_committer freebsd_triage 2018-01-05 10:23:56 UTC
That is still an issue with llvm50, not mesa-libs.  I'm not even sure it is related to this.

I had no problems building llvm50 as part of the mesa stack when I tested.
Comment 9 Daniel Eischen freebsd_committer freebsd_triage 2018-01-06 21:55:45 UTC
I agree this is an llvm port problem.  Another data point, llvm40 also fails to build in the same poudriere build environment.  llvm40/50 also failed for me in the 2017Q4 branch.  I don't understand why many others don't have the same problem.  I don't have anything special in poudriere, jails are created by poudriere and ports tree also by poudriere (svn checkout).  I'll open up new bugs when I get back to work, where the systems are.

The system compiler is clang40, it sucks having to rebuild the same compiler for just a few ports when the system compiler should be sufficient.
Comment 10 Brooks Davis freebsd_committer freebsd_triage 2018-01-07 22:03:36 UTC
This feels like a cmake or some other build infrastructure problem to me.  The llvm ports haven't changed recently.

You are rebuilding llvm not to get clang but to get LLVM support libraries that can not be shipped with the base system because they have neither stable APIs or ABIs.
Comment 11 Val Packett 2018-01-08 11:19:14 UTC
(In reply to Brooks Davis from comment #10)
The LLVM ports haven't changed, but they should! See bug 223191. To sum up:

- the ports set the LLVM_BUILD_LLVM_DYLIB CMake option but not LLVM_LINK_LLVM_DYLIB, which makes llvm-config pretty broken (it defaults to static linking, and when asked for dynamic linking with --link-shared it outputs nonexistent tiny libraries libLLVM(Whatever).so instead of the big monolithic libLLVM that actually exists). They should set LLVM_LINK_LLVM_DYLIB!
- CMake's standard library finds the whole path to libexecinfo: https://github.com/Kitware/CMake/blob/40dea7e4b2e1c4518337bba284a233bf6f788a1a/Modules/FindBacktrace.cmake#L75 LLVM uses that result as if it were a library name, creating the broken "-l/usr/lib/libexecinfo.so" linker flag in llvm-config's output for static linking.

LLVM consumers also should change:

- LLVM can add git version suffixes (this thread). Consumers should handle them. E.g. Meson 0.44 already does: https://github.com/mesonbuild/meson/pull/2787 — and Mesa will handle that for older Mesons https://lists.freedesktop.org/archives/mesa-dev/2018-January/181200.html We're on stable release Mesa, so we should fix this in the old (autoconf) Mesa build system.
Comment 12 Daniel Eischen freebsd_committer freebsd_triage 2018-01-08 17:54:22 UTC
In response to Brooks:

"You are rebuilding llvm not to get clang but to get LLVM support libraries that can not be shipped with the base system because they have neither stable APIs or ABIs."

If you only need mesa-libs, then llvm is only a build_depends.  Doesn't that mean that you don't need llvm installed to run your dependent applications?  And if llvm is not installed, then neither are its support libraries.  Unless ports statically link to llvm support libraries when they are built, why are they needed for only a build_depends?
Comment 13 Daniel Eischen freebsd_committer freebsd_triage 2018-01-11 22:23:55 UTC
In my case, though it looks like the same problem with the GIT_CEILING_DIRECTORIES, it is because poudriere was configured to use tmpfs(5) for wrkdir, and my system having 2GB of RAM and 2GB of swap space apparently wasn't enough for building llvm40 or llvm50.  Setting USE_TMPFS=no in poudriere.conf solved the problem.
Comment 14 Brooks Davis freebsd_committer freebsd_triage 2018-02-26 21:36:51 UTC
(In reply to Jan Beich from comment #2)
devel/llvm* has never intentionally embedded a git version in the llvm-config output and I just verified that the current version don't embed on when built in a git checkout (manual build).

Is there anything actionable left in this PR?
Comment 15 Brooks Davis freebsd_committer freebsd_triage 2018-05-07 17:29:35 UTC
I just build llvm50 in a ports tree in a git repo and the git behavior does not occur.  I've never been able to reproduce the other issues mention here other than the dylib stuff that I believe is fixed.