Bug 283155 - Link fails with -flto: error loading plugin: Cannot open "/usr/bin/../lib/LLVMgold.so" (port multimedia/kew)
Summary: Link fails with -flto: error loading plugin: Cannot open "/usr/bin/../lib/LL...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 14.1-STABLE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-toolchain (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-12-06 02:12 UTC by Yuri Victorovich
Modified: 2024-12-06 18:41 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2024-12-06 02:12:11 UTC
Link fails when the patch multimedia/kew/patch-Makefile is removed:

> /usr/local/bin/x86_64-unknown-freebsd14.1-ld: /usr/bin/../lib/LLVMgold.so: error loading plugin: Cannot open "/usr/bin/../lib/LLVMgold.so"
Comment 1 Ed Maste freebsd_committer freebsd_triage 2024-12-06 02:57:54 UTC
This is an issue with devel/binutils presumably, please indicate the version you are using.
Comment 2 Yuri Victorovich freebsd_committer freebsd_triage 2024-12-06 03:54:50 UTC
(In reply to Ed Maste from comment #1)

binutils-2.43.1,1
Comment 3 Mark Millard 2024-12-06 04:10:45 UTC
I explored what happens to exist in my context, which is for main FreeBSD:
(I'll note that lang/gcc14 and devel/llvm19 are part of this
environment. The system c++ is also LLVM19 based.)

# find / -name '*LLVMgold*' -print
/usr/local/llvm19/lib/LLVMgold.so

# pkg-static info -l llvm19 | grep gold
	/usr/local/llvm19/lib/LLVMgold.so

The system toolchain does not provide LLVMgold.so --and never has
if I understand right.

# find / -name '*ld.gold*' -print
/usr/local/x86_64-portbld-freebsd15.0/bin/ld.gold
/usr/local/bin/ld.gold

# pkg-static info -l binutils | grep gold
	/usr/local/bin/ld.gold
	/usr/local/x86_64-portbld-freebsd15.0/bin/ld.gold

# pkg-static which /usr/local/bin/ld.gold
/usr/local/bin/ld.gold was installed by package binutils-2.43.1,1

# pkg-static which /usr/local/x86_64-portbld-freebsd15.0/bin/ld.gold
/usr/local/x86_64-portbld-freebsd15.0/bin/ld.gold was installed by package binutils-2.43.1,1

# find / -name '*-unknown-*-ld' -print
/usr/local/bin/aarch64-unknown-freebsd15.0-ld

pkg-static which /usr/local/bin/aarch64-unknown-freebsd15.0-ld
/usr/local/bin/aarch64-unknown-freebsd15.0-ld was installed by package aarch64-binutils-2.43.1,1

# pkg info | grep binutil
aarch64-binutils-2.43.1,1      GNU binary tools
arm-gnueabi-binutils-2.43.1,1  GNU binary tools
binutils-2.43.1,1              GNU binary tools

# grep -A2 FLAVORS= /usr/ports/devel/binutils/Makefile
FLAVORS=	native aarch64 aarch64_none_elf amd64 arm_gnueabi arm_none_eabi \
		avr i386 mingw32 mips mips64 powerpc powerpc64 powerpc64le riscv64 \
		riscv64_none_elf s390x riscv32_unknown_elf

So devel/binutils@amd64 seems to be what would supply
x86_64-unknown-freebsd*-ld . My context does not happen
to have that.
Comment 4 Mark Millard 2024-12-06 04:26:43 UTC
(In reply to Mark Millard from comment #3)

From what I can tell, /usr/ports/devel/binutils/Makefile needs
to deal with specifying and using a specific version of
devel/llvm* in order to use its specific LLVMgold.so .

(That note presumes that LLVMgold.so use is really intended,
rather than a mistake of its own.)

I see no evidence of /usr/ports/devel/binutils/Makefile
doing so but it needs to. It may need to track the default
devel/llvm* rather than a specific number.
Comment 5 Mark Millard 2024-12-06 04:48:22 UTC
Why is LLVMgold.so not part of FreeBSD? The answer is, likely, licensing:

QUOTE ( from https://llvm.org/docs/GoldPlugin.html )
Gold is licensed under the GPLv3. LLVMgold uses the interface file plugin-api.h from gold which means that the resulting LLVMgold.so binary is also GPLv3. This can still be used to link non-GPLv3 programs just as much as gold could without the plugin.
END QUOTE
Comment 6 Mark Millard 2024-12-06 05:26:48 UTC
Hmm. Having LLVMgold.so in the installed devel/llvm19 implies
that the devel/llvm19 licensing should reference GPLv3 . But:

# pkg info llvm19
llvm19-19.1.3
Name           : llvm19
Version        : 19.1.3
Installed on   : Thu Nov 28 00:09:17 2024 PST
Origin         : devel/llvm19
Architecture   : FreeBSD:15:amd64
Prefix         : /usr/local
Categories     : lang devel
Licenses       : REGEX and PD and MIT and LLVM2 and LLVM and BSD3CLAUSE'
. . .

Also:

/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFINE_amd64=	GOLD
/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFINE_powerpc=	GOLD
/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFINE_powerpc64=	GOLD
/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFINE_powerpc64le=	GOLD
/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFAULT_amd64+=	GOLD
/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFAULT_powerpc+=	GOLD
/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFAULT_powerpc64+=	GOLD
/usr/ports/devel/llvm19/Makefile:OPTIONS_DEFAULT_powerpc64le+=	GOLD
/usr/ports/devel/llvm19/Makefile:_ALL_OPTIONS=	CLANG COMPILER_RT DOCS EXTRAS FLANG GOLD LIT LLD LLDB MLIR \
Comment 7 Dimitry Andric freebsd_committer freebsd_triage 2024-12-06 13:27:56 UTC
I don't see it here. With files/patch-Makefile deleted, a poudriere bulk build of multimedia/kew works just fine.

Final link command line:
cc -o kew src/obj/common_ui.o src/obj/sound.o src/obj/directorytree.o src/obj/soundcommon.o src/obj/m4a.o src/obj/search_ui.o src/obj/playlist_ui.o src/obj/player.o src/obj/soundbuiltin.o src/obj/mpris.o src/obj/playerops.o src/obj/utils.o src/obj/file.o src/obj/imgfunc.o src/obj/cache.o src/obj/songloader.o src/obj/playlist.o src/obj/term.o src/obj/settings.o src/obj/visuals.o src/obj/kew.o src/obj/tagLibWrapper.o -L/usr/lib -lm -lopusfile -lglib-2.0 -lpthread -L/usr/local/lib -lgio-2.0 -lgobject-2.0 -lchafa -lfftw3f -lopus -lopusfile -logg -lvorbis -lvorbisfile -lglib-2.0 -lintl -ltag -lz -lstdc++ -lnotify -L/usr/local/lib -lfaad -logg -lz -flto

I'm unsure why it would even attempt to use the gold plugin here?
Comment 8 Mark Millard 2024-12-06 14:00:57 UTC
(In reply to Dimitry Andric from comment #7)

Probably how to reproduce the LLVMgold.so use attempt for
direct make style builds: have devel/binutils@amd64
installed at the time so that the:

/usr/local/bin/*-unknown-freebsd*-ld

exists when the cc based link command is executed.

Turns out that I've reported the issue to the lists in the past. An
example is:

https://lists.freebsd.org/archives/freebsd-toolchain/2022-September/000898.html

It even shows the example (aarch64 context, not amd64):

/usr/local/bin/aarch64-unknown-freebsd14.0-ld: /usr/bin/../lib/LLVMgold.so: error loading plugin: Cannot open "/usr/bin/../lib/LLVMgold.so"

Sorry it took so long to remember.
Comment 9 Dimitry Andric freebsd_committer freebsd_triage 2024-12-06 14:33:02 UTC
(In reply to Mark Millard from comment #8)
Yes, so the first problem is that other ports use the prefix ${arch}-portbld-freebsdX.Y-, while devel/binutils tries to be different, and uses ${arch}-unknown-freebsdX.Y-:

$ ls -l /usr/local/bin/x8*
-rwxr-xr-x  4 root wheel 1720112 2024-11-23 23:37:08 /usr/local/bin/x86_64-portbld-freebsd15.0-c++13*
-rwxr-xr-x  4 root wheel 1720112 2024-11-23 23:37:08 /usr/local/bin/x86_64-portbld-freebsd15.0-g++13*
-rwxr-xr-x  3 root wheel 1716464 2024-11-24 00:07:52 /usr/local/bin/x86_64-portbld-freebsd15.0-gcc-13.3.0*
-rwxr-xr-x  2 root wheel   26208 2024-11-24 00:07:55 /usr/local/bin/x86_64-portbld-freebsd15.0-gcc-ar13*
-rwxr-xr-x  2 root wheel   26144 2024-11-24 00:07:55 /usr/local/bin/x86_64-portbld-freebsd15.0-gcc-nm13*
-rwxr-xr-x  2 root wheel   26144 2024-11-24 00:07:55 /usr/local/bin/x86_64-portbld-freebsd15.0-gcc-ranlib13*
-rwxr-xr-x  3 root wheel 1716464 2024-11-24 00:07:52 /usr/local/bin/x86_64-portbld-freebsd15.0-gcc13*
-rwxr-xr-x  2 root wheel 1719056 2024-11-23 23:37:14 /usr/local/bin/x86_64-portbld-freebsd15.0-gfortran13*
-r-xr-xr-x  1 root wheel 1212928 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-addr2line*
-r-xr-xr-x  2 root wheel 1245144 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-ar*
-r-xr-xr-x  2 root wheel 1955200 2024-12-06 15:17:54 /usr/local/bin/x86_64-unknown-freebsd15.0-as*
-r-xr-xr-x  1 root wheel 1211304 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-c++filt*
-r-xr-xr-x  1 root wheel   38968 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-elfedit*
-r-xr-xr-x  1 root wheel 1275096 2024-12-06 15:17:54 /usr/local/bin/x86_64-unknown-freebsd15.0-gprof*
-r-xr-xr-x  4 root wheel 2236920 2024-12-06 15:17:54 /usr/local/bin/x86_64-unknown-freebsd15.0-ld*
-r-xr-xr-x  4 root wheel 2236920 2024-12-06 15:17:54 /usr/local/bin/x86_64-unknown-freebsd15.0-ld.bfd*
-r-xr-xr-x  2 root wheel 1229232 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-nm*
-r-xr-xr-x  2 root wheel 1360832 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-objcopy*
-r-xr-xr-x  2 root wheel 2533640 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-objdump*
-r-xr-xr-x  2 root wheel 1245144 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-ranlib*
-r-xr-xr-x  2 root wheel 1112456 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-readelf*
-r-xr-xr-x  1 root wheel 1214704 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-size*
-r-xr-xr-x  1 root wheel 1218776 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-strings*
-r-xr-xr-x  2 root wheel 1360848 2024-12-06 15:17:53 /usr/local/bin/x86_64-unknown-freebsd15.0-strip*

This particular bug would simply go away if devel/binutils used ${arch}-portbld-freebsdX.Y- too.

The second problem is that the GNU ld driver searches in the wrong path for LLVMgold.so. If somebody would actually want to use LLVMgold.so (not really recommended), they'd have to first choose an LLVM version to use, then try any of:

/usr/local/llvm15/lib/LLVMgold.so
/usr/local/llvm16/lib/LLVMgold.so
/usr/local/llvm17/lib/LLVMgold.so
/usr/local/llvm18/lib/LLVMgold.so
/usr/local/llvm19/lib/LLVMgold.so

That said, the better solution seems fixing binutils to me.
Comment 10 Mark Millard 2024-12-06 16:21:38 UTC
(In reply to Dimitry Andric from comment #9)

In order to avoid conflicts with lang/gcc* the following in devel/
use -unknown- instead of -portbld- as I remember :

d---------	freebsd-gcc12	535	log plain
d---------	freebsd-gcc13	535	log plain
d---------	freebsd-gcc14	529	log plain

I believe devel/binutils is set to match the devel/freebsd-gcc* .

(Otherwise it would likely be lang/binutils ?)

In the long term history, the change from -portbld- to
-unknown- was on an earlier freebsd-gcc* that is no
longer visible in devel/ --if I remember right.

But it is possible that my memory is mixing in the old base/
history as well.
Comment 11 John Baldwin freebsd_committer freebsd_triage 2024-12-06 17:43:10 UTC
I would really like it if in-tree clang would just always use the in-tree linker and not try to find a "better" linker using the triple.  I think though you would still have the problem of clang from ports also happily using ld.bfd from binutils instead of its associated ld.lld by default.  Ideally the LLVM ports toolchains would prefer to use all their own bits as well unless told otherwise via -B, etc.
Comment 12 Mark Millard 2024-12-06 18:26:52 UTC
(In reply to John Baldwin from comment #11)

One type of handling that avoids the likes of use
of x86_64-unknown-freebsd14.1-ld is to use, say,
-fuse-ld=lld on the command line. (Avoids adjusting
LLVM source.)

But that is very specific to ld.lld and the clang code
for finding tools is more general:

  const ToolChain::path_list &List = TC.getProgramPaths();
  for (const auto &TargetSpecificExecutable : TargetSpecificExecutables) {
    // For each possible name of the tool look for it in
    // program paths first, then the path.
    // Higher priority names will be first, meaning that
    // a higher priority name in the path will be found
    // instead of a lower priority name in the program path.
    // E.g. <triple>-gcc on the path will be found instead
    // of gcc in the program path
    for (const auto &Path : List) {
      SmallString<128> P(Path);
      if (ScanDirForExecutable(P, TargetSpecificExecutable))
        return std::string(P);
    }

Does FreeBSD's built-in toolchain have any intended-to-be-supported
potential use of <triple>-TOOL overriding TOOL from earlier in the
path? Any deliberate use of such already?
Comment 13 Mark Millard 2024-12-06 18:41:54 UTC
(In reply to Mark Millard from comment #12)

I should have listed that TargetSpecificExecutables traces back
to being filled in via:

void Driver::generatePrefixedToolNames(
    StringRef Tool, const ToolChain &TC,
    SmallVectorImpl<std::string> &Names) const {
  // FIXME: Needs a better variable than TargetTriple
  Names.emplace_back((TargetTriple + "-" + Tool).str());
  Names.emplace_back(Tool);
}

So my final question traces back to if the FreeBSD
system toolchain context has any need for the:

  Names.emplace_back((TargetTriple + "-" + Tool).str())

(The ports are a different matter, not being limited to the
FreeBSD system toolchain's intended range of usage.)

Even the FIXME note might be suggesting that TargetTriple
content was not completely obvious as the best of choices
for forming the special name.