Bug 221808

Summary: archivers/lua51-zlib: port passes $LDFLAGS to direct linker invocation, not compiler driver
Product: Ports & Packages Reporter: Ed Maste <emaste>
Component: Individual Port(s)Assignee: freebsd-ports-bugs (Nobody) <ports-bugs>
Status: Closed FIXED    
Severity: Affects Only Me CC: sirl33tname, w.schwarzenfeld
Priority: --- Keywords: patch
Version: LatestFlags: bugzilla: maintainer-feedback? (sirl33tname)
Hardware: Any   
OS: Any   
Bug Depends on:    
Bug Blocks: 214864    
Attachments:
Description Flags
patch LD flags none

Description Ed Maste freebsd_committer freebsd_triage 2017-08-25 16:55:02 UTC
I've been investigating building the ports tree with LLVM's lld installed as /usr/bin/ld (via the exp-run in PR 214864). Through this I've found a few common issues, some of which are due to limitations of lld or choices made by its authors, and some of which are problems in the ports themselves. audio/oss is affected by one of the latter cases.

Normally $LDFLAGS contains flags passed to the compiler driver when invoked for linking. One issue that affects a small number of ports is passing $LDFLAGS to direct linker invocation (i.e, $LD $LDFLAGS ...), when $LDFLAGS contains flags that are appropriate only for the compiler driver, not the linker.

At one point in the build -fPIC is passed to ld. Normally -fPIC is accepted by GNU BFD ld but results in a bogus DT_AUXILIARY entry in the output (from another port):

% readelf -d work/.build/target/lib/libossmix.so

Dynamic section at offset 0x5000 contains 15 entries:
  Tag                Type                  Name/Value
 0x000000007ffffffd AUXILIARY            Auxiliary library: [PIC]
...

In the case of archivers/lua51-zlib this is masked by another issue: ld's -O flag requires an optimization level:

       -O level
           If level is a numeric values greater than zero ld optimizes the
           output.  This might take significantly longer and therefore
           probably should only be enabled for the final binary.

For this port ld is invoked as:

ld -O -fPIC -shared -L/usr/local/lib/lua/5.1 -L/usr/local/lib/lua/5.1 lua_zlib.o -lz -lm -o zlib.so

it appears that this is interpreted as optimize with level "-fPIC" which is not a numeric value grater than zero, and so just silently consumed (with BFD ld), making "-O -fPIC" a no-op. If I remove the -O flag zlib.so indeed gains the bogus DT_AUXILIARY PIC entry.

For lld the build fails with:

ld -O -fPIC -shared -L/usr/local/lib/lua/5.1 -L/usr/local/lib/lua/5.1 lua_zlib.o -lz -lm -o zlib.so
ld: error: unknown argument: -fPIC
ld: error: -O: number expected, but got
ld: error: unable to find library -lz
ld: error: unable to find library -lm

(the missing libraries with lld are a separate issue which I hope can be addressed in ports infrastructure)
Comment 1 Sir l33tname 2017-08-27 15:34:19 UTC
Created attachment 185819 [details]
patch LD flags

This patch should address your first two points. I built it 
and tested it and it seems to work fine when you link it with out -fPIC.

But I didn't found a quick way to test it with ldd so I need either feedback from you how to setup a ldd test env. Or some one who is confirming that this patch solves the problem.
Comment 2 Ed Maste freebsd_committer freebsd_triage 2017-08-29 17:27:32 UTC
(In reply to Sir l33tname from comment #1)
Thanks for the patch - I will review and test it here.

For the majority of ports that invoke the compiler driver (${CC}) to link the easiest way to test with lld is to add -fuse-ld=lld to LDFLAGS. For ports that invoke the linker directly (as this one appears to) setting LD=ld.lld in the environment is probably sufficient.

You can also make /usr/bin/ld a link to one of ld.bfd and ld.lld to select the default linker. In -current /usr/bin/ld is a hard link to ld.bfd by default, or a symlink to ld.lld if WITH_LLD_IS_LD is set.

You can verify that lld was used for linking by checking the .comment section. lld inserts a "Linker: ' comment with a version string. For example:

nuc% readelf --string-dump=.comment /bin/ls

String dump of section '.comment':
  [     1]  $FreeBSD$
  [     b]  FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
  [    5d]  FreeBSD clang version 5.0.0 (trunk 306956) (based on LLVM 5.0.0svn)
  [    a1]  Linker: LLD 5.0.0 (FreeBSD 311606)
Comment 3 Ed Maste freebsd_committer freebsd_triage 2017-08-29 17:34:43 UTC
Comment on attachment 185819 [details]
patch LD flags

The patch LGTM (although -fPIC should also be removed from LNX_LDFLAGS upstream).

If I change the BSD_LDFLAGS to
BSD_LDFLAGS = -O2 -shared $(LUALIBDIR) -L/usr/lib
the port builds for me with lld as the linker, although we might want to consider this further still (as 15 or so ports share the same issue).

I think your current patch is suitable for committing and posting upstream though.
Comment 4 Ed Maste freebsd_committer freebsd_triage 2017-11-21 01:56:09 UTC
It looks like the attached patch was included in the update in r452729

The port still fails to build when ld is lld, with:

ld: error: unable to find library -lz
ld: error: unable to find library -lm

because lld does not include a compiled-in search path. I managed to build the port by changing the Makefile to specify

BSD_LDFLAGS = -O2 -shared $(LUALIBDIR) -L/usr/lib
Comment 5 commit-hook freebsd_committer freebsd_triage 2017-12-20 01:58:55 UTC
A commit references this bug:

Author: emaste
Date: Wed Dec 20 01:58:39 UTC 2017
New revision: 456771
URL: https://svnweb.freebsd.org/changeset/ports/456771

Log:
  archivers/lua51-zlib: set LLD_UNSAFE

  The port invokes the linker directly (not via the compiler driver).
  lld does not include any default search paths (unlike the GNU BFD linker
  so cannot find libraries specified as e.g. -lm or -lz.

  This can be addressed by invoking the linker via the compiler driver,
  or by specifying the search path explicitly with -L. For now just avoid
  using lld to link the port.

  PR:		214864, 221808
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/archivers/lua51-zlib/Makefile
Comment 6 Walter Schwarzenfeld freebsd_triage 2018-02-07 18:21:53 UTC
Forgot to close (or is it not fixed?).
Comment 7 Sir l33tname 2018-02-08 21:04:34 UTC
(In reply to w.schwarzenfeld from comment #6)
I thinks it's not really working but since I don't have a setup to test it and no idea how it should work I'm not really sure.
Comment 8 Ed Maste freebsd_committer freebsd_triage 2018-03-27 20:29:40 UTC
(In reply to Sir l33tname from comment #7)
FreeBSD 11 and later has lld available as /usr/bin/ld.lld. You can temporarily use it via `ln -fs ld.lld /usr/bin/ld`.

Via tobik@ in ports r465725, BINARY_ALIAS=ld=ld.bfd may be an effective workaround if LLD_UNSAFE does not work.