Bug 226872 - lang/ghc: fails to build with lld as /usr/bin/ld
Summary: lang/ghc: fails to build with lld as /usr/bin/ld
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Ed Maste
URL: https://reviews.llvm.org/D46623
Keywords:
Depends on:
Blocks: 214864
  Show dependency treegraph
 
Reported: 2018-03-23 13:04 UTC by Ed Maste
Modified: 2018-10-31 18:49 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (haskell)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer freebsd_triage 2018-03-23 13:04:00 UTC
-CURRENT will soon be switching to LLVM's lld linker as /usr/bin/ld, and ghc fails to build in this case. Build log tail:
---
<no location info>:
    Warning: Couldn't figure out linker information!
             Make sure you're using GNU ld, GNU gold or the built in OS X linker, etc.
Segmentation fault (core dumped)
*** Error code 139
---
Log from exp-run in PR 214864, http://package18.nyi.freebsd.org/data/headamd64PR214864-default/2018-03-21_17h43m11s/logs/errors/ghc-8.0.2_3.log

The common technique of specifying linker via LD / LDFLAGS does not work (although this probably isn't surprising for non-C ports).
Comment 1 Gleb Popov freebsd_committer freebsd_triage 2018-03-23 14:06:22 UTC
I'm aware of this problem, someone asked on IRC about this. The problem is that even simple hello world Haskell programs linked with LLD are segfaulting.

I saw some LLD support in recent 8.4.1 release, so the current plan is to upgrade port to the latest version and then see how it works with LLD.
Comment 2 Ed Maste freebsd_committer freebsd_triage 2018-03-27 20:43:52 UTC
Via tobik@ in ports r465725, BINARY_ALIAS=ld=ld.bfd may be an effective workaround if LLD_UNSAFE does not work.
Comment 3 Ed Maste freebsd_committer freebsd_triage 2018-03-29 17:02:35 UTC
(In reply to arrowd from comment #1)
> I'm aware of this problem, someone asked on IRC about this. The problem is that > even simple hello world Haskell programs linked with LLD are segfaulting.

Can we configure ghc to invoke ld.bfd by default perhaps?

It appears that building the compiler itself picks up LD=/usr/bin/ld.bfd as it should, but the built ghc (somewhat unsurprisingly) defaults to invoking ld.

Of course the best case scenario is if ghc can just use lld as the linker (and perhaps achieve a 10x speedup).
Comment 4 Ed Maste freebsd_committer freebsd_triage 2018-04-02 15:18:54 UTC
For reference, 558 dependent ports are skipped as a result of this issue when using lld as the system linker (http://package18.nyi.freebsd.org/build.html?mastername=headamd64PR214864-default&build=2018-03-21_17h43m11s)
Comment 6 Gleb Popov freebsd_committer freebsd_triage 2018-04-07 18:29:27 UTC
I've posted a preliminary diff to update GHC to 8.4.1 [1]. I repackaged bootstrap compiler changed both to use ld.bfd as their linker. However, this isn't ready to be committed yet, because ~400 hs* ports also need updating. And then at least 3 exp runs.

To make the current port use ld.bfd try a change I made on lang/ghc/Makefile:100 and lang/ghc/Makefile:41, that is removing --with-ld=${LD} configure argument and passing LD env var.

[1] https://reviews.freebsd.org/D15005
Comment 7 Ed Maste freebsd_committer freebsd_triage 2018-04-10 19:15:22 UTC
(In reply to arrowd from comment #6)

When LLD_UNSAFE is set LD should be set to /usr/bin/ld.bfd so the --with-ld=${LD} should still be acceptable.
Comment 8 Gleb Popov freebsd_committer freebsd_triage 2018-04-10 19:43:56 UTC
(In reply to Ed Maste from comment #7)
--with-ld is not a support ./configure option anymore in GHC, that's why it isn't picked up.
Comment 9 Ed Maste freebsd_committer freebsd_triage 2018-05-08 00:25:36 UTC
Grab; this is an lld issue.
Comment 10 Ed Maste freebsd_committer freebsd_triage 2018-05-08 01:13:52 UTC
LLVM bug report at https://bugs.llvm.org/show_bug.cgi?id=37361
Comment 11 Gleb Popov freebsd_committer freebsd_triage 2018-05-08 05:48:13 UTC
(In reply to Ed Maste from comment #9)
Yep, and IIRC, it is present even when compiling hello world written in haskell. You don't actually need to build GHC itself, just configure bootstrap compiler with lld and use it to compile helloworld.hs.

Anyway, thanks for digging into this and filing the bug. I still messing with GHC update and have no change to look at this linker problem.
Comment 12 Ed Maste freebsd_committer freebsd_triage 2018-05-08 20:20:54 UTC
I have a workaround (have rtld ignore these bogus PT_ELF segments) at https://github.com/emaste/freebsd/commit/df17f86fc178f8cb9d1d84b064d35977cd36c638

I have that patch on my laptop, and I applied your 8.4.2 ghc patch to my ports tree, and lang/ghc builds.
Comment 13 Ed Maste freebsd_committer freebsd_triage 2018-05-08 20:21:36 UTC
(In reply to Ed Maste from comment #12)
Err, bogus PT_NOTE segments
Comment 14 Gleb Popov freebsd_committer freebsd_triage 2018-05-08 20:46:25 UTC
(In reply to Ed Maste from comment #13)
That most likely means that it indeed the source of the problem. If you wish, you can try building devel/stack with built GHC to be sure. Stack is pretty heavy dependency-wise, so it is a good candidate to test toolchain.
Comment 15 Ed Maste freebsd_committer freebsd_triage 2018-05-08 22:05:33 UTC
(In reply to arrowd from comment #14)
Thank you.

Given a few other things in progress I suspect it will be after your PR227968 update to 8.4.2 gets committed that I get back to this. Then I'll try a local build of devel/stack and ask for another lld-is-ld exp-run (both with with either my rtld workaround or lld bug fix, if it then exists.

Finally, commit the workaround or bugfix, and switch /usr/bin/ld to lld.
Comment 16 commit-hook freebsd_committer freebsd_triage 2018-05-09 11:17:21 UTC
A commit references this bug:

Author: emaste
Date: Wed May  9 11:17:02 UTC 2018
New revision: 333401
URL: https://svnweb.freebsd.org/changeset/base/333401

Log:
  lld: Omit PT_NOTE for SHT_NOTE without SHF_ALLOC

  A non-alloc note section should not have a PT_NOTE program header.

  Found while linking ghc (Haskell compiler) with lld on FreeBSD.  Haskell
  emits a .debug-ghc-link-info note section (as the name suggests, it
  contains link info) as a SHT_NOTE section without SHF_ALLOC set.

  For this case ld.bfd does not emit a PT_NOTE segment for
  .debug-ghc-link-info.  lld previously emitted a PT_NOTE with p_vaddr = 0
  and FreeBSD's rtld segfaulted when trying to parse a note at address 0.

  LLVM PR:	https://llvm.org/pr37361
  LLVM review:	https://reviews.llvm.org/D46623

  PR:		226872
  Reviewed by:	dim
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/contrib/llvm/tools/lld/ELF/Writer.cpp
  head/lib/clang/include/lld/Common/Version.inc
Comment 17 commit-hook freebsd_committer freebsd_triage 2018-07-16 05:36:02 UTC
A commit references this bug:

Author: dim
Date: Mon Jul 16 05:35:40 UTC 2018
New revision: 336331
URL: https://svnweb.freebsd.org/changeset/base/336331

Log:
  MFC r333401 (by emaste):

  lld: Omit PT_NOTE for SHT_NOTE without SHF_ALLOC

  A non-alloc note section should not have a PT_NOTE program header.

  Found while linking ghc (Haskell compiler) with lld on FreeBSD.  Haskell
  emits a .debug-ghc-link-info note section (as the name suggests, it
  contains link info) as a SHT_NOTE section without SHF_ALLOC set.

  For this case ld.bfd does not emit a PT_NOTE segment for
  .debug-ghc-link-info.  lld previously emitted a PT_NOTE with p_vaddr = 0
  and FreeBSD's rtld segfaulted when trying to parse a note at address 0.

  LLVM PR:	https://llvm.org/pr37361
  LLVM review:	https://reviews.llvm.org/D46623

  PR:		226872
  Reviewed by:	dim
  Sponsored by:	The FreeBSD Foundation

Changes:
_U  stable/11/
  stable/11/contrib/llvm/tools/lld/ELF/Writer.cpp
  stable/11/lib/clang/include/lld/Common/Version.inc
Comment 18 commit-hook freebsd_committer freebsd_triage 2018-10-26 21:20:33 UTC
A commit references this bug:

Author: dim
Date: Fri Oct 26 21:20:06 UTC 2018
New revision: 483054
URL: https://svnweb.freebsd.org/changeset/ports/483054

Log:
  Add all patches from base llvm/clang/lld/lldb 6.0 to devel/llvm60

  This adds all the patches that were applied in the past to head, under
  contrib/llvm.  After these, there only minimal diffs left between the
  port sources and the base sources.

  Most of these remaining diffs are due to #ifdef shortcuts in the base
  sources, because we don't compile certain features in.  Other diffs are
  because the port has applied a few changes that we don't have in base.

  While here, use Makefile.LICENSE from the devel/llvm-devel port.

  Approved by:	brooks (maintainer)
  Reviewed by:	brooks
  PR:		212343, 225128, 225471, 226388, 226658, 226872, 229050, 230444, 230604, 231355
  MFH:		2018Q4
  Differential Revision: https://reviews.freebsd.org/D17702

Changes:
  head/devel/llvm60/Makefile
  head/devel/llvm60/files/clang/patch-head-r331066.diff
  head/devel/llvm60/files/clang/patch-head-r336227.diff
  head/devel/llvm60/files/clang/patch-head-r338697.diff
  head/devel/llvm60/files/clang/patch-head-r339019.diff
  head/devel/llvm60/files/lld/
  head/devel/llvm60/files/lld/patch-head-r331731.diff
  head/devel/llvm60/files/lld/patch-head-r333401.diff
  head/devel/llvm60/files/lld/patch-head-r336664.diff
  head/devel/llvm60/files/lld/patch-head-r336972.diff
  head/devel/llvm60/files/lld/patch-head-r337282.diff
  head/devel/llvm60/files/lld/patch-head-r338251.diff
  head/devel/llvm60/files/lld/patch-head-r338682.diff
  head/devel/llvm60/files/lld/patch-head-r339013.diff
  head/devel/llvm60/files/lld/patch-head-r339304.diff
  head/devel/llvm60/files/lldb/
  head/devel/llvm60/files/lldb/patch-head-r332849.diff
  head/devel/llvm60/files/lldb/patch-head-r332965.diff
  head/devel/llvm60/files/patch-head-r308867.diff
  head/devel/llvm60/files/patch-head-r330686.diff
  head/devel/llvm60/files/patch-head-r331065.diff
  head/devel/llvm60/files/patch-head-r331366.diff
  head/devel/llvm60/files/patch-head-r336969.diff
  head/devel/llvm60/files/patch-head-r336970.diff
  head/devel/llvm60/files/patch-head-r337615.diff
  head/devel/llvm60/files/patch-head-r338689.diff
Comment 19 commit-hook freebsd_committer freebsd_triage 2018-10-31 18:49:44 UTC
A commit references this bug:

Author: dim
Date: Wed Oct 31 18:49:07 UTC 2018
New revision: 483602
URL: https://svnweb.freebsd.org/changeset/ports/483602

Log:
  MFH: r481120

  Update to a new snapshot.

  Update LICENSE data per mailing list feedback and move to a seperate
  Makefile.LICENSE for use by other llvm ports.

  MFH: r483054

  Add all patches from base llvm/clang/lld/lldb 6.0 to devel/llvm60

  This adds all the patches that were applied in the past to head, under
  contrib/llvm.  After these, there only minimal diffs left between the
  port sources and the base sources.

  Most of these remaining diffs are due to #ifdef shortcuts in the base
  sources, because we don't compile certain features in.  Other diffs are
  because the port has applied a few changes that we don't have in base.

  While here, use Makefile.LICENSE from the devel/llvm-devel port.

  Approved by:	portmgr (miwi)
  Reviewed by:	brooks
  PR:		212343, 225128, 225471, 226388, 226658, 226872, 229050, 230444, 230604, 231355
  Differential Revision: https://reviews.freebsd.org/D17702

Changes:
_U  branches/2018Q4/
  branches/2018Q4/devel/llvm-devel/Makefile
  branches/2018Q4/devel/llvm-devel/Makefile.LICENSE
  branches/2018Q4/devel/llvm-devel/Makefile.snapshot
  branches/2018Q4/devel/llvm-devel/distinfo
  branches/2018Q4/devel/llvm-devel/files/lldb-patch-tools_lldb_source_Plugins_Process_FreeBSD_ProcessFreeBSD.cpp
  branches/2018Q4/devel/llvm-devel/pkg-plist
  branches/2018Q4/devel/llvm60/Makefile
  branches/2018Q4/devel/llvm60/files/clang/patch-head-r331066.diff
  branches/2018Q4/devel/llvm60/files/clang/patch-head-r336227.diff
  branches/2018Q4/devel/llvm60/files/clang/patch-head-r338697.diff
  branches/2018Q4/devel/llvm60/files/clang/patch-head-r339019.diff
  branches/2018Q4/devel/llvm60/files/lld/
  branches/2018Q4/devel/llvm60/files/lldb/
  branches/2018Q4/devel/llvm60/files/patch-head-r308867.diff
  branches/2018Q4/devel/llvm60/files/patch-head-r330686.diff
  branches/2018Q4/devel/llvm60/files/patch-head-r331065.diff
  branches/2018Q4/devel/llvm60/files/patch-head-r331366.diff
  branches/2018Q4/devel/llvm60/files/patch-head-r336969.diff
  branches/2018Q4/devel/llvm60/files/patch-head-r336970.diff
  branches/2018Q4/devel/llvm60/files/patch-head-r337615.diff
  branches/2018Q4/devel/llvm60/files/patch-head-r338689.diff