Bug 250783 - ld segmentation fault
Summary: ld segmentation fault
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 12.1-STABLE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-toolchain (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-11-01 17:10 UTC by Nick Venenga
Modified: 2020-11-15 11:29 UTC (History)
2 users (show)

See Also:
dim: mfc-stable12+
dim: mfc-stable11+

Attachments
Log from failed build (16.24 KB, text/plain)
2020-11-01 17:10 UTC, Nick Venenga 		no flags Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Venenga 2020-11-01 17:10:28 UTC 
Created attachment 219280 [details]
Log from failed build

ld segfaults when trying to build `maturin` Python package

```
curl -LO https://files.pythonhosted.org/packages/35/e1/d6a9c170f925fb8f4f97d0bed27e2c43a5ba73d7f88a10ed63cd3f8f7968/maturin-0.8.3.tar.gz
mkdir maturin
cd mautrin
tar xf ../maturin-0.8.3.tar.gz
cargo rustc --bin maturin --manifest-path Cargo.toml -- -C link-arg=-s
```

```
(gdb) bt
#0  0x0000000000f99607 in operator() ()
    at /usr/src/contrib/llvm/tools/lld/ELF/MarkLive.cpp:207
#1  callback_fn<(lambda at /usr/src/contrib/llvm/tools/lld/ELF/MarkLive.cpp:194:18)>(void) () at /usr/src/contrib/llvm/include/llvm/ADT/STLExtras.h:126
#2  0x0000000000f96363 in forEachSuccessor<llvm::object::ELFType<llvm::support::little, true> > () at /usr/src/contrib/llvm/tools/lld/ELF/MarkLive.cpp:97
#3  doGcSections<llvm::object::ELFType<llvm::support::little, true> > ()
    at /usr/src/contrib/llvm/tools/lld/ELF/MarkLive.cpp:264
#4  markLive<llvm::object::ELFType<llvm::support::little, true> > ()
    at /usr/src/contrib/llvm/tools/lld/ELF/MarkLive.cpp:312
#5  0x0000000000f28de6 in link<llvm::object::ELFType<llvm::support::little, true> > () at /usr/src/contrib/llvm/tools/lld/ELF/Driver.cpp:1636
#6  0x0000000000f1ed26 in main ()
    at /usr/src/contrib/llvm/tools/lld/ELF/Driver.cpp:443
#7  0x0000000000f1d619 in link ()
    at /usr/src/contrib/llvm/tools/lld/ELF/Driver.cpp:103
#8  0x000000000106792b in main ()
    at /usr/src/contrib/llvm/tools/lld/tools/lld/lld.cpp:129
```

I'm not very well-versed in C++/rust but if I naively replace /usr/bin/ld with /usr/local/bin/ld from gcc9 the build seems to work
I'm not sure if a core dump is helpful here, https://files.nickv.me/maturin-ld.lld.core.xz

This is on a fresh VM from http://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/ (with git, gcc9, rust, and python38)
Comment 1 Nick Venenga 2020-11-01 17:11:06 UTC 
Here is a related conversation https://github.com/PyO3/maturin/issues/363
Comment 2 Dimitry Andric freebsd_committer freebsd_triage 2020-11-01 22:01:59 UTC 
If this happens on 12.1, can you try the same on a more recent 12.2-RELEASE or snapshot image, or even on 13-CURRENT? 12.1 has lld 8.0.1 which is rather old now. Alternatively, you can install the llvm 10 or llvm 11 packages, and try to use the lld from those.
Comment 3 Nick Venenga 2020-11-01 22:21:05 UTC 
error: linking with `cc` failed: exit code: 254
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,--eh-frame-hdr" "-L" "/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib" "/root/maturin/target/debug/deps/maturin-fe53e6c49e774afc.1agzv0ps7sudcbsb.rcgu.o" "/root/maturin/target/debug/deps/maturin-fe53e6c49e774afc.1cu2a2eew775gbht.rcgu.o" "/root/matur"
  = note: Stack dump:
          0.    Program arguments: /usr/bin/ld -pie --eh-frame-hdr -dynamic-linker /libexec/ld-elf.so.1 --hash-style=both --enable-new-dtags -o /root/maturin/target/debug/deps/maturin-fe53e6c49e774afc /usr/lib/Scrt1.o /usr/lib/crti.o /usr/lib/crtbeginS.o -L/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib -L/root/ma 
          #0 0x0000000001b362ae PrintStackTrace /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:564:13
          #1 0x0000000001b34525 RunSignalHandlers /usr/src/contrib/llvm-project/llvm/lib/Support/Signals.cpp:69:18
          #2 0x0000000001b36e40 SignalHandler /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
          #3 0x0000000002b9eb70 handle_signal /usr/src/lib/libthr/thread/thr_sig.c:0:3
          cc: error: unable to execute command: Segmentation fault (core dumped)
          cc: error: linker command failed due to signal (use -v to see invocation)


error: aborting due to previous error

error: could not compile `maturin`.

To learn more, run the command again with --verbose.
root@freebsd:~/maturin # ld --version
LLD 10.0.1 (FreeBSD llvmorg-10.0.1-0-gef32c611aa2-1200012) (compatible with GNU linkers)
root@freebsd:~/maturin # uname -a
FreeBSD freebsd 12.2-STABLE FreeBSD 12.2-STABLE r367116 GENERIC  amd64
Comment 4 Nick Venenga 2020-11-01 22:22:55 UTC 
(gdb) bt
#0  0x000000000125e9b2 in enqueue ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:194
#1  resolveReloc<llvm::object::Elf_Rel_Impl<llvm::object::ELFType<llvm::support::little, true>, true> const> ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:108
#2  0x000000000125e560 in mark ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:279
#3  0x0000000001259e3c in run ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:242
#4  markLive<llvm::object::ELFType<llvm::support::little, true> > ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:378
#5  0x00000000011d56c4 in link<llvm::object::ELFType<llvm::support::little, true> > () at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:1970
#6  0x00000000011caa47 in main ()
    at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:514
#7  0x00000000011c9055 in link ()
    at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:111
#8  0x0000000001364145 in main ()
    at /usr/src/contrib/llvm-project/lld/tools/lld/lld.cpp:151
Comment 5 Nick Venenga 2020-11-01 23:25:06 UTC 
I tried the FreeBSD 13.0 VM

# cargo rustc --bin maturin --manifest-path Cargo.toml \-- -C link-arg=-s
   Compiling maturin v0.8.3 (/root/maturin)                   
error: linking with `cc` failed: exit code: 254
  |                              
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,--eh-frame-hdr" "-L" "/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib" "/root/maturin/target/debug/deps/maturin-fe53e6c49e774afc.1agzv0ps7sudcbsb.rcgu.o" "/root/maturin/target/debug/deps/maturin-fe53e6c49e774afc.1cu2a2eew775gbht.rcgu.o" "/root/matur"
  = note: PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace.
          Stack dump:                                                                                                                                          
          0.    Program arguments: /usr/bin/ld -pie --eh-frame-hdr -dynamic-linker /libexec/ld-elf.so.1 --hash-style=both --enable-new-dtags -o /root/maturin/target/debug/deps/maturin-fe53e6c49e774afc /usr/lib/Scrt1.o /usr/lib/crti.o /usr/lib/crtbeginS.o -L/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib -L/root/ma 
          #0 0x0000000001d1f23e PrintStackTrace /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:564:13
          #1 0x0000000001d1d5a5 RunSignalHandlers /usr/src/contrib/llvm-project/llvm/lib/Support/Signals.cpp:69:18
          #2 0x0000000001d1f930 SignalHandler /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
          #3 0x0000000803561b90 handle_signal /usr/src/lib/libthr/thread/thr_sig.c:0:3
          cc: error: unable to execute command: Segmentation fault (core dumped)
          cc: error: linker command failed due to signal (use -v to see invocation)


error: aborting due to previous error

error: could not compile `maturin`.


Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000000000151245d in enqueue ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:194
194     /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp: No such file or directory.
[Current thread is 1 (LWP 100106)]
(gdb) bt
#0  0x000000000151245d in enqueue ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:194
#1  resolveReloc<llvm::object::Elf_Rel_Impl<llvm::object::ELFType<llvm::support::little, true>, true> const> ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:108
#2  0x0000000001511eb0 in mark ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:279
#3  0x000000000150d6fc in run ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:242
#4  markLive<llvm::object::ELFType<llvm::support::little, true> > ()
    at /usr/src/contrib/llvm-project/lld/ELF/MarkLive.cpp:379
#5  0x000000000147dcf8 in link<llvm::object::ELFType<llvm::support::little, true> > () at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:2114
#6  0x00000000014715bc in main ()
    at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:545
#7  0x000000000146fa7d in link ()
    at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:114
#8  0x000000000161aa45 in main ()
    at /usr/src/contrib/llvm-project/lld/tools/lld/lld.cpp:146
(gdb) quit
root@freebsd:~/maturin # uname -a
FreeBSD freebsd 13.0-CURRENT FreeBSD 13.0-CURRENT #0 b9403d7aae8-c254071(main): Thu Oct 29 08:06:03 UTC 2020     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
root@freebsd:~/maturin # ld --version
LLD 11.0.0 (FreeBSD llvmorg-11.0.0-0-g176249bd673-1300007) (compatible with GNU linkers)
Comment 6 Dimitry Andric freebsd_committer freebsd_triage 2020-11-02 12:38:11 UTC 
Ok, to have something to submit to upstream, we need a reproduction case, and lld has an option for this, which produces a tarball:

  --reproduce=<value>     Write a tar file containing input files and command line options to reproduce link

So can you repeat the link operation you did on your FreeBSD 13.0 VM with lld 11.0.0, and add an additional flag to the "linking with cc" line, e.g.:

  -Wl,--reproduce=bug250783.tar

If you re-run the link, it should produce a bug250783.tar file with everything in it to be able reproduce the failure. Please compress this with xz and attach it here.
Comment 7 Nick Venenga 2020-11-04 01:07:59 UTC 
The file was 70mb so I uploaded it to my website here https://files.nickv.me/bug250783.tar.xz
Comment 8 Dimitry Andric freebsd_committer freebsd_triage 2020-11-04 14:16:49 UTC 
(In reply to Nick Venenga from comment #7)
Reported upstream as https://bugs.llvm.org/show_bug.cgi?id=48071, as this also occurs with the latest llvm 12.0.0 builds as of today.
Comment 9 commit-hook freebsd_committer freebsd_triage 2020-11-04 14:21:57 UTC 
A commit references this bug:

Author: dim
Date: Wed Nov  4 14:21:44 UTC 2020
New revision: 554081
URL: https://svnweb.freebsd.org/changeset/ports/554081

Log:
  devel/binutils: Apply upstream patch to fix "undefined symbol:
  bswap_identity_64" and similar errors, when building devel/binutils in
  certain environments.

  This has been sitting in a bug for almost a year now without any action,
  which is why I am committing it now.

  MFH:		2020Q4
  PR:		250783

Changes:
  head/devel/binutils/Makefile
  head/devel/binutils/files/patch-libctf_swap.h
Comment 10 Dimitry Andric freebsd_committer freebsd_triage 2020-11-04 14:24:43 UTC 
(In reply to commit-hook from comment #9)
Oops, wrong PR number. That commit was supposed to be for bug 242053!
Comment 11 commit-hook freebsd_committer freebsd_triage 2020-11-12 19:25:59 UTC 
A commit references this bug:

Author: dim
Date: Thu Nov 12 19:25:31 UTC 2020
New revision: 367623
URL: https://svnweb.freebsd.org/changeset/base/367623

Log:
  Merge commit 8df4e6094 from llvm git (by Fangrui Song):

    [ELF] Don't consider SHF_ALLOC ".debug*" sections debug sections

    Fixes PR48071

    * The Rust compiler produces SHF_ALLOC `.debug_gdb_scripts` (which
      normally does not have the flag)
    * `.debug_gdb_scripts` sections are removed from `inputSections` due
      to --strip-debug/--strip-all
    * When processing --gc-sections, pieces of a SHF_MERGE section can be
      marked live separately

    `=>` segfault when marking liveness of a `.debug_gdb_scripts` which
    is not split into pieces (because it is not in `inputSections`)

    This patch circumvents the problem by not treating SHF_ALLOC
    ".debug*" as debug sections (to prevent --strip-debug's stripping)
    (which is still useful on its own).

    Reviewed By: grimar

    Differential Revision: https://reviews.llvm.org/D91291

  This should fix lld segfaulting when linking the rust-based parts of the
  devel/py-maturin port.

  Reported by:	Nick Venenga <nijave@gmail.com>
  PR:		250783
  MFC after:	3 days

Changes:
  head/contrib/llvm-project/lld/ELF/InputSection.h
Comment 12 commit-hook freebsd_committer freebsd_triage 2020-11-15 11:28:23 UTC 
A commit references this bug:

Author: dim
Date: Sun Nov 15 11:28:03 UTC 2020
New revision: 367704
URL: https://svnweb.freebsd.org/changeset/base/367704

Log:
  MFC r367623:

  Merge commit 8df4e6094 from llvm git (by Fangrui Song):

    [ELF] Don't consider SHF_ALLOC ".debug*" sections debug sections

    Fixes PR48071

    * The Rust compiler produces SHF_ALLOC `.debug_gdb_scripts` (which
      normally does not have the flag)
    * `.debug_gdb_scripts` sections are removed from `inputSections` due
      to --strip-debug/--strip-all
    * When processing --gc-sections, pieces of a SHF_MERGE section can be
      marked live separately

    `=>` segfault when marking liveness of a `.debug_gdb_scripts` which
    is not split into pieces (because it is not in `inputSections`)

    This patch circumvents the problem by not treating SHF_ALLOC
    ".debug*" as debug sections (to prevent --strip-debug's stripping)
    (which is still useful on its own).

    Reviewed By: grimar

    Differential Revision: https://reviews.llvm.org/D91291

  This should fix lld segfaulting when linking the rust-based parts of the
  devel/py-maturin port.

  Reported by:	Nick Venenga <nijave@gmail.com>
  PR:		250783

Changes:
_U  stable/11/
  stable/11/contrib/llvm-project/lld/ELF/InputSection.h
_U  stable/12/
  stable/12/contrib/llvm-project/lld/ELF/InputSection.h