Bug 217189

Summary: emulators/qemu-user-static: lld 4.0 crashes aarch64 emulator but works fine on ThunderX
Product: Ports & Packages Reporter: Jan Beich <jbeich>
Component: Individual Port(s)Assignee: Sean Bruno <sbruno>
Status: Closed FIXED    
Severity: Affects Only Me CC: bdrewery, emaste, jonc, mikael, pi, swills, wolfgang.meyer
Priority: --- Keywords: needs-patch
Version: LatestFlags: bugzilla: maintainer-feedback? (sbruno)
Hardware: arm64   
OS: Any   
Bug Depends on:    
Bug Blocks: 216008    

Description Jan Beich freebsd_committer freebsd_triage 2017-02-18 06:42:49 UTC
Soon /head is going to upgrade lld to 4.0 which no longer works, so poudriere is blocked by ports-mgmt/pkg failing to build. Downgrading to lld to 3.9 or using GNU binutils appears to help. I can reproduce on 11.0 aarch64 + lld 4.0 on qemu-user-static but not on ref12-aarch64.

# projects/clang400-import branch
$ echo 'int main() {}' >a.c
$ cc a.c
/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-506a08c/tcg/tcg.c:2017: tcg fatal error
cc: error: unable to execute command: Abort trap
cc: error: linker command failed due to signal (use -v to see invocation)

# On earlier releases
$ pkg install llvm40
$ clang40 -fuse-ld=lld a.c
/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-506a08c/tcg/tcg.c:2017: tcg fatal error
clang-4.0: error: unable to execute command: Abort trap
Comment 1 Jan Beich freebsd_committer freebsd_triage 2017-03-15 15:30:10 UTC
*** Bug 217753 has been marked as a duplicate of this bug. ***
Comment 2 Bryan Drewery freebsd_committer freebsd_triage 2017-06-01 23:55:30 UTC
Something missing is that native-xtools is not building lld as /usr/bin/ld
for aarch64.  I am adding that support now.  Then poudriere jail -x will
have a native lld for use rather than hitting the qemu bug.
Comment 3 commit-hook freebsd_committer freebsd_triage 2017-06-02 00:58:50 UTC
A commit references this bug:

Author: bdrewery
Date: Fri Jun  2 00:58:00 UTC 2017
New revision: 319485
URL: https://svnweb.freebsd.org/changeset/base/319485

Log:
  native-xtools: Connect lld.

  This will ensure that aarch64 gets a working native /usr/bin/ld rather
  than requiring the aarch64-binutils hack in Poudriere, or emulating
  the aarch64 lld.

  PR:		217189
  Reported by:	swills, jbeich

Changes:
  head/Makefile.inc1
Comment 4 Bryan Drewery freebsd_committer freebsd_triage 2017-06-02 00:59:52 UTC
native-xtools will now install a proper native lld binary.  I'll leave
this bug for the qemu issue though as it is still relevant.
Comment 5 Jan Beich freebsd_committer freebsd_triage 2017-07-08 10:25:58 UTC
After ports r445254 the error changed:

# Tested on 11.1-RC1
$ echo 'int main() {}' >a.c
$ cc a.c
qemu: unhandled CPU exception 0x10005 - aborting
PC=0000004002208e70  SP=0000000869705e50
X00=00000040021ec498 X01=0000000000018cb7 X02=0000000000000000 X03=0000000000000400
X04=00000000ffffffff X05=0000000000000000 X06=0000000000000000 X07=0000000000000000
X08=0000000000000000 X09=8000000000000000 X10=0000000000000002 X11=0000000000000000
X12=0000000000000004 X13=0000000000000427 X14=0000000869705b00 X15=0000004002406ff8
X16=00000040021cb250 X17=0000004002208e48 X18=0000000000000000 X19=00000040021ec498
X20=0000004002ea1c08 X21=0000000080000010 X22=0000000000000000 X23=0000000000000000
X24=0000000000000000 X25=0000000080000010 X26=0000000000000000 X27=0000000000000000
X28=0000000000000000 X29=0000000869705e60 X30=000000400217a754
PSTATE=80000000 N--- EL0t
cc: error: unable to execute command: Abort trap
cc: error: linker command failed due to signal (use -v to see invocation)
Comment 7 Jan Beich freebsd_committer freebsd_triage 2017-07-08 16:09:07 UTC
(In reply to mikael.urankar from comment #6)
## aarch64 after
$ cc a.c

## armv6
# before
$ cc -fuse-ld=lld a.c
/usr/bin/ld.lld: error: failed to open a.out: Unknown error -1

$ after
$ cc -fuse-ld=lld a.c
/usr/bin/ld.lld: error: failed to open a.out: Invalid argument
cc: error: linker command failed with exit code 1 (use -v to see invocation)

So, lld works fine on aarch64 where it's default linker but not on armv6 where bfd is still default.
Comment 8 Bryan Drewery freebsd_committer freebsd_triage 2017-07-08 18:22:45 UTC
(In reply to commit-hook from comment #3)
> A commit references this bug:
> 
> Author: bdrewery
> Date: Fri Jun  2 00:58:00 UTC 2017
> New revision: 319485
> URL: https://svnweb.freebsd.org/changeset/base/319485
> 
> Log:
>   native-xtools: Connect lld.
> 
>   This will ensure that aarch64 gets a working native /usr/bin/ld rather
>   than requiring the aarch64-binutils hack in Poudriere, or emulating
>   the aarch64 lld.
> 
>   PR:		217189
>   Reported by:	swills, jbeich
> 
> Changes:
>   head/Makefile.inc1

After this fix, the tcg error goes away for me cross-building arm64 with QEMU
with native-xtools (poudriere jail -x) since QEMU is avoided.  With using
/usr/bin/ld (arm64 binary) via QEMU it still hits the error.
Comment 9 Mikael Urankar freebsd_committer freebsd_triage 2017-07-09 08:50:54 UTC
(In reply to Jan Beich from comment #7)
The armv6 problem should be solved with:
https://github.com/MikaelUrankar/qemu-bsd-user/commit/845d047b974f4687044f2237f0d009578c814cf2
Comment 10 Jan Beich freebsd_committer freebsd_triage 2017-07-09 12:00:05 UTC
(In reply to mikael.urankar from comment #9)
I confirm, with all 3 patches applied both aarch64 and armv6 are able to build using lld.
Comment 11 Sean Bruno freebsd_committer freebsd_triage 2017-07-11 17:35:50 UTC
(In reply to Jan Beich from comment #10)
I'll add these patches to the next release of qemu-user-static.  Doing build tests and runtime tests now.