Bug 223009

Summary: lang/rust: fails to build on aarch64
Product: Ports & Packages Reporter: Jan Beich <jbeich>
Component: Individual Port(s)Assignee: FreeBSD Rust Team <rust>
Status: Closed FIXED    
Severity: Affects Only Me CC: allanjude, clusteradm, dumbbell, emaste, freebsd-arm, linimon, marklmi26-fbsd, riggs, rust, sid
Priority: --- Keywords: needs-patch, needs-qa
Version: LatestFlags: dumbbell: maintainer-feedback+
dumbbell: maintainer-feedback+
Hardware: arm64   
OS: Any   
Bug Depends on:    
Bug Blocks: 201763    

Description Jan Beich freebsd_committer freebsd_triage 2017-10-14 17:20:14 UTC
running: build/aarch64-unknown-freebsd/stage0/bin/cargo build --manifest-path src/bootstrap/Cargo.toml --verbose --frozen
error: process didn't exit successfully: `build/aarch64-unknown-freebsd/stage0/bin/rustc -vV` (exit code: 1)
--- stdout
build/aarch64-unknown-freebsd/stage0/lib/../lib/librustc_llvm-21c27decc38d9d13.so: Unhandled relocation 1031

--- stderr
Fatal error

http://thunderx1.nyi.freebsd.org/data/110arm64-default/452015/logs/errors/rust-1.20.0.log
http://thunderx1.nyi.freebsd.org/data/110arm64-quarterly/451997/logs/errors/rust-1.19.0_2.log
http://thunderx1.nyi.freebsd.org/data/latest-per-pkg/rust/
Comment 1 Bugzilla Automation freebsd_committer freebsd_triage 2017-10-14 17:20:14 UTC
Maintainer informed via mail
Comment 2 Jan Beich freebsd_committer freebsd_triage 2017-10-14 17:36:20 UTC
The issue appeared since rust-1.19.0 but doesn't affect FreeBSD 12.0.
Comment 3 Jan Beich freebsd_committer freebsd_triage 2017-10-14 17:42:23 UTC
"Unhandled relocation" comes from BFD, so LLD may not be affected.

aarch64 switched from ld.bfd to ld.lld by default since 11.1. portmgr is probably going to switch all 11.0 builders to 11.1 shortly after 2017-10-26 (11.0 EOL). Unfortunately, before that happens confirming the fix will be complicated due to bug 221185.
Comment 4 Jan Beich freebsd_committer freebsd_triage 2017-10-14 17:54:37 UTC
Jean, did you generate Rust bootstrap on a ref*-aarch64 machine? Both ref11-aarch64 and ref12-aarch64 are currently using ld.lld by default which may confuse rtld(1) on 11.0.
Comment 5 Jan Beich freebsd_committer freebsd_triage 2017-10-14 18:09:30 UTC
clusteradm, can you keep ref11-aarch64 on the oldest supported release? Too late for 11.0 but for 11.1 life upgrading to -STABLE may render binaries built on the reference machine unusable for consumption on the package builder due to lack of *forward* ABI compatibility.
Comment 6 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-10-16 08:24:18 UTC
Hi!

Yes, I use ref11-aarch64 to build the bootstrap. I can look at how to use Binutils' ld explicitely if it's available, but I don't know when as I really lack enough spare time...

Thank you for providing all this informations already!
Comment 7 Jan Beich freebsd_committer freebsd_triage 2017-10-21 09:44:07 UTC
core@, why did you (or was it security-officer) break the policy[1] and extend 3 months support to 4 months[2]? 11.1 was released on 2017-07-26, so the expected EOL for 11.0 would be 2017-10-26, not 2017-11-26 or 2017-11-30.

I guess, rust package won't be available on 11.* for 1 month more, likely because clusteradm@ carelessly upgraded ref11-aarch64 machine. firefox-56.* package (which depends on rust) will never be built for 11.* aarch64 as firefox-57.* will land around 2017-11-14.

[1] https://lists.freebsd.org/pipermail/freebsd-announce/2015-February/001624.html
[2] https://lists.freebsd.org/pipermail/freebsd-announce/2017-October/001808.html
Comment 8 Allan Jude freebsd_committer freebsd_triage 2017-10-21 15:07:31 UTC
(In reply to Jan Beich from comment #7)
It was decided to push the deadline for upgrading to 11.1 back an additional month because the announcement did not go out as early as planned. Giving people less than 2 weeks notice before the EoL was decided to be too short, especially for the first EoL under the new policy. It would not have happened that way if the announcement has gone out earlier. We apologize for the issue this has caused you.
Comment 9 Jan Beich freebsd_committer freebsd_triage 2017-10-22 03:02:16 UTC
clusteadm@, nevermind comment 5. Can you downgrade ref11-aarch64 to 11.0-RELEASE? 1100122 ABI is still supported for 1 month more. You can't upgrade before that without risking to sabotage ports/ work.

  # poudriere jail (qemu-user + native-xtools)
  $ cc -v
  FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
  Target: aarch64-unknown-freebsd11.0
  Thread model: posix
  InstalledDir: /usr/bin
  $ cc -Wl,--version
  GNU ld (GNU Binutils) 2.28

  $ echo 'int main() {}' >a.c
  $ cc a.c
  $ file a.out
  a.out: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.0 (1100122), FreeBSD-style, not stripped

vs.

  $ ssh ref11-aarch64.freebsd.org
  $ cc -v
  FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
  Target: aarch64-unknown-freebsd11.1
  Thread model: posix
  InstalledDir: /usr/bin
  $ cc -Wl,--version
  LLD 4.0.0 (FreeBSD 297347) (compatible with GNU linkers)

  $ echo 'int main() {}' >a.c
  $ cc a.c
  $ file a.out
  a.out: ELF 64-bit LSB executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.0 (1100513), FreeBSD-style, not stripped

Notice how ABI was unintentionally upgraded in ports r446815:

  $ for f in /usr/ports/distfiles/rust/*/rustc-*-aarch64-unknown-freebsd.tar.gz; do tar xkf $f; done
  $ file */rustc/bin/rustc
  rustc-1.17.0-aarch64-unknown-freebsd/rustc/bin/rustc: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.0 (1100122), FreeBSD-style, not stripped
  rustc-1.18.0-aarch64-unknown-freebsd/rustc/bin/rustc: ELF 64-bit LSB shared object, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.0 (1100513), FreeBSD-style, not stripped
  rustc-1.19.0-aarch64-unknown-freebsd/rustc/bin/rustc: ELF 64-bit LSB shared object, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.0 (1100513), FreeBSD-style, not stripped

Obviously, downgrading ref11-aarch64 to 11.0 would require starting over from the last bootstrap with 1100122 ABI. Maybe the maintainer wouldn't mind doing it as part of bug 223123.
Comment 10 sid 2017-10-29 09:32:19 UTC
Softlinking from /usr/local/bin/c++ to /usr/local/bin/clang40 or /usr/local/bin/clang50, and softlinking /usr/local/llvm40/bin/c++ or /usr/local/llvm50/bin/c++ to /usr/local/llvm40/clang or /usr/local/llvm50/clang lets it build.

This is not just for aarch64, this link needs to be fixed for many architectures  (amd64 etc) to compile rust properly.
Comment 11 Mark Millard 2017-10-29 10:26:09 UTC
(In reply to sid from comment #10)

Quoting Stefan Esser on freebsd-toolchain
(he replied to my clang++50 material instead of
Sid's original clang++40 material):

This situation should be covered by the new BINARY_ALIAS feature,
which makes a symbolic link to some binary available under a new
name and adds the location of that link to the PATH:

https://svnweb.freebsd.org/ports?view=revision&revision=451772

In this particular case:

BINARY_ALIAS	c++=clang++50

END QUOTE


(It is new to me: I've not used it. My guess
is that it would go in /etc/make.conf or the
like to work around the problem until lang/rust
is fixed to not require the command c++ .)

(Comment 10 has some missing ++'s as far as I
can tell, for example those in:
/usr/local/bin/clang++50 .)
Comment 12 sid 2017-10-29 10:50:35 UTC
Mark is right about the first one lacking ++

In /usr/local/bin/ it should be
from c++ to clang++40 or clang++50




The second example goes to clang as well, because here clang++ also links to it, which links to clang-5.0 or clang-4.0.

/usr/local/llvm40/bin/c++ or /usr/local/llvm50/bin/c++ to /usr/local/llvm40/clang or /usr/local/llvm50/clang


In /usr/local/llvm?0/bin/, clang++ links to clang, which links to clang-?.0

Now that you wrote it out, BINARY_ALIAS makes sense.


I'm not sure anymore that this is the same bug as above.
Comment 13 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-10-31 08:02:52 UTC
I'm not sure I understand what's described here.

First, there is the issue that the aarch64 boostrap is created on 11.1 (ref11-aarch64.freebsd.org) and thus can't run on 11.0. Then, there is the (in)ability to use clang from ports. It looks like two separate issues, doesn't it?

For the first issue, I will try to cross-compile a bootstrap from amd64 using 11.0 aarch64 userland. Anyway, I can't ssh to any ref*-aarch64.freebsd.org for a week now, I don't know what's going on with those hosts.

For the second problem, I'll study that after.
Comment 14 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-11-09 14:20:37 UTC
Quick update: I couldn't create a bootstrap on amd64 (even one targetting amd64) because rustc segfaults early in the build. I have no solution for now and won't have much time to look for one :(
Comment 15 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-11-26 13:06:39 UTC
As part of creating a new aarch64 bootstrap for Rust 1.22.1, I spawned a FreeBSD 11.0 amd64 EC2 VM and will use it for this purpose. Hopefully, it should solve the issue.

In parallel, I could build a bootstrap on ref11-aarch64.freebsd.org which is back apparently.
Comment 16 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-11-27 00:28:09 UTC
I finally was able to create a new aarch64 bootstrap on 11.0-RELEASE. Dist files in LOCAL/dumbbell should be available tomorrow morning, I'll commit Rust 1.22.1 at that time.
Comment 17 commit-hook freebsd_committer freebsd_triage 2017-11-27 08:21:37 UTC
A commit references this bug:

Author: dumbbell
Date: Mon Nov 27 08:20:33 UTC 2017
New revision: 454938
URL: https://svnweb.freebsd.org/changeset/ports/454938

Log:
  lang/rust: Update to 1.22.1

  Support for LibreSSL up-to 2.6.3 is now included, thus we can remove the
  patches to the `openssl` and `openssl-sys` crates.

  The renewed bootstrap for aarch64 was created on 11.0-RELEASE, instead
  of 11.1-STABLE. It should allow it to run on all 11.x and thus fix the
  build on 11.0-RELEASE. [1]

  PR:		223009 [1]

Changes:
  head/lang/rust/Makefile
  head/lang/rust/distinfo
  head/lang/rust/files/patch-src_vendor_openssl-sys_build.rs
  head/lang/rust/files/patch-src_vendor_openssl-sys_src_lib.rs
  head/lang/rust/files/patch-src_vendor_openssl-sys_src_libressl_mod.rs
  head/lang/rust/files/patch-src_vendor_openssl_build.rs
  head/lang/rust/files/patch-src_vendor_openssl_src_ssl_mod.rs
  head/lang/rust/files/patch-src_vendor_openssl_src_ssl_tests_mod.rs
Comment 18 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-11-27 18:02:25 UTC
I believe this is fixed with the new bootstrap created for Rust 1.22.1 (r454938 above).
Comment 19 Jan Beich freebsd_committer freebsd_triage 2017-12-03 19:36:07 UTC
11.0 EOL fixed Rust build on /quarterly:
http://thunderx1.nyi.freebsd.org/data/111arm64-quarterly/455339/logs/rust-1.19.0_3.log
Comment 20 Jan Beich freebsd_committer freebsd_triage 2017-12-12 04:47:12 UTC
Ditto for Rust on /latest set. FreeBSD 11.* aarch64 should have Firefox Quantum as binary package in a day. ;)
http://thunderx1.nyi.freebsd.org/data/111arm64-default/455387/logs/rust-1.22.1_2.log
http://thunderx1.nyi.freebsd.org/data/111arm64-default/455387/logs/firefox-57.0.1,1.log