Bug 229826

Summary: lang/rust: Update to 1.29
Product: Ports & Packages Reporter: Jan Beich <jbeich>
Component: Individual Port(s)Assignee: FreeBSD Rust Team <rust>
Status: Closed FIXED    
Severity: Affects Only Me CC: dumbbell, mikael
Priority: --- Keywords: needs-patch
Version: LatestFlags: dumbbell: maintainer-feedback+
Hardware: Any   
OS: Any   
URL: https://blog.rust-lang.org/2018/09/13/Rust-1.29.html
See Also: https://bugzilla.mozilla.org/show_bug.cgi?id=1450078
Bug Depends on: 231411    
Bug Blocks: 229893    

Description Jan Beich freebsd_committer freebsd_triage 2018-07-17 11:33:22 UTC
Firefox 63 (scheduled on 2018-10-23) will require Rust 1.28 (scheduled on 2018-08-02) to build. To avoid MFH pain the port needs to be updated before 2018Q4 (scheduled on 2018-10-01) is branched.

https://searchfox.org/mozilla-central/search?q=rustc_min_ver
Comment 1 Jan Beich freebsd_committer freebsd_triage 2018-07-17 11:39:33 UTC
OTOH, Firefox 62 (scheduled on 2018-09-05) still requires Rust 1.24.
Comment 2 Jan Beich freebsd_committer freebsd_triage 2018-08-09 22:11:20 UTC
Firefox 63 cannot be built downstream anymore while upstream build relies on https://rustup.rs/ which only supports amd64 hosts.
https://hg.mozilla.org/mozilla-central/rev/0260bc2917dd
Comment 3 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-03 18:28:23 UTC
Hi!

I see I never updated this report since I started to work on Rust 1.28.0.

So, Rust 1.28.0 does not compile out-of-the-box unfortunately.

The change behind this breakage is probably this one:
https://github.com/rust-lang/rust/commit/7c14a54bc81d8e259b43ac8077f2e851c7769753

In particular, the way libbacktrace is compiled changed:

* In Rust 1.27.x, `src/libstd/build.rs` runs libbacktrace's `configure` script.
* In Rust 1.28.0, `src/libstd/build.rs` compiles libbacktrace directory with its own compiler flags and source files list.

The problem is that the `configure` script was testing the presence and usability of libunwind. This check was always failing on FreeBSD even if libunwind was installed because none of our compilers know about /usr/local... So libbacktrace was compiled without unwind support.

Now `src/libstd/build.rs` assumes libunwind is available somehow and `backtrace.c` is compiled: this fails because the compiler has no idea how to find `unwind.h`.

Once this is fixed, the build fails later because it doesn't know where to find libunwind.so. I couldn't find a way to extend the search directories so far.

Anyway, libunwind support was never used/tested on FreeBSD so it's probably not a good time to add it. I'm going to try ways to compile libbacktrace as if the `configure` script did run.

Another fix I had to do is revert the changes made by FreeBSD 10 autotools fix to a few files: they are not used but are verified by Cargo's checksum facility.
Comment 4 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-13 20:58:50 UTC
Rust 1.29.0 was just released.

Unfortunately I still couldn't build Rust 1.28.0 because it wants libunwind and I can't find a way to tell our compiler to find our libunwind in the default location.

Trying with Rust 1.29.0 now, but I'm almost sure it will fail the same way.
Comment 5 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-13 22:04:23 UTC
In fact, good news, Rust 1.29.0 built successfully with the patch I started to write for 1.28.0!

    $ (...)/lang/rust/work/stage/usr/local/bin/rustc --version
    rustc 1.29.0

I'm going to run it through Poudriere as well as try a few Rust-based ports (like Firefox, ripgrep and alacritty) tomorrow.
Comment 6 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-14 15:47:08 UTC
The Rust port update is tested and ready to land.

However, Firefox fails to configure. The issue and the patch are here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1479540

With the patch, the configure script succeeds. I'm now waiting on Poudriere to finish.
Comment 7 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-14 16:31:42 UTC
After applying the patch from the mentionned issue, Firefox builds fine with Rust 1.29.0.
Comment 8 Jan Beich freebsd_committer freebsd_triage 2018-09-14 18:57:51 UTC
(In reply to Jean-Sébastien Pédron from comment #7)
Probably all USE_GECKO ports that depend on lang/rust need upstream fix:
- mail/thunderbird
- www/firefox
- www/firefox-esr
- www/seamonkey (when RUST option is enabled, firefox code is under mozilla/ prefix)
- www/waterfox

For testing it maybe enough to build only one then check the rest via "make clean configure WITH=RUST" in each port directory (skipping full build). Personally, I'd probably still test full build of www/waterfox (as unlike www/seamonkey) it has Stylo which is a lot of Rust code.

For landing the only requirement is to name the patch after the upstream bug. This is important in order to efficiently determine what to rebase and what to prune on update. Ideally, the patch format is based on "raw" patch from hgweb (e.g., www/firefox-esr/files/patch-bug1467882) or massaged via hg/git log -p. If you land the patch separately from lang/rust update then documenting build error via excerpt to serve as the rationale would be nice.
Comment 9 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-15 14:06:50 UTC
Thanks! I'll prepare a patch and post it on reviews.freebsd.org for you to review it.
Comment 10 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-15 16:23:32 UTC
The patch was also required in www/firefox-esr and mail/thunderbird. www/waterfox or www/seamonkey didn't need it.

Here is the review:
https://reviews.freebsd.org/D17178
Comment 11 Jan Beich freebsd_committer freebsd_triage 2018-09-15 17:10:36 UTC
(In reply to Jean-Sébastien Pédron from comment #10)
> www/waterfox or www/seamonkey didn't need it.

Probably, because https://bugzilla.mozilla.org/show_bug.cgi?id=1319332 landed in Firefox 58 while Waterfox 56.* is based on Firefox 56 and SeaMonkey 2.49.* is based Firefox 52.
Comment 12 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2018-09-15 17:15:48 UTC
Do you prefer I commit the Firefox & friends patches in the same revision as the Rust 1.29.0 update?
Comment 13 Jan Beich freebsd_committer freebsd_triage 2018-09-15 18:10:06 UTC
Doesn't matter much. Landing separately is more work: in order to avoid commit message being too ambiguous or too repetitive you'd need to include build error excerpt and in case of MFH not forget about commit dependencies. For one, I land fixes separately during updates with many consumers (e.g., bug 229569) to keep rebase churn low (and my bus factor high) but use a single commit for more simple stuff (e.g., ports r476802) that can be done within a day or two.
Comment 14 commit-hook freebsd_committer freebsd_triage 2018-09-16 07:57:33 UTC
A commit references this bug:

Author: dumbbell
Date: Sun Sep 16 07:57:09 UTC 2018
New revision: 479877
URL: https://svnweb.freebsd.org/changeset/ports/479877

Log:
  lang/rust: Update to 1.29.0

  Release notes:
  * https://blog.rust-lang.org/2018/07/20/Rust-1.27.2.html
  * https://blog.rust-lang.org/2018/08/02/Rust-1.28.html
  * https://blog.rust-lang.org/2018/09/13/Rust-1.29.html

  Up to and including Rust 1.27.x, the Rust build system shelled out to
  a configure script to detect the presence and usability of libunwind.
  Since Rust 1.28.0, it's using a static result in a `build.rs` file and
  expects libunwind to be used. It was not the case on FreeBSD so far, so
  we need a patch to this `build.rs` to disable that. We still need to
  study if the FreeBSD port should use libunwind and what to do with this
  patch. But this problem prevented the update to Rust 1.28.0 already, so
  enough delay.

  The update also comes with a patch to a few `USE_GECKO`-based ports such
  as Firefox [1]. Their configure script has some asumptions on the output
  of `rustc --print target-list` which are not true anymore. The patch was
  already committed upstream.

  The aarch64 version is still marked as BROKEN because I didn't find the
  time to work on it. As a consequence, there is also no aarch64 bootstrap
  for Rust 1.29.0.

  PR:		229826
  Approved by:	jbeich [1]
  Obtained from:	https://bugzilla.mozilla.org/show_bug.cgi?id=1479540 [1]
  Differential Revision:	https://reviews.freebsd.org/D17178

Changes:
  head/lang/rust/Makefile
  head/lang/rust/distinfo
  head/lang/rust/files/patch-src_libstd_build.rs
  head/lang/rust/files/patch-src_vendor_libgit2-sys_libgit2_src_streams_openssl.c
  head/lang/rust/files/patch-src_vendor_libgit2-sys_libgit2_src_streams_openssl.h
  head/mail/thunderbird/files/patch-bug1479540
  head/www/firefox/files/patch-bug1479540
  head/www/firefox-esr/files/patch-bug1479540
Comment 15 commit-hook freebsd_committer freebsd_triage 2018-09-23 18:54:43 UTC
A commit references this bug:

Author: jbeich
Date: Sun Sep 23 18:53:49 UTC 2018
New revision: 480524
URL: https://svnweb.freebsd.org/changeset/ports/480524

Log:
  lang/rust-nightly: no longer builds after r479877

  mv: rename /wrkdirs/usr/ports/lang/rust-nightly/work/rustc-nightly-src/src/vendor/backtrace-sys/src/libbacktrace/config/libtool.m4 to /wrkdirs/usr/ports/lang/rust-nightly/work/rustc-nightly-src/src/vendor/backtrace-sys/src/libbacktrace/config/libtool.m4.dont-fix: No such file or directory
  *** Error code 1

  As the port appears to be a staging ground for future updates don't
  deprecate yet.

  PR:		229826
  Reported by:	pkg-fallout

Changes:
  head/lang/rust-nightly/Makefile
Comment 16 commit-hook freebsd_committer freebsd_triage 2018-09-28 17:59:57 UTC
A commit references this bug:

Author: jbeich
Date: Fri Sep 28 17:59:31 UTC 2018
New revision: 480871
URL: https://svnweb.freebsd.org/changeset/ports/480871

Log:
  lang/rust: update to 1.29.1

  - Force rebuild all consumers as standard library is statically linked

  Changes:	https://blog.rust-lang.org/2018/09/25/Rust-1.29.1.html
  PR:		229826 (follow up)
  Approved by:	dumbbell
  Differential Revision:	https://reviews.freebsd.org/D17337

Changes:
  head/databases/puppetdb-cli/Makefile
  head/devel/bingrep/Makefile
  head/devel/pijul/Makefile
  head/devel/racer/Makefile
  head/devel/racerd/Makefile
  head/devel/rust-bindgen/Makefile
  head/devel/rust-cbindgen/Makefile
  head/devel/tokei/Makefile
  head/dns/doh-proxy/Makefile
  head/editors/xi-core/Makefile
  head/games/genact/Makefile
  head/lang/rust/Makefile
  head/lang/rust/distinfo
  head/mail/thunderbird/Makefile
  head/security/kr/Makefile
  head/sysutils/exa/Makefile
  head/sysutils/fd/Makefile
  head/sysutils/flowgger/Makefile
  head/sysutils/potnet/Makefile
  head/textproc/bat/Makefile
  head/textproc/ripgrep/Makefile
  head/textproc/xsv-rs/Makefile
  head/www/firefox/Makefile
  head/www/firefox-esr/Makefile
  head/www/geckodriver/Makefile
  head/www/seamonkey/Makefile
  head/www/waterfox/Makefile
  head/www/websocat/Makefile
  head/x11/admiral/Makefile
  head/x11/alacritty/Makefile
Comment 17 commit-hook freebsd_committer freebsd_triage 2018-10-12 19:01:33 UTC
A commit references this bug:

Author: jbeich
Date: Fri Oct 12 19:01:09 UTC 2018
New revision: 481921
URL: https://svnweb.freebsd.org/changeset/ports/481921

Log:
  lang/rust: update to 1.29.2

  - Force rebuild all consumers in case some are affected by miscompilation

  Changes:	https://blog.rust-lang.org/2018/10/12/Rust-1.29.2.html
  PR:		229826 (follow up)
  Approved by:	rust (tobik)
  MFH:		2018Q4
  Differential Revision:	https://reviews.freebsd.org/D17477

Changes:
  head/databases/puppetdb-cli/Makefile
  head/devel/bingrep/Makefile
  head/devel/pijul/Makefile
  head/devel/racer/Makefile
  head/devel/racerd/Makefile
  head/devel/rust-bindgen/Makefile
  head/devel/rust-cbindgen/Makefile
  head/devel/sccache/Makefile
  head/devel/tokei/Makefile
  head/dns/doh-proxy/Makefile
  head/editors/xi-core/Makefile
  head/games/genact/Makefile
  head/lang/rust/Makefile
  head/lang/rust/distinfo
  head/lang/rust/files/patch-src_vendor_libssh2-sys_libssh2_src_CMakeLists.txt
  head/mail/thunderbird/Makefile
  head/net-im/fractal/Makefile
  head/security/cargo-audit/Makefile
  head/security/kr/Makefile
  head/sysutils/exa/Makefile
  head/sysutils/fd/Makefile
  head/sysutils/flowgger/Makefile
  head/sysutils/potnet/Makefile
  head/textproc/bat/Makefile
  head/textproc/ripgrep/Makefile
  head/textproc/xsv-rs/Makefile
  head/www/firefox/Makefile
  head/www/firefox-esr/Makefile
  head/www/geckodriver/Makefile
  head/www/seamonkey/Makefile
  head/www/websocat/Makefile
  head/x11/admiral/Makefile
  head/x11/alacritty/Makefile
Comment 18 commit-hook freebsd_committer freebsd_triage 2018-10-13 09:51:33 UTC
A commit references this bug:

Author: jbeich
Date: Sat Oct 13 09:51:29 UTC 2018
New revision: 481972
URL: https://svnweb.freebsd.org/changeset/ports/481972

Log:
  MFH: r481921 r481969

  lang/rust: update to 1.29.2

  - Force rebuild all consumers in case some are affected by miscompilation

  Changes:	https://blog.rust-lang.org/2018/10/12/Rust-1.29.2.html
  PR:		229826 (follow up)
  Approved by:	rust (tobik)
  Approved by:	ports-secteam (miwi)
  Differential Revision:	https://reviews.freebsd.org/D17477

Changes:
_U  branches/2018Q4/
  branches/2018Q4/databases/puppetdb-cli/Makefile
  branches/2018Q4/devel/bingrep/Makefile
  branches/2018Q4/devel/pijul/Makefile
  branches/2018Q4/devel/racer/Makefile
  branches/2018Q4/devel/racerd/Makefile
  branches/2018Q4/devel/rust-bindgen/Makefile
  branches/2018Q4/devel/rust-cbindgen/Makefile
  branches/2018Q4/devel/tokei/Makefile
  branches/2018Q4/dns/doh-proxy/Makefile
  branches/2018Q4/editors/xi-core/Makefile
  branches/2018Q4/games/genact/Makefile
  branches/2018Q4/lang/rust/Makefile
  branches/2018Q4/lang/rust/distinfo
  branches/2018Q4/lang/rust/files/patch-src_vendor_libssh2-sys_libssh2_src_CMakeLists.txt
  branches/2018Q4/mail/thunderbird/Makefile
  branches/2018Q4/sysutils/exa/Makefile
  branches/2018Q4/sysutils/fd/Makefile
  branches/2018Q4/sysutils/flowgger/Makefile
  branches/2018Q4/sysutils/potnet/Makefile
  branches/2018Q4/textproc/bat/Makefile
  branches/2018Q4/textproc/ripgrep/Makefile
  branches/2018Q4/textproc/xsv-rs/Makefile
  branches/2018Q4/www/firefox/Makefile
  branches/2018Q4/www/firefox-esr/Makefile
  branches/2018Q4/www/geckodriver/Makefile
  branches/2018Q4/www/seamonkey/Makefile
  branches/2018Q4/www/websocat/Makefile
  branches/2018Q4/x11/admiral/Makefile
  branches/2018Q4/x11/alacritty/Makefile