Bug 221088 - lang/rust: Fails to build with "error: the listed checksum of `.../config.rpath` has changed"
Summary: lang/rust: Fails to build with "error: the listed checksum of `.../config.rpa...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Jean-Sébastien Pédron
URL: https://reviews.freebsd.org/D11783
Keywords:
: 220968 221516 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-07-29 12:22 UTC by Jochen Neumeister
Modified: 2017-09-28 21:38 UTC (History)
12 users (show)

See Also:
koobs: merge-quarterly?


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jochen Neumeister freebsd_committer freebsd_triage 2017-07-29 12:22:19 UTC
Building stage0 tool rust-installer (x86_64-unknown-freebsd)
running: "/usr/ports/lang/rust/work/rustc-1.19.0-src/build/x86_64-unknown-freebsd/stage0/bin/cargo" "build" "-j" "4" "--target" "x86_64-unknown-freebsd" "-v" "--release" "--frozen" "--manifest-path" "/usr/ports/lang/rust/work/rustc-1.19.0-src/src/tools/rust-installer/Cargo.toml"
error: the listed checksum of `/usr/ports/lang/rust/work/rustc-1.19.0-src/src/vendor/lzma-sys/xz-5.2.3/build-aux/config.rpath` has changed:
expected: c8b4c017079da9dfb3086a0583e60ffe736184d89005dc5973f0bb0fd17c04bb
actual:   561b00eb30ecaef2c9da17bc195e7d2a7ea63facea38ea9849fbb0ed340bebba

directory sources are not intended to be edited, if modifications are required then it is recommended that [replace] is used with a forked copy of the source


command did not execute successfully: "/usr/ports/lang/rust/work/rustc-1.19.0-src/build/x86_64-unknown-freebsd/stage0/bin/cargo" "build" "-j" "4" "--target" "x86_64-unknown-freebsd" "-v" "--release" "--frozen" "--manifest-path" "/usr/ports/lang/rust/work/rustc-1.19.0-src/src/tools/rust-installer/Cargo.toml"
expected success, got: exit code: 101


Traceback (most recent call last):
  File "/usr/ports/lang/rust/work/rustc-1.19.0-src/x.py", line 20, in <module>
    bootstrap.main()
  File "/usr/ports/lang/rust/work/rustc-1.19.0-src/src/bootstrap/bootstrap.py", line 678, in main
    bootstrap()
  File "/usr/ports/lang/rust/work/rustc-1.19.0-src/src/bootstrap/bootstrap.py", line 670, in bootstrap
    run(args, env=env, verbose=rb.verbose)
  File "/usr/ports/lang/rust/work/rustc-1.19.0-src/src/bootstrap/bootstrap.py", line 142, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /usr/ports/lang/rust/work/rustc-1.19.0-src/build/bootstrap/debug/bootstrap install --verbose --config ./config.toml --jobs 4
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/lang/rust
*** Error code 1

Stop.
make: stopped in /usr/ports/lang/rust

===>>> make stage failed for lang/rust
===>>> Aborting update

===>>> Update for lang/rust failed
===>>> Aborting update
Comment 1 Bugzilla Automation freebsd_committer freebsd_triage 2017-07-29 12:22:19 UTC
Maintainer informed via mail
Comment 2 Rainer Hurling freebsd_committer freebsd_triage 2017-07-29 15:20:30 UTC
Please try to update your ports tree again. This problem should have gone after r446861 :)
Comment 3 Walter Schwarzenfeld freebsd_triage 2017-07-29 15:41:16 UTC
Had the same problem, solved it with WITHOUT_FBSD10_FIX=yes.
Comment 4 Jochen Neumeister freebsd_committer freebsd_triage 2017-07-29 17:22:04 UTC
(In reply to Rainer Hurling from comment #2)

i use r446882, but the same error as i reported :-)
Comment 5 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-07-30 11:04:13 UTC
Hi!

Ok, I could reproduce the issue, but I'm not sure how :)

Crates have a `.cargo-checksum.json` file which lists all files and their checksum, and Cargo verifies those checksums before trying to compile a crate. Therefore, everyone should be impacted by this problem, but that's obviously not the case.

I'll continue investigating and will keep you posted.
Comment 6 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-07-30 11:06:28 UTC
*** Bug 220968 has been marked as a duplicate of this bug. ***
Comment 7 fullermd 2017-07-31 16:22:17 UTC
It seems like the magic sudo-hackaround stuff in the build system probably has something to do with it.  A `make` for me dies inside of a minute with the checksum error.  Hoever, an `env -u SUDO_USER make` runs for nearly 15 minutes before SIGSEGV'ing elsewhere.
Comment 8 Walter Schwarzenfeld freebsd_triage 2017-07-31 16:38:48 UTC
I wrote above it works if I add WITHOUT_FBSD10_FIX=yes (in this case to the 
commandline). And this is working.
It is the question what bsd.ports.mk FBSD10 autotools fix does (I am not really
clear about this).
Comment 9 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-07-31 16:55:36 UTC
The "FreeBSD 10" autotools fix is required because some shell scripts copied  by autotools check the target's triple to determine if some compiler/linker features are to be used or not.

The problem is that many scripts used too generic patterns, such as "freebsd1*", when they wanted to match against FreeBSD 1.x. Therefore those patterns also match FreeBSD 10+ (so 11, 12, etc. as well) and the scripts pick compiler/linker flags which are goo for the FreeBSD 1.x era.

The automatic fix finds all those scripts and patches them to transform the pattern to "freebsd1.*" (note the dot after the digit).

I prepared the following patch which fixes that checksum problem among other things:
https://reviews.freebsd.org/D11783

The solution I choose is simply to patch the `.cargo-checksum.json` files to have the new checksums.
Comment 10 fullermd 2017-07-31 18:48:31 UTC
For me, the FBSD10_FIX stuff has no impact one way or the other; it's purely a result of the rust build's "I-see-you're-sudoing-let-me-change-something".  With SUDO_USER masked out, it runs past that just fine.

The segfault I was getting appears to have been a local issue with needing to rebuild llvm40 (and with PORT_LLVM set).  On a stable/10 box with SUDO_USER removed it build fine, and after rebuilding llvm40 on my head box it built fine as well.
Comment 11 Walter Schwarzenfeld freebsd_triage 2017-07-31 19:14:46 UTC
Sorry, but how to remove SUDO_USER?
Comment 12 fullermd 2017-08-01 07:10:24 UTC
(In reply to w.schwarzenfeld from comment #11)

Just use env to clear it out.

% env -u SUDO_USER make

[or portupgrade or portmaster or whatever other command you're using to build it]
Comment 13 Jochen Neumeister freebsd_committer freebsd_triage 2017-08-01 07:29:01 UTC
(In reply to Jean-Sébastien Pédron from comment #9)

hm, on CURRENT doesn't build, see http://joneumbox.org/data/12i386-ports/2017-07-31_22h33m23s/logs/errors/rust-1.19.0_1.log
Comment 14 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-08-01 07:46:58 UTC
(In reply to Jochen Neumeister from comment #13)
Good catch, I fixed the `extra-ino64` in my patch.

It's not committed yet, I'm still waiting for approval for changes outside of the port.
Comment 15 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-08-01 07:49:36 UTC
(In reply to fullermd from comment #12)

Is it with the patch in https://reviews.freebsd.org/D11783 ?

In the source archive, I see that if $SUDO_USER is set, the build process will use the bundled crates, not the one in Cargo registry. My patch enables the use of the bundled crates for all situations. I'm wondering if you hit the failure in this case.
Comment 16 fullermd 2017-08-01 08:02:55 UTC
(In reply to Jean-Sébastien Pédron from comment #15)

No, just out of ports.
Comment 17 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-08-01 08:15:01 UTC
(In reply to fullermd from comment #16)

Could you please try with the patch?
Comment 18 fullermd 2017-08-01 10:08:26 UTC
(In reply to Jean-Sébastien Pédron from comment #17)

Probably be a couple days before I get a chance.  I don't expect my system's that much different than any other -CURRENT box though.
Comment 19 fullermd 2017-08-06 19:55:45 UTC
(In reply to Jean-Sébastien Pédron from comment #17)

With the lang/rust bits of the patch, it fails independent of sudo (plain sudo, with env -u SUDO_USER, or just as my regular user) with checksum errors.  Perhaps that's because it's off in a path nowhere near /usr/ports/?  Though moving it to another dir gives the same differing result...

At any rate, the 'actual' values it gives are the same whatever the sudo-state, so it seems a reasonable guess that that part is resolved at least.
Comment 20 ru_M1cRO 2017-08-07 19:40:39 UTC
(In reply to w.schwarzenfeld from comment #3)
Helped me too.
Comment 21 david 2017-08-08 13:21:14 UTC
Thanks to the hints here, I was able to circumvent the problem by (e.g.):

sudo script -a typescript
Script started on Tue Aug  8 05:41:47 2017
root@g1-252:/common/home/david # env | grep -i sudo
SUDO_COMMAND=/usr/bin/script -a portmaster
SUDO_USER=david
SUDO_UID=1001
SUDO_GID=1001
root@g1-252:/common/home/david # su -
root@g1-252:~ # env | grep -i sudo
root@g1-252:~ # portmaster --no-term-title -ad
...
===>>> The following actions will be taken if you choose to proceed:
        Upgrade firefox-54.0.1_1,1 to firefox-55.0,1
        Install devel/cargo
        Install lang/rust

===>>> Proceed? y/n [y] 
...
===>>> Dependency check complete for lang/rust

===>>> All >> firefox-54.0.1_1,1 >> devel/cargo >> lang/rust (3/3)

===>  Cleaning for rust-1.19.0
...
===>>> Starting check for runtime dependencies
===>>> Gathering dependency list for lang/rust from ports
===>>> Dependency check complete for lang/rust

===>>> All >> firefox-54.0.1_1,1 >> devel/cargo >> lang/rust (3/3)

===>  Installing for rust-1.19.0
===>   Registering installation for rust-1.19.0 as automatic
Installing rust-1.19.0...
======================================================================

Printing Rust backtraces requires procfs(5) mounted on /proc .
If you have not already done so, please do the following:

        mount -t procfs proc /proc

To make it permanent, you need the following lines in /etc/fstab:

        proc    /proc           procfs          rw      0       0

======================================================================
...
===>>> Installation of lang/rust (rust-1.19.0) succeeded

===>>> Deleting stale distfile: rust/rust-registry-1.16.0.tar.xz

===>>> All >> firefox-54.0.1_1,1 >> devel/cargo (3/3)
Comment 22 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-08-10 21:53:50 UTC
I updated the patch in the review (https://reviews.freebsd.org/D11783). Could you please give it another try? I tested it successfully with sudo(1).
Comment 23 Jan Beich freebsd_committer freebsd_triage 2017-08-14 11:13:18 UTC
*** Bug 221516 has been marked as a duplicate of this bug. ***
Comment 24 Jean-Sébastien Pédron freebsd_committer freebsd_triage 2017-09-15 16:52:10 UTC
I'm still waiting for approval from portmgr to land this patch. Sorry for the delay.
Comment 25 commit-hook freebsd_committer freebsd_triage 2017-09-15 18:02:23 UTC
A commit references this bug:

Author: dumbbell
Date: Fri Sep 15 18:01:19 UTC 2017
New revision: 449914
URL: https://svnweb.freebsd.org/changeset/ports/449914

Log:
  lang/rust: Install Cargo + use bundled crates

  This port now provides Cargo. This is the recommended now because Cargo
  won't be provided separately in the future.

  To build Cargo, we set `extended = true` in `config.toml`. As a side
  effect, this flag also installs Rust source code. The port has a new
  `SOURCES` option (disabled by default) to keep those sources.

  As a consequence of this, `devel/cargo` is removed. Several ports
  and Makefiles in Mk were updated to depend on `lang/rust` instead of
  `devel/cargo`.

  The other big change in this patch is the use of the bundled crates,
  instead of relying on Cargo's registry (which was part of the distfiles,
  in order to allow offline builds). So now, we don't need to prepare the
  registry when updating this port.

  This has several other benefits:
      * It fixes the build with sudo(8).
      * It fixes the use of the ino-64 patch (it was not applied to the
        registry, thus not used).

  Compilation errors were fixed in the ino-64 patch.

  Various `.cargo-checksum.json` files are updated after the sources are
  patched (FBSD10_FIX, ino-64, and so on). This fixes builds which were
  failing with errors such as:

      error: the listed checksum of `.../rustc-1.19.0-src/src/vendor/lzma-sys/xz-5.2.3/build-aux/config.rpath` has changed:
      expected: c8b4c017079da9dfb3086a0583e60ffe736184d89005dc5973f0bb0fd17c04bb
      actual:   561b00eb30ecaef2c9da17bc195e7d2a7ea63facea38ea9849fbb0ed340bebba

  PR:		221088
  Reported by:	joneum@, nwhitehorn@, romain@,
  		Ekaterina Vaartis <vaartis@cock.li>,
  		david@catwhisker.org,
  		fullermd@over-yonder.net,
  		rum1cro@yandex.ru,
  		w.schwarzenfeld@utanet.at
  Differential Revision:	https://reviews.freebsd.org/D11783

Changes:
  head/MOVED
  head/Mk/Uses/cargo.mk
  head/Mk/bsd.gecko.mk
  head/UPDATING
  head/devel/Makefile
  head/devel/cargo/
  head/lang/rust/Makefile
  head/lang/rust/distinfo
  head/lang/rust/files/config.toml
  head/lang/rust/files/extra-patch-ino64
  head/multimedia/librespot/Makefile
  head/www/firefox/Makefile