Bug 217309

Summary: lang/rust (1.15.1): Error installing on 10.3 (pkg-static: No such file or directory)
Product: Ports & Packages Reporter: Vladimir Omelchuk <admin>
Component: Individual Port(s)Assignee: Jean-Sébastien Pédron <dumbbell>
Status: Closed FIXED    
Severity: Affects Many People CC: admin, anindya49, debdrup, drhowarddrfine, dumbbell, fastmint, freebsd-ports, freebsd, fullermd, gja822, godevilove, ksbeattie, lumiwa, mbeis, mercenary02
Priority: --- Flags: bugzilla: maintainer-feedback? (riggs)
Version: Latest   
Hardware: Any   
OS: Any   
URL: https://reviews.freebsd.org/D9816
Attachments:
Description Flags
portmaster build output
none
Alternative pkg-plist none

Description Vladimir Omelchuk 2017-02-23 11:12:44 UTC
Part of log:
…
gmake[4]: Leaving directory '/usr/ports/lang/rust/work/rustc-1.15.1-src'
====> Compressing man pages (compress-man)
===>  Installing for rust-1.15.1
===>  Checking if rust already installed
===>   Registering installation for rust-1.15.1 as automatic
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libarena-22adfce669fd9832.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libflate-e1e6e55ceb7251ef.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libfmt_macros-ba80d211c09981a2.so: No such file or directory
…
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib/libunwind-58a2f7d37c62243d.rlib: No such file or directory
*** Error code 74

Stop.
make[3]: stopped in /usr/ports/lang/rust
*** Error code 1

<uname>
FreeBSD 10.3-RELEASE-p16 amd64

<make.conf>
CPUTYPE?=core2
OPTIONS_SET=CUPS
WITHOUT_LPR=YES
DEFAULT_VERSIONS+=apache=2.4
QT4_OPTIONS=CUPS NAS QGTKSTYLE
LOCALIZED_LANG=ru
MAKE_JOBS_UNSAFE=yes
DEFAULT_VERSIONS+=mysql=5.7
.if ${.CURDIR:M*/ports/devel/ccache}
NO_CCACHE=yes
.endif
DEFAULT_VERSIONS+=ssl=openssl
CC=clang39
CXX=clang++39
CPP=clang-cpp39
.if ${.CURDIR:M*/emulators/linux-c6}
OPTIONS_SET=NVIDIA_GL
.endif
.if ${.CURDIR:M*/graphics/linux-c6-glx-utils}
OPTIONS_SET=NVIDIA_GL
.endif
.if ${.CURDIR:M*/x11-toolkits/linux-c6-qt47-x11}
OPTIONS_SET=NVIDIA_GL
.endif
Comment 1 Vladimir Omelchuk 2017-02-23 11:37:53 UTC
# pwd
/usr/ports/lang/rust/work/stage/usr/local/lib
# ls -1 | nl
     1	X11
     2	debug
     3	libarena-bd1dd1b27e05eddb.so
     4	libflate-5e0dc9817d302cc9.so
     5	libfmt_macros-4c44d4d1529f8323.so
     6	libgetopts-292bb6f8bcd3d4b3.so
     7	libgraphviz-ebc0ff18a0e057f6.so
     8	liblog-e6f2f74016bcd952.so
     9	libproc_macro-cb2c9b5fd89a7edd.so
    10	libproc_macro_plugin-0de7e89fa791738c.so
    11	libproc_macro_tokens-b26e17ac487cd24f.so
    12	librustc-6dcb579f26b362a0.so
    13	librustc_back-2112a22d2ae4a5bc.so
    14	librustc_borrowck-0609cadf5e73c60c.so
    15	librustc_const_eval-cd8c23719660cfef.so
    16	librustc_const_math-b49b9b6b0bd886fe.so
    17	librustc_data_structures-8745a829af74b4d2.so
    18	librustc_driver-e82ba1883bac3dd5.so
    19	librustc_errors-ee741e29591f5c66.so
    20	librustc_incremental-007046afe7182f00.so
    21	librustc_lint-e414d8a5d7bd5160.so
    22	librustc_llvm-58cf621fa38a08a4.so
    23	librustc_metadata-5999597c2b0ae6cf.so
    24	librustc_mir-ddf1c2541e25d21e.so
    25	librustc_passes-eb06b619f40e0cbe.so
    26	librustc_platform_intrinsics-41c3b0ad63b5d848.so
    27	librustc_plugin-fe20ad21fd3cee82.so
    28	librustc_privacy-6f927e13085a1757.so
    29	librustc_resolve-cf2db2538fa0bf4b.so
    30	librustc_save_analysis-e08c766b3f9bad3e.so
    31	librustc_trans-4819e3dfe19ebdb5.so
    32	librustc_typeck-84dc5b70cefda914.so
    33	librustdoc-495a8e59ae53d5f1.so
    34	libserialize-fdb4eca770ee8d94.so
    35	libstd-bbad147d04f57248.so
    36	libsyntax-5a7ae94a24348cb5.so
    37	libsyntax_ext-58615868c4f73daf.so
    38	libsyntax_pos-8f8d1498c8d72620.so
    39	libterm-025bc0da01299fdf.so
    40	libtest-113e0fd198f23bd0.so
    41	rustlib
Comment 2 Vladimir Omelchuk 2017-02-23 11:40:13 UTC
# pwd
/usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib
# ls -1 | nl
     1	liballoc-478c2c96518c7e5b.rlib
     2	liballoc_jemalloc-d208bd02daa6fc0f.rlib
     3	liballoc_system-4f23e2b4b0719747.rlib
     4	libarena-bd1dd1b27e05eddb.so
     5	libcollections-a5d82f499e10453e.rlib
     6	libcompiler_builtins-0d2e45d31725a5f2.rlib
     7	libcore-86c54730ad3f2aa7.rlib
     8	libflate-5e0dc9817d302cc9.so
     9	libfmt_macros-4c44d4d1529f8323.so
    10	libgetopts-292bb6f8bcd3d4b3.rlib
    11	libgetopts-292bb6f8bcd3d4b3.so
    12	libgraphviz-ebc0ff18a0e057f6.so
    13	liblibc-c7efa20fc219365e.rlib
    14	liblog-e6f2f74016bcd952.rlib
    15	liblog-e6f2f74016bcd952.so
    16	libpanic_abort-0cfc93d731f216c0.rlib
    17	libpanic_unwind-4d12e1e8c8617774.rlib
    18	libproc_macro-cb2c9b5fd89a7edd.so
    19	libproc_macro_plugin-0de7e89fa791738c.so
    20	libproc_macro_tokens-b26e17ac487cd24f.so
    21	librand-2dd5f86b98340e25.rlib
    22	librustc-6dcb579f26b362a0.so
    23	librustc_back-2112a22d2ae4a5bc.so
    24	librustc_bitflags-e537db9c7903b152.rlib
    25	librustc_borrowck-0609cadf5e73c60c.so
    26	librustc_const_eval-cd8c23719660cfef.so
    27	librustc_const_math-b49b9b6b0bd886fe.so
    28	librustc_data_structures-8745a829af74b4d2.so
    29	librustc_driver-e82ba1883bac3dd5.so
    30	librustc_errors-ee741e29591f5c66.so
    31	librustc_incremental-007046afe7182f00.so
    32	librustc_lint-e414d8a5d7bd5160.so
    33	librustc_llvm-58cf621fa38a08a4.so
    34	librustc_metadata-5999597c2b0ae6cf.so
    35	librustc_mir-ddf1c2541e25d21e.so
    36	librustc_passes-eb06b619f40e0cbe.so
    37	librustc_platform_intrinsics-41c3b0ad63b5d848.so
    38	librustc_plugin-fe20ad21fd3cee82.so
    39	librustc_privacy-6f927e13085a1757.so
    40	librustc_resolve-cf2db2538fa0bf4b.so
    41	librustc_save_analysis-e08c766b3f9bad3e.so
    42	librustc_trans-4819e3dfe19ebdb5.so
    43	librustc_typeck-84dc5b70cefda914.so
    44	librustdoc-495a8e59ae53d5f1.so
    45	libserialize-fdb4eca770ee8d94.rlib
    46	libserialize-fdb4eca770ee8d94.so
    47	libstd-bbad147d04f57248.rlib
    48	libstd-bbad147d04f57248.so
    49	libstd_shim-03acaaa5c5981473.rlib
    50	libstd_unicode-1534993a7053ece8.rlib
    51	libsyntax-5a7ae94a24348cb5.so
    52	libsyntax_ext-58615868c4f73daf.so
    53	libsyntax_pos-8f8d1498c8d72620.so
    54	libterm-025bc0da01299fdf.rlib
    55	libterm-025bc0da01299fdf.so
    56	libtest-113e0fd198f23bd0.rlib
    57	libtest-113e0fd198f23bd0.so
    58	libtest_shim-6f4be40d6bb889a5.rlib
    59	libunwind-aea74c20e3a347a7.rlib
Comment 3 drhowarddrfine 2017-02-23 12:38:11 UTC
Created attachment 180240 [details]
portmaster build output
Comment 4 gja822 2017-02-23 18:30:01 UTC
The same is here, at least with LLNEXTGEN and PORT_LLVM options on FreeBSD 10.3-STABLE #0 r313683M amd64.
Comment 5 Marco Beishuizen 2017-02-23 19:52:23 UTC
Same here on FreeBSD yokozuna.lan 11.0-STABLE FreeBSD 11.0-STABLE #0 r314104: Thu Feb 23 08:17:42 CET 2017     root@yokozuna.lan:/usr/obj/usr/src/sys/YOKOZUNA  amd64

===>  Installing for rust-1.15.1
===>  Checking if rust already installed
===>   Registering installation for rust-1.15.1
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libarena-22adfce669fd9832.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libflate-e1e6e55ceb7251ef.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libfmt_macros-ba80d211c09981a2.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libgetopts-09faa2be228f2ddb.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/libgraphviz-6303af3dd8d718e9.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/liblog-761db57e43e641cb.so: No such file or directory
[...]
6_64-unknown-freebsd/lib/libterm-bed969971b58b7ed.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib/libtest-098357d8bb203fcf.rlib: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib/libtest-098357d8bb203fcf.so: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib/libtest_shim-0dc4520584a37d49.rlib: No such file or directory
pkg-static: Unable to access file /usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib/libunwind-58a2f7d37c62243d.rlib: No such file or directory
*** Error code 74

Stop.
make[1]: stopped in /usr/ports/lang/rust
*** Error code 1

Stop.
make: stopped in /usr/ports/lang/rust
Comment 6 Jean-Sébastien Pédron freebsd_committer 2017-02-23 22:22:18 UTC
Hi!

Thanks to all of you for the report!

Rust build is influenced by installed ports. I don't know which yet (currently investigating how the library suffix is created). You need to use an isolated clean environment to build Rust, like Poudriere.

We'll keep this report opened until we understand this suffix.
Comment 7 freebsd 2017-02-23 23:04:55 UTC
Created attachment 180252 [details]
Alternative pkg-plist

FWIW, the following pkg-plist matches my system.
Comment 8 Jean-Sébastien Pédron freebsd_committer 2017-02-23 23:44:58 UTC
Ok, so the library suffix isn't influenced by installed ports, but by the build path :(

The hash is computed here (and following lines):
https://github.com/rust-lang/cargo/blob/master/src/cargo/ops/cargo_rustc/context.rs#L398-L400

It's based on the "PackageId" which itself is based on the "SourceId".

The SourceId is hashed here:
https://github.com/rust-lang/cargo/blob/master/src/cargo/core/source.rs#L444-452

And the SourceId is created here:
https://github.com/rust-lang/cargo/blob/master/src/cargo/core/source.rs#L138-174

So the path of the crate sources is used as the URL. I tried to compile a crate (cloned from Git) from two different directories and both copies have different suffixes.

The immediate fix would be a more generic or generated plist.
Comment 9 fullermd 2017-02-25 14:17:27 UTC
(In reply to Jean-Sébastien Pédron from comment #8)
> Ok, so the library suffix isn't influenced by installed ports, but by the
> build path :(

I get it failing on both a stable/11 and head machine, and both get the same
set of hashes as each other and with/without PORT_LLVM.  Both also have
identical

WRKDIRPREFIX=/usr/ports.work

in make.conf.  However, when I do a build without WRKDIRPREFIX set (so it keeps
building directly under /usr/ports), it still fails, and the hashes seem to be
the same as they were with WRKDIRPREFIX.  Interestingly, in both cases they
appear to be the same as in the OP's attached logs.
Comment 10 gja822 2017-02-25 17:02:22 UTC
Exactly, I have the same missing filenames.

...
/usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib/libtest_shim-0dc4520584a37d49.rlib:No such file or directory

 /usr/ports/lang/rust/work/stage/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib/libunwind-58a2f7d37c62243d.rlib:No such file or directory
Comment 11 fullermd 2017-02-25 18:57:49 UTC
(In reply to fullermd from comment #9)

Nevermind, I was looking at the wrong files.  So, yes, it looks purely path dependent from the symptoms here; on both systems with identical WRKDIRPREFIX, they get the same names, but then a build without it gets different hashes.
Comment 12 lumiwa 2017-02-26 12:48:06 UTC
The same problem on FreeBSD11-RELEASE (amd64) build with Synth.
Comment 13 Jean-Sébastien Pédron freebsd_committer 2017-02-26 21:49:01 UTC
Here is a patch which should fix the problem:
https://reviews.freebsd.org/D9816
Comment 14 Vladimir Omelchuk 2017-02-27 07:50:02 UTC
(In reply to Jean-Sébastien Pédron from comment #13)

Rust (1.15.1) with this patch installing without any errors!
Thanks!

root@vladiom:~ # pkg version -v | grep rust
rust-1.15.1                        =   up-to-date with index

<uname>
FreeBSD 10.3-RELEASE-p17 amd64
Comment 15 D. Ebdrup 2017-02-27 10:23:52 UTC
I can also confirm that review D9816 mentioned in comment #13 fixes the issue in the bug report.
Comment 16 fullermd 2017-02-27 13:47:22 UTC
With the patch from review D9816 it seems to package right for me too.
Comment 17 Anindya Mukherjee 2017-02-27 18:14:32 UTC
Confirming it works on my synth build.
Comment 18 Jens Grassel 2017-02-28 09:26:52 UTC
I can also confirm that the patch works on 11.0-RELEASE-p8.
Comment 19 Jean-Sébastien Pédron freebsd_committer 2017-03-01 07:55:43 UTC
Thank you to all of you for testing the patch!

I'm still waiting for approval from my mentors and lang/rust maintainer to commit the patch.
Comment 20 Reface 2017-03-03 16:28:34 UTC
Path is good only for rust. But when you have rust and firefox there is a problem.
For example, rust was installed with your diff. Then I make portupgrade -arR --batch and then I see the error:
===>   Registering installation for rust-1.15.1 as automatic
Installing rust-1.15.1...
======================================================================

Printing Rust backtraces requires procfs(5) mounted on /proc .
If you have not already done so, please do the following:

        mount -t procfs proc /proc

To make it permanent, you need the following lines in /etc/fstab:

        proc    /proc           procfs          rw      0       0

======================================================================

===>   firefox-52.0_1,1 depends on executable: rustc - not found
*** Error code 1

Stop.
make: stopped in /usr/ports/www/firefox
** Command failed [exit code 1]: /usr/bin/script -qa /tmp/portupgrade20170303-68342-k27ls3 env UPGRADE_TOOL=portupgrade UPGRADE_PORT=firefox-51.0.1_2,1 UPGRADE_PORT_VER=51.0.1_2,1 make BATCH=yes
Comment 21 Vladimir Omelchuk 2017-03-03 17:37:15 UTC
(In reply to Reface from comment #20)

I just today updated firefox (with rust 1.15.1) without any errors!

root@vladiom:~ # pkg version -v | grep -e rust -e firefox
firefox-52.0_1,1                   =   up-to-date with index
rust-1.15.1                        =   up-to-date with index
Comment 22 Vladimir Omelchuk 2017-03-03 17:39:40 UTC
# make showconfig
===> The following configuration options are available for firefox-52.0_1,1:
     BUNDLED_CAIRO=on: Use bundled fork of cairo-1.9.5
     CANBERRA=off: Sound theme alerts
     DBUS=on: D-Bus IPC system support
     DEBUG=off: Build with debugging support
     DTRACE=on: Build with DTrace probes
     FFMPEG=on: FFmpeg support (WMA, AIFF, AC3, APE...)
     GCONF=off: GConf configuration backend support
     GNOMEUI=off: libgnomeui support module
     INTEGER_SAMPLES=off: Integer audio sample format
     LIBPROXY=off: Proxy support via libproxy
     OPTIMIZED_CFLAGS=on: Use extra compiler optimizations
     PGO=off: Use Profile-Guided Optimization
     PROFILE=off: Build with profiling support
     RUST=on: Build with components written in Rust language
     TEST=off: Build and/or run tests
====> Options available for the multi AUDIO: you have to choose at least one of them
     ALSA=on: ALSA audio architecture support
     PULSEAUDIO=on: PulseAudio sound server support
     SNDIO=off: Sndio audio support
====> Options available for the single TOOLKIT: you have to select exactly one of them
     GTK2=on: GTK+ 2 GUI toolkit support
     GTK3=off: GTK+ 3 GUI toolkit support
===> Use 'make config' to modify these settings
Comment 23 Thomas Zander freebsd_committer 2017-03-04 11:15:54 UTC
*** Bug 217439 has been marked as a duplicate of this bug. ***
Comment 24 Anindya Mukherjee 2017-03-04 20:51:31 UTC
Firefox upgrade works fine for me too. I build in ports-mgmt/synth.
Comment 25 commit-hook freebsd_committer 2017-03-06 13:28:43 UTC
A commit references this bug:

Author: dumbbell
Date: Mon Mar  6 13:27:57 UTC 2017
New revision: 435537
URL: https://svnweb.freebsd.org/changeset/ports/435537

Log:
  lang/rust, lang/rust-nightly: Generate PLIST in post-install

  Several libraries have their filename computed and based on the absolute
  path to source files. Therefore, we need to generate the PLIST. Rust
  installer already produces manifests listing files it installs. The port
  now uses those files to complete `${TMPPLIST}`.

  While the port built fine in Poudriere, it failed when built directly on
  the host (regular make or with portmaster(1)) or using a different tool
  such as Synth. This commit fixes the build for those methods.

  Handling of DOCS-specific files is also unified with normal files. This
  gets rid of code duplication.

  How to do reproducible builds will be studied later.

  PR:		217309
  Reported by:	Several people on freebsd-ports@ or Bugzilla
  Tested by:	Almost everyone who reported the issue
  Approved by:	antoine (mentor), riggs (maintainer of lang/rust)
  Differential Revision:	https://reviews.freebsd.org/D9816

Changes:
  head/lang/rust/Makefile
  head/lang/rust/pkg-plist.amd64
  head/lang/rust/pkg-plist.i386
  head/lang/rust-nightly/pkg-plist.amd64
  head/lang/rust-nightly/pkg-plist.i386
Comment 26 Jean-Sébastien Pédron freebsd_committer 2017-03-06 13:30:07 UTC
Thank you to everyone who reported the problem and/or tested the patch!