Bug 243766 - lang/rust: Fails to build when Python (lang/python37) environment has no lzma module
Summary: lang/rust: Fails to build when Python (lang/python37) environment has no lzma...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: FreeBSD Rust Team
Depends on:
Reported: 2020-01-31 20:24 UTC by doctor
Modified: 2020-04-28 04:35 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (rust)

this should show you the failure (10.01 KB, text/plain)
2020-01-31 20:24 UTC, doctor
no flags Details
this should show you the failure (35.66 KB, text/plain)
2020-01-31 23:05 UTC, doctor
no flags Details
as requested (2.18 KB, text/plain)
2020-02-01 07:33 UTC, doctor
no flags Details
And an issue in Ython pooped up (815.58 KB, text/plain)
2020-02-02 19:11 UTC, doctor
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description doctor 2020-01-31 20:24:39 UTC
Created attachment 211227 [details]
this should show you the failure

Comment 1 Tobias Kortkamp freebsd_committer 2020-01-31 22:25:32 UTC
I cannot reproduce and builds fine on the cluster too:

> this should show you the failure

Log is not helpful with all the interesting bits missing.  What
exactly happened in fetch and patch that causes it to attempt to
download things from static.rust-lang.org during the build phase?
Please post the *full* output of the following commands:

cat /etc/make.conf
make clean patch
Comment 2 doctor 2020-01-31 23:05:48 UTC
Created attachment 211230 [details]
this should show you the failure

as requested
Comment 3 Tobias Kortkamp freebsd_committer 2020-01-31 23:17:02 UTC
I see now that it picks tar.gz as bootstrap extension on your system.  This
basically should not happen.  What happens when you run

python3.7 -c 'import tarfile; tarfile.open("/tmp/foo", "w:xz")'
Comment 4 doctor 2020-02-01 07:33:38 UTC
Created attachment 211238 [details]
as requested

Comment 5 Tobias Kortkamp freebsd_committer 2020-02-01 08:16:22 UTC
Comment on attachment 211238 [details]
as requested

> ImportError: /usr/contrib/lib/python3.7/lib-dynload/_lzma.so: invalid file format

So the real problem has little to do with lang/rust and your python37
installation seems to be broken.

Please reinstall it and once the command I gave you in comment #3 is successful,
i.e., returns nothing instead of an exception, reattempt to build lang/rust.
Comment 6 doctor 2020-02-02 19:11:12 UTC
Created attachment 211293 [details]
And an issue in Ython pooped up
Comment 7 Tobias Kortkamp freebsd_committer 2020-02-02 19:20:41 UTC
> *** WARNING: renaming "_lzma" since importing it failed: build/lib.freebsd-12.1-RELEASE-p2-amd64-3.7-pydebug/_lzma.so: Undefined symbol "lzma_check_is_supported"

I'm reassigning this to the Python team since the problem is not really
in lang/rust and I cannot help you with Python problems.
Comment 8 Tobias Kortkamp freebsd_committer 2020-02-02 19:23:23 UTC
*** Bug 243817 has been marked as a duplicate of this bug. ***
Comment 9 Kubilay Kocak freebsd_committer freebsd_triage 2020-02-03 00:55:19 UTC
(In reply to Tobias Kortkamp from comment #7)

Is there any way to force rust to use / explicitly specify a particular format/extension (since it seems we want a specific format?), bypassing the fallback mechanism in the event lzma is not available in the Python environment? 

Further, while having an lzma module is certainly common, it is by no means certain that this module is available, either in terms of the system environment the Python installation is built on, or whether the environment has been modified intentionally to preclude certain common.shared libraries, of which lzma is one.
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2020-02-03 00:58:28 UTC
Going to work on this separately in bug 243817
Comment 11 commit-hook freebsd_committer 2020-04-28 04:34:33 UTC
A commit references this bug:

Author: tobik
Date: Tue Apr 28 04:33:36 UTC 2020
New revision: 533234
URL: https://svnweb.freebsd.org/changeset/ports/533234

  lang/rust: Let the framework extract the bootstraps

  It is possible for lang/python37 to be built in such a way that it
  installs an unloadable lzma module which then causes Rust to fallback
  to trying to fetch/extract the tar.gz bootstraps instead.

  As a workaround and since it also simplifies some things, let the
  ports framework extract the bootstraps and "install" them under
  WRKDIR.  We point the build to them in config.toml.  This is similar
  to how things are hooked up in lang/rust-bootstrap and Rust will
  then not try to fetch and extract the bootstraps on its own.

  PR:		243766
  Reviewed by:	mikael
  Differential Revision:	https://reviews.freebsd.org/D24582