Bug 223300 - www/firefox (56+) fails to build using lang/rust (1.21+) built with PORT_LLVM=on: error[E0432]: unresolved import `x86::sse2`
Summary: www/firefox (56+) fails to build using lang/rust (1.21+) built with PORT_LLVM...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-gecko mailing list
URL:
Keywords:
: 223302 223570 (view as bug list)
Depends on: 223342
Blocks:
  Show dependency treegraph
 
Reported: 2017-10-29 10:54 UTC by Dimitry Andric
Modified: 2017-12-17 23:08 UTC (History)
10 users (show)

See Also:
bugzilla: maintainer-feedback? (gecko)


Attachments
Poudriere log for www/firefox (40.06 KB, application/x-xz)
2017-10-29 10:54 UTC, Dimitry Andric
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dimitry Andric freebsd_committer 2017-10-29 10:54:30 UTC
Created attachment 187555 [details]
Poudriere log for www/firefox

After updating my ports tree to r453055, lang/rust got updated to 1.21.0, and www/firefox to 56.0.2_3,1.  Building under poudriere now results in an error (it used to build OK in the past):

...
gmake[6]: Entering directory '/wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/obj-x86_64-unknown-freebsd11.1/toolkit/library/rust'
force-cargo-library-build
env   RUSTC_BOOTSTRAP=1 RUSTFLAGS=' '  CARGO_TARGET_DIR=/wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/obj-x86_64-unknown-freebsd11.1/toolkit/library RUSTC=/usr/local/bin/rustc MOZ_SRC=/wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2 MOZ_DIST=/wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/obj-x86_64-unknown-freebsd11.1/dist LIBCLANG_PATH="/usr/local/llvm40/lib" CLANG_PATH="/usr/local/llvm40/bin/clang" PKG_CONFIG_ALLOW_CROSS=1 RUST_BACKTRACE=1 MOZ_TOPOBJDIR=/wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/obj-x86_64-unknown-freebsd11.1 MOZ_CARGO_WRAP_LDFLAGS="-pthread -Wl,--as-needed -fstack-protector -Wl,-z,noexecstack -Wl,-z,text -Wl,-z,relro -Wl,-rpath-link,/wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/obj-x86_64-unknown-freebsd11.1/dist/bin -Wl,-rpath-link,/usr/local/lib" MOZ_CARGO_WRAP_LD=" /usr/bin/cc -std=gnu99" CARGO_TARGET_X86_64_UNKNOWN_FREEBSD_LINKER=/wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/build/cargo-linker /usr/local/bin/cargo build  --release --frozen --manifest-path /wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/toolkit/library/rust/Cargo.toml --lib --target=x86_64-unknown-freebsd --features "servo bindgen simd-accel no-static-ideograph-encoder-tables"
   Compiling nom v1.2.4
   Compiling rayon-core v1.2.0
   Compiling nsstring_vendor v0.1.0 (file:///wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/servo/components/style/gecko_bindings/nsstring_vendor)
   Compiling either v1.1.0
   Compiling smallvec v0.4.0
   Compiling stable_deref_trait v1.0.0
   Compiling same-file v0.1.3
   Compiling precomputed-hash v0.1.0
   Compiling scopeguard v0.3.2
   Compiling void v1.0.2
   Compiling regex-syntax v0.4.0
   Compiling fnv v1.0.5
   Compiling odds v0.2.25
   Compiling libc v0.2.24
   Compiling rustc-serialize v0.3.22
   Compiling simd v0.2.0
error[E0432]: unresolved import `x86::sse2`
  --> /wrkdirs/usr/ports/www/firefox/work/firefox-56.0.2/third_party/rust/simd/src/common.rs:17:10
   |
17 | use x86::sse2::common;
   |          ^^^^ Could not find `sse2` in `x86`

   Compiling unicode-xid v0.0.4
   Compiling unicode-normalization v0.1.4
error: aborting due to previous error

error: Could not compile `simd`.

I guess either the sse2 component got removed from rust, or the firefox component got updated to now include it?
Comment 1 fullermd 2017-10-29 12:04:16 UTC
I can generally confirm it as rust related; I rebuild the new PORTREVISION of firefox on one system before updating rust, and it went fine; doing it again from the same ports tree after the rust update fails as listed.
Comment 2 fullermd 2017-10-29 12:06:39 UTC
(In reply to fullermd from comment #1)

"before updating rust" meaning with 1.20.0.
Comment 3 VVD 2017-10-29 12:39:08 UTC
I have the same error on amd64 too.

$ grep ^OPTIONS /var/db/ports/www_firefox/options
OPTIONS_FILE_SET+=BUNDLED_CAIRO
OPTIONS_FILE_UNSET+=CANBERRA
OPTIONS_FILE_UNSET+=DBUS
OPTIONS_FILE_UNSET+=DEBUG
OPTIONS_FILE_UNSET+=DTRACE
OPTIONS_FILE_SET+=FFMPEG
OPTIONS_FILE_SET+=GCONF
OPTIONS_FILE_UNSET+=INTEGER_SAMPLES
OPTIONS_FILE_SET+=LIBPROXY
OPTIONS_FILE_SET+=OPTIMIZED_CFLAGS
OPTIONS_FILE_UNSET+=PGO
OPTIONS_FILE_UNSET+=PROFILE
OPTIONS_FILE_UNSET+=TEST
OPTIONS_FILE_SET+=ALSA
OPTIONS_FILE_UNSET+=JACK
OPTIONS_FILE_UNSET+=PULSEAUDIO
OPTIONS_FILE_UNSET+=SNDIO

$ grep CPUTYPE?= /etc/make.conf 
CPUTYPE?=core2

With rust 1.20 compiled fine, with rust 1.21 - this error.
Comment 4 Jan Beich freebsd_committer 2017-10-29 12:41:16 UTC
Comment on attachment 187555 [details]
Poudriere log for www/firefox

Hmm, I can't reproduce on 11.1-RELEASE amd64 using the same port options and the same CPUTYPE set. Build log: http://sprunge.us/EYDH
Comment 5 Jan Beich freebsd_committer 2017-10-29 12:49:02 UTC
Can you try rebuilding lang/rust without CPUTYPE defined in make.conf?
Comment 6 Jan Beich freebsd_committer 2017-10-29 13:12:12 UTC
*** Bug 223302 has been marked as a duplicate of this bug. ***
Comment 7 Jan Beich freebsd_committer 2017-10-29 13:17:22 UTC
Why does Summary field mentions i386 when every reporter so far is on amd64?
Comment 8 gja822 2017-10-29 16:57:48 UTC
I have

CPUTYPE?=bdver2

in make.conf and the same error. (And without poudriere, just from make command within /usr/ports/www/firefox directory.)

Options:
OPTIONS_FILE_UNSET+=BUNDLED_CAIRO
OPTIONS_FILE_UNSET+=CANBERRA
OPTIONS_FILE_SET+=DBUS
OPTIONS_FILE_UNSET+=DEBUG
OPTIONS_FILE_UNSET+=DTRACE
OPTIONS_FILE_SET+=FFMPEG
OPTIONS_FILE_SET+=GCONF
OPTIONS_FILE_UNSET+=INTEGER_SAMPLES
OPTIONS_FILE_UNSET+=LIBPROXY
OPTIONS_FILE_SET+=OPTIMIZED_CFLAGS
OPTIONS_FILE_UNSET+=PGO
OPTIONS_FILE_UNSET+=PROFILE
OPTIONS_FILE_SET+=TEST
OPTIONS_FILE_SET+=ALSA
OPTIONS_FILE_UNSET+=JACK
OPTIONS_FILE_SET+=PULSEAUDIO
OPTIONS_FILE_UNSET+=SNDIO
Comment 9 Dimitry Andric freebsd_committer 2017-10-29 17:38:53 UTC
(In reply to Jan Beich from comment #7)
> Why does Summary field mentions i386 when every reporter so far is on amd64?

Sorry about it, that was a complete misinterpretation on my side.  I had accidentally thought that the system I saw this on was i386, but it was amd64, in fact.  I have now fixed the subject.
Comment 10 Keren Sky 2017-10-29 17:54:26 UTC
I too have the same issue building Firefox. FreeBSD 11.1 amd64. I believe it occurred after updating rust to 1.21 
Thank you.
Comment 11 VVD 2017-10-29 18:19:04 UTC
(In reply to Jan Beich from comment #4)
Removed CPUTYPE from /etc/make.conf, rebuilded rust 1.21.0_1, firefox build with same error:
   Compiling simd v0.2.0
error[E0432]: unresolved import `x86::sse2`
  --> /tmp/work/usr/ports/www/firefox/work/firefox-56.0.2/third_party/rust/simd/src/common.rs:17:10
   |
17 | use x86::sse2::common;
   |          ^^^^ Could not find `sse2` in `x86`

error: aborting due to previous error

error: Could not compile `simd`.

P.S. FreeBSD 11.1, amd64.
Comment 12 VVD 2017-10-29 18:21:33 UTC
For information:
$ grep ^OPTIONS /var/db/ports/lang_rust/options 
OPTIONS_FILE_UNSET+=DOCS
OPTIONS_FILE_UNSET+=GDB
OPTIONS_FILE_UNSET+=LLNEXTGEN
OPTIONS_FILE_SET+=PORT_LLVM
OPTIONS_FILE_UNSET+=SOURCES
Comment 13 Jan Beich freebsd_committer 2017-10-29 19:34:14 UTC
(In reply to vvd from comment #12)
> OPTIONS_FILE_SET+=PORT_LLVM

Can you try without? Rust uses patched LLVM 4.0 by default which maybe required for cfg_target_feature to detect SSE2.

https://github.com/rust-lang/rust/pull/31709
https://github.com/rust-lang/rust/blob/690ff045949a/src/rustllvm/PassWrapper.cpp#L200
https://github.com/rust-lang/rust/blob/690ff045949a/src/bootstrap/compile.rs#L543
https://github.com/rust-lang/rust/blob/690ff045949a/src/bootstrap/lib.rs#L505
Comment 14 Jan Beich freebsd_committer 2017-10-29 19:41:52 UTC
Rust 1.21.0 has https://github.com/rust-lang/rust/commit/881a724660 which would explain why simd crate built fine with Rust 1.20.0.
Comment 15 Jan Beich freebsd_committer 2017-10-29 19:50:18 UTC
If you really want PORT_LLVM=on maybe drop --enable-rust-simd from Mk/bsd.gecko.mk. It's mainly used by https://bugzilla.mozilla.org/show_bug.cgi?id=1261841
Comment 16 VVD 2017-10-29 20:14:06 UTC
(In reply to Jan Beich from comment #13)
> Can you try without?
In progress. Will write result in about 1 hour… I hope. :-D

> If you really want PORT_LLVM=on
Too long time for compile the rust with embedded llvm. And I already have the llvm 4.0.1 installed. IMHO, it's bad solution to build and install a huge port llvm twice in the system…
Comment 17 Jan Beich freebsd_committer 2017-10-29 20:33:01 UTC
(In reply to vvd from comment #16)
> Too long time for compile the rust with embedded llvm.

It's a stripped down version: no compiler-rt, no clang, no extra clang tools, no lld, no lldb, no openmp. lang/rust also takes a lot of time to build because it does a poor job of parallelizing release builds.

cf. https://internals.rust-lang.org/t/help-test-out-thinlto/6017

> And I already have the llvm 4.0.1 installed.

PORT_LLVM=on statically links LLVM, so you don't save space. Given both bundled llvm and devel/llvm* have similar dependencies if one of those is obsolete poudriere would force a rebuild unless you trick it into not looking at the whole dependency chain.
Comment 18 Jan Beich freebsd_committer 2017-10-29 21:31:02 UTC
I could reproduce in 11.1-RELEASE i386 jail. PORT_LLVM=on is really the culprit.

$ git clone https://github.com/rust-lang-nursery/simd
$ cd simd
$ git checkout v0.2.0
$ export RUSTC_BOOTSTRAP=1 # cheat Nightly requirement
$ cargo build --release
   Compiling simd v0.2.0 (file:///tmp/simd)
error[E0432]: unresolved import `x86::sse2`
  --> src/common.rs:17:10
   |
17 | use x86::sse2::common;
   |          ^^^^ Could not find `sse2` in `x86`

error: aborting due to previous error

$ rustc --print target-features
Target features help is not supported by this LLVM version.
Comment 19 Jan Beich freebsd_committer 2017-10-29 22:55:13 UTC
FWIW, PORT_LLVM=on also means you're going to miss out on bug 223314.
Comment 20 VVD 2017-10-30 00:28:36 UTC
(In reply to vvd from comment #16)
>>> OPTIONS_FILE_SET+=PORT_LLVM
>> Can you try without?
> In progress. Will write result in about 1 hour… I hope. :-D
Tested. Firefox build fine on amd64 using rust with OPTIONS_FILE_UNSET+=PORT_LLVM and CPUTYPE?=core2 in /etc/make.conf.

Testing i386.
Comment 21 Dimitry Andric freebsd_committer 2017-10-30 07:45:10 UTC
Right, after rebuilding rust without PORT_LLVM, I get the features list:

$ rustc --print target-features
Available features for this target:
    16bit-mode             - 16-bit mode (i8086).
    32bit-mode             - 32-bit mode (80386).
    3dnow                  - Enable 3DNow! instructions.
    3dnowa                 - Enable 3DNow! Athlon instructions.
    64bit                  - Support 64-bit instructions.
    64bit-mode             - 64-bit mode (x86_64).
    adx                    - Support ADX instructions.
    aes                    - Enable AES instructions.
    atom                   - Intel Atom processors.
    avx                    - Enable AVX instructions.
    avx2                   - Enable AVX2 instructions.
    avx512bw               - Enable AVX-512 Byte and Word Instructions.
    avx512cd               - Enable AVX-512 Conflict Detection Instructions.
    avx512dq               - Enable AVX-512 Doubleword and Quadword Instructions.
    avx512er               - Enable AVX-512 Exponential and Reciprocal Instructions.
    avx512f                - Enable AVX-512 instructions.
    avx512ifma             - Enable AVX-512 Integer Fused Multiple-Add.
    avx512pf               - Enable AVX-512 PreFetch Instructions.
    avx512vbmi             - Enable AVX-512 Vector Byte Manipulation Instructions.
    avx512vl               - Enable AVX-512 Vector Length eXtensions.
    bmi                    - Support BMI instructions.
    bmi2                   - Support BMI2 instructions.
    call-reg-indirect      - Call register indirect.
    clflushopt             - Flush A Cache Line Optimized.
    clwb                   - Cache Line Write Back.
    cmov                   - Enable conditional move instructions.
    cx16                   - 64-bit with cmpxchg16b.
    f16c                   - Support 16-bit floating point conversion instructions.
    fast-lzcnt             - LZCNT instructions are as fast as most simple integer ops.
    fast-partial-ymm-write - Partial writes to YMM registers are fast.
    fast-scalar-fsqrt      - Scalar SQRT is fast (disable Newton-Raphson).
    fast-vector-fsqrt      - Vector SQRT is fast (disable Newton-Raphson).
    fma                    - Enable three-operand fused multiple-add.
    fma4                   - Enable four-operand fused multiple-add.
    fsgsbase               - Support FS/GS Base instructions.
    fxsr                   - Support fxsave/fxrestore instructions.
    hle                    - Support HLE.
    idivl-to-divb          - Use 8-bit divide for positive values less than 256.
    idivq-to-divl          - Use 32-bit divide for positive values less than 2^32.
    invpcid                - Invalidate Process-Context Identifier.
    lea-sp                 - Use LEA for adjusting the stack pointer.
    lea-uses-ag            - LEA instruction needs inputs at AG stage.
    lzcnt                  - Support LZCNT instruction.
    mmx                    - Enable MMX instructions.
    movbe                  - Support MOVBE instruction.
    mpx                    - Support MPX instructions.
    mwaitx                 - Enable MONITORX/MWAITX timer functionality.
    pad-short-functions    - Pad short functions.
    pclmul                 - Enable packed carry-less multiplication instructions.
    pcommit                - Enable Persistent Commit.
    pku                    - Enable protection keys.
    popcnt                 - Support POPCNT instruction.
    prefetchwt1            - Prefetch with Intent to Write and T1 Hint.
    prfchw                 - Support PRFCHW instructions.
    rdrnd                  - Support RDRAND instruction.
    rdseed                 - Support RDSEED instruction.
    rtm                    - Support RTM instructions.
    sahf                   - Support LAHF and SAHF instructions.
    sgx                    - Enable Software Guard Extensions.
    sha                    - Enable SHA instructions.
    slm                    - Intel Silvermont processors.
    slow-bt-mem            - Bit testing of memory is slow.
    slow-incdec            - INC and DEC instructions are slower than ADD and SUB.
    slow-lea               - LEA instruction with certain arguments is slow.
    slow-pmulld            - PMULLD instruction is slow.
    slow-shld              - SHLD instruction is slow.
    slow-unaligned-mem-16  - Slow unaligned 16-byte memory access.
    slow-unaligned-mem-32  - Slow unaligned 32-byte memory access.
    smap                   - Supervisor Mode Access Protection.
    soft-float             - Use software floating point features..
    sse                    - Enable SSE instructions.
    sse-unaligned-mem      - Allow unaligned memory operands with SSE instructions.
    sse2                   - Enable SSE2 instructions.
    sse3                   - Enable SSE3 instructions.
    sse4.1                 - Enable SSE 4.1 instructions.
    sse4.2                 - Enable SSE 4.2 instructions.
    sse4a                  - Support SSE 4a instructions.
    ssse3                  - Enable SSSE3 instructions.
    tbm                    - Enable TBM instructions.
    vmfunc                 - VM Functions.
    x87                    - Enable X87 float instructions.
    xop                    - Enable XOP instructions.
    xsave                  - Support xsave instructions.
    xsavec                 - Support xsavec instructions.
    xsaveopt               - Support xsaveopt instructions.
    xsaves                 - Support xsaves instructions.

Use +feature to enable a feature, or -feature to disable it.
For example, rustc -C -target-cpu=mycpu -C target-feature=+feature1,-feature2

And then www/firefox does build to the end.
Comment 22 VVD 2017-10-30 10:35:14 UTC
(In reply to vvd from comment #20)
>>>> OPTIONS_FILE_SET+=PORT_LLVM
>>> Can you try without?
>> In progress. Will write result in about 1 hour… I hope. :-D
> Tested. Firefox build fine on amd64 using rust with OPTIONS_FILE_UNSET+=PORT_LLVM and CPUTYPE?=core2 in /etc/make.conf.
>
> Testing i386.
Tested on i386: the firefox rebuilded without errors with same configuration for the rust: OPTIONS_FILE_UNSET+=PORT_LLVM.

The rust's option PORT_LLVM is the key.
Comment 23 jakub_lach 2017-10-30 17:58:06 UTC
Same here, PORT_LLVM. FreeBSD 11.1-STABLE #0 r324952 amd64, core2 CPU.

If I'm not mistaken, PORT_LLVM uses already installed port's LLVM if present (not building the one bundled with rust), thus speeding up things considerably?
Comment 24 Jan Beich freebsd_committer 2017-11-13 09:31:01 UTC
*** Bug 223570 has been marked as a duplicate of this bug. ***
Comment 25 commit-hook freebsd_committer 2017-11-27 18:16:38 UTC
A commit references this bug:

Author: dumbbell
Date: Mon Nov 27 18:16:15 UTC 2017
New revision: 454983
URL: https://svnweb.freebsd.org/changeset/ports/454983

Log:
  lang/rust: Remove the LLVM_PORT option

  FTR, this option allowed to used LLVM from ports instead of building the
  bundled copy.

  The problem is that this combination isn't really tested upstream. This
  led to regressions which are difficult to diagnose. For instance, in
  Rust 1.21.0, the bundled LLVM provided a new API to query the features
  supported by the target arch. The equivalent code inside Rust was
  removed to use that new API. Unfortunately, building Rust 1.21.0+ with a
  copy of LLVM not providing this API didn't failed but instead made that
  list of CPU features empty. This resulted in the following obscure build
  failure in Firefox:

      error[E0432]: unresolved import `x86::sse2`

  To avoid future pain for both end users and maintainers, we decided to
  remove that option. Yes, it will increase the (already long) time to
  build Rust, but it should save time wasted on debugging what is not
  really supported anyway.

  PR:		223342, 223300
  Reported by:	Many users

Changes:
  head/lang/rust/Makefile
  head/lang/rust/files/config.toml
Comment 26 Jean-Sébastien Pédron freebsd_committer 2017-11-27 18:19:38 UTC
This is "fixed" in the sense that the LLVM_PORT option is now gone.
Comment 27 VVD 2017-11-28 10:34:49 UTC
After last update of rust-1.22.1 to 1.22.1_1 and firefox-57.0_2,1 to 57.0_3,1 error was returned:
   Compiling simd v0.2.0
error[E0432]: unresolved import `x86::sse2`
  --> /usr/obj/usr/ports/www/firefox/work/firefox-57.0/third_party/rust/simd/src/common.rs:17:10
   |
17 | use x86::sse2::common;
   |          ^^^^ Could not find `sse2` in `x86`

   Compiling term v0.4.5
error: aborting due to previous error

error: Could not compile `simd`.
warning: build failed, waiting for other jobs to finish...
error: build failed
========================
It's FreeBSD 11.1 i386. On 11.1 amd64 no errors.
Comment 28 Jean-Sébastien Pédron freebsd_committer 2017-11-28 10:52:48 UTC
What is the output of `rustc --print target-features`?
Comment 29 VVD 2017-11-28 11:15:53 UTC
I think this commit broke ff build:
https://svnweb.freebsd.org/ports/head/lang/rust/files/patch-src_librustc__back_target_i686__unknown__freebsd.rs?revision=454995&view=markup&sortby=date

The test in progress - I'll confirm it soon.
Comment 30 Jan Beich freebsd_committer 2017-11-28 11:47:54 UTC
Probably because -march= and -C target-cpu= are out of sync e.g., when CPUTYPE is defined in make.conf. The fix is to pass RUSTFLAGS+=-C target-feature=+sse2 explicitly on i386 when MACHINE_CPU contains sse2.
Comment 31 VVD 2017-11-28 11:52:57 UTC
Ofc CPUTYPE defined: CPUTYPE?=prescott
The fix is to check CPUTYPE and turn on/off SSE2 automatically instead of turn off it on ALL i386 by default.
Comment 32 commit-hook freebsd_committer 2017-11-28 12:58:43 UTC
A commit references this bug:

Author: jbeich
Date: Tue Nov 28 12:57:57 UTC 2017
New revision: 455039
URL: https://svnweb.freebsd.org/changeset/ports/455039

Log:
  gecko: respect CPUTYPE for Rust code

  After r454995 build may fail if SSE2 is only enabled for C/C++ code as
  simd crate expects SSE2 based on MACHINE_CPU.

  PR:		223415 223300
  Reported by:	vvd@unislabs.com

Changes:
  head/Mk/bsd.gecko.mk
Comment 33 Jan Beich freebsd_committer 2017-11-28 12:59:36 UTC
Now rustc can emit SSE3 for CPUTYPE=prescott as well. Whether it will probably depends on auto-vectorizer.
Comment 34 VVD 2017-11-28 14:07:45 UTC
$ rustc --print target-features | grep sse
    sse                    - Enable SSE instructions.
    sse-unaligned-mem      - Allow unaligned memory operands with SSE instructions.
    sse2                   - Enable SSE2 instructions.
    sse3                   - Enable SSE3 instructions.
    sse4.1                 - Enable SSE 4.1 instructions.
    sse4.2                 - Enable SSE 4.2 instructions.
    sse4a                  - Support SSE 4a instructions.
    ssse3                  - Enable SSSE3 instructions.

With rust-1.22.1 firefox-57.0_3,1 build fine.

Testing build with rust-1.22.1_1 and new Mk/bsd.gecko.mk now.
Comment 35 VVD 2017-11-28 18:39:23 UTC
> Testing build with rust-1.22.1_1 and new Mk/bsd.gecko.mk now.
Test passed without errors.
Comment 36 Mikhail T. 2017-12-15 14:06:02 UTC
(In reply to commit-hook from comment #32)

The commit changes gecko.mk to derive the target-cpu setting given to rustc from the CFLAGS. This is a very elegant solution and would've been perfect, if not for the common case of "-march=native".

Because, sadly, "native" to rustc currently means "generic", instead of "check the features of the currently-used processor and enable them all". And "generic" means "no SSE2".

And, because it means generic, the firefox build fails with that same error about SSE2, when CPUTYPE is set to "native".

I'm not sure, how to fix it correctly, but a warning advising user to set CPUTYPE explicitly may be helpful.
Comment 37 Jan Beich freebsd_committer 2017-12-15 14:13:16 UTC
(In reply to Mikhail T. from comment #36)
> "native" to rustc currently means "generic"

I cannot reproduce.

$ RUSTC_BOOTSTRAP=1 rustc -C target-cpu=native --print cfg
debug_assertions
target_arch="x86"
target_endian="little"
target_env=""
target_family="unix"
target_feature="avx"
target_feature="avx2"
target_feature="bmi"
target_feature="bmi2"
target_feature="fma"
target_feature="lzcnt"
target_feature="popcnt"
target_feature="rdrnd"
target_feature="rdseed"
target_feature="sse"
target_feature="sse2"
target_feature="sse3"
target_feature="sse4.1"
target_feature="sse4.2"
target_feature="ssse3"
target_has_atomic="16"
target_has_atomic="32"
target_has_atomic="64"
target_has_atomic="8"
target_has_atomic="ptr"
target_os="freebsd"
target_pointer_width="32"
target_vendor="unknown"
unix
Comment 38 Mikhail T. 2017-12-15 14:41:43 UTC
> I cannot reproduce.

> $ RUSTC_BOOTSTRAP=1 rustc -C target-cpu=native --print cfg

For me the above command prints something different:

    debug_assertions
    target_arch="x86"
    target_endian="little"
    target_env=""
    target_family="unix"
    target_os="freebsd"
    target_pointer_width="32"
    unix

Maybe, that's because I compiled rust itself (1.22.1) with "march=native"? 

BTW, if you are trying to set RUSTFLAGS intelligently in bsd.gecko.mk, you may also want to add the `-C codegen-units=${MAKE_JOBS_NUMBER}` to parallelize the compilations.
Comment 39 Mikhail T. 2017-12-15 14:44:13 UTC
(In reply to Mikhail T. from comment #38)
> For me the above command prints something different:

Err, sorry, with the RUSTC_BOOTSTRAP=1 I get the longer list below -- but still without SSE2:

debug_assertions
target_arch="x86"
target_endian="little"
target_env=""
target_family="unix"
target_has_atomic="16"
target_has_atomic="32"
target_has_atomic="64"
target_has_atomic="8"
target_has_atomic="ptr"
target_os="freebsd"
target_pointer_width="32"
target_vendor="unknown"
unix
Comment 40 Jan Beich freebsd_committer 2017-12-15 14:55:35 UTC
(In reply to Mikhail T. from comment #38)
> Maybe, that's because I compiled rust itself (1.22.1) with "march=native"? 

Well, I use CPUTYPE=native myself (on -CURRENT amd64) and on head-i386 poudriere jail. Obviously, MACHINE_CPU had to be populated manually because bsd.cpu.mk is dumb (see bug 112997).

> may also want to add the `-C codegen-units=${MAKE_JOBS_NUMBER}` to parallelize the compilations.

No, -C codegen-units pessimizes LTO which is enabled by default for release builds. ThinLTO isn't ready yet.

https://internals.rust-lang.org/t/help-test-out-thinlto/6017
Comment 41 Mikhail T. 2017-12-15 16:08:26 UTC
(In reply to Jan Beich from comment #40)
> Well, I use CPUTYPE=native myself (on -CURRENT amd64) and on head-i386

So, why would my rustc have a different cfg-output? I have it built with the following options:

# Options for rust-1.22.1_2
_OPTIONS_READ=rust-1.22.1_2
_FILE_COMPLETE_OPTIONS_LIST=DOCS GDB LLNEXTGEN SOURCES
OPTIONS_FILE_UNSET+=DOCS
OPTIONS_FILE_UNSET+=GDB
OPTIONS_FILE_SET+=LLNEXTGEN
OPTIONS_FILE_UNSET+=SOURCES

The CPUs in this machine are old -- could it be, it does not find SSE3 (or some other later addition) and presumes, SSE2 is not there either?
Comment 42 Jan Beich freebsd_committer 2017-12-15 17:11:12 UTC
Try using freebsd.org package. If it works bisect your environment by replicating it inside a poudriere jail or find another user with the same issue. If it doesn't work try checking if llvm40 has working -march=native then try booting a FreeBSD kernel from newer release (keep COMPAT_* in config).