Bug 194641

Summary: [EFI] boot/loader.efi: miscompilation on Intel Haswell with AVX2 and CPUTYPE=native
Product: Base System Reporter: O. Hartmann <ohartmann>
Component: kernAssignee: Warner Losh <imp>
Status: Closed FIXED    
Severity: Affects Some People CC: bcran, emaste, grahamperrin, imp
Priority: --- Keywords: uefi
Version: CURRENTFlags: imp: mfc-stable13+
imp: mfc-stable12+
imp: mfc-stable11-
Hardware: amd64   
OS: Any   
Bug Depends on:    
Bug Blocks: 231027    

Description O. Hartmann 2014-10-27 19:24:42 UTC
Compileing world on Intel Haswell with CPUTYPE?=native set in either /etc/make.conf and/or /etc/src.conf renders /boot/loader.efi to fail: the EFI boot process stops and freezes when loader.efi is loaded.

The problem affects only Haswell-based hardware (from my point of view, it seems to be AVX2), since Sandy_bridge and Ivy-Bridge based systems do not show the problem, even if world/kernel is compiled with -O3 and -march=native as I have tested on several CURRENT systems with most recent CURRENT as of today (FreeBSD 11.0-CURRENT #2 r273743: Mon Oct 27 19:51:01 CET 2014 amd64).

The problem has been discussed recently on the CURRENT mailing list, but no PR has been issued so far.

Please look at 

http://freebsd.1045724.n5.nabble.com/CURRENT-EFI-boot-failure-td5949387.html

oh
Comment 1 Ed Maste freebsd_committer 2014-11-26 19:34:28 UTC
Note, adding -mno-avx2 reported not to help:
https://lists.freebsd.org/pipermail/freebsd-current/2014-September/052351.html

It appears the flags are not passed into libefi.
Comment 2 Warner Losh freebsd_committer 2017-11-20 21:06:55 UTC
Does CPUTYPE=haswell also cause this issue?

And more importantly, this was from 3 years ago...  Can someone confirm the problem still exists...
Comment 3 Warner Losh freebsd_committer 2017-11-20 23:43:22 UTC
I think that https://svnweb.freebsd.org/changeset/base/326038 fixed this. Can someone who is in a position to test please confirm?
Comment 4 Eitan Adler freebsd_committer freebsd_triage 2018-05-20 23:50:02 UTC
For bugs matching the following conditions:
- Status == In Progress
- Assignee == "bugs@FreeBSD.org"
- Last Modified Year <= 2017

Do
- Set Status to "Open"
Comment 5 Ed Maste freebsd_committer 2018-08-31 18:25:52 UTC
I just tried this again in my WIP branch and loader.efi still hangs with CPUTYPE?=native in src.conf.
Comment 6 Ed Maste freebsd_committer 2018-08-31 18:35:52 UTC
comparing builds with and without the src.conf setting shows -march=native in the cc commandline
Comment 7 Warner Losh freebsd_committer 2018-08-31 19:52:41 UTC
I know how to paper over this bug :)
Comment 8 Ed Maste freebsd_committer 2019-09-16 19:31:24 UTC
Any update @imp?
Comment 9 Rebecca Cran freebsd_committer 2019-09-16 19:39:14 UTC
I'm starting to think there's a miscompilation or other build problem in general, too. I've been seeing memory corruption when using the new HTTP boot feature that appears to be in our binary.
Comment 10 Warner Losh freebsd_committer 2021-07-15 00:55:55 UTC
Does this still happen?
Comment 11 Warner Losh freebsd_committer 2021-07-15 02:03:15 UTC
https://reviews.freebsd.org/D31187

looks like it is still the case, but I get there by inspection. The above review will fix it, however.
Comment 12 commit-hook freebsd_committer 2021-07-15 03:07:20 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=7a0c0ff7ee25ff1efa3dbedcb20d2c80178639cc

commit 7a0c0ff7ee25ff1efa3dbedcb20d2c80178639cc
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2021-07-15 03:06:08 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2021-07-15 03:06:19 +0000

    loader: make sure CPUTYPE is ignored when building

    CPUTYPE?=native causes -march=native to be added to the command
    line. When the host machine is haswell, this causes some versions of
    clang to generate code that can't execute in the efi boot loader
    environment. Set _CPUCFLAGS= to undo what's done bsd.cpu.mk. bsd.cpu.mk
    is included too early to control with NO_CPU_CFLAGS here. The only other
    option is to put that in all the Makefiles, and this is less tedious and
    error prone.

    PR:                     194641
    Sponsored by:           Netflix
    Differential Revision:  https://reviews.freebsd.org/D31187
    MFC After:              1 week

 stand/defs.mk | 8 ++++++++
 1 file changed, 8 insertions(+)
Comment 13 Warner Losh freebsd_committer 2021-07-15 03:11:17 UTC
Please try out the committed changed. It looks to fix the problem for me.
It's in my MFC queue. If there's still an issue, please reopen. Sorry it took so long to resolve.
Comment 14 O. Hartmann 2021-07-15 10:57:56 UTC
Thanks.
Comment 15 Kubilay Kocak freebsd_committer freebsd_triage 2021-07-20 06:04:40 UTC
^Triage: Re-open pending merge
Comment 16 commit-hook freebsd_committer 2021-07-21 16:14:27 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=5ad4296780ded135e8cd48a3877b52c4a7f4cb1d

commit 5ad4296780ded135e8cd48a3877b52c4a7f4cb1d
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2021-07-15 03:06:08 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2021-07-21 16:13:10 +0000

    loader: make sure CPUTYPE is ignored when building

    CPUTYPE?=native causes -march=native to be added to the command
    line. When the host machine is haswell, this causes some versions of
    clang to generate code that can't execute in the efi boot loader
    environment. Set _CPUCFLAGS= to undo what's done bsd.cpu.mk. bsd.cpu.mk
    is included too early to control with NO_CPU_CFLAGS here. The only other
    option is to put that in all the Makefiles, and this is less tedious and
    error prone.

    PR:                     194641
    Sponsored by:           Netflix
    Differential Revision:  https://reviews.freebsd.org/D31187
    MFC After:              1 week

    (cherry picked from commit 7a0c0ff7ee25ff1efa3dbedcb20d2c80178639cc)

 stand/defs.mk | 8 ++++++++
 1 file changed, 8 insertions(+)
Comment 17 commit-hook freebsd_committer 2021-07-21 16:19:34 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=ee6908d78275449011f8fb1b6d73bde75c02f705

commit ee6908d78275449011f8fb1b6d73bde75c02f705
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2021-07-15 03:06:08 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2021-07-21 16:16:32 +0000

    loader: make sure CPUTYPE is ignored when building

    CPUTYPE?=native causes -march=native to be added to the command
    line. When the host machine is haswell, this causes some versions of
    clang to generate code that can't execute in the efi boot loader
    environment. Set _CPUCFLAGS= to undo what's done bsd.cpu.mk. bsd.cpu.mk
    is included too early to control with NO_CPU_CFLAGS here. The only other
    option is to put that in all the Makefiles, and this is less tedious and
    error prone.

    PR:                     194641
    Sponsored by:           Netflix
    Differential Revision:  https://reviews.freebsd.org/D31187
    MFC After:              1 week

    (cherry picked from commit 7a0c0ff7ee25ff1efa3dbedcb20d2c80178639cc)

 stand/defs.mk | 8 ++++++++
 1 file changed, 8 insertions(+)
Comment 18 Warner Losh freebsd_committer 2021-07-21 16:47:42 UTC
fixed in all supported branches. I no longer support nanobsd in stable/11 since that branch is at end of life.