Bug 211864 - LLVM is using AVX instructions for CPUTYPE=btver1 (AMD Bobcat)
Summary: LLVM is using AVX instructions for CPUTYPE=btver1 (AMD Bobcat)
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 11.0-RC1
Hardware: amd64 Any
: Normal Affects Some People
Assignee: Dimitry Andric
URL:
Keywords: needs-qa, patch, regression
Depends on:
Blocks:
 
Reported: 2016-08-15 09:58 UTC by Michel Depeige
Modified: 2016-08-20 13:37 UTC (History)
4 users (show)

See Also:
dim: mfc-stable9-
dim: mfc-stable10-
dim: mfc-stable11+


Attachments
Import upstream fix to stop using xsave/avx on btver1 CPUs (1.47 KB, patch)
2016-08-15 17:21 UTC, Dimitry Andric
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michel Depeige 2016-08-15 09:58:11 UTC
Hello,

While rebuilding some ports after upgrading to 11.0-RC1 (with freebsd-update), almost every binary is crashing with SIGILL 

The system hardware is bobcat based :
CPU: AMD C-70 APU with Radeon(tm) HD Graphics (1000.02-MHz K8-class CPU)

Bobcat CPUs have only MMX, SSE, SSE2, SSE3, SSSE3, SSE4A instructions.

Compiling without CPUTYPE results in working binaries.
With CPUTYPE=btver1, the binaries does not work, as the CPU does not have any AVX instructions :

root@square:~ # lldb /usr/local/bin/sudo
(lldb) target create "/usr/local/bin/sudo"
Current executable set to '/usr/local/bin/sudo' (x86_64).
(lldb) run test
Process 71447 launching
Process 71447 launched: '/usr/local/bin/sudo' (x86_64)
Process 71447 stopped
* thread #1: tid = 101195, 0x0000000800a5421e libsudo_util.so.0`??? + 286, stop reason = signal SIGILL: privileged instruction
    frame #0: 0x0000000800a5421e libsudo_util.so.0`??? + 286
libsudo_util.so.0`???:
->  0x800a5421e <+286>: vzeroupper 
    0x800a54221 <+289>: callq  0x800a511e4               ; symbol stub for: sudo_debug_exit_int_v1
    0x800a54226 <+294>: movq   0x20ad93(%rip), %rax      ; libsudo_util.so.0..got + 72
    0x800a5422d <+301>: movq   (%rax), %rax

Some information :

root@square:~ # uname -a
FreeBSD square 11.0-RC1 FreeBSD 11.0-RC1 #0 r303979: Fri Aug 12 02:28:24 UTC 2016     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

root@square:~ # cc -v
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
Target: x86_64-unknown-freebsd11.0
Thread model: posix
InstalledDir: /usr/bin

Everything was working fine while running 10.3-RELEASE

this bug might be related to this one :
https://reviews.llvm.org/D17682

Regards
Comment 1 Dimitry Andric freebsd_committer 2016-08-15 17:21:57 UTC
Created attachment 173705 [details]
Import upstream fix to stop using xsave/avx on btver1 CPUs

Michel, can you please try out this patch?  As far as I can see, this causes clang to not generate AVX instructions anymore, at least for that sudo port.

But I would appreciate independent verification. :)
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2016-08-16 11:07:05 UTC
Setting high MFC expectations, but at least stable/11 (unknown impact on previous branches)

@Dim please set mfc-stableX to + *after* MFC, or set to - with comment why its not necessary/possible
Comment 3 Michel Depeige 2016-08-17 19:50:43 UTC
Quick follow : I'm trying to rebuild the world with the patch applied, but the box is overheating. I'm trying to sort that out before winter :)
Comment 4 commit-hook freebsd_committer 2016-08-17 21:58:00 UTC
A commit references this bug:

Author: dim
Date: Wed Aug 17 21:57:11 UTC 2016
New revision: 304319
URL: https://svnweb.freebsd.org/changeset/base/304319

Log:
  Pull in r262772 from upstream clang trunk (by Simon Pilgrim):

    [X86] AMD Bobcat CPU (btver1) doesn't support XSAVE

    btver1 is a SSSE3/SSE4a only CPU - it doesn't have AVX and doesn't
    support XSAVE.

    Differential Revision: http://reviews.llvm.org/D17682

  Pull in r262782 from upstream llvm trunk (by Simon Pilgrim):

    [X86] AMD Bobcat CPU (btver1) doesn't support XSAVE

    btver1 is a SSSE3/SSE4a only CPU - it doesn't have AVX and doesn't
    support XSAVE.

    Differential Revision: http://reviews.llvm.org/D17683

  This ensures clang does not emit AVX instructions for CPUTYPE=btver1.

  Reported by:	Michel Depeige <demik+freebsd@lostwave.net>
  PR:		211864
  MFC after:	3 days

Changes:
  head/contrib/llvm/lib/Target/X86/X86.td
  head/contrib/llvm/tools/clang/lib/Basic/Targets.cpp
  head/lib/clang/freebsd_cc_version.h
Comment 5 Dimitry Andric freebsd_committer 2016-08-17 22:16:45 UTC
This only applies to clang 3.8.0, there's no need to merge it back to stable/9 and stable/10, as they have clang 3.4.1.
Comment 6 Michel Depeige 2016-08-18 19:30:42 UTC
Hello,

the patch is working as expected : no more vzeroupper !
I have rebuild successfully all my ports, even kernel, everything is working fine

Thanks for your help !

Will this fix be in RC2 ?

Regards
Comment 7 Dimitry Andric freebsd_committer 2016-08-18 20:01:53 UTC
(In reply to Michel Depeige from comment #6)
> Will this fix be in RC2 ?

I will merge it to stable/11 on August 20, when the timeout expires.  Then I will contact the Release Engineering team for permission to merge it to the 11.0 release branch.

This is not really a showstopper though, since you can work around it relatively easily by setting your CPUTYPE to something else.  It is up to the Release Engineering team to decide.
Comment 8 commit-hook freebsd_committer 2016-08-20 12:49:36 UTC
A commit references this bug:

Author: dim
Date: Sat Aug 20 12:49:06 UTC 2016
New revision: 304528
URL: https://svnweb.freebsd.org/changeset/base/304528

Log:
  MFC r304319:

  Pull in r262772 from upstream clang trunk (by Simon Pilgrim):

    [X86] AMD Bobcat CPU (btver1) doesn't support XSAVE

    btver1 is a SSSE3/SSE4a only CPU - it doesn't have AVX and doesn't
    support XSAVE.

    Differential Revision: http://reviews.llvm.org/D17682

  Pull in r262782 from upstream llvm trunk (by Simon Pilgrim):

    [X86] AMD Bobcat CPU (btver1) doesn't support XSAVE

    btver1 is a SSSE3/SSE4a only CPU - it doesn't have AVX and doesn't
    support XSAVE.

    Differential Revision: http://reviews.llvm.org/D17683

  This ensures clang does not emit AVX instructions for CPUTYPE=btver1.

  Reported by:	Michel Depeige <demik+freebsd@lostwave.net>
  PR:		211864

Changes:
_U  stable/11/
  stable/11/contrib/llvm/lib/Target/X86/X86.td
  stable/11/contrib/llvm/tools/clang/lib/Basic/Targets.cpp
  stable/11/lib/clang/freebsd_cc_version.h
Comment 9 commit-hook freebsd_committer 2016-08-20 13:30:41 UTC
A commit references this bug:

Author: dim
Date: Sat Aug 20 13:29:59 UTC 2016
New revision: 304529
URL: https://svnweb.freebsd.org/changeset/base/304529

Log:
  MFC r304319:

  Pull in r262772 from upstream clang trunk (by Simon Pilgrim):

    [X86] AMD Bobcat CPU (btver1) doesn't support XSAVE

    btver1 is a SSSE3/SSE4a only CPU - it doesn't have AVX and doesn't
    support XSAVE.

    Differential Revision: http://reviews.llvm.org/D17682

  Pull in r262782 from upstream llvm trunk (by Simon Pilgrim):

    [X86] AMD Bobcat CPU (btver1) doesn't support XSAVE

    btver1 is a SSSE3/SSE4a only CPU - it doesn't have AVX and doesn't
    support XSAVE.

    Differential Revision: http://reviews.llvm.org/D17683

  This ensures clang does not emit AVX instructions for CPUTYPE=btver1.

  Approved by:	re (kib)
  Reported by:	Michel Depeige <demik+freebsd@lostwave.net>
  PR:		211864

Changes:
_U  releng/11.0/
  releng/11.0/contrib/llvm/lib/Target/X86/X86.td
  releng/11.0/contrib/llvm/tools/clang/lib/Basic/Targets.cpp
  releng/11.0/lib/clang/freebsd_cc_version.h