Bug 207837 - www/firefox: clang34 and clang35 crash on i386 with -O2 -fstack-protector (OPTIMIZED_CFLAGS=off)
Summary: www/firefox: clang34 and clang35 crash on i386 with -O2 -fstack-protector (OP...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: i386 Any
: --- Affects Only Me
Assignee: freebsd-gecko (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-03-09 11:52 UTC by Vikash Badal
Modified: 2016-03-13 19:13 UTC (History)
4 users (show)

See Also:
jbeich: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Vikash Badal 2016-03-09 11:52:02 UTC
www/firefox fails on i386 in poudriere

fatal error: error in backend: ran out of registers during register allocation
c++: error: clang frontend command failed with exit code 70 (use -v to see invocation)
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
Target: i386-unknown-freebsd10.2
Thread model: posix
c++: note: diagnostic msg: PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace, preprocessed source, and associated run script.
c++: error: unable to execute command: Segmentation fault (core dumped)
c++: note: diagnostic msg: Error generating preprocessed source(s).
/wrkdirs/usr/ports/www/firefox/work/firefox-45.0/config/rules.mk:956: recipe for target 'Unified_cpp_protocol_websocket0.o' failed

how to replicate:

poudriere testport -j RELENG_10_2_i386 -o www/firefox

full log:
http://anger.where-ever.za.net/~vikashb/firefox-45.0_1,1.log
Comment 1 Jan Beich freebsd_committer 2016-03-09 12:36:26 UTC
(In reply to Vikash Badal from comment #0)
> http://anger.where-ever.za.net/~vikashb/firefox-45.0_1,1.log

fetch(1) says "Connection refused".

> fatal error: error in backend: ran out of registers during register allocation
[...]
> diagnostic msg: PLEASE submit ... and include the crash backtrace, preprocessed source, and associated run script.

I don't have 10.2 i386 jail set up but at least 10.1 i386 builds fine[1]. Can you follow the instruction, so clang maintainer has more chance reproducing the crash outside of firefox build?

[1] http://beefy5.nyi.freebsd.org/data/101i386-default/410591/logs/firefox-45.0_1,1.log
Comment 2 Vikash Badal 2016-03-09 14:24:36 UTC
no sure how to obtain this info

lldb37 c++.core

(lldb) bt
error: invalid process


please advise
Comment 3 Jan Beich freebsd_committer 2016-03-10 06:53:39 UTC
lldb37 $(which c++) --core c++.core and if you've built world with symbols (e.g. DEBUG_FLAGS=-g) it'll show something like the following. After checking your full log I can reproduce it on lang/clang34, lang/clang35, /usr/bin/clang on 10.1, 10.2, 10.3 with -m32 -O2 -fstack-protector. It doesn't crash with -O0, -O1, -O3. As a workaround try building with OPTIMIZED_CFLAGS=on, using lang/clang3[6-8] or lang/gcc*.

  (lldb) bt
  * thread #1: tid = 100230, 0x00000008064fae9a libc.so.7`thr_kill + 10 at thr_kill.S:3, name = 'clang', stop reason = signal SIGABRT
    * frame #0: 0x00000008064fae9a libc.so.7`thr_kill + 10 at thr_kill.S:3
      frame #1: 0x00000008064fae6b libc.so.7`__raise(s=6) + 59 at raise.c:52 [opt]
      frame #2: 0x00000008064fae26 libc.so.7`abort + 150 at abort.c:77 [opt]
      frame #3: 0x000000080657e931 libc.so.7`__assert(func=<unavailable>, file=<unavailable>, line=<unavailable>, failedexpr=<unavailable>) + 81 at assert.c:51 [opt]
      frame #4: 0x0000000001f63483 clang`clang::Lexer::resetExtendedTokenMode(this=0x00007fffffff8680) + 67 at Lexer.cpp:134
      frame #5: 0x0000000001f6c958 clang`clang::Lexer::LexEndOfFile(this=0x00007fffffff8680, Result=0x00007fffffff85c8, CurPtr="") + 88 at Lexer.cpp:2463
      frame #6: 0x0000000001f6dbb4 clang`clang::Lexer::LexTokenInternal(this=0x00007fffffff8680, Result=0x00007fffffff85c8, TokAtPhysicalStartOfLine=false) + 420 at Lexer.cpp:2915
      frame #7: 0x0000000001f6c8a8 clang`clang::Lexer::Lex(this=0x00007fffffff8680, Result=0x00007fffffff85c8) + 216 at Lexer.cpp:2866
      frame #8: 0x000000000063c2f3 clang`clang::Lexer::LexFromRawLexer(this=0x00007fffffff8680, Result=0x00007fffffff85c8) + 83 at Lexer.h:156
      frame #9: 0x0000000001aa50e5 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 1624), FileType=C_User) + 3989 at InclusionRewriter.cpp:495
      frame #10: 0x0000000001aa4667 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 1622), FileType=C_User) + 1303 at InclusionRewriter.cpp:401
      frame #11: 0x0000000001aa4667 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 667), FileType=C_User) + 1303 at InclusionRewriter.cpp:401
      frame #12: 0x0000000001aa4667 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 666), FileType=C_User) + 1303 at InclusionRewriter.cpp:401
      frame #13: 0x0000000001aa4667 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 665), FileType=C_User) + 1303 at InclusionRewriter.cpp:401
      frame #14: 0x0000000001aa4667 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 9), FileType=C_User) + 1303 at InclusionRewriter.cpp:401
      frame #15: 0x0000000001aa4667 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 8), FileType=C_User) + 1303 at InclusionRewriter.cpp:401
      frame #16: 0x0000000001aa4667 clang`(anonymous namespace)::InclusionRewriter::Process(this=0x0000000806c5f0f0, FileId=(ID = 1), FileType=C_User) + 1303 at InclusionRewriter.cpp:401
      frame #17: 0x0000000001aa3f83 clang`clang::RewriteIncludesInInput(PP=0x0000000806c50800, OS=0x0000000806c1a980, Opts=0x0000000806c37408) + 579 at InclusionRewriter.cpp:548
      frame #18: 0x0000000001aa19bf clang`clang::RewriteIncludesAction::ExecuteAction(this=0x0000000806c1a0c0) + 175 at FrontendActions.cpp:190
      frame #19: 0x0000000000633b37 clang`clang::FrontendAction::Execute(this=0x0000000806c1a0c0) + 183 at FrontendAction.cpp:378
      frame #20: 0x00000000005f196e clang`clang::CompilerInstance::ExecuteAction(this=0x0000000806c34000, Act=0x0000000806c1a0c0) + 846 at CompilerInstance.cpp:707
      frame #21: 0x00000000005b0996 clang`clang::ExecuteCompilerInvocation(Clang=0x0000000806c34000) + 1958 at ExecuteCompilerInvocation.cpp:236
      frame #22: 0x000000000059a281 clang`cc1_main(ArgBegin=0x00007fffffffd028, ArgEnd=0x00007fffffffd400, Argv0="/usr/local/llvm34/bin/clang", MainAddr=0x00000000005a78e0) + 993 at cc1_main.cpp:100
      frame #23: 0x00000000005a7cc6 clang`main(argc_=125, argv_=0x00007fffffffd888) + 806 at driver.cpp:314
      frame #24: 0x00000000005994cf clang`_start(ap=<unavailable>, cleanup=<unavailable>) + 383 at crt1.c:72 [opt]
Comment 4 Jan Beich freebsd_committer 2016-03-10 07:04:38 UTC
(In reply to Vikash Badal from comment #0)
> c++: note: diagnostic msg: Error generating preprocessed source(s).

This probably means we have to shave Unified_cpp_protocol_websocket0.cpp of extra code manually. Firefox no longer supports non-unified build.

Steps to reproduce:
$ cd www/firefox
$ make
$ pkg install clang34
$ cd $(make -V WRKSRC)
$ clang++34 -m32 -o Unified_cpp_protocol_websocket0.o -c -I obj-*/dist/stl_wrappers -I obj-*/dist/system_wrappers -include config/gcc_hidden.h -DOS_POSIX=1 -DOS_FREEBSD=1 -DOS_BSD=1 -DSTATIC_EXPORTABLE_JS_API -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -Inetwerk/protocol/websocket -I. -I obj-*/ipc/ipdl/_ipdlheaders -Iipc/chromium/src -Iipc/glue -Idom/base -Inetwerk/base -I obj-*/dist/include  -I/usr/local/include/nspr -I/usr/local/include/nss -I/usr/local/include/nss/nss -I/usr/local/include -I/usr/local/include  -I/usr/local/include/pixman-1   -fPIC  -DMOZILLA_CLIENT -include obj-*/mozilla-config.h -Qunused-arguments -isystem/usr/local/include -DLIBICONV_PLUG  -Qunused-arguments -Wall -Wempty-body -Woverloaded-virtual -Wsign-compare -Wwrite-strings -Wno-invalid-offsetof -Wno-inline-new-delete -Wno-c++0x-extensions -Wno-extended-offsetof -Wno-unknown-warning-option -Wno-return-type-c-linkage -O2 -pipe -DLIBICONV_PLUG -fstack-protector -fno-strict-aliasing -DLIBICONV_PLUG -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -std=gnu++0x -pipe  -DNDEBUG -DTRIMMED -fno-omit-frame-pointer      obj-*/netwerk/protocol/websocket/Unified_cpp_protocol_websocket0.cpp
Comment 5 Dimitry Andric freebsd_committer 2016-03-12 21:59:59 UTC
Some bisection shows that this has been fixed by upstream llvm commit r219512 [1], as a fix for LLVM PR 18883.  It looks like this fix applies without fuzz on clang 3.4, and probably also on clang 3.5.

I'm currently building stable/9 world with it, to see if it causes any problems, but I don't really expect them.  If it works, I will merge it to stable/9.

Brooks, we should probably add this fix to both the llvm34 and llvm35 ports.

[1] https://github.com/llvm-mirror/llvm/commit/d3aa46a1bc5d195f8399d109a13353378516138b
[2] https://llvm.org/bugs/show_bug.cgi?id=18883
Comment 6 commit-hook freebsd_committer 2016-03-13 04:16:53 UTC
A commit references this bug:

Author: brooks
Date: Sun Mar 13 04:15:58 UTC 2016
New revision: 410942
URL: https://svnweb.freebsd.org/changeset/ports/410942

Log:
  Apply r219512 from LLVM which reportedly fixes a crash building firefox.

  PR:		207837

Changes:
  head/devel/llvm34/Makefile
  head/devel/llvm34/files/patch-svn-219512
  head/devel/llvm35/Makefile
  head/devel/llvm35/files/patch-svn-219512
Comment 7 Vikash Badal 2016-03-13 12:12:59 UTC
(In reply to Jan Beich from comment #3)


I tried adding OPTIMIZED_CFLAGS=on to the make.conf for the poudriere jail

---Begin make.conf---
MACHINE=i386
MACHINE_ARCH=i386
ARCH=${MACHINE_ARCH}
USE_PACKAGE_DEPENDS=yes
BATCH=yes
WRKDIRPREFIX=/wrkdirs
PORTSDIR=/usr/ports
PACKAGES=/packages
DISTDIR=/distfiles
#### /usr/local/etc/poudriere.d/RELENG_10_2_i386-make.conf ####
WITH_PKGNG=yes
WITH_GALLIUM="yes"
DEFAULT_VERSIONS=apache=2.2 php=5.6
OPTIMIZED_CFLAGS=on

still fails

full log:

http://anger.where-ever.za.net/~vikashb/firefox-45.0_3,1-OPTIMIZED_CFLAGS_off.log

not sure if i missed the plot here
Comment 8 Jan Beich freebsd_committer 2016-03-13 12:31:57 UTC
(In reply to Vikash Badal from comment #7)
> ---Begin OPTIONS List---
> [...]
>     OPTIMIZED_CFLAGS=off: Use extra compiler optimizations

$ make config -C /usr/ports/www/firefox
<toggle OPTIMIZED_CFLAGS>

I'll add a workaround later via USES=compiler:c++14-lang.
Comment 9 commit-hook freebsd_committer 2016-03-13 14:43:04 UTC
A commit references this bug:

Author: jbeich
Date: Sun Mar 13 14:42:59 UTC 2016
New revision: 410973
URL: https://svnweb.freebsd.org/changeset/ports/410973

Log:
  www/firefox: work around Clang 3.4 crash with OPTIMIZED_CFLAGS=off

  Pretend we want C++14 to pull more modern version of Clang on 10.x i386.

  PR:		207837
  MFH:		2016Q1

Changes:
  head/Mk/bsd.gecko.mk
Comment 10 commit-hook freebsd_committer 2016-03-13 15:17:12 UTC
A commit references this bug:

Author: jbeich
Date: Sun Mar 13 15:17:03 UTC 2016
New revision: 410995
URL: https://svnweb.freebsd.org/changeset/ports/410995

Log:
  MFH: r410973

  www/firefox: work around Clang 3.4 crash with OPTIMIZED_CFLAGS=off

  Pretend we want C++14 to pull more modern version of Clang on 10.x i386.

  PR:		207837
  Approved by:	ports-secteam (junovitch)

Changes:
_U  branches/2016Q1/
  branches/2016Q1/Mk/bsd.gecko.mk
Comment 11 Dimitry Andric freebsd_committer 2016-03-13 15:34:23 UTC
(In reply to Vikash Badal from comment #7)
> still fails
> 
> full log:
> 
> http://anger.where-ever.za.net/~vikashb/firefox-45.0_3,1-
> OPTIMIZED_CFLAGS_off.log
> 
> not sure if i missed the plot here

The patch hasn't been applied to stable/10 nor stable/9 yet, so unless you manually applied it, this is expected.

Alternatively, build with either the clang34 or clang35 port, after updating to ports r410942.
Comment 12 Dimitry Andric freebsd_committer 2016-03-13 16:26:48 UTC
Strangely, I tried building www/firefox on stable/10, from before r410973 was applied, and it builds just fine for me, even the problematic Unified_cpp_protocol_websocket0.cpp file.  I'm unsure what is different in my environment from the original submitter, though.  This is on pretty plain stable/10 r294049, as of 2016-01-21.
Comment 13 Dimitry Andric freebsd_committer 2016-03-13 16:31:10 UTC
Hm, for some reason my compiles get an additional -O3, and a -fomit-frame-pointer.  When the frame pointer is omitted, there is a chance that more registers are available for general use, avoiding the original "ran out of registers" error.

Anybody have an idea why the firefox port gets different compilation flags on different systems?  Note that I don't have any CFLAGS in my /etc/make.conf, so that's not it. :)
Comment 14 Jan Beich freebsd_committer 2016-03-13 17:00:27 UTC
(In reply to Dimitry Andric from comment #13)
OPTIMIZED_CFLAGS is enabled by default. It doesn't pass just -O3 but also --enable-optimize which lets vendor decide whether to pass -fomit-frame-pointer and (in later versions) rust -O.

https://dxr.mozilla.org/mozilla-central/search?q=MOZ_OPTIMIZE
Comment 15 commit-hook freebsd_committer 2016-03-13 18:32:32 UTC
A commit references this bug:

Author: dim
Date: Sun Mar 13 18:32:11 UTC 2016
New revision: 296800
URL: https://svnweb.freebsd.org/changeset/base/296800

Log:
  Pull in r219512 from upstream llvm trunk (by Hal Finkel):

    [MiSched] Fix a logic error in tryPressure()

    Fixes a logic error in the MachineScheduler found by Steve Montgomery
    (and confirmed by Andy). This has gone unfixed for months because the
    fix has been found to introduce some small performance regressions.
    However, Andy has recommended that, at this point, we fix this to
    avoid further dependence on the incorrect behavior (and then
    follow-up separately on any regressions), and I agree.

    Fixes PR18883.

  This fixes a possible "ran out of registers" error when compiling
  www/firefox 45.0 on i386.

  Direct commit to stable/10, because head already has this fix since the
  llvm/clang 3.6.0 import.

  PR:		207837

Changes:
  stable/10/contrib/llvm/lib/CodeGen/MachineScheduler.cpp
Comment 16 commit-hook freebsd_committer 2016-03-13 18:32:35 UTC
A commit references this bug:

Author: dim
Date: Sun Mar 13 18:32:18 UTC 2016
New revision: 296801
URL: https://svnweb.freebsd.org/changeset/base/296801

Log:
  Pull in r219512 from upstream llvm trunk (by Hal Finkel):

    [MiSched] Fix a logic error in tryPressure()

    Fixes a logic error in the MachineScheduler found by Steve Montgomery
    (and confirmed by Andy). This has gone unfixed for months because the
    fix has been found to introduce some small performance regressions.
    However, Andy has recommended that, at this point, we fix this to
    avoid further dependence on the incorrect behavior (and then
    follow-up separately on any regressions), and I agree.

    Fixes PR18883.

  This fixes a possible "ran out of registers" error when compiling
  www/firefox 45.0 on i386.

  Direct commit to stable/9, because head already has this fix since the
  llvm/clang 3.6.0 import.

  PR:		207837

Changes:
  stable/9/contrib/llvm/lib/CodeGen/MachineScheduler.cpp
Comment 17 Dimitry Andric freebsd_committer 2016-03-13 18:39:27 UTC
I don't think it is likely this change will end up in 10.3, so the workaround may need to be kept in place.  Shall we close this bug now?
Comment 18 Jan Beich freebsd_committer 2016-03-13 19:12:56 UTC
(In reply to Dimitry Andric from comment #17)
> Shall we close this bug now?

Maybe only adjust ports workaround for /stable/10 fix. According to Mk/bsd.ssp.mk -fstack-protector isn't enabled on /stable/9.
Comment 19 commit-hook freebsd_committer 2016-03-13 19:13:40 UTC
A commit references this bug:

Author: jbeich
Date: Sun Mar 13 19:13:07 UTC 2016
New revision: 411019
URL: https://svnweb.freebsd.org/changeset/ports/411019

Log:
  www/firefox: limit r410973 to OSVERSION before the fix

  PR:		207837

Changes:
  head/Mk/bsd.gecko.mk