Bug 206384

Summary: llvm libunwind requires larger stack than old unwinder (segfault while building lang/polyml)
Product: Base System Reporter: Ed Maste <emaste>
Component: miscAssignee: Ed Maste <emaste>
Status: Closed FIXED    
Severity: Affects Only Me Flags: emaste: mfc-stable11?
emaste: mfc-stable10-
emaste: mfc-stable9-
Priority: ---    
Version: CURRENT   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206039
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206381

Description Ed Maste freebsd_committer freebsd_triage 2016-01-18 22:04:40 UTC
As reported in PR206039 the lang/polyml build fails when using LLVM's libunwind, due to a segfault during the build. The segfault can be reproduced by running (in the workdir):

echo | LD_LIBRARY_PATH=libpolyml/.libs .libs/polyimport -H 50 polytemp.txt -I .

The crash looks like:
volta% lldb .libs/polyimport -c polyimport.core 
(lldb) target create ".libs/polyimport" --core "polyimport.core"
Core file '/home/emaste/src/ports/lang/polyml/work/polyml.5.5.2/polyimport.core' (x86_64) was loaded.
(lldb) bt
* thread #1: tid = 0, 0x0000000800efa731 libgcc_s.so.1`libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::setInfoBasedOnIPRegister(this=0x00007fffdfffd648, isReturnAddress=false) + 33 at UnwindCursor.hpp:1171, name = 'polyimport', stop reason = signal SIGSEGV
  * frame #0: 0x0000000800efa731 libgcc_s.so.1`libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::setInfoBasedOnIPRegister(this=0x00007fffdfffd648, isReturnAddress=false) + 33 at UnwindCursor.hpp:1171
    frame #1: 0x0000000800ef960e libgcc_s.so.1`::unw_init_local(cursor=0x00007fffdfffd648, context=0x00007fffdfffdae0) + 190 at libunwind.cpp:73
    frame #2: 0x0000000800ef8ed6 libgcc_s.so.1`unwind_phase2_forced(uc=<unavailable>, exception_object=<unavailable>, stop=<unavailable>, stop_parameter=<unavailable>) + 54 at UnwindLevel1.c:244
    frame #3: 0x0000000800ef923c libgcc_s.so.1`_Unwind_ForcedUnwind(exception_object=0x0000000801e155f0, stop=0x0000000800ab4dd0, stop_parameter=0x0000000000000000) + 108 at UnwindLevel1.c:414
    frame #4: 0x0000000800ab4c55 libthr.so.3`_pthread_exit_mask [inlined] _Unwind_ForcedUnwind(ex=<unavailable>, stop_func=<unavailable>, stop_arg=<unavailable>) + 501 at thr_exit.c:106
    frame #5: 0x0000000800ab4c46 libthr.so.3`_pthread_exit_mask [inlined] thread_unwind + 41 at thr_exit.c:167
    frame #6: 0x0000000800ab4c1d libthr.so.3`_pthread_exit_mask(status=<unavailable>, mask=<unavailable>) + 445 at thr_exit.c:242
    frame #7: 0x0000000800ab4a5b libthr.so.3`_pthread_exit(status=<unavailable>) + 11 at thr_exit.c:188
    frame #8: 0x0000000800aa982c libthr.so.3`thread_start(curthread=<unavailable>) + 300 at thr_create.c:288
Comment 1 Ed Maste freebsd_committer freebsd_triage 2016-01-19 19:02:50 UTC
This is happening because libunwind steps on the thread stack guard page.

libpolyml/sighandler.cpp for the signal handler thread calls pthread_attr_setstacksize(min(4096, PTHREAD_STACK_MIN)) - on FreeBSD/x86 PTHREAD_STACK_MIN is 2k so we end up with the 4k min, but it seems llvm-libunwind needs more stack space for the _Unwind_ForcedUnwind.
Comment 2 Ed Maste freebsd_committer freebsd_triage 2016-01-19 20:14:25 UTC
The large stack usage comes from this function -- note sub $0x878,%rsp:

template <typename A, typename R>
void UnwindCursor<A, R>::setInfoBasedOnIPRegister(bool isReturnAddress) {
    92d0:       55                      push   %rbp
    92d1:       48 89 e5                mov    %rsp,%rbp
    92d4:       41 57                   push   %r15
    92d6:       41 56                   push   %r14
    92d8:       41 55                   push   %r13
    92da:       41 54                   push   %r12
    92dc:       53                      push   %rbx
    92dd:       48 81 ec 78 08 00 00    sub    $0x878,%rsp

from the stack-allocated typename CFI_Parser<A>::PrologInfo prolog

  1229  #if _LIBUNWIND_SUPPORT_DWARF_UNWIND
  1230    // There is no static unwind info for this pc. Look to see if an FDE was
  1231    // dynamically registered for it.
  1232    pint_t cachedFDE = DwarfFDECache<A>::findFDE(0, pc);
  1233    if (cachedFDE != 0) {
  1234      CFI_Parser<LocalAddressSpace>::FDE_Info fdeInfo;
  1235      CFI_Parser<LocalAddressSpace>::CIE_Info cieInfo;
  1236      const char *msg = CFI_Parser<A>::decodeFDE(_addressSpace,
  1237                                                  cachedFDE, &fdeInfo, &cieInfo);
  1238      if (msg == NULL) {
  1239        typename CFI_Parser<A>::PrologInfo prolog;
  1240        if (CFI_Parser<A>::parseFDEInstructions(_addressSpace, fdeInfo, cieInfo,
  1241                                                                  pc, &prolog)) {

(lldb) p sizeof(PrologInfo)
(unsigned long) $10 = 1952
Comment 3 Ed Maste freebsd_committer freebsd_triage 2016-01-19 21:33:29 UTC
By comparison, the equivalent struct in the GCC unwinder:

(lldb) p sizeof(_Unwind_FrameState)
(unsigned long) $3 = 384

as it has storage for only 18 registers:

(lldb) p sizeof(fs.regs.reg) / sizeof(fs.regs.reg[0])
(unsigned long) $9 = 18

while LLVM's libunwind has storage for:
kMaxRegisterNumber = 120
Comment 4 Ed Maste freebsd_committer freebsd_triage 2016-07-08 21:20:09 UTC
Fixed by llvm-libunwind update in r302450
Comment 5 commit-hook freebsd_committer freebsd_triage 2016-07-09 00:35:47 UTC
A commit references this bug:

Author: emaste
Date: Sat Jul  9 00:35:21 UTC 2016
New revision: 302475
URL: https://svnweb.freebsd.org/changeset/base/302475

Log:
  libunwind: limit stack usage in unwind cursor

  This may be reworked upstream but in the interim should address the
  stack usage issue reported in the PR.

  PR:		206384
  MFC after:	1 week
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/contrib/llvm/projects/libunwind/include/__libunwind_config.h
  head/contrib/llvm/projects/libunwind/src/DwarfParser.hpp
Comment 6 commit-hook freebsd_committer freebsd_triage 2016-07-22 17:35:03 UTC
A commit references this bug:

Author: emaste
Date: Fri Jul 22 17:34:29 UTC 2016
New revision: 303196
URL: https://svnweb.freebsd.org/changeset/base/303196

Log:
  MFC libunwind improvements

  r302450: libunwind: update to upstream snapshot r272680

  The key improvement is that it may be built without cross-unwinding
  support, which significantly reduces the stack space requirement.

  r302456: libunwind: enable only the native unwinder by default

  This significantly reduces stack space requirements, and runtimes
  require only native unwinding.

  r302475: libunwind: limit stack usage in unwind cursor

  This may be reworked upstream but in the interim should address the
  stack usage issue reported in the PR.

  r303016: llvm-libunwind: use conventional (non-Darwin) X86 register numbers

  For historical reasons Darwin/i386 has ebp and esp swapped in the
  eh_frame register numbering.  That is:

                Darwin      Other
      Reg #    eh_frame    eh_frame    DWARF
      =====    ========    ========    =====
        4        ebp         esp        esp
        5        esp         ebp        ebp

  Although the UNW_X86_* constants are not supposed to be coupled to
  DWARF / eh_frame numbering they are currently conflated in LLVM
  libunwind, and thus we require the non-Darwin numbering.

  PR:		206384
  Approved by:	re (kib)
  Sponsored by:	The FreeBSD Foundation

Changes:
_U  stable/11/
  stable/11/contrib/llvm/projects/libunwind/include/__libunwind_config.h
  stable/11/contrib/llvm/projects/libunwind/include/libunwind.h
  stable/11/contrib/llvm/projects/libunwind/src/AddressSpace.hpp
  stable/11/contrib/llvm/projects/libunwind/src/CompactUnwinder.hpp
  stable/11/contrib/llvm/projects/libunwind/src/Registers.hpp
  stable/11/contrib/llvm/projects/libunwind/src/Unwind-EHABI.cpp
  stable/11/contrib/llvm/projects/libunwind/src/UnwindCursor.hpp
  stable/11/contrib/llvm/projects/libunwind/src/UnwindLevel1.c
  stable/11/contrib/llvm/projects/libunwind/src/UnwindRegistersRestore.S
  stable/11/contrib/llvm/projects/libunwind/src/UnwindRegistersSave.S
  stable/11/contrib/llvm/projects/libunwind/src/config.h
  stable/11/contrib/llvm/projects/libunwind/src/libunwind.cpp
  stable/11/gnu/lib/libgcc/Makefile
Comment 7 commit-hook freebsd_committer freebsd_triage 2016-07-25 19:37:35 UTC
A commit references this bug:

Author: emaste
Date: Mon Jul 25 19:37:11 UTC 2016
New revision: 303318
URL: https://svnweb.freebsd.org/changeset/base/303318

Log:
  Merge LLVM libunwind fixes

  r302475: libunwind: limit stack usage in unwind cursor

  This may be reworked upstream but in the interim should address the
  stack usage issue reported in the PR.

  r303061: libunwind: Properly align _Unwind_Exception.

  _Unwind_Exception is required to be double word aligned.  GCC has
  interpreted this to mean "use the maximum useful alignment for the
  target" so follow that lead.

  PR:		206384 (r302475)
  Obtained from:	LLVM review D22543 (r303061)
  Approved by:	re (gjb)

Changes:
_U  stable/11/
  stable/11/contrib/llvm/projects/libunwind/include/__libunwind_config.h
  stable/11/contrib/llvm/projects/libunwind/include/unwind.h
  stable/11/contrib/llvm/projects/libunwind/src/DwarfParser.hpp