Bug 246322 - ld.bfd error on releng/11.4 confuses many autoconf scripts
Summary: ld.bfd error on releng/11.4 confuses many autoconf scripts
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 11.4-RELEASE
Hardware: i386 Any
: --- Affects Only Me
Assignee: freebsd-toolchain (Nobody)
URL: https://bugs.freebsd.org/bugzilla/sho...
Keywords: regression
Depends on:
Blocks: 240629
  Show dependency treegraph
 
Reported: 2020-05-09 07:19 UTC by Jan Beich
Modified: 2020-08-09 17:26 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Beich freebsd_committer 2020-05-09 07:19:53 UTC
Some configure scripts check compiler output when testing flags. Often this disables POSIX threading support.

$ cc --version
FreeBSD clang version 10.0.0 (git@github.com:llvm/llvm-project.git llvmorg-10.0.0-0-gd32170dbd5b)
Target: i386-unknown-freebsd11.4
Thread model: posix
InstalledDir: /usr/bin
$ cc -Wl,--version
GNU ld 2.17.50 [FreeBSD] 2007-07-03
Copyright 2007 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License.  This program has absolutely no warranty.

$ echo 'int main() {}' >a.c

$ cc a.c
/usr/bin/ld: error in /usr/lib/crt1.o(.eh_frame); no .eh_frame_hdr table will be created.
$ echo $?
0

$ cc -fuse-ld=lld a.c
$ echo $?
0

From devel/nspr:
checking for pthread_create in -lpthreads... no
checking whether cc accepts -pthread... no
[...]
/usr/bin/ld: cannot find -lc_r
cc: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[4]: *** [../../config/rules.mk:308: libnspr4.so] Error 1

From graphics/cairo:
checking for cairo's pthread feature... 
checking whether cairo's pthread feature could be enabled... no (can't link with -lpthread or -pthread)
[...]
./cairo-mutex-impl-private.h:262:3: error: "XXX: No mutex implementation found.  Cairo will not work with multiple threads.  Define CAIRO_NO_MUTEX to 1 to acknowledge and accept this limitation and compile cairo without thread-safety support."
# error "XXX: No mutex implementation found.  Cairo will not work with multiple threads.  Define CAIRO_NO_MUTEX to 1 to acknowledge and accept this limitation and compile cairo without thread-safety support."
  ^
./cairo-mutex-impl-private.h:271:11: error: unknown type name 'cairo_mutex_impl_t'
  typedef cairo_mutex_impl_t cairo_recursive_mutex_impl_t;
          ^
Comment 1 Jan Beich freebsd_committer 2020-05-09 07:35:44 UTC
Clang 9 from 11.4-BETA1 is also affected but Clang 8 from 11.4-PRERELEASE (2020-04-30) is not.
Comment 2 Jan Beich freebsd_committer 2020-05-09 07:44:48 UTC
Hmm, I can't reproduce on amd64.
Comment 3 Dimitry Andric freebsd_committer 2020-05-09 17:16:16 UTC
As emaste noted, it looks exactly like this very old DragonFly issue:
https://bugs.dragonflybsd.org/issues/2511. The workaround they used eventually was to compile the crt objects with -fno-asynchronous-unwind-tables.

But this is very likely a ld.bfd bug, as there seems to be nothing wrong with the .eh_frame section in crt1.o. Dwarfdump gives me:

.eh_frame

fde:
<    0><0x00000020:0x00000039><_fini><cie offset 0x0000001c::cie index     0><fde offset 0x00000018 length: 0x00000018>
       <eh aug data len 0x0>
        0x00000020: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x00000023: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x00000025: <off cfa=04(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
<    1><0x00000074:0x000001a3><><cie offset 0x0000001c::cie index     1><fde offset 0x0000004c length: 0x0000001c>
       <eh aug data len 0x0>
        0x00000074: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x00000075: <off cfa=08(r4) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x00000077: <off cfa=08(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x0000007d: <off cfa=08(r5) > <off r3=-12(cfa) > <off r5=-8(cfa) > <off r6=-20(cfa) > <off r7=-16(cfa) > <off r8=-4(cfa) >
<    2><0x000001c4:0x00000297><><cie offset 0x0000003c::cie index     1><fde offset 0x0000006c length: 0x00000024>
       <eh aug data len 0x0>
        0x000001c4: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x000001c5: <off cfa=08(r4) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000001c7: <off cfa=08(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000001cd: <off cfa=08(r5) > <off r3=-12(cfa) > <off r5=-8(cfa) > <off r6=-20(cfa) > <off r7=-16(cfa) > <off r8=-4(cfa) >
        0x00000296: <off cfa=04(r4) > <off r3=-12(cfa) > <off r5=-8(cfa) > <off r6=-20(cfa) > <off r7=-16(cfa) > <off r8=-4(cfa) >
<    3><0x000002cc:0x00000311><><cie offset 0x00000064::cie index     1><fde offset 0x00000094 length: 0x00000020>
       <eh aug data len 0x0>
        0x000002cc: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x000002cd: <off cfa=08(r4) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000002cf: <off cfa=08(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000002d0: <off cfa=08(r5) > <off r5=-8(cfa) > <off r6=-12(cfa) > <off r8=-4(cfa) >
        0x000002ed: <off cfa=04(r4) > <off r5=-8(cfa) > <off r6=-12(cfa) > <off r8=-4(cfa) >
        0x000002f2: <off cfa=08(r5) > <off r5=-8(cfa) > <off r6=-12(cfa) > <off r8=-4(cfa) >

cie:
<    0> version                         1
        cie section offset              0 0x00000000
        augmentation                    zR
        code_alignment_factor           1
        data_alignment_factor           -4
        return_address_register         8
        eh aug data len 0x1 bytes 0x1b
        bytes of initial instructions   7
        cie length                      20
        initial instructions
         0 DW_CFA_def_cfa r4 4
         3 DW_CFA_offset r8 -4
         5 DW_CFA_nop
         6 DW_CFA_nop
<    1> version                         1
        cie section offset              52 0x00000034
        augmentation                    zR
        code_alignment_factor           1
        data_alignment_factor           -4
        return_address_register         8
        eh aug data len 0x1 bytes 0x1b
        bytes of initial instructions   7
        cie length                      20
        initial instructions
         0 DW_CFA_def_cfa r4 4
         3 DW_CFA_offset r8 -4
         5 DW_CFA_nop
         6 DW_CFA_nop
Comment 4 Dimitry Andric freebsd_committer 2020-05-09 19:44:47 UTC
In any case clang 8 does not produce any .cfi directives in crt*.o, while clang 9 and 10 do.

For instance /usr/obj/usr/src/lib/csu/i386/crt1_c.s with clang becomes:

        .text
        .file   "crt1_c.c"
                                        # Start of file scope inline assembly
        .ident  "$FreeBSD: stable/11/lib/csu/i386/crt1_c.c 292000 2015-12-08 19:32:58Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/crtbrand.c 339300 2018-10-11 00:26:15Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/ignore_init.c 339300 2018-10-11 00:26:15Z emaste $"
        .hidden _start1

                                        # End of file scope inline assembly
        .globl  _start1                 # -- Begin function _start1
        .p2align        4, 0x90
        .type   _start1,@function
_start1:                                # @_start1
# %bb.0:
...

With clang 10 this becomes:

       .text
        .file   "crt1_c.c"
                                        # Start of file scope inline assembly
        .ident  "$FreeBSD: stable/11/lib/csu/i386/crt1_c.c 292000 2015-12-08 19:32:58Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/crtbrand.c 339300 2018-10-11 00:26:15Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/ignore_init.c 339300 2018-10-11 00:26:15Z emaste $"
        .hidden _start1

                                        # End of file scope inline assembly
        .globl  _start1                 # -- Begin function _start1
        .p2align        4, 0x90
        .type   _start1,@function
_start1:                                # @_start1
        .cfi_startproc
# %bb.0:
...

The directives got turned on by emaste in https://github.com/llvm/llvm-project/commit/cb1761465a0d, referring in his commit message to bug 241562 ("failing test case: lib.libexecinfo.backtrace_test.backtrace_fmt_basic").
Comment 5 Dimitry Andric freebsd_committer 2020-05-10 08:57:47 UTC
(In reply to Dimitry Andric from comment #4)
So what shall we do here? I've gone a little bit into the rabbit hole that is BFD ld, but the bug does not "jump out" at me. So fixing it is likely not trivial.

We could:
1) Patch out the error message in BFD ld (possibly only on stable/11)
2) Spend time to figure why BFD ld is choking on this .eh_frame section, get a fix, submit it upstream, merge it downstream, etc
3) Add -fno-asynchronous-unwind-tables to lib/csu in general
4) Add -fno-asynchronous-unwind-tables to lib/csu/i386 only
5) Add -fno-asynchronous-unwind-tables to lib/csu/i386 only, and only on stable/11

My preference would be for 5). Note that we'd need to MFS any fix to releng/11.4 too.
Comment 6 Konstantin Belousov freebsd_committer 2020-05-10 09:06:47 UTC
(In reply to Dimitry Andric from comment #5)
Look at r209294.
I think 3 is best.
Comment 7 commit-hook freebsd_committer 2020-05-11 19:37:26 UTC
A commit references this bug:

Author: dim
Date: Mon May 11 19:36:39 UTC 2020
New revision: 360915
URL: https://svnweb.freebsd.org/changeset/base/360915

Log:
  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  PR:		246322
  MFC after:	3 days
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
  head/lib/csu/Makefile.inc
Comment 8 commit-hook freebsd_committer 2020-05-14 19:09:30 UTC
A commit references this bug:

Author: dim
Date: Thu May 14 19:09:14 UTC 2020
New revision: 361046
URL: https://svnweb.freebsd.org/changeset/base/361046

Log:
  MFC r360915:

  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  PR:		246322
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
_U  stable/12/
  stable/12/lib/csu/Makefile.inc
Comment 9 commit-hook freebsd_committer 2020-05-14 19:09:32 UTC
A commit references this bug:

Author: dim
Date: Thu May 14 19:09:26 UTC 2020
New revision: 361047
URL: https://svnweb.freebsd.org/changeset/base/361047

Log:
  MFC r360915:

  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  PR:		246322
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
_U  stable/11/
  stable/11/lib/csu/Makefile.inc
Comment 10 commit-hook freebsd_committer 2020-05-14 19:33:35 UTC
A commit references this bug:

Author: dim
Date: Thu May 14 19:32:42 UTC 2020
New revision: 361050
URL: https://svnweb.freebsd.org/changeset/base/361050

Log:
  MF11 r361047:

  MFC r360915:

  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  Approved by:	re (gjb)
  PR:		246322
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
_U  releng/11.4/
  releng/11.4/lib/csu/Makefile.inc
Comment 11 Dimitry Andric freebsd_committer 2020-05-14 19:35:19 UTC
Fixed after base r361050, which should end up soon in 11.4-BETA2.
Comment 12 Conrad Meyer freebsd_committer 2020-05-18 16:10:53 UTC
(In reply to Dimitry Andric from comment #4)
> The directives got turned on by emaste in https://github.com/llvm/llvm-
> project/commit/cb1761465a0d, referring in his commit message to bug 241562
> ("failing test case: lib.libexecinfo.backtrace_test.backtrace_fmt_basic").

Sort of; they were on before that, but only for amd64.  That change just
unified the behavior for all arch.
Comment 13 Conrad Meyer freebsd_committer 2020-05-18 16:12:45 UTC
(Link to regression introduced by this change: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246537 )
Comment 14 Conrad Meyer freebsd_committer 2020-08-05 15:28:04 UTC
Could the cflags be more constrained to the specific problematic CUs?  For gcc crt (probably dead now?), the exclusion should only be needed for crtend.o/So.  That could either be a Makefile change, or we could #if 0 these lines from crtstuff.c?

  https://svnweb.freebsd.org/base/head/contrib/gcc/crtstuff.c?revision=169690&view=markup&pathrev=209294#l483  (inside a defined(CRT_END) section starting at line 446).

For llvm csu, the issue reported was specific to i386 crt1.o; globally disabling unwind seems heavy-handed.  Ideally we get a _start frame on i386, but as a first step we should undisable these frames on !i386.

I don't see any obvious reason llvm csu would actually emit any special last-CIE eh_frame value for i386; unlike the GNU one above, there isn't any eh_frame section symbol with magic zero value in any of the lib/csu code.  The weirdest thing about i386 crt1.o is that it has two DW_TAG_compile_units and also two CIEs.

Supposing that is ld.bfd's problem, could crt1 be a .a?

Alternatively, we could disable the unwind directives in crt1_s.S, keeping unwind directives for the rest of crt1.o (_start1, etc).  We'd lose _start, but still have a _start1 frame below main.

(There is a special __EH_FRAME_LIST_END__ in contrib/llvm-project/compiler-rt/lib/crt/crtend.c, but as far as I can tell we don't build that file into userspace anywhere and it wouldn't be impacted by lib/csu/Makefile.inc CFLAGS.  We use the one in lib/csu/common/crtend.c.)
Comment 15 Conrad Meyer freebsd_committer 2020-08-05 15:44:09 UTC
Also, this doesn't seem to reproduce with modern (ports) binutils?

$ llvm-readelf10 --sections /obj/usr/home/conrad/src/freebsd/i386.i386/lib/csu/i386/crt1.o
...
  [ 4] .eh_frame         PROGBITS        00000000 0003fc 0000b8 00   A  0   0  4

$  cat main.c
int main(int argc, char **argv) { return 1; }

$ cc -Wall -Wextra -O2 -g -m32 -c main.c
$ ld.bfd -melf_i386_fbsd /obj/usr/home/conrad/src/freebsd/i386.i386/lib/csu/i386/crt{1,i,n}.o main.o /obj/usr/home/conrad/src/freebsd/i386.i386/tmp/usr/lib/libc.a   -o main.test
(no error)

$ ld.bfd --version
GNU ld (GNU Binutils) 2.33.1

If this issue is specific to ancient base ld.bfd on 11 (and maybe 12?), I think we should just revert this in CURRENT.
Comment 16 Dimitry Andric freebsd_committer 2020-08-09 17:26:47 UTC
It's strange, but whatever I do I cannot reproduce the original issue on 11.4 anymore, even if I comment out the -fno-asynchronous-unwind-tables line from lib/csu/Makefile.inc and rebuild it.

jbeich@ are you able to still reproduce this?