Bug 246322 - ld.bfd error on releng/11.4 confuses many autoconf scripts
Summary: ld.bfd error on releng/11.4 confuses many autoconf scripts
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 11.4-RELEASE
Hardware: i386 Any
: --- Affects Only Me
Assignee: freebsd-toolchain (Nobody)
URL: https://bugs.freebsd.org/bugzilla/sho...
Keywords: regression
Depends on:
Blocks: 240629
  Show dependency treegraph
 
Reported: 2020-05-09 07:19 UTC by Jan Beich
Modified: 2023-06-29 16:53 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Beich freebsd_committer freebsd_triage 2020-05-09 07:19:53 UTC
Some configure scripts check compiler output when testing flags. Often this disables POSIX threading support.

$ cc --version
FreeBSD clang version 10.0.0 (git@github.com:llvm/llvm-project.git llvmorg-10.0.0-0-gd32170dbd5b)
Target: i386-unknown-freebsd11.4
Thread model: posix
InstalledDir: /usr/bin
$ cc -Wl,--version
GNU ld 2.17.50 [FreeBSD] 2007-07-03
Copyright 2007 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License.  This program has absolutely no warranty.

$ echo 'int main() {}' >a.c

$ cc a.c
/usr/bin/ld: error in /usr/lib/crt1.o(.eh_frame); no .eh_frame_hdr table will be created.
$ echo $?
0

$ cc -fuse-ld=lld a.c
$ echo $?
0

From devel/nspr:
checking for pthread_create in -lpthreads... no
checking whether cc accepts -pthread... no
[...]
/usr/bin/ld: cannot find -lc_r
cc: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[4]: *** [../../config/rules.mk:308: libnspr4.so] Error 1

From graphics/cairo:
checking for cairo's pthread feature... 
checking whether cairo's pthread feature could be enabled... no (can't link with -lpthread or -pthread)
[...]
./cairo-mutex-impl-private.h:262:3: error: "XXX: No mutex implementation found.  Cairo will not work with multiple threads.  Define CAIRO_NO_MUTEX to 1 to acknowledge and accept this limitation and compile cairo without thread-safety support."
# error "XXX: No mutex implementation found.  Cairo will not work with multiple threads.  Define CAIRO_NO_MUTEX to 1 to acknowledge and accept this limitation and compile cairo without thread-safety support."
  ^
./cairo-mutex-impl-private.h:271:11: error: unknown type name 'cairo_mutex_impl_t'
  typedef cairo_mutex_impl_t cairo_recursive_mutex_impl_t;
          ^
Comment 1 Jan Beich freebsd_committer freebsd_triage 2020-05-09 07:35:44 UTC
Clang 9 from 11.4-BETA1 is also affected but Clang 8 from 11.4-PRERELEASE (2020-04-30) is not.
Comment 2 Jan Beich freebsd_committer freebsd_triage 2020-05-09 07:44:48 UTC
Hmm, I can't reproduce on amd64.
Comment 3 Dimitry Andric freebsd_committer freebsd_triage 2020-05-09 17:16:16 UTC
As emaste noted, it looks exactly like this very old DragonFly issue:
https://bugs.dragonflybsd.org/issues/2511. The workaround they used eventually was to compile the crt objects with -fno-asynchronous-unwind-tables.

But this is very likely a ld.bfd bug, as there seems to be nothing wrong with the .eh_frame section in crt1.o. Dwarfdump gives me:

.eh_frame

fde:
<    0><0x00000020:0x00000039><_fini><cie offset 0x0000001c::cie index     0><fde offset 0x00000018 length: 0x00000018>
       <eh aug data len 0x0>
        0x00000020: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x00000023: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x00000025: <off cfa=04(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
<    1><0x00000074:0x000001a3><><cie offset 0x0000001c::cie index     1><fde offset 0x0000004c length: 0x0000001c>
       <eh aug data len 0x0>
        0x00000074: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x00000075: <off cfa=08(r4) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x00000077: <off cfa=08(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x0000007d: <off cfa=08(r5) > <off r3=-12(cfa) > <off r5=-8(cfa) > <off r6=-20(cfa) > <off r7=-16(cfa) > <off r8=-4(cfa) >
<    2><0x000001c4:0x00000297><><cie offset 0x0000003c::cie index     1><fde offset 0x0000006c length: 0x00000024>
       <eh aug data len 0x0>
        0x000001c4: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x000001c5: <off cfa=08(r4) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000001c7: <off cfa=08(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000001cd: <off cfa=08(r5) > <off r3=-12(cfa) > <off r5=-8(cfa) > <off r6=-20(cfa) > <off r7=-16(cfa) > <off r8=-4(cfa) >
        0x00000296: <off cfa=04(r4) > <off r3=-12(cfa) > <off r5=-8(cfa) > <off r6=-20(cfa) > <off r7=-16(cfa) > <off r8=-4(cfa) >
<    3><0x000002cc:0x00000311><><cie offset 0x00000064::cie index     1><fde offset 0x00000094 length: 0x00000020>
       <eh aug data len 0x0>
        0x000002cc: <off cfa=04(r4) > <off r8=-4(cfa) >
        0x000002cd: <off cfa=08(r4) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000002cf: <off cfa=08(r5) > <off r5=-8(cfa) > <off r8=-4(cfa) >
        0x000002d0: <off cfa=08(r5) > <off r5=-8(cfa) > <off r6=-12(cfa) > <off r8=-4(cfa) >
        0x000002ed: <off cfa=04(r4) > <off r5=-8(cfa) > <off r6=-12(cfa) > <off r8=-4(cfa) >
        0x000002f2: <off cfa=08(r5) > <off r5=-8(cfa) > <off r6=-12(cfa) > <off r8=-4(cfa) >

cie:
<    0> version                         1
        cie section offset              0 0x00000000
        augmentation                    zR
        code_alignment_factor           1
        data_alignment_factor           -4
        return_address_register         8
        eh aug data len 0x1 bytes 0x1b
        bytes of initial instructions   7
        cie length                      20
        initial instructions
         0 DW_CFA_def_cfa r4 4
         3 DW_CFA_offset r8 -4
         5 DW_CFA_nop
         6 DW_CFA_nop
<    1> version                         1
        cie section offset              52 0x00000034
        augmentation                    zR
        code_alignment_factor           1
        data_alignment_factor           -4
        return_address_register         8
        eh aug data len 0x1 bytes 0x1b
        bytes of initial instructions   7
        cie length                      20
        initial instructions
         0 DW_CFA_def_cfa r4 4
         3 DW_CFA_offset r8 -4
         5 DW_CFA_nop
         6 DW_CFA_nop
Comment 4 Dimitry Andric freebsd_committer freebsd_triage 2020-05-09 19:44:47 UTC
In any case clang 8 does not produce any .cfi directives in crt*.o, while clang 9 and 10 do.

For instance /usr/obj/usr/src/lib/csu/i386/crt1_c.s with clang becomes:

        .text
        .file   "crt1_c.c"
                                        # Start of file scope inline assembly
        .ident  "$FreeBSD: stable/11/lib/csu/i386/crt1_c.c 292000 2015-12-08 19:32:58Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/crtbrand.c 339300 2018-10-11 00:26:15Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/ignore_init.c 339300 2018-10-11 00:26:15Z emaste $"
        .hidden _start1

                                        # End of file scope inline assembly
        .globl  _start1                 # -- Begin function _start1
        .p2align        4, 0x90
        .type   _start1,@function
_start1:                                # @_start1
# %bb.0:
...

With clang 10 this becomes:

       .text
        .file   "crt1_c.c"
                                        # Start of file scope inline assembly
        .ident  "$FreeBSD: stable/11/lib/csu/i386/crt1_c.c 292000 2015-12-08 19:32:58Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/crtbrand.c 339300 2018-10-11 00:26:15Z emaste $"
        .ident  "$FreeBSD: stable/11/lib/csu/common/ignore_init.c 339300 2018-10-11 00:26:15Z emaste $"
        .hidden _start1

                                        # End of file scope inline assembly
        .globl  _start1                 # -- Begin function _start1
        .p2align        4, 0x90
        .type   _start1,@function
_start1:                                # @_start1
        .cfi_startproc
# %bb.0:
...

The directives got turned on by emaste in https://github.com/llvm/llvm-project/commit/cb1761465a0d, referring in his commit message to bug 241562 ("failing test case: lib.libexecinfo.backtrace_test.backtrace_fmt_basic").
Comment 5 Dimitry Andric freebsd_committer freebsd_triage 2020-05-10 08:57:47 UTC
(In reply to Dimitry Andric from comment #4)
So what shall we do here? I've gone a little bit into the rabbit hole that is BFD ld, but the bug does not "jump out" at me. So fixing it is likely not trivial.

We could:
1) Patch out the error message in BFD ld (possibly only on stable/11)
2) Spend time to figure why BFD ld is choking on this .eh_frame section, get a fix, submit it upstream, merge it downstream, etc
3) Add -fno-asynchronous-unwind-tables to lib/csu in general
4) Add -fno-asynchronous-unwind-tables to lib/csu/i386 only
5) Add -fno-asynchronous-unwind-tables to lib/csu/i386 only, and only on stable/11

My preference would be for 5). Note that we'd need to MFS any fix to releng/11.4 too.
Comment 6 Konstantin Belousov freebsd_committer freebsd_triage 2020-05-10 09:06:47 UTC
(In reply to Dimitry Andric from comment #5)
Look at r209294.
I think 3 is best.
Comment 7 commit-hook freebsd_committer freebsd_triage 2020-05-11 19:37:26 UTC
A commit references this bug:

Author: dim
Date: Mon May 11 19:36:39 UTC 2020
New revision: 360915
URL: https://svnweb.freebsd.org/changeset/base/360915

Log:
  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  PR:		246322
  MFC after:	3 days
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
  head/lib/csu/Makefile.inc
Comment 8 commit-hook freebsd_committer freebsd_triage 2020-05-14 19:09:30 UTC
A commit references this bug:

Author: dim
Date: Thu May 14 19:09:14 UTC 2020
New revision: 361046
URL: https://svnweb.freebsd.org/changeset/base/361046

Log:
  MFC r360915:

  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  PR:		246322
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
_U  stable/12/
  stable/12/lib/csu/Makefile.inc
Comment 9 commit-hook freebsd_committer freebsd_triage 2020-05-14 19:09:32 UTC
A commit references this bug:

Author: dim
Date: Thu May 14 19:09:26 UTC 2020
New revision: 361047
URL: https://svnweb.freebsd.org/changeset/base/361047

Log:
  MFC r360915:

  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  PR:		246322
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
_U  stable/11/
  stable/11/lib/csu/Makefile.inc
Comment 10 commit-hook freebsd_committer freebsd_triage 2020-05-14 19:33:35 UTC
A commit references this bug:

Author: dim
Date: Thu May 14 19:32:42 UTC 2020
New revision: 361050
URL: https://svnweb.freebsd.org/changeset/base/361050

Log:
  MF11 r361047:

  MFC r360915:

  Use -fno-asynchronous-unwind-tables to compile lib/csu

  Summary:
  In r209294 kib added -fno-asynchronous-unwind-tables to the compile
  flags for the GNU C startup components. This was done to work around a
  BFD ld assertion, "no .eh_frame_hdr table will be created", which is
  produced because of the layout of the startup objects.

  Add the same flag to lib/csu too, for the same reason. And similarly to
  r209294, also add -fno-omit-frame-pointer.

  This is primarily meant to quickly MFC to stable/11, so it can end up in
  the 11.4 release, as a fix for https://bugs.freebsd.org/246322.

  Approved by:	re (gjb)
  PR:		246322
  Differential Revision: https://reviews.freebsd.org/D24797

Changes:
_U  releng/11.4/
  releng/11.4/lib/csu/Makefile.inc
Comment 11 Dimitry Andric freebsd_committer freebsd_triage 2020-05-14 19:35:19 UTC
Fixed after base r361050, which should end up soon in 11.4-BETA2.
Comment 12 Conrad Meyer freebsd_committer freebsd_triage 2020-05-18 16:10:53 UTC
(In reply to Dimitry Andric from comment #4)
> The directives got turned on by emaste in https://github.com/llvm/llvm-
> project/commit/cb1761465a0d, referring in his commit message to bug 241562
> ("failing test case: lib.libexecinfo.backtrace_test.backtrace_fmt_basic").

Sort of; they were on before that, but only for amd64.  That change just
unified the behavior for all arch.
Comment 13 Conrad Meyer freebsd_committer freebsd_triage 2020-05-18 16:12:45 UTC
(Link to regression introduced by this change: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246537 )
Comment 14 Conrad Meyer freebsd_committer freebsd_triage 2020-08-05 15:28:04 UTC
Could the cflags be more constrained to the specific problematic CUs?  For gcc crt (probably dead now?), the exclusion should only be needed for crtend.o/So.  That could either be a Makefile change, or we could #if 0 these lines from crtstuff.c?

  https://svnweb.freebsd.org/base/head/contrib/gcc/crtstuff.c?revision=169690&view=markup&pathrev=209294#l483  (inside a defined(CRT_END) section starting at line 446).

For llvm csu, the issue reported was specific to i386 crt1.o; globally disabling unwind seems heavy-handed.  Ideally we get a _start frame on i386, but as a first step we should undisable these frames on !i386.

I don't see any obvious reason llvm csu would actually emit any special last-CIE eh_frame value for i386; unlike the GNU one above, there isn't any eh_frame section symbol with magic zero value in any of the lib/csu code.  The weirdest thing about i386 crt1.o is that it has two DW_TAG_compile_units and also two CIEs.

Supposing that is ld.bfd's problem, could crt1 be a .a?

Alternatively, we could disable the unwind directives in crt1_s.S, keeping unwind directives for the rest of crt1.o (_start1, etc).  We'd lose _start, but still have a _start1 frame below main.

(There is a special __EH_FRAME_LIST_END__ in contrib/llvm-project/compiler-rt/lib/crt/crtend.c, but as far as I can tell we don't build that file into userspace anywhere and it wouldn't be impacted by lib/csu/Makefile.inc CFLAGS.  We use the one in lib/csu/common/crtend.c.)
Comment 15 Conrad Meyer freebsd_committer freebsd_triage 2020-08-05 15:44:09 UTC
Also, this doesn't seem to reproduce with modern (ports) binutils?

$ llvm-readelf10 --sections /obj/usr/home/conrad/src/freebsd/i386.i386/lib/csu/i386/crt1.o
...
  [ 4] .eh_frame         PROGBITS        00000000 0003fc 0000b8 00   A  0   0  4

$  cat main.c
int main(int argc, char **argv) { return 1; }

$ cc -Wall -Wextra -O2 -g -m32 -c main.c
$ ld.bfd -melf_i386_fbsd /obj/usr/home/conrad/src/freebsd/i386.i386/lib/csu/i386/crt{1,i,n}.o main.o /obj/usr/home/conrad/src/freebsd/i386.i386/tmp/usr/lib/libc.a   -o main.test
(no error)

$ ld.bfd --version
GNU ld (GNU Binutils) 2.33.1

If this issue is specific to ancient base ld.bfd on 11 (and maybe 12?), I think we should just revert this in CURRENT.
Comment 16 Dimitry Andric freebsd_committer freebsd_triage 2020-08-09 17:26:47 UTC
It's strange, but whatever I do I cannot reproduce the original issue on 11.4 anymore, even if I comment out the -fno-asynchronous-unwind-tables line from lib/csu/Makefile.inc and rebuild it.

jbeich@ are you able to still reproduce this?
Comment 17 commit-hook freebsd_committer freebsd_triage 2023-06-29 16:39:32 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=c969310c992a12459ed4025c1cd8b22f29c763b5

commit c969310c992a12459ed4025c1cd8b22f29c763b5
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2023-06-29 16:34:39 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2023-06-29 16:34:39 +0000

    csu: Implement _start using as to satisfy unwinders on x86_64

    The right unwinding stop indicator should be CFI-undefined PC.
    https://dwarfstd.org/doc/Dwarf3.pdf - page 118:
    If a Return Address register is defined in the virtual unwind table,
    and its rule is undefined (for example, by DW_CFA_undefined), then
    there is no return address and no call address, and the virtual
    unwind of stack activations is complete.

    This requires the crt code be built with unwind tables, for that remove
    -fno-asynchronous-unwind-tables to enable unwind tables generation.

    PR:                     241562, 246322, 246537
    Reviewed by:            kib
    Differential Revision:  https://reviews.freebsd.org/D40780

 lib/csu/Makefile.inc         | 20 +++++-----
 lib/csu/amd64/Makefile       |  3 ++
 lib/csu/amd64/crt1_c.c       | 22 -----------
 lib/csu/amd64/crt1_s.S (new) | 88 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+), 31 deletions(-)
Comment 18 commit-hook freebsd_committer freebsd_triage 2023-06-29 16:53:40 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=a18b956b73cee784e5c422d20fd0e4dabebd7eee

commit a18b956b73cee784e5c422d20fd0e4dabebd7eee
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2023-06-29 16:53:07 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2023-06-29 16:53:07 +0000

    libexecinfo: Enable backtrace_test.backtrace_fmt_basic on amd64 again

    Due to unwind tables generation enabled after c969310c for csu.

    PR:                     241562, 246322, 246537
    Reviewed by:            kib, ngie
    Differential Revision:  https://reviews.freebsd.org/D40758

 contrib/netbsd-tests/lib/libexecinfo/t_backtrace.c | 5 -----
 1 file changed, 5 deletions(-)