One difference between a GNU ld-linked libc.so and an lld-linked libc.so is that the former's _end symbol's section index is SHN_ABS, while the latter's is that of the bss section. For some reason, GNU ld won't emit an _end symbol when creating an executable linked with an lld-linked libc. This results in libc's internal definition being used for curbrk, which is wrong and breaks brk() and sbrk(). As an experiment, I hacked lld to emit _end with a section index of SHN_ABS, and used it to link libc.so. Using that, GNU ld behaves as desired.
This shows up as a build failure of news/nntpcache (PR 225128). See also this related LLVM PR: https://bugs.llvm.org/show_bug.cgi?id=35570
I asked about the GNU ld behaviour here: https://sourceware.org/ml/binutils/2018-05/msg00300.html I suspect that this will have to be worked around in lld regardless, but I'd like to understand the problem better first.
Looks like the binutils maintainers consider our use of _end to be a bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23161 I don't see how we can implement brk()/sbrk() without it though. Looking at glibc, they initialize curbrk using a seemingly undocumented kernel interface: on Linux, the brk() system call returns the current break address if the input break value is 0. Our SYS_break system call currently does not have a return value... perhaps the right solution here is to add a similar hack and use that in libc to initialize curbrk.
The solution proposed by the binutils developers is to emit _end, etc., only in executables. To integrate that change, we'd have to remove _end from libc.so's version map. However, this change would require lld to adopt the same behaviour, since otherwise libc.so will use the internal definition of _end when linked with lld, which will always be wrong. lld also needs to be tweaked to emit _end if libc.so contains an undefined _end symbol. Currently it will not emit _end in an executable if libc.so has an undefined reference to _end.
I proposed removing the dependency on _end here: https://reviews.freebsd.org/D15663 That change indirectly addresses this PR.
A commit references this bug: Author: markj Date: Mon Jun 4 19:35:20 UTC 2018 New revision: 334626 URL: https://svnweb.freebsd.org/changeset/base/334626 Log: Reimplement brk() and sbrk() to avoid the use of _end. Previously, libc.so would initialize its notion of the break address using _end, a special symbol emitted by the static linker following the bss section. Compatibility issues between lld and ld.bfd could cause the wrong definition of _end (libc.so's definition rather than that of the executable) to be used, breaking the brk()/sbrk() interface. Avoid this problem and future interoperability issues by simply not relying on _end. Instead, modify the break() system call to return the kernel's view of the current break address, and have libc initialize its state using an extra syscall upon the first use of the interface. As a side effect, this appears to fix brk()/sbrk() usage in executables run with rtld direct exec, since the kernel and libc.so no longer maintain separate views of the process' break address. PR: 228574 Reviewed by: kib (previous version) MFC after: 2 months Differential Revision: https://reviews.freebsd.org/D15663 Changes: head/lib/libc/amd64/Symbol.map head/lib/libc/amd64/sys/Makefile.inc head/lib/libc/amd64/sys/brk.S head/lib/libc/amd64/sys/sbrk.S head/lib/libc/arm/Symbol.map head/lib/libc/arm/sys/Makefile.inc head/lib/libc/arm/sys/brk.S head/lib/libc/arm/sys/sbrk.S head/lib/libc/i386/Symbol.map head/lib/libc/i386/sys/Makefile.inc head/lib/libc/i386/sys/brk.S head/lib/libc/i386/sys/sbrk.S head/lib/libc/mips/Symbol.map head/lib/libc/mips/sys/Makefile.inc head/lib/libc/mips/sys/brk.S head/lib/libc/mips/sys/sbrk.S head/lib/libc/powerpc/Symbol.map head/lib/libc/powerpc/sys/Makefile.inc head/lib/libc/powerpc/sys/brk.S head/lib/libc/powerpc/sys/sbrk.S head/lib/libc/powerpc64/Symbol.map head/lib/libc/powerpc64/sys/Makefile.inc head/lib/libc/powerpc64/sys/brk.S head/lib/libc/powerpc64/sys/sbrk.S head/lib/libc/riscv/sys/Makefile.inc head/lib/libc/sparc64/Symbol.map head/lib/libc/sparc64/sys/Makefile.inc head/lib/libc/sparc64/sys/brk.S head/lib/libc/sparc64/sys/sbrk.S head/lib/libc/sys/Makefile.inc head/lib/libc/sys/brk.2 head/lib/libc/sys/brk.c head/lib/libc/tests/sys/Makefile head/lib/libc/tests/sys/brk_test.c head/sys/compat/freebsd32/syscalls.master head/sys/kern/syscalls.master head/sys/vm/vm_unix.c
Mark, what do you think we should do with this issue in FreeBSD? Just leave it as is, awaiting ld.bfd 2.17.50 retirement?
(In reply to Ed Maste from comment #7) That was my intention, yeah. I'm not aware of any other code that makes use of _end, so with r334626 this bug is effectively fixed.
Marking fixed. I haven't seen any fallout from r334626, but I don't have a good reason to MFC it either (and my attempt to do so resulted in a lot of conflicts). The problem also only exists when using lld to bootstrap the system and ld.bfd to link applications.