Bug 233361 - mips build with Clang + LLD fails
Summary: mips build with Clang + LLD fails
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: CURRENT
Hardware: mips Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
Depends on:
Reported: 2018-11-20 22:05 UTC by Ed Maste
Modified: 2021-08-23 16:32 UTC (History)
4 users (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer 2018-11-20 22:05:38 UTC
From _.mips.mips.buildworld (mips* same):

--- crtbeginS.o ---
<inline asm>:9:1: error: no .cprestore used in PIC mode [-Werror,-Winline-asm]
jal     __do_global_dtors_aux
<inline asm>:18:1: error: no .cprestore used in PIC mode [-Werror,-Winline-asm]
jal     register_classes

_.mips.mips64.buildworld (mips64* same):
/scratch/tmp/emaste/obj/scratch/tmp/emaste/freebsd/mips.mips64/tmp/usr/bin/ld: /scratch/tmp/emaste/obj/scratch/tmp/emaste/freebsd/mips.mips64/tmp/usr/lib/crtbeginS.o: CALL16 reloc at 0x18 not against global symbol
/scratch/tmp/emaste/obj/scratch/tmp/emaste/freebsd/mips.mips64/tmp/usr/lib/crtbeginS.o: could not read symbols: Bad value
cc: error: linker command failed with exit code 1 (use -v to see invocation)

/scratch/tmp/emaste/obj/scratch/tmp/emaste/freebsd/mips.mipsn32/tmp/usr/bin/ld: /scratch/tmp/emaste/obj/scratch/tmp/emaste/freebsd/mips.mipsn32/tmp/usr/lib/crtbeginS.o: CALL16 reloc at 0x14 not against global symbol
/scratch/tmp/emaste/obj/scratch/tmp/emaste/freebsd/mips.mipsn32/tmp/usr/lib/crtbeginS.o: could not read symbols: Bad value
cc: error: linker command failed with exit code 1 (use -v to see invocation)
Comment 1 Andrew Turner freebsd_committer 2018-11-21 14:15:23 UTC
What build flags need to be set to reproduce this?
Comment 2 Ed Maste freebsd_committer 2018-11-21 14:29:46 UTC
(In reply to Andrew Turner from comment #1)
I saw this in a tinderbox build from my wipbsd branch https://github.com/emaste/freebsd/tree/wipbsd.20181109

I enabled Clang for mips in https://github.com/emaste/freebsd/commit/d676ebee94e317b93d61d3fc346eaab75bb8b5be

I think that it should be reproducible with these src.conf knobs:

(I used to have some additional compiler fixes for mips in an earlier wipbsd branch but I seem to have lost them.)
Comment 3 John Baldwin freebsd_committer freebsd_triage 2019-01-21 21:05:11 UTC
This appears to be a clang 7.0 regression.  External GCC is able to build this fine.  In addition, the old crt bits in contrib/gcc use the same logic (.cpload without a .cprestore), and while clang compiled that during a buildworld WITHOUT_BSD_CRTBEGIN=yes, clang now chokes trying to link libc:

ld: error: can't create dynamic relocation R_MIPS_32 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in cancelpoints_sem_new.pico
>>> referenced by cancelpoints_sem_new.c
>>>               cancelpoints_sem_new.pico:(.eh_frame+0x1C)

So, clang 7.0 seems to be busted compared to clang 6 + patches. :-/  I haven't tried mips64 yet, only 32-bit mips.  Cheri only tests 64-bit mips, so it might be that 64-bit works ok and only 32-bit mips is broken.
Comment 4 Dan Nelson 2019-07-27 16:03:46 UTC
A gcc build actually does generate the same warning message, but doesn't fail the compile.  It looks like gcc doesn't add "--fatal-warnings" to the gas commandline when gcc is in -Werror mode, so the assembly warnings are ignored.
Comment 5 Kyle Evans freebsd_committer 2019-09-11 14:00:50 UTC
I use the following patch in my local mips-llvm branch:

commit 9783fdf63c8a4e1775a494b47b7f3648b65779ab
Author: Kyle Evans <kevans@FreeBSD.org>
Date:   Mon Aug 19 13:18:47 2019 -0500

    Hack around .cprestore anger

diff --git a/lib/csu/mips/crt.h b/lib/csu/mips/crt.h
index 1d967b73ffe..a90d8661f34 100644
--- a/lib/csu/mips/crt.h
+++ b/lib/csu/mips/crt.h
@@ -29,12 +29,14 @@
 #define        HAVE_CTORS
 #define        CTORS_CONSTRUCTORS
 #ifdef __mips_o32
+/* The .cprestore is bogus, as we do elsewhere */
 #define        INIT_CALL_SEQ(func)                                             \
     ".set noreorder            \n"                                     \
     "bal       1f              \n"                                     \
     "nop                       \n"                                     \
     "1:                                \n"                                     \
     ".cpload $ra               \n"                                     \
+    ".cprestore 12             \n"                                     \
     ".set reorder              \n"                                     \
     ".local    " __STRING(func) "\n"                                   \
     "jal       " __STRING(func)

I fake the .cprestore based on what we do for _mcount in ^/sys/mips/include/profile.h. Whether this is wrong or not, I do not know, but it results in a functional world along with the rest of my mips-llvm patches (https://github.com/freebsd/freebsd/compare/master...kevans91:mips-llvm?expand=1), some of which were stolen from jhb. This branch results in a functional CLANG_BOOTSTRAP+LLD_BOOTSTRAP world and kernel for mips32, but mips64 kernel is horribly broken in weird ways with both llvm8 and projects/clang900-import and CLANG_BOOTSTRAP+LD_BOOTSTRAP.
Comment 6 Ed Maste freebsd_committer 2019-11-08 18:03:35 UTC
Update headline to remove mention of BSD_CRTBEGIN, it is on by default for MIPS now and works.
Comment 7 Ed Maste freebsd_committer 2021-08-23 16:32:36 UTC
all in-tree archs build with clang & lld now