Context: amd64 -> powerpc cross build of -r319936 (one of my usual clang-based experiments): --- all_subdir_sys --- Building /usr/obj/powerpcvtsc_clang/powerpc.powerpc/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf --- boot1.elf --- boot1.o: In function `fsread_size': /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3' /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3' cc: error: linker command failed with exit code 1 (use -v to see invocation) --- all_subdir_lib --- Building /usr/obj/powerpcvtsc_clang/powerpc.powerpc/usr/src/lib/msun/s_remquo.po --- all_subdir_sys --- *** [boot1.elf] Error code 1 make[6]: stopped in /usr/src/sys/boot/powerpc/boot1.chrp .ERROR_TARGET='boot1.elf' .ERROR_META_FILE='/usr/obj/powerpcvtsc_clang/powerpc.powerpc/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta' # Meta data file /usr/obj/powerpcvtsc_clang/powerpc.powerpc/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta CMD cc -target powerpc-unknown-freebsd12.0 --sysroot=/usr/obj/powerpcvtsc_clang/powerpc.powerpc/usr/src/tmp -B/usr/obj/powerpcvtsc_clang/powerpc.powerpc/usr/src/tmp/usr/bin -ffreestanding -msoft-float -I/usr/src/sys/boot/powerpc/boot1.chrp/../../common -I/usr/src/sys/boot/powerpc/boot1.chrp/../../../ -D_STANDALONE -std=gnu99 -Qunused-arguments -nostdlib -static -Wl,-N -o boot1.elf boot1.o ashldi3.o syncicache.o CWD /usr/obj/powerpcvtsc_clang/powerpc.powerpc/usr/src/sys/boot/powerpc/boot1.chrp TARGET boot1.elf -- command output -- boot1.o: In function `fsread_size': /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3' /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3' cc: error: linker command failed with exit code 1 (use -v to see invocation) *** Error code 1 Note: This was -j16 for the build. I updated /usr/src and amd64 to -r319991 and then retried cross building for powerpc: same result. Note: I'd frozen at -r317820 until this update. Back then I could buildworld and buildkernel via clang (although I could not boot the clang-based kernel and so had to build a gcc 4.2.1 based one and use it). Note: A buildworld buildkernel via gcc 4.2.1 did not have this problem. It is clang tool chain specific. Clang based build-context details: # more ~/sys_build_scripts.amd64-host/make_powerpcvtsc_nodebug_clang_bootstrap-amd64-host.sh kldload -n filemon && \ script ~/sys_typescripts/typescript_make_powerpcvtsc_nodebug_clang_bootstrap-amd64-host-$(date +%Y-%m-%d:%H:%M:%S) \ env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" SRC_ENV_CONF="/root/src.configs/src.conf.powerpc-clang-bootstrap.amd64-host" \ WITH_META_MODE=yes \ MAKEOBJDIRPREFIX="/usr/obj/powerpcvtsc_clang" \ make $* # more /root/src.configs/src.conf.powerpc-clang-bootstrap.amd64-host TO_TYPE=powerpc # KERNCONF=GENERICvtsc-NODBG TARGET=${TO_TYPE} .if ${.MAKE.LEVEL} == 0 TARGET_ARCH=${TO_TYPE} .export TARGET_ARCH .endif # WITH_CROSS_COMPILER= WITHOUT_SYSTEM_COMPILER= # WITH_LIBCPLUSPLUS= WITH_BINUTILS_BOOTSTRAP= WITH_ELFTOOLCHAIN_BOOTSTRAP= WITH_CLANG_BOOTSTRAP= WITH_CLANG= WITH_CLANG_IS_CC= WITH_CLANG_FULL= WITH_CLANG_EXTRAS= WITH_LLD= # lldb requires missing atomic 8-byte operations for powerpc (non-64) WITHOUT_LLDB= # WITH_BOOT= WITHOUT_LIB32= # WITHOUT_GCC_BOOTSTRAP= WITHOUT_GCC= WITHOUT_GCC_IS_CC= WITHOUT_GNUCXX= # NO_WERROR= # # Use WERROR to avoid stopping at the likes of: # error: implicit conversion from 'int' to 'int8_t' (aka 'signed char') changes value from 128 to -128 [-Werror,-Wconstant-conversion] WERROR= MALLOC_PRODUCTION= # WITH_REPRODUCIBLE_BUILD= WITH_DEBUG_FILES= So the system binutils tools are in used. Even though I build lld, last I tried lld could not be used so it is not the linker used by the above.
(In reply to Mark Millard from comment #0) boot1.o: In function `fsread_size': /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3' /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3' cc: error: linker command failed with exit code 1 (use -v to see invocation) is from the buildworld part of the overall build attempt. I'll note that back in -r317820 and before I was able to buildworld and installworld based on the system clang and use the installed world on an old PowerMac (other than handling thrown C++ exceptions). Also: trying buildkernel separately after the buildworld failure worked fine. [That does not mean that such a clang-based kernel would complete the boot sequence on an old PowerMac.]
(In reply to Mark Millard from comment #1) powerpc64 has the same buildworld problem: --- boot1.elf --- boot1.o: In function `fsread_size': /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3' /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3' cc: error: linker command failed with exit code 1 (use -v to see invocation) --- all_subdir_usr.sbin --- Building /usr/obj/powerpc64vtsc_clang/powerpc.powerpc64/usr/src/usr.sbin/fstyp/ext2fs.o --- all_subdir_sys --- *** [boot1.elf] Error code 1 make[6]: stopped in /usr/src/sys/boot/powerpc/boot1.chrp .ERROR_TARGET='boot1.elf' .ERROR_META_FILE='/usr/obj/powerpc64vtsc_clang/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta'
(In reply to Mark Millard from comment #2) The prior powerpc64 example was using the system binutils, just like I had done for powerpc (32-bit). The problem still occurs when the likes of: -B/usr/local/powerpc64-freebsd/bin/ is in use instead: --- boot1.elf --- boot1.o: In function `fsread_size': /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3' /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3' cc: error: linker command failed with exit code 1 (use -v to see invocation) --- all_subdir_tests --- Building /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/tests/sys/mqueue/mqtest2.full --- all_subdir_sys --- *** [boot1.elf] Error code 1 make[6]: stopped in /usr/src/sys/boot/powerpc/boot1.chrp .ERROR_TARGET='boot1.elf' .ERROR_META_FILE='/usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta' # more /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta # Meta data file /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta CMD cc -target powerpc64-unknown-freebsd12.0 --sysroot=/usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/tmp -B/usr/local/powerpc64-freebsd/bin/ -ffreestanding -msoft-float -I/usr/src/sys/boot/powerpc/boot1.chrp/../../common -I/usr/src/sys/boot/powerpc/boot1.chrp/../../../ -D_STANDALONE -m32 -mcpu=powerpc -m32 -mcpu=powerpc -std=gnu99 -Qunused-arguments -nostdlib -static -Wl,-N -Wl,-m -Wl,elf32ppc_fbsd -Wl,-m -Wl,elf32ppc_fbsd -o boot1.elf boot1.o ashldi3.o syncicache.o CWD /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp TARGET boot1.elf -- command output -- boot1.o: In function `fsread_size': /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3' /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3' cc: error: linker command failed with exit code 1 (use -v to see invocation) *** Error code 1 # more ~/sys_build_scripts.amd64-host/make_powerpc64vtsc_nodebug_clang_altbinutils-amd64-host.sh kldload -n filemon && \ script ~/sys_typescripts/typescript_make_powerpc64vtsc_nodebug_clang_altbinutils-amd64-host-$(date +%Y-%m-%d:%H:%M:%S) \ env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" SRC_ENV_CONF="/root/src.configs/src.conf.powerpc64-clang_altbinutils-bootstrap.amd64-host" \ WITH_META_MODE=yes \ MAKEOBJDIRPREFIX="/usr/obj/powerpc64vtsc_clang_altbinutils" \ make $* # more /root/src.configs/src.conf.powerpc64-clang_altbinutils-bootstrap.amd64-host TO_TYPE=powerpc64 TOOLS_TO_TYPE=${TO_TYPE} VERSION_CONTEXT=12.0 # KERNCONF=GENERIC64vtsc-NODBG TARGET=powerpc .if ${.MAKE.LEVEL} == 0 TARGET_ARCH=${TO_TYPE} .export TARGET_ARCH .endif # WITH_CROSS_COMPILER= WITHOUT_SYSTEM_COMPILER= # WITH_LIBCPLUSPLUS= WITHOUT_BINUTILS_BOOTSTRAP= WITH_ELFTOOLCHAIN_BOOTSTRAP= WITH_CLANG_BOOTSTRAP= WITH_CLANG= WITH_CLANG_IS_CC= WITH_CLANG_FULL= WITH_CLANG_EXTRAS= WITHOUT_LLD_BOOTSTRAP= WITH_LLD= WITHOUT_LLD_IS_LD= WITH_LLDB= # WITH_BOOT= WITH_LIB32= # WITHOUT_GCC_BOOTSTRAP= WITHOUT_GCC= WITHOUT_GCC_IS_CC= WITHOUT_GNUCXX= # NO_WERROR= MALLOC_PRODUCTION= # # Avoid converts between pointers to integer types with different sign [-Werror,-Wpointer-sign] # and such from blocking the build. WERROR= # WITH_REPRODUCIBLE_BUILD= WITH_DEBUG_FILES= # # # For TO (so-called "cross") stages . . . # So-called-cross via ${TO_TYPE}-xtoolchain-gcc/${TO_TYPE}-gcc. . . # TOOLS_TO_TYPE based on ${TO_TYPE}-xtoolchain-gcc related binutils. . . # CROSS_BINUTILS_PREFIX=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ .if ${.MAKE.LEVEL} == 0 # # Note: The WITH_CROSS_COMPILER picks up the CROSS_BINUTILS_PREFIX # binding automatically. # XAS=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/as XAR=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ar XNM=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/nm XOBJCOPY=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objcopy XOBJDUMP=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objdump XRANLIB=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ranlib XSIZE=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/size #NO-SUCH: XSTRINGS=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/strings XSTRINGS=/usr/local/bin/${TOOLS_TO_TYPE}-freebsd-strings .export XAS .export XAR .export XNM .export XOBJCOPY .export XOBJDUMP .export XRANLIB .export XSIZE .export XSTRINGS XLD=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ld .export XLD .endif
Turns out clang is generating the __udivdi3 references because of the INO64 changes being involved in fsread_size --at least if I understand right. And that explains why this was not a problem in the likes of -r317820 (before INO64). For reference: part of a note that I sent out on the lists. . . buildworld via clang for powerpc64 and powerpc fails for lack of `__udivdi3' referenced in sys/boot/common/ufsread.c fsread_size code. But this lead to me looking around and I found a conceptually separate possible issue. . . sys/sys/_types.h : typedef __uint64_t __ino_t; /* inode number */ # find /usr/src/sys/ -exec grep __ino_t {} \; -print | more typedef __ino_t ino_t; /usr/src/sys/sys/stat.h typedef __ino_t ino_t; /* inode number */ /usr/src/sys/sys/types.h typedef __uint64_t __ino_t; /* inode number */ /usr/src/sys/sys/_types.h typedef __ino_t ino_t; /usr/src/sys/sys/dirent.h sys/boot/common/ufsread.c : . . . #include <ufs/ufs/dinode.h> #include <ufs/ufs/dir.h> #include <ufs/ffs/fs.h> . . . typedef uint32_t ufs_ino_t; . . . Note the 32-bit type above. The headers included have use of the 64-bit ino_t type as well, for example: sys/ufs/ufs/diniode.h : . . . #define UFS_ROOTINO ((ino_t)2) . . . #define UFS_WINO ((ino_t)1) . . . sys/ufs/ffs/fs.h : . . . #define ino_to_cg(fs, x) (((ino_t)(x)) / (fs)->fs_ipg) #define ino_to_fsba(fs, x) \ ((ufs2_daddr_t)(cgimin(fs, ino_to_cg(fs, (ino_t)(x))) + \ (blkstofrags((fs), ((((ino_t)(x)) % (fs)->fs_ipg) / INOPB(fs)))))) #define ino_to_fsbo(fs, x) (((ino_t)(x)) % INOPB(fs)) . . . I believe the powerpc64/powerpc issue gives evidence of ino_t being used in addition to ufs_ino_t in sys/boot/common/ufsread.c 's fsread_size .
(In reply to Mark Millard from comment #4) I should have noted up front that: /usr/src/sys/boot/powerpc/boot1.chrp/boot1.c does: #include "ufsread.c" and that is the context of the __udivdi3 use that is rejected at link time when clang is used to buildworld for powerpc or powerpc64. As for potential INO64 issues beyond __udivdi3 use and needin got have it available for linking: My original comment #4 might really trace back to boot1.c needing to be different even if ufsread.c stays the same.
(In reply to Mark Millard from comment #5) Konstantin Belousov has reported on the lists that the mix of ino_t (64-bit) and 32-bit ino types should be fine for this ufs context, mostly due to ufs being limited to 32 bit inodes in the first place. So if the __udivddi3 is supplied so that the linking can complete for building boot1.elf via clang then the result should be okay.
(In reply to Mark Millard from comment #6) Konstantin Belousov later wrote: I never said that. I apparently read too much of my overall purpose into his reply to what I asked about if the types needed to be changed in fsread.c .
(In reply to Mark Millard from comment #7) For reference Konstantin Belousov's original words that I misrepresented were (copied and pasted): UFS uses 32bit inodes, changing to 64bit is both pointless currently, and causes on-disk layout incompatibilities. As a consequence, use of ino_t (64bit) or uint32_t for inode numbers are almost always interchangeable, unless used for specifying on-disk layout. UFS correctly uses (and was changed to use) uint32_t for inode numbers in the disk-layout definitions. Other places, which calculate inode numbers from inode block numbers, or do some other calculations with inodes, are fine with either width. That is, I believe that all instances which I looked at during the ino64 preparation are fine.
Created attachment 183694 [details] Makefile adjustment to compile/link missing routines As of INO64 involving 64-bit integer types two more sources need to be compiled (and the .o's linked in): udivdi3.c which in turn requires adding: qdivrem.c So overall SRCS ends up as: SRCS= boot1.c qdivrem.c udivdi3.c ashldi3.c syncicache.c With this change I was able to build TARGET_ARCH=powerpc and TARGET_ARCH=powerpc64 .
Is this patch still relevant? fwiw, the new location of this file appears to be ./stand/powerpc/boot1.chrp/Makefile .
(In reply to Mark Linimon from comment #10) I've still got the patch in my environment: # svnlite diff stand/powerpc/boot1.chrp/Makefile Index: stand/powerpc/boot1.chrp/Makefile =================================================================== --- stand/powerpc/boot1.chrp/Makefile (revision 338341) +++ stand/powerpc/boot1.chrp/Makefile (working copy) @@ -7,7 +7,7 @@ INSTALLFLAGS= -b FILES= boot1.hfs -SRCS= boot1.c ashldi3.c syncicache.c +SRCS= boot1.c qdivrem.c udivdi3.c ashldi3.c syncicache.c CFLAGS+=-I${LDRSRC} LDFLAGS=-nostdlib -static -Wl,-N but I do not have access to the powerpc64's or powerpc's currently and likely will not for weeks or months. I do amd64 -> powerpc* cross-build buildworld buildkernel for them on occasion. I've not tried without the patch, however.
This belongs in libsa/Makefile so we don't have to add it to every boot/loader program...
(In reply to Warner Losh from comment #12) Okay. I'll note that both stand/libsa/Makefile and stand/powerpc/boot1.chrp/Makefile list SRCS has including ashldi3.c : .if ${MACHINE_CPUARCH} == "powerpc" .PATH: ${LIBC_SRC}/quad SRCS+= ashldi3.c ashrdi3.c lshrdi3.c SRCS+= syncicache.c .endif vs. (without my patch): SRCS= boot1.c ashldi3.c syncicache.c But also note the += vs. = use. Some old material may also then be moved for where it is listed, not just qdivrem.c and udivdi3.c .
(In reply to Mark Millard from comment #13) Hmm. Looking around shows stand/powerpc possibly not using libsa* at all: # grep -r stand/libsa /usr/src/stand/ | grep -v "BSD[(:]" | more /usr/src/stand/userboot/userboot/Makefile.depend: stand/libsa \ /usr/src/stand/efi/boot1/Makefile.depend: stand/libsa \ /usr/src/stand/efi/loader/Makefile.depend: stand/libsa \ /usr/src/stand/i386/gptzfsboot/Makefile.depend: stand/libsa32 \ /usr/src/stand/i386/gptboot/Makefile.depend: stand/libsa32 \ /usr/src/stand/i386/loader/Makefile.depend: stand/libsa32 \ /usr/src/stand/i386/zfsboot/Makefile.depend: stand/libsa32 \ # grep -r SRCS /usr/src/stand/powerpc/ /usr/src/stand/powerpc/kboot/Makefile:SRCS= conf.c vers.c main.c ppc64_elf_freebsd.c /usr/src/stand/powerpc/kboot/Makefile:SRCS+= host_syscall.S hostcons.c hostdisk.c kerneltramp.S kbootfdt.c /usr/src/stand/powerpc/kboot/Makefile:SRCS+= ucmpdi2.c /usr/src/stand/powerpc/boot1.chrp/Makefile:SRCS= boot1.c ashldi3.c syncicache.c /usr/src/stand/powerpc/ofw/Makefile:SRCS= conf.c vers.c main.c elf_freebsd.c ppc64_elf_freebsd.c start.c /usr/src/stand/powerpc/ofw/Makefile:SRCS+= ucmpdi2.c /usr/src/stand/powerpc/ofw/Makefile:SRCS+= ofwfdt.c /usr/src/stand/powerpc/uboot/Makefile:SRCS= start.S conf.c vers.c /usr/src/stand/powerpc/uboot/Makefile:SRCS+= ucmpdi2.c (My patch is omitted above.) So Warner's note would seem to be asking for a bigger reorganization for stand/powerpc/*/ I'll note that building for amd64, aarch64, armv7, powerpc64, and powerpc it was only powerpc* boot1.chrp that failed historically. But I've not built for all architectures. ashldi3.c in stand/powerpc/boot1.chrp/Makefile would overlap with stand/libsa/Makefile also listing it as stands: .if ${MACHINE_CPUARCH} == "powerpc" .PATH: ${LIBC_SRC}/quad SRCS+= ashldi3.c ashrdi3.c lshrdi3.c SRCS+= syncicache.c .endif The same for syncicache.c .
For head -r358966 it builds fine with no patch to the Makefile and the .meta file shows no use of qdivrem.o or udivdi3.o for such a build. This is true for both 32-bit powerpc and for powerpc64 in my tests. I do not know when the change occurred, but I'm closing this as fixed.