Bug 220024

(In reply to Mark Millard from comment #0)

boot1.o: In function `fsread_size':
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3'
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3'
cc: error: linker command failed with exit code 1 (use -v to see invocation)

is from the buildworld part of the overall build
attempt.

I'll note that back in -r317820 and before I was
able to buildworld and installworld based on the
system clang and use the installed world on an
old PowerMac (other than handling thrown C++
exceptions).


Also: trying buildkernel separately after the
buildworld failure worked fine. [That does not
mean that such a clang-based kernel would
complete the boot sequence on an old PowerMac.]

(In reply to Mark Millard from comment #1)

powerpc64 has the same buildworld problem:

--- boot1.elf ---
boot1.o: In function `fsread_size':
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3'
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
--- all_subdir_usr.sbin ---
Building /usr/obj/powerpc64vtsc_clang/powerpc.powerpc64/usr/src/usr.sbin/fstyp/ext2fs.o
--- all_subdir_sys ---
*** [boot1.elf] Error code 1

make[6]: stopped in /usr/src/sys/boot/powerpc/boot1.chrp
.ERROR_TARGET='boot1.elf'
.ERROR_META_FILE='/usr/obj/powerpc64vtsc_clang/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta'

(In reply to Mark Millard from comment #2)

The prior powerpc64 example was using the system
binutils, just like I had done for powerpc (32-bit).

The problem still occurs when the likes of:

-B/usr/local/powerpc64-freebsd/bin/

is in use instead:

--- boot1.elf ---
boot1.o: In function `fsread_size':
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3'
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
--- all_subdir_tests ---
Building /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/tests/sys/mqueue/mqtest2.full
--- all_subdir_sys ---
*** [boot1.elf] Error code 1

make[6]: stopped in /usr/src/sys/boot/powerpc/boot1.chrp
.ERROR_TARGET='boot1.elf'
.ERROR_META_FILE='/usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta'


# more /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta
# Meta data file /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp/boot1.elf.meta
CMD cc -target powerpc64-unknown-freebsd12.0 --sysroot=/usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/tmp -B/usr/local/powerpc64-freebsd/bin/ -ffreestanding -msoft-float -I/usr/src/sys/boot/powerpc/boot1.chrp/../../common -I/usr/src/sys/boot/powerpc/boot1.chrp/../../../ -D_STANDALONE -m32 -mcpu=powerpc -m32 -mcpu=powerpc -std=gnu99 -Qunused-arguments -nostdlib -static -Wl,-N -Wl,-m -Wl,elf32ppc_fbsd -Wl,-m -Wl,elf32ppc_fbsd -o boot1.elf boot1.o ashldi3.o syncicache.o  
CWD /usr/obj/powerpc64vtsc_clang_altbinutils/powerpc.powerpc64/usr/src/sys/boot/powerpc/boot1.chrp
TARGET boot1.elf
-- command output --
boot1.o: In function `fsread_size':
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x14b8): undefined reference to `__udivdi3'
/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c:(.text+0x1508): undefined reference to `__udivdi3'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1


# more ~/sys_build_scripts.amd64-host/make_powerpc64vtsc_nodebug_clang_altbinutils-amd64-host.sh 
kldload -n filemon && \
script ~/sys_typescripts/typescript_make_powerpc64vtsc_nodebug_clang_altbinutils-amd64-host-$(date +%Y-%m-%d:%H:%M:%S) \
env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" SRC_ENV_CONF="/root/src.configs/src.conf.powerpc64-clang_altbinutils-bootstrap.amd64-host" \
WITH_META_MODE=yes \
MAKEOBJDIRPREFIX="/usr/obj/powerpc64vtsc_clang_altbinutils" \
make $*


# more /root/src.configs/src.conf.powerpc64-clang_altbinutils-bootstrap.amd64-host
TO_TYPE=powerpc64
TOOLS_TO_TYPE=${TO_TYPE}
VERSION_CONTEXT=12.0
#
KERNCONF=GENERIC64vtsc-NODBG
TARGET=powerpc
.if ${.MAKE.LEVEL} == 0
TARGET_ARCH=${TO_TYPE}
.export TARGET_ARCH
.endif
#
WITH_CROSS_COMPILER=
WITHOUT_SYSTEM_COMPILER=
#
WITH_LIBCPLUSPLUS=
WITHOUT_BINUTILS_BOOTSTRAP=
WITH_ELFTOOLCHAIN_BOOTSTRAP=
WITH_CLANG_BOOTSTRAP=
WITH_CLANG=
WITH_CLANG_IS_CC=
WITH_CLANG_FULL=
WITH_CLANG_EXTRAS=
WITHOUT_LLD_BOOTSTRAP=
WITH_LLD=
WITHOUT_LLD_IS_LD=
WITH_LLDB=
#
WITH_BOOT=
WITH_LIB32=
#
WITHOUT_GCC_BOOTSTRAP=
WITHOUT_GCC=
WITHOUT_GCC_IS_CC=
WITHOUT_GNUCXX=
#
NO_WERROR=
MALLOC_PRODUCTION=
#
# Avoid converts between pointers to integer types with different sign [-Werror,-Wpointer-sign]
# and such from blocking the build.
WERROR=
#
WITH_REPRODUCIBLE_BUILD=
WITH_DEBUG_FILES=
#
#
# For TO (so-called "cross") stages . . .
# So-called-cross via ${TO_TYPE}-xtoolchain-gcc/${TO_TYPE}-gcc. . .
# TOOLS_TO_TYPE based on ${TO_TYPE}-xtoolchain-gcc related binutils. . .
#
CROSS_BINUTILS_PREFIX=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/
.if ${.MAKE.LEVEL} == 0
#
# Note: The WITH_CROSS_COMPILER picks up the CROSS_BINUTILS_PREFIX
#       binding automatically.
#
XAS=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/as
XAR=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ar
XNM=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/nm
XOBJCOPY=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objcopy
XOBJDUMP=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objdump
XRANLIB=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ranlib
XSIZE=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/size
#NO-SUCH: XSTRINGS=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/strings
XSTRINGS=/usr/local/bin/${TOOLS_TO_TYPE}-freebsd-strings
.export XAS
.export XAR
.export XNM
.export XOBJCOPY
.export XOBJDUMP
.export XRANLIB
.export XSIZE
.export XSTRINGS
XLD=/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ld
.export XLD
.endif

Turns out clang is generating the __udivdi3
references because of the INO64 changes being
involved in fsread_size --at least if I
understand right.

And that explains why this was not a problem
in the likes of -r317820 (before INO64).

For reference: part of a note that I sent out
on the lists. . .

buildworld via clang for powerpc64 and powerpc fails for lack of
`__udivdi3' referenced in sys/boot/common/ufsread.c fsread_size
code. But this lead to me looking around and I found a conceptually
separate possible issue. . .

sys/sys/_types.h :

typedef	__uint64_t	__ino_t;	/* inode number */

# find /usr/src/sys/ -exec grep __ino_t {} \; -print | more
typedef __ino_t         ino_t;
/usr/src/sys/sys/stat.h
typedef __ino_t         ino_t;          /* inode number */
/usr/src/sys/sys/types.h
typedef __uint64_t      __ino_t;        /* inode number */
/usr/src/sys/sys/_types.h
typedef __ino_t         ino_t;
/usr/src/sys/sys/dirent.h


sys/boot/common/ufsread.c :

. . .
#include <ufs/ufs/dinode.h>
#include <ufs/ufs/dir.h>
#include <ufs/ffs/fs.h>
. . .
typedef uint32_t        ufs_ino_t;
. . .

Note the 32-bit type above. The headers included
have use of the 64-bit ino_t type as well, for
example:

sys/ufs/ufs/diniode.h :

. . .
#define UFS_ROOTINO     ((ino_t)2)
. . .
#define UFS_WINO        ((ino_t)1)
. . .

sys/ufs/ffs/fs.h :

. . .
#define ino_to_cg(fs, x)        (((ino_t)(x)) / (fs)->fs_ipg)
#define ino_to_fsba(fs, x)                                              \
       ((ufs2_daddr_t)(cgimin(fs, ino_to_cg(fs, (ino_t)(x))) +         \
           (blkstofrags((fs), ((((ino_t)(x)) % (fs)->fs_ipg) / INOPB(fs))))))
#define ino_to_fsbo(fs, x)      (((ino_t)(x)) % INOPB(fs))
. . .


I believe the powerpc64/powerpc issue
gives evidence of ino_t being used in
addition to ufs_ino_t in
sys/boot/common/ufsread.c 's fsread_size .

(In reply to Mark Millard from comment #4)

I should have noted up front that:

/usr/src/sys/boot/powerpc/boot1.chrp/boot1.c

does:

#include "ufsread.c"

and that is the context of the __udivdi3
use that is rejected at link time when
clang is used to buildworld for powerpc
or powerpc64.



As for potential INO64 issues beyond
__udivdi3 use and needin got have it
available for linking:

My original comment #4 might really trace
back to boot1.c needing to be different
even if ufsread.c stays the same.

(In reply to Mark Millard from comment #5)

Konstantin Belousov has reported on the lists
that the mix of ino_t (64-bit) and 32-bit ino
types should be fine for this ufs context, mostly
due to ufs being limited to 32 bit inodes in the
first place.

So if the __udivddi3 is supplied so that the
linking can complete for building boot1.elf
via clang then the result should be okay.

(In reply to Mark Millard from comment #6)

Konstantin Belousov later wrote:

I never said that.



I apparently read too much of my
overall purpose into his reply to
what I asked about if the types
needed to be changed in fsread.c .

(In reply to Mark Millard from comment #7)

For reference Konstantin Belousov's original
words that I misrepresented were (copied and
pasted):

UFS uses 32bit inodes, changing to 64bit is both pointless currently, and
causes on-disk layout incompatibilities.

As a consequence, use of ino_t (64bit) or uint32_t for inode numbers are
almost always interchangeable, unless used for specifying on-disk layout.
UFS correctly uses (and was changed to use) uint32_t for inode numbers
in the disk-layout definitions.  Other places, which calculate inode
numbers from inode block numbers, or do some other calculations with
inodes, are fine with either width.

That is, I believe that all instances which I looked at during the
ino64 preparation are fine.

Created attachment 183694 [details]
Makefile adjustment to compile/link missing routines

As of INO64 involving 64-bit integer types
two more sources need to be compiled (and
the .o's linked in):

udivdi3.c

which in turn requires adding:

qdivrem.c

So overall SRCS ends up as:

SRCS=		boot1.c qdivrem.c udivdi3.c ashldi3.c syncicache.c

With this change I was able to build TARGET_ARCH=powerpc
and TARGET_ARCH=powerpc64 .

Comment 10 Mark Linimon 2018-08-29 20:04:34 UTC

Is this patch still relevant?

fwiw, the new location of this file appears to be ./stand/powerpc/boot1.chrp/Makefile .

(In reply to Mark Linimon from comment #10)

I've still got the patch in my environment:

# svnlite diff stand/powerpc/boot1.chrp/Makefile
Index: stand/powerpc/boot1.chrp/Makefile
===================================================================
--- stand/powerpc/boot1.chrp/Makefile	(revision 338341)
+++ stand/powerpc/boot1.chrp/Makefile	(working copy)
@@ -7,7 +7,7 @@
 INSTALLFLAGS=   -b
 
 FILES=		boot1.hfs
-SRCS=		boot1.c ashldi3.c syncicache.c
+SRCS=		boot1.c qdivrem.c udivdi3.c ashldi3.c syncicache.c
 
 CFLAGS+=-I${LDRSRC}
 LDFLAGS=-nostdlib -static -Wl,-N

but I do not have access to the powerpc64's or powerpc's
currently and likely will not for weeks or months.

I do amd64 -> powerpc* cross-build buildworld buildkernel for
them on occasion. I've not tried without the patch, however.

Comment 12 Warner Losh 2018-08-29 22:20:48 UTC

This belongs in libsa/Makefile so we don't have to add it to every  boot/loader program...

(In reply to Warner Losh from comment #12)

Okay.

I'll note that both stand/libsa/Makefile and stand/powerpc/boot1.chrp/Makefile
list SRCS has including ashldi3.c :

.if ${MACHINE_CPUARCH} == "powerpc"
.PATH: ${LIBC_SRC}/quad
SRCS+=  ashldi3.c ashrdi3.c lshrdi3.c
SRCS+=  syncicache.c
.endif

vs. (without my patch):

SRCS=           boot1.c ashldi3.c syncicache.c

But also note the  += vs. = use.

Some old material may also then be moved for where
it is listed, not just qdivrem.c and udivdi3.c .

(In reply to Mark Millard from comment #13)

Hmm. Looking around shows stand/powerpc possibly
not using libsa* at all:

# grep -r stand/libsa /usr/src/stand/ | grep -v "BSD[(:]" | more
/usr/src/stand/userboot/userboot/Makefile.depend:       stand/libsa \
/usr/src/stand/efi/boot1/Makefile.depend:       stand/libsa \
/usr/src/stand/efi/loader/Makefile.depend:      stand/libsa \
/usr/src/stand/i386/gptzfsboot/Makefile.depend: stand/libsa32 \
/usr/src/stand/i386/gptboot/Makefile.depend:    stand/libsa32 \
/usr/src/stand/i386/loader/Makefile.depend:     stand/libsa32 \
/usr/src/stand/i386/zfsboot/Makefile.depend:    stand/libsa32 \

# grep -r SRCS /usr/src/stand/powerpc/
/usr/src/stand/powerpc/kboot/Makefile:SRCS=		conf.c vers.c main.c ppc64_elf_freebsd.c
/usr/src/stand/powerpc/kboot/Makefile:SRCS+=		host_syscall.S hostcons.c hostdisk.c kerneltramp.S kbootfdt.c
/usr/src/stand/powerpc/kboot/Makefile:SRCS+=		ucmpdi2.c
/usr/src/stand/powerpc/boot1.chrp/Makefile:SRCS=		boot1.c ashldi3.c syncicache.c
/usr/src/stand/powerpc/ofw/Makefile:SRCS=		conf.c vers.c main.c elf_freebsd.c ppc64_elf_freebsd.c start.c
/usr/src/stand/powerpc/ofw/Makefile:SRCS+=		ucmpdi2.c
/usr/src/stand/powerpc/ofw/Makefile:SRCS+=		ofwfdt.c
/usr/src/stand/powerpc/uboot/Makefile:SRCS=		start.S conf.c vers.c
/usr/src/stand/powerpc/uboot/Makefile:SRCS+=		ucmpdi2.c

(My patch is omitted above.)

So Warner's note would seem to be asking for a bigger
reorganization for stand/powerpc/*/

I'll note that building for amd64, aarch64, armv7, powerpc64,
and powerpc it was only powerpc* boot1.chrp that failed
historically. But I've not built for all architectures.



ashldi3.c in stand/powerpc/boot1.chrp/Makefile would
overlap with stand/libsa/Makefile also listing it as
stands:

.if ${MACHINE_CPUARCH} == "powerpc"
.PATH: ${LIBC_SRC}/quad
SRCS+=  ashldi3.c ashrdi3.c lshrdi3.c
SRCS+=  syncicache.c
.endif

The same for syncicache.c .

For head -r358966 it builds fine with no
patch to the Makefile and the .meta file
shows no use of qdivrem.o or udivdi3.o 
for such a build. This is true for both
32-bit powerpc and for powerpc64 in my
tests.

I do not know when the change occurred,
but I'm closing this as fixed.