Bug 229744

Summary: WITHOUT=LIB32 breaks installworld
Product: Base System Reporter: Mikhail Teterin <mi>
Component: miscAssignee: Warner Losh <imp>
Status: Closed FIXED    
Severity: Affects Only Me CC: emaste, freebsd-2024, imp, kevans
Priority: ---    
Version: 11.2-RELEASE   
Hardware: Any   
OS: Any   

Description Mikhail Teterin freebsd_committer freebsd_triage 2018-07-12 21:45:31 UTC
I don't have any 32-bit binaries and added LIB32 to the WITHOUT-list. The installworld broke:

....
===> stand/i386/libi386 (install)
===> stand/i386/loader (install)
cc -target x86_64-unknown-freebsd11.2 --sysroot=/usr/obj/symbion/src/tmp -B/usr/obj/symbion/src/tmp/usr/bin -O2 -pipe -march=ivybridge -I/symbion/src/stand/i386/btx/lib -nostdinc -I/usr/obj/symbion/src/stand/libsa32 -I/symbion/src/stand/libsa -D_STANDALONE -I/symbion/src/sys -Ddouble=jagged-little-pill -Dfloat=floaty-mcfloatface -DLOADER_GELI_SUPPORT -I/symbion/src/stand/geli -DLOADER_DISK_SUPPORT -m32 -ffreestanding -mno-mmx -mno-sse -mno-avx -msoft-float -march=i386 -I. -I/symbion/src/stand/common -I/symbion/src/stand/ficl -I/symbion/src/stand/ficl/i386 -I/symbion/src/stand/common -DBF_DICTSIZE=15000 -DLOADER_CD9660_SUPPORT -DLOADER_EXT2FS_SUPPORT -DLOADER_MSDOS_SUPPORT -DLOADER_UFS_SUPPORT -DLOADER_GZIP_SUPPORT -DLOADER_BZIP2_SUPPORT -DLOADER_NET_SUPPORT -DLOADER_NFS_SUPPORT -DLOADER_TFTP_SUPPORT -DLOADER_GPT_SUPPORT -DLOADER_MBR_SUPPORT -Wall -I/symbion/src/stand/i386 -DLOADER_PREFER_AMD64 -std=gnu99 -Wsystem-headers -Werror -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unused-local-typedef -Wno-address-of-packed-member -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parentheses -Qunused-arguments ERROR-tried-to-rebuild-during-make-install  -nostdlib -static -Ttext 0x0 -o loader.sym /usr/obj/symbion/src/stand/i386/btx/lib/crt0.o main.o conf.o vers.o chain.o boot.o commands.o console.o devopen.o interp.o interp_backslash.o interp_parse.o ls.o misc.o module.o load_elf32.o load_elf32_obj.o reloc_elf32.o load_elf64.o load_elf64_obj.o reloc_elf64.o disk.o part.o dev_net.o bcache.o isapnp.o pnp.o interp_forth.o  /usr/obj/symbion/src/stand/ficl32/libficl.a   /usr/obj/symbion/src/stand/i386/libi386/libi386.a /usr/obj/symbion/src/stand/geli/libgeliboot.a /usr/obj/symbion/src/stand/libsa32/libsa32.a
/tmp/install.XPLrpoDl/sh: cc: not found
*** Error code 127

Stop.
make[6]: stopped in /symbion/src/stand/i386/loader
*** Error code 1

Stop.
make[5]: stopped in /symbion/src/stand/i386
*** Error code 1
Comment 1 Kyle Evans freebsd_committer freebsd_triage 2018-07-12 21:48:33 UTC
CC imp@, seems reasonable that we should probably be honored MK_LIB32 before we add the LIB32LIST to SUBDIR in stand/Makefile.
Comment 2 Kyle Evans freebsd_committer freebsd_triage 2018-07-13 02:45:41 UTC
(In reply to Kyle Evans from comment #1)

It occurs to me that it's trying to compile during install, which is generally a no-no and I thought we had some measures to prevent that.

I think WITHOUT_LIB32 is a red herring, and I don't think it actually makes sense to gate the 32-bit libs here behind it since they're not installed but used to build other important bits.
Comment 3 Warner Losh freebsd_committer freebsd_triage 2018-07-13 03:48:38 UTC
confirm your system time is good. We've had issues in the past when the time is in the past and so some files are always newer than the sources which causes builds during install...
Comment 4 Mikhail Teterin freebsd_committer freebsd_triage 2018-07-13 16:22:37 UTC
(In reply to Warner Losh from comment #3)
Confirming. The build is local -- no NFS involved at all.

install succeeded after I manually did "make" in stand/i386/loader and then in another child of stand/i386/pxeldr.
Comment 5 Mikhail Teterin freebsd_committer freebsd_triage 2021-07-18 00:31:46 UTC
This remains a problem three years later -- and on a different computer, building today's 12.x tree... When WITHOUT_LIB32 is set, buildworld succeeds, but installworld fails.

Is it really so difficult to reproduce -- and fix? Am I crazy NOT wanting the 32-bit libraries on a 64-bit machine?
Comment 6 Warner Losh freebsd_committer freebsd_triage 2021-07-18 01:21:58 UTC
I'll note it works in -current.

This is likely a missing MFC for the crazy dependencies we used to have in stand/i386 which I fixed in e713d3a013882893fceb84dd14569052271497a9.

Can you apply that to stable/12 and let me know if that fixes things?
Comment 7 Mikhail Teterin freebsd_committer freebsd_triage 2021-07-18 01:23:39 UTC
> e713d3a013882893fceb84dd14569052271497a9

Sorry, could I trouble to either attach the actual patch to this ticket, or link to it? Thank you.
Comment 8 Warner Losh freebsd_committer freebsd_triage 2021-07-18 01:27:01 UTC
"git cherry-pick e713d3a01388" applies cleanly.


But you can download the patch from https://cgit.freebsd.org/src/commit/?id=e713d3a013882893fceb84dd14569052271497a9
Comment 9 Warner Losh freebsd_committer freebsd_triage 2021-07-18 01:28:52 UTC
Note: if the cherry-pick works "git reset --hard HEAD^" will undo the commit that's done as part of that process... This will be important the next time you do a git pull from upstream...
Comment 10 Warner Losh freebsd_committer freebsd_triage 2021-07-18 01:29:26 UTC
I'll take care of MFCing this and have it assigned to me.
Sorry for letting it linger so long.
Comment 11 Warner Losh freebsd_committer freebsd_triage 2021-07-18 02:19:26 UTC
Also

% cd src
% git show e713d3a01388 | patch -p1

will do the same thing as a cherry-pick in this case w/o committing, in case you have other changes in your tree you don't want to resolve to do the cherry-pick.
Comment 12 Mikhail Teterin freebsd_committer freebsd_triage 2021-07-18 02:21:59 UTC
(In reply to Warner Losh from comment #11)
Thanks for the advice, but my 12.x src-tree is, actually, under Subversion control still :) I'm a git-dissident, as it were.

The clean build WITHOUT_LIB32 is running as I type this -- will post the results, when I have them.
Comment 13 Warner Losh freebsd_committer freebsd_triage 2021-07-18 04:01:33 UTC
(In reply to Mikhail Teterin from comment #12)
Ah! Now your initial reply makes sense. I look forward to good news :)
Comment 14 Mikhail Teterin freebsd_committer freebsd_triage 2021-07-18 04:39:03 UTC
Perhaps, something very similar needs to be done for loader_4th/?

===> stand/i386/loader_4th (install)
installing DIRS BINDIR
install  -d -m 0755 -o root  -g wheel  /boot
cc -target x86_64-unknown-freebsd12.2 --sysroot=/spare/green/src/12/amd64.amd64/tmp -B/spare/green/src/12/amd64.amd64/tmp/usr/bin -O2 -pipe -fno-common -march=ivybridge -I/green/src/12/stand/i386/btx/lib -nostdinc -I/spare/green/src/12/amd64.amd64/stand/libsa32 -I/green/src/12/stand/libsa -D_STANDALONE -I/green/src/12/sys -Ddouble=jagged-little-pill -Dfloat=floaty-mcfloatface -ffunction-sections -fdata-sections -DLOADER_GELI_SUPPORT -I/green/src/12/stand/libsa/geli -DLOADER_DISK_SUPPORT -m32 -ffreestanding -mno-mmx -mno-sse -mno-avx -mno-avx2 -msoft-float -march=i386 -I. -Iinclude -I/green/src/12/stand/common -I/green/src/12/stand/ficl -I/green/src/12/stand/ficl/i386 -I/green/src/12/stand/common -DBF_DICTSIZE=15000 -DLOADER_CD9660_SUPPORT -DLOADER_EXT2FS_SUPPORT -DLOADER_MSDOS_SUPPORT -DLOADER_UFS_SUPPORT -DLOADER_GZIP_SUPPORT -DLOADER_BZIP2_SUPPORT -DLOADER_NET_SUPPORT -DLOADER_NFS_SUPPORT -DLOADER_TFTP_SUPPORT -DLOADER_GPT_SUPPORT -DLOADER_MBR_SUPPORT -DLOADER_ZFS_SUPPORT -I/green/src/12/stand/libsa/zfs -I/green/src/12/sys/cddl/boot/zfs -Wall -I/green/src/12/stand/i386 -DLOADER_PREFER_AMD64 -std=gnu99 -Wno-format-zero-length -Wsystem-headers -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unused-local-typedef -Wno-address-of-packed-member -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parentheses -Oz -Qunused-arguments ERROR-tried-to-rebuild-during-make-install  -nostdlib -static -Wl,--defsym,ORG=0x0,-T,/green/src/12/stand/i386/boot.ldscript -Wl,--gc-sections /spare/green/src/12/amd64.amd64/stand/i386/btx/lib/crt0.o -Wl,--no-rosegment -o loader_4th.sym main.o conf.o vers.o chain.o boot.o commands.o console.o devopen.o interp.o interp_backslash.o interp_parse.o ls.o misc.o module.o load_elf32.o load_elf32_obj.o reloc_elf32.o load_elf64.o load_elf64_obj.o reloc_elf64.o disk.o part.o vdisk.o dev_net.o bcache.o isapnp.o pnp.o interp_forth.o zfs_cmd.o  /spare/green/src/12/amd64.amd64/stand/ficl32/libficl.a  /spare/green/src/12/amd64.amd64/stand/i386/libi386/libi386.a /spare/green/src/12/amd64.amd64/stand/libsa32/libsa32.a
/tmp/install.yJ5H0ZMe/sh: cc: not found
*** Error code 127
Comment 15 Warner Losh freebsd_committer freebsd_triage 2021-07-18 15:30:36 UTC
(In reply to Mikhail Teterin from comment #14)
It just includes the makefile that was patched.

I'll look into this further.
Comment 16 Warner Losh freebsd_committer freebsd_triage 2021-07-18 16:57:44 UTC
(In reply to Warner Losh from comment #15)

I can't recreate this locally.

When building, how many -j jobs do you have? What's the speed of the host you are building on?
Comment 17 Mikhail T. 2021-07-18 17:30:16 UTC
Build runs with -j7, but there is no problem building.

It is the installworld, that fails. And that runs sequentially…
Comment 18 Warner Losh freebsd_committer freebsd_triage 2023-09-17 14:24:14 UTC
Closing. This works for me.
And it's been working for a while.
Please open a new bug if there's still some shortcoming.