Bug 203094 - Unable to make installworld without disabling tests and setting COMPILER_TYPE=clang
Summary: Unable to make installworld without disabling tests and setting COMPILER_TYPE...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: tests (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-testing mailing list
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2015-09-14 08:05 UTC by Juan Ramón Molina Menor
Modified: 2018-09-30 05:44 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Juan Ramón Molina Menor 2015-09-14 08:05:03 UTC
The standard build process works just as normal with a fresh source (September 13th) but fails with make installworld:

=========
mkdir -p /tmp/install.H63Fp2Ty
progs=$(for prog in [ awk cap_mkdb cat chflags chmod chown cmp cp  date echo egrep find grep id install   ln lockf make mkdir mtree mv pwd_mkdb  rm sed services_mkdb sh strip sysctl test true uname wc zic tzsetup   makewhatis; do  if progpath=`which $prog`; then  echo $progpath;  else  echo "Required tool $prog not found in PATH." >&2;  exit 1;  fi;  done);  libs=$(ldd -f "%o %p\n" -f "%o %p\n" $progs 2>/dev/null | sort -u |  while read line; do  set -- $line;  if [ "$2 $3" != "not found" ]; then  echo $2;  else  echo "Required library $1 not found." >&2;  exit 1;  fi;  done);  cp $libs $progs /tmp/install.H63Fp2Ty
cp -R ${PATH_LOCALE:-"/usr/share/locale"} /tmp/install.H63Fp2Ty/locale
cd /usr/src; MAKEOBJDIRPREFIX=/usr/obj MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE= GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/tmp/install.H63Fp2Ty  LD_LIBRARY_PATH=/tmp/install.H63Fp2Ty  PATH_LOCALE=/tmp/install.H63Fp2Ty/locale make -f Makefile.inc1    __MAKE_SHELL=/tmp/install.H63Fp2Ty/sh reinstall;  MAKEOBJDIRPREFIX=/usr/obj MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE= GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/tmp/install.H63Fp2Ty  LD_LIBRARY_PATH=/tmp/install.H63Fp2Ty  PATH_LOCALE=/tmp/install.H63Fp2Ty/locale rm -rf /tmp/install.H63Fp2Ty
make[2]: "/usr/src/share/mk/bsd.compiler.mk" line 48: Unable to determine compiler type for /usr/local/libexec/ccache/world/cc.  Consider setting COMPILER_TYPE.
*** Error code 1

Stop.
make[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src
=========

Setting 'make COMPILER_TYPE=clang installworld' allows advancing further, but still fails:

=========
install -s -o root -g wheel -m 555   h_memmove /usr/tests/lib/libc/ssp/h_memmove
install -o root  -g wheel -m 555  ssp_test  /usr/tests/lib/libc/ssp/ssp_test
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_memset  make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSING_PROGS=  SUBDIR= PROG=h_memset  install)
install -s -o root -g wheel -m 555   h_memset /usr/tests/lib/libc/ssp/h_memset
install -o root  -g wheel -m 555  ssp_test  /usr/tests/lib/libc/ssp/ssp_test
(cd /usr/src/lib/libc/tests/ssp &&  DEPENDFILE=.depend.h_raw  make -f /usr/src/lib/libc/tests/ssp/Makefile _RECURSING_PROGS=  SUBDIR= PROG=h_raw  install)
install -s -o root -g wheel -m 555   h_raw /usr/tests/lib/libc/ssp/h_raw
install: h_raw: No such file or directory
*** Error code 71

Stop.
make[8]: stopped in /usr/src/lib/libc/tests/ssp
*** Error code 1

Stop.
make[7]: stopped in /usr/src/lib/libc/tests/ssp
*** Error code 1

Stop.
make[6]: stopped in /usr/src/lib/libc/tests
*** Error code 1

Stop.
make[5]: stopped in /usr/src/lib/libc
*** Error code 1

Stop.
make[4]: stopped in /usr/src/lib
*** Error code 1

Stop.
make[3]: stopped in /usr/src
*** Error code 1

Stop.
make[2]: stopped in /usr/src
*** Error code 1

Stop.
make[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src
=========

I have to disable installing tests to make it work: 'make COMPILER_TYPE=clang -DWITHOUT_TESTS installworld'

I use ccache, but I have tested without it and the result is the same.
Comment 1 Enji Cooper freebsd_committer 2015-09-14 08:07:39 UTC
- How did you do `make buildworld`?
- Did you update your source tree between when you ran buildworld vs installworld?
- How is your source tree/object tree mounted (noatime, atime, etc)?
Comment 2 Enji Cooper freebsd_committer 2015-09-14 08:09:41 UTC
(In reply to Garrett Cooper,425-314-3911 from comment #1)

Exact commands are needed. In particular, if you are calling make installworld without sudo -E and are using env/export to export variables, that won't work necessarily as expected (but you would have run into issues later on with mismatched sources/objects ..)
Comment 3 Enji Cooper freebsd_committer 2015-09-14 08:12:08 UTC
Finally, this *should* not happen, but I'm wondering if somehow the build vs installed version of ${CC} is affecting what's being built:

29 # This testcase doesn't run properly when not compiled with -fsantize=bounds
30 # with clang, which is currently contingent on a compiler_rt update
31 .if ${COMPILER_TYPE} == "clang" && ${COMPILER_VERSION} < 30500
32 PROGS+=         h_raw
33 .endif

This would make sense and it would be a huge bug with whatever we do as far as COMPILER_VERSION gating logic is concerned (it affects a few handfuls of areas of the tree).
Comment 4 Juan Ramón Molina Menor 2015-09-14 08:25:59 UTC
(In reply to Garrett Cooper,425-314-3911 from comment #1)
First, I just did 'make -j2 buildworld' and it stopped on 'make installworld' as reported. When I saw that setting COMPILER_TYPE allowed the install to proceed further (till the h_raw error), I tried starting again with 'make COMPILER_TYPE=clang -j2 buildworld', which worked, but installworld failed again on the h_raw error. I have not tried disabling the tests at the buildworld stage.
Comment 5 Juan Ramón Molina Menor 2015-09-14 08:29:20 UTC
(In reply to Garrett Cooper,425-314-3911 from comment #1)
- Did you update your source tree between when you ran buildworld vs installworld?

No.

- How is your source tree/object tree mounted (noatime, atime, etc)?

This is a plain vanilla install HEAD snapshot install downloaded Saturday:

# cat /etc/fstab
# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/ada0s1a    /               ufs     rw      1       1
/dev/ada0s1b    none            swap    sw      0       0
Comment 6 Enji Cooper freebsd_committer 2015-09-14 08:30:05 UTC
(In reply to Juan Ramón Molina Menor from comment #4)
What version are you upgrading from/to?
Comment 7 Juan Ramón Molina Menor 2015-09-14 08:50:47 UTC
(In reply to Garrett Cooper,425-314-3911 from comment #6)

This is a fresh install of one of the latest snapshots:
FreeBSD-11.0-CURRENT-amd64-20150903-r287437-mini-memstick.img.xz

Then I download source:
/usr/src # svnlite info
Path: .
Working Copy Root Path: /usr/src
URL: https://svn.freebsd.org/base/head
Relative URL: ^/head
Repository Root: https://svn.freebsd.org/base
Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
Revision: 287761
Node Kind: directory
Schedule: normal
Last Changed Author: jilles
Last Changed Rev: 287761
Last Changed Date: 2015-09-13 19:17:52 +0200 (Sun, 13 Sep 2015)

Then I followed the upgrade procedure stated in /usr/src/UPDATING:
make -j2 buildworld
make kernel
<reboot in single user>
mergemaster -Fp
make installworld

…and it failed as described.

Only two things depart from a plain install: I installed devel/ccache and NFS-exported /usr/src and /usr/obj, as the aim is to upgrade a USB stick I’m using to test head on my ThinkPad S440. But I have tried the above procedure without ccache and without NFS exports (commented-out ccache in /etc/make.conf and NFS lines in /etc/rc.conf and /etc/exports, then rebooted computer) and the failure is still the same.

The machine is on and I can make as many tests as you wish.
Comment 8 Enji Cooper freebsd_committer 2015-09-14 08:56:04 UTC
Still not enough info. What version are you upgrading the base system from? The version that's on the USB/CD isn't important, unless it's in sync with the version that you're installing to on the base system.

Please note: building/installing on the base system requires some degree of finesse. Try using a kernel greater than the install version, chrooting into the install tree, then running make installworld (yes, it would be a doc bug, but as a rule of thumb I chroot into directories and run installworld because otherwise.. it's running binaries off the install media).
Comment 9 Juan Ramón Molina Menor 2015-09-14 09:06:03 UTC
(In reply to Garrett Cooper,425-314-3911 from comment #8)
> Still not enough info. What version are you upgrading the base system from?
> The version that's on the USB/CD isn't important, unless it's in sync with
> the version that you're installing to on the base system.

Sorry, I must be obtuse, but I don’t understand your question: This is a just a FreeBSD-11.0-CURRENT-amd64-20150903-r287437 mini-memstick image booted-up and installed on a wiped-out hard disk. Then I downloaded sources from the SVN repository (2015-09-13 r287761, ten days older) with svnlite, then tried to upgrade.
Comment 10 Enji Cooper freebsd_committer 2015-09-15 08:08:06 UTC
(In reply to Juan Ramón Molina Menor from comment #9)

Just to be clear -- did you build/install from the rebooted system or the USB live image?
Comment 11 Juan Ramón Molina Menor 2015-09-15 08:38:50 UTC
(In reply to Garrett Cooper,425-314-3911 from comment #10)
I did build/install from the rebooted system, the USB (mini-memstick, whithout sources) was not even connected.
Comment 12 Juan Ramón Molina Menor 2015-10-01 10:09:25 UTC
This is still happening with the system updated to base r288424.

Process as usual:
# cd /usr/src
# svnlite up
…updates from r287761 to r288424…
# make -j2 buildworld
# make -j2 kernel
# shutdown now
# make -j2 installworld
…fails as previously reported…
# make COMPILER_TYPE=clang -j2 -DWITHOUT_TESTS installworld
…works…

It’s really curious I am the only one seeing this.
Comment 13 Pierre Kraschinski 2016-03-29 09:41:49 UTC
I'm able to confirm and reproduce (a similar/this) problem during the build-/installworld process in combination with ccache. 

1. TEST-ENV
- ThinkPad X220 (4290-DD1)
- Fresh installed FreeBSD 11.0-CURRENT amd64 r296485

2. WORKING REFERENCE
- check out freebsd-current (r297375)
  # svnlite co -r 297375 https://svn.freebsd.org/base/head/ /usr/src
- modify /etc/make.conf
  # vi /etc/make.conf

<--
MALLOC_PRODUCTION=YES
-->

- start build and install process
  # make -j16 buildworld
  # make -j16 buildkernel KERNCONF=GENERIC-NODEBUG
  <reboot in s.u. mode> 
  # mergemaster -Fp
  # make installworld
  # mergemaster -Fi
  # make delete-old
  <reboot>
- Everything works as usual

2. NON-WORKING EXAMPLE
- install ccache version 3.2.4
  # pkg install ccache
- modify /etc/make.conf
  # vi /etc/make.conf

<--
MALLOC_PRODUCTION=YES
WITH_CCACHE_BUILD=YES

.if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
.if !defined(NOCCACHE) && exists(/usr/local/libexec/ccache/world/cc)
CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
.endif
.endif
-->

- modify /etc/csh.cshrc
  # vi /etc/csh.cshrc

<--
setenv PATH /usr/local/libexec/ccache:$PATH
setenv CCACHE_PATH /usr/bin:/usr/local/bin
-->

- check out freebsd-current (r297375)
  # svnlite co -r 297375 https://svn.freebsd.org/base/head/ /usr/src
- start build and install process
  # make -j16 buildworld
  # make -j16 buildkernel KERNCONF=GENERIC-NODEBUG
  <reboot in s.u. mode> 
  # mergemaster -Fp
  # make installworld

- the problem occurs
<--
mkdir -p /tmp/install.KI6wjBV6
progs=$(for prog in [ awk cap_mkdb cat chflags chmod chown cmp cp  date echo egrep find grep id install   ln make mkdir mtree mv pwd_mkdb  rm sed services_mkdb sh strip sysctl test true uname wc zic tzsetup   makewhatis; do  if progpath=`which $prog`; then  echo $progpath;  else  echo "Required tool $prog not found in PATH." >&2;  exit 1;  fi;  done);  libs=$(ldd -f "%o %p\n" -f "%o %p\n" $progs 2>/dev/null | sort -u |  while read line; do  set -- $line;  if [ "$2 $3" != "not found" ]; then  echo $2;  else  echo "Required library $1 not found." >&2;  exit 1;  fi;  done);  cp $libs $progs /tmp/install.KI6wjBV6
cp -R ${PATH_LOCALE:-"/usr/share/locale"} /tmp/install.KI6wjBV6/locale
cd /usr/src; MAKEOBJDIRPREFIX=/usr/obj MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE= GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac CC="/usr/local/libexec/ccache/world/cc -target x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin" CXX="/usr/local/libexec/ccache/world/c++ -target x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin " DEPFLAGS="" CPP="cpp -target x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin" AS="as" AR="ar" LD="ld" NM=nm OBJDUMP=objdump OBJCOPY="objcopy" RANLIB=ranlib STRINGS= SIZE="size" PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/tmp/install.KI6wjBV6  LD_LIBRARY_PATH=/tmp/install.KI6wjBV6  PATH_LOCALE=/tmp/install.KI6wjBV6/locale /usr/obj/usr/src/make.amd64/bmake -f Makefile.inc1    __MAKE_SHELL=/tmp/install.KI6wjBV6/sh reinstall;  MAKEOBJDIRPREFIX=/usr/obj MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE= GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac CC="/usr/local/libexec/ccache/world/cc -target x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin" CXX="/usr/local/libexec/ccache/world/c++ -target x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin " DEPFLAGS="" CPP="cpp -target x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin" AS="as" AR="ar" LD="ld" NM=nm OBJDUMP=objdump OBJCOPY="objcopy" RANLIB=ranlib STRINGS= SIZE="size" PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/tmp/install.KI6wjBV6  LD_LIBRARY_PATH=/tmp/install.KI6wjBV6  PATH_LOCALE=/tmp/install.KI6wjBV6/locale rm -rf /tmp/install.KI6wjBV6
ccache: error: Could not find compiler "cc" in PATH
bmake[2]: "/usr/src/share/mk/bsd.compiler.mk" line 141: Unable to determine compiler type for /usr/local/libexec/ccache/world/cc -target x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin.  Consider setting COMPILER_TYPE.
*** Error code 1

Stop.
bmake[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src
-->

- NOTE: to proceed at this step it is necessary to set "COMPILER_TYPE=clang -DWITHOUT_TESTS" as Juan wrote before.
  # make COMPILER_TYPE=clang -DWITHOUT_TESTS installworld
  ...
Comment 14 Pierre Kraschinski 2016-03-29 17:03:12 UTC
During my research I've found SVN:290526 (https://svnweb.freebsd.org/base?view=revision&revision=290526). Bryan added ccache build support via WITH_CCACHE_BUILD option to "base". This removes the need to add the following lines (as suggested in /usr/local/share/doc/ccache/ccache-howto-freebsd.txt) to /etc/make.conf:

<--
.if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
.if !defined(NOCCACHE) && exists(/usr/local/libexec/ccache/world/cc)
CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
.endif
.endif
-->

Remove them and just add WITH_CCACHE_BUILD to /etc/make.conf:

<--
WITH_CCACHE_BUILD=YES
-->

Regarding to SVN:290526 Bryans work should have been fixed all known issues with buildworld and ccache. 

After modifying my /etc/make.conf I was able to rebuild world and install it as usual.
Comment 15 Chris Collins 2017-06-08 12:01:58 UTC
Something is still broken.

Here is my situation.

Base system is 10-STABLE dated Sep 26 2016
New src code is RELENG 11.0 P9

ccache is installed on the system

I first couldnt even get buildworld to complete, very odd error rejecting cputype of nehalem, this error appeared even with no cputype set in make.conf.
I proceeded to uncomment the CC= lines that set as below

.if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
.if !defined(NOCCACHE) && exists(/usr/local/libexec/ccache/world/cc)
CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
.endif
.endif

After I did this the buildworld proceeded but very slow, I discovered ccache was operating, as the misses were tallying up and I had disk thrashing from the cache objects been written.

So I then commented out WITH_CCACHE_BUILD=YES in make.conf, the line is not present in src.conf at all.

This time buildworld completed and at normal speed.

But now I get the error on make installworld.

So the question is if none of the cache lines are active in either make.conf or src.conf and one gets this error.

Is it a bug or operator error?  

bmake[2]: "/usr/src/110/share/mk/bsd.compiler.mk" line 132: warning: "/usr/local/bin/ccache cc  --version || echo 0.0.0" returned non-zero status
bmake[2]: "/usr/src/110/share/mk/bsd.compiler.mk" line 145: Unable to determine compiler type for CC=/usr/local/bin/ccache cc .  Consider setting COMPILER_TYPE.

So here is 2 observations.

1 - The compiler used during buildworld was not "/usr/local/bin/ccache cc"it was just "cc"
2 - Why is it showing "/usr/local/bin/ccache cc" when I have nothing enabling ccache active in either src.conf or make.conf, is simply having the ccache port installed enough to make the world tools try to use it?
Comment 16 Chris Collins 2017-06-08 12:04:58 UTC
Here is error

bmake[2]: "/usr/src/110/share/mk/bsd.compiler.mk" line 132: warning: "cc  --version || echo 0.0.0" returned non-zero status
bmake[2]: "/usr/src/110/share/mk/bsd.compiler.mk" line 145: Unable to determine compiler type for CC=cc .  Consider setting COMPILER_TYPE.
*** Error code 1

The above is when I tried to add to src.conf following the advise in earlier reply.
Comment 17 Chris Collins 2017-06-08 12:07:35 UTC
So I added CC=clang to src.conf

Now more weirdness

bmake[2]: "/usr/src/110/share/mk/bsd.compiler.mk" line 132: warning: "clang --version || echo 0.0.0" returned non-zero status
bmake[2]: "/usr/src/110/share/mk/bsd.compiler.mk" line 149: warning: "echo "" | awk -F. '{print $1 * 10000 + $2 * 100 + $3;}'" returned non-zero status
bmake[2]: "/usr/src/110/share/mk/bsd.compiler.mk" line 154: warning: "{ echo "__FreeBSD_cc_version" | clang -E - 2>/dev/null || echo __FreeBSD_cc_version; } | sed -n '$p'" returned non-zero status
bmake[2]: "/usr/src/110/Makefile.inc1" line 385: warning: "MK_AUTO_OBJ=no MAKEFLAGS= CPUTYPE=nehalem /usr/obj/usr/src/110/make.amd64/bmake  -f /dev/null -m /usr/src/110/share/mk -V CPUTYPE" returned non-zero status
bmake[2]: "/usr/src/110/Makefile.inc1" line 387: CPUTYPE global should be set with ?=.
*** Error code 1

The weirdness is the fact CPUTYPE?= is actually set.