Bug 213428 - make buildworld relies on /usr/include i.e. -nostdinc breaks buildworld
Summary: make buildworld relies on /usr/include i.e. -nostdinc breaks buildworld
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.3-STABLE
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-bugs mailing list
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2016-10-12 23:25 UTC by frf
Modified: 2016-10-13 04:07 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description frf 2016-10-12 23:25:51 UTC
buildworld fails because it relies on /usr/include instead of /usr/src/include. i.e. buildworld does not rely upon a good source tree, but rather relies upon /usr/include to be correct for /usr/src. We used to have -nostdinc as a compiler flag, but the addition of this flag now causes buildworld to fail in all cases that I tested.
I don't know when this change was made. It could have been many years ago, and/or with the adoption of llvm, but it is a bug that should be corrected. I should not need to rely upon /usr/include being clean to make world when I have a perfectly good /usr/src. For example, I could have a system crash during an installworld leaving /usr/include in an uncertain state that would then prohibit me from rebuilding a system. 
This is a regression from past stability that needs to be corrected.
Comment 1 Mark Millard 2016-10-13 00:57:15 UTC
(In reply to frf from comment #0)

FYI:

buildworld normally depends on a build tree under somewhere like (say) /usr/obj/ someplace that has its own usr/src/tmp/usr/include/ directory tree that buildworld first fills in with target-specific files before they are used (when things are working).

I wonder if you are somehow not getting the proper redirections into that build tree include area during buildworld. Has the build tree usr/src/tmp/usr/include/ area been filled in correctly?

You have not published enough material in this submittal from build logs and the like to see what example compiler commands look like and what the -I's and such are like. Such publishing is likely required for someone to figure out what is going on in your context.

You have not published your src.conf and make.conf content in this submittal either. Do you have any other build tailoring that should be published? It would be helpful to folks that try to help for you to provide such material so they know in detail what contributed to the build attempts.


Notes:

For cross builds this sort of build tree usr/src/tmp/usr/include/ area need not match some host file versions in various ways.
Comment 2 frf 2016-10-13 01:35:36 UTC
Try this:

# uname -a
FreeBSD snif01 10.3-STABLE FreeBSD 10.3-STABLE #0 r307159: Wed Oct 12 23:30:20 UTC 2016     root@snif01:/usr/obj/usr/src/sys/GENERIC  amd64

# mv /etc/src.conf /
# mv /etc/make.conf /
# pwd
/usr/src
# make buildworld
--------------------------------------------------------------
>>> World build started on Thu Oct 13 01:12:12 UTC 2016
--------------------------------------------------------------

--------------------------------------------------------------
>>> Rebuilding the temporary build tree
--------------------------------------------------------------
rm -rf /usr/obj/usr/src/tmp
rm -rf /usr/obj/usr/src/lib32
mkdir -p /usr/obj/usr/src/tmp/lib
mkdir -p /usr/obj/usr/src/tmp/usr
mkdir -p /usr/obj/usr/src/tmp/legacy/bin
mkdir -p /usr/obj/usr/src/tmp/legacy/usr
mtree -deU -f /usr/src/etc/mtree/BSD.usr.dist  -p /usr/obj/usr/src/tmp/legacy/usr >/dev/null
mtree -deU -f /usr/src/etc/mtree/BSD.groff.dist  -p /usr/obj/usr/src/tmp/legacy/usr >/dev/null
mtree -deU -f /usr/src/etc/mtree/BSD.usr.dist  -p /usr/obj/usr/src/tmp/usr >/dev/null
mtree -deU -f /usr/src/etc/mtree/BSD.include.dist  -p /usr/obj/usr/src/tmp/usr/include >/dev/null
ln -sf /usr/src/sys /usr/obj/usr/src/tmp

--------------------------------------------------------------
>>> stage 1.1: legacy release compatibility shims
--------------------------------------------------------------
cd /usr/src; MAKEOBJDIRPREFIX=/usr/obj/usr/src/tmp  INSTALL="sh /usr/src/tools/install.sh" PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/legacy/bin:/sbin:/bin:/usr/sbin:/usr/bin  WORLDTMP=/usr/obj/usr/src/tmp  VERSION="FreeBSD 10.3-STABLE amd64 1003508" MAKEFLAGS="-m /usr/src/tools/build/mk  -m /usr/src/share/mk" COMPILER_TYPE=clang make  -f Makefile.inc1  DESTDIR=  BOOTSTRAPPING=0 SSP_CFLAGS=  -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT -DWITHOUT_MAN  -DNO_PIC -DNO_PROFILE -DNO_SHARED  _BOOTSTRAP_MAKEINFO=yes  -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS legacy
===> tools/build (obj,includes,depend,all,install)
grep: /usr/include/stdio.h: No such file or directory
grep: /usr/include/pwd.h: No such file or directory
grep: /usr/include/vis.h: No such file or directory
/usr/obj/usr/src/tmp/usr/src/tools/build created for /usr/src/tools/build
grep: /usr/include/stdio.h: No such file or directory
grep: /usr/include/pwd.h: No such file or directory
grep: /usr/include/vis.h: No such file or directory
set -e; cd /usr/src/tools/build; make buildincludes; make installincludes
grep: /usr/include/stdio.h: No such file or directory
grep: /usr/include/pwd.h: No such file or directory
grep: /usr/include/vis.h: No such file or directory
grep: /usr/include/stdio.h: No such file or directory
grep: /usr/include/pwd.h: No such file or directory
grep: /usr/include/vis.h: No such file or directory
grep: /usr/include/stdio.h: No such file or directory
grep: /usr/include/pwd.h: No such file or directory
grep: /usr/include/vis.h: No such file or directory
make[3]: don't know how to make getline.c. Stop

make[3]: stopped in /usr/src/tools/build
*** Error code 2

Stop.
make[2]: stopped in /usr/src
*** Error code 1

Stop.
make[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src
# 



Or this:

# uname -a
FreeBSD snif01 10.3-STABLE FreeBSD 10.3-STABLE #0 r307159: Wed Oct 12 23:30:20 UTC 2016     root@snif01:/usr/obj/usr/src/sys/GENERIC  amd64

# make CFLAGS='CFLAGS -nostdinc' buildworld
--------------------------------------------------------------
>>> World build started on Thu Oct 13 01:28:49 UTC 2016
--------------------------------------------------------------

--------------------------------------------------------------
>>> Rebuilding the temporary build tree
--------------------------------------------------------------
rm -rf /usr/obj/usr/src/tmp
rm -rf /usr/obj/usr/src/lib32
mkdir -p /usr/obj/usr/src/tmp/lib
mkdir -p /usr/obj/usr/src/tmp/usr
mkdir -p /usr/obj/usr/src/tmp/legacy/bin
mkdir -p /usr/obj/usr/src/tmp/legacy/usr
mtree -deU -f /usr/src/etc/mtree/BSD.usr.dist  -p /usr/obj/usr/src/tmp/legacy/usr >/dev/null
mtree -deU -f /usr/src/etc/mtree/BSD.groff.dist  -p /usr/obj/usr/src/tmp/legacy/usr >/dev/null
mtree -deU -f /usr/src/etc/mtree/BSD.usr.dist  -p /usr/obj/usr/src/tmp/usr >/dev/null
mtree -deU -f /usr/src/etc/mtree/BSD.include.dist  -p /usr/obj/usr/src/tmp/usr/include >/dev/null
ln -sf /usr/src/sys /usr/obj/usr/src/tmp

--------------------------------------------------------------
>>> stage 1.1: legacy release compatibility shims
--------------------------------------------------------------
cd /usr/src; MAKEOBJDIRPREFIX=/usr/obj/usr/src/tmp  INSTALL="sh /usr/src/tools/install.sh" PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/legacy/bin:/sbin:/bin:/usr/sbin:/usr/bin  WORLDTMP=/usr/obj/usr/src/tmp  VERSION="FreeBSD 10.3-STABLE amd64 1003508" MAKEFLAGS="-m /usr/src/tools/build/mk  -m /usr/src/share/mk" COMPILER_TYPE=clang make  -f Makefile.inc1  DESTDIR=  BOOTSTRAPPING=1003508 SSP_CFLAGS=  -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT -DWITHOUT_MAN  -DNO_PIC -DNO_PROFILE -DNO_SHARED  _BOOTSTRAP_MAKEINFO=no  -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS legacy
===> tools/build (obj,includes,depend,all,install)
/usr/obj/usr/src/tmp/usr/src/tools/build created for /usr/src/tools/build
set -e; cd /usr/src/tools/build; make buildincludes; make installincludes
rm -f .depend
mkdep -f .depend -a   -nostdinc -I/usr/src/tools/build/../../contrib/libc-pwcache -I/usr/src/tools/build/../../lib/libc/include -DNDEBUG -I/usr/obj/usr/src/tmp/legacy/usr/include -std=gnu99 /usr/src/tools/build/../../contrib/libc-pwcache/pwcache.c
/usr/src/tools/build/../../contrib/libc-pwcache/pwcache.c:72:10: fatal error: 'sys/cdefs.h' file not found
#include <sys/cdefs.h>
         ^
1 error generated.
mkdep: compile failed
*** Error code 1

Stop.
make[3]: stopped in /usr/src/tools/build
*** Error code 1

Stop.
make[2]: stopped in /usr/src
*** Error code 1

Stop.
make[1]: stopped in /usr/src
*** Error code 1

Stop.
make: stopped in /usr/src
#
Comment 3 frf 2016-10-13 01:42:01 UTC
Sorry. typo in the second post.

After 
# pwd
/usr/src

insert 
# rm /usr/include/*

Removing /usr/include immediately breaks buildworld. 
This has not always been the case for us.
Comment 4 Mark Millard 2016-10-13 03:00:38 UTC
(In reply to frf from comment #3)

As I understand buildworld does initially build some programs to execute on the host under the original operating system version that is running on the host --to be executed during the build itself. The bootstrap clang itself is/can-be an example (even if it is built to target a different TARGET_ARCH [cross build]).

Thus I would not expect the

# rm /usr/include/*

example to work correctly in general. Avoiding /usr/include is specific to certain (most) stages of buildworld, but not to all of it as I understand.

While most of what buildworld does targets the new operating system version (if it changed) and the TARGET_ARCH, such is not the case for everything it does.


Notes:

Your examples do not change the TARGET_ARCH. In some respects I'm referencing structure that is there to allow other alternatives than you are using. The operating system version's of files can still apply to your case.

Thanks for adding comment #3: I was very confused about the first part of comment #2 before the extra information.