Bug 223564

Summary: Using a test script with filename "null" prevent "make install"
Product: Base System Reporter: Olivier Cochard <olivier>
Component: testsAssignee: Bryan Drewery <bdrewery>
Status: Open ---    
Severity: Affects Only Me CC: asomers, emaste, sjg
Priority: ---    
Version: CURRENT   
Hardware: Any   
OS: Any   

Description Olivier Cochard freebsd_committer 2017-11-09 13:53:58 UTC
I'm writing tests and one file (script shell) has the name "null" and this prevent makefile to run:

How to reproduce the bug:

===================================
1) First, create a dummy test service
===================================

mkdir -p /usr/src/tests/bug/testing
cat > /usr/src/tests/bug/Makefile <<EOF
TESTSDIR=\${TESTSBASE}/bug
TESTS_SUBDIRS+=testing
.include <bsd.test.mk>
EOF

cat > /usr/src/tests/bug/testing/Makefile <<EOF
PACKAGE=tests
TESTSDIR=\${TESTSBASE}/bug/testing
ATF_TESTS_SH+= null
.include <bsd.test.mk>
EOF

cat > /usr/src/tests/bug/testing/null.sh <<EOF
# empty
atf_test_case "dummy" "cleanup"
dummy_head() {
atf_set descr 'dummy test'
}
dummy_body() {
atf_check -s exit:0 true 
}
EOF

==============================================
2) build and (try) install it
============================================

mkdir -p /usr/tests/bug/testing
cd /usr/src/tests/bug
make & make install
root@lame4:/usr/src/tests/bug # make install
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/bug/Kyuafile
===> testing (install)
make[1]: "/usr/obj/usr/src/amd64.amd64/tests/bug/testing/null" line 3: Need an operator
make[1]: "/usr/obj/usr/src/amd64.amd64/tests/bug/testing/null" line 4: Need an operator
make[1]: "/usr/obj/usr/src/amd64.amd64/tests/bug/testing/null" line 5: Need an operator
make[1]: "/usr/obj/usr/src/amd64.amd64/tests/bug/testing/null" line 6: Need an operator
make[1]: "/usr/obj/usr/src/amd64.amd64/tests/bug/testing/null" line 7: Need an operator
make[1]: "/usr/obj/usr/src/amd64.amd64/tests/bug/testing/null" line 9: Need an operator
make[1]: Fatal errors encountered -- cannot continue
make[1]: stopped in /usr/src/tests/bug/testing
*** Error code 1

Stop.
make: stopped in /usr/src/tests/bug

=> The makefile try to interpret this file


===========================================
3) Fixing, by renaming filename "null" to "empty"
=============================================

rm -rf /usr/obj/usr/src/amd64.amd64/tests/bug/
mv /usr/src/tests/bug/testing/null.sh /usr/src/tests/bug/testing/empty.sh
sed -i "" -e "s/null/empty/" /usr/src/tests/bug/testing/Makefile

make & make install
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/bug/Kyuafile
===> testing (install)
install  -o root  -g wheel -m 555  empty  /usr/tests/bug/testing/empty
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/bug/testing/Kyuafile

=> No more bug
Comment 1 Alan Somers freebsd_committer 2017-12-01 17:55:33 UTC
I can confirm the bug.  I ran "make && sudo make -dA install" in both the working and non-working case and compared the output.  What I found was very surprising.  In the working (file name is not named "null") case, I see this:

Applying[.MAKE.DEPENDFILE] :T to "/dev/null"
  Result[.MAKE.DEPENDFILE] of :T is "null"
  Searching for null ...
     failed.
  Searching for null ...
     /usr/home/alans/freebsd/head/share/mk ...
     /usr/share/mk ...
     failed.


In the non-working (file name is "null") case, I see this:

Applying[.MAKE.DEPENDFILE] :T to "/dev/null"
  Result[.MAKE.DEPENDFILE] of :T is "null"
  Global:.MAKE.MAKEFILES = /usr/home/alans/freebsd/head/share/mk/sys.mk /usr/home/alans/freebsd/head/share/mk/local.sys.env.mk /usr/home/alans/freebsd/head/share/mk/src.sys.env.mk /usr/home/alans/freebsd/head/s
  Parse_SetInput: file /usr/obj/usr/home/alans/freebsd/head/amd64.amd64/tests/bug/testing/null, line 0, fd -1, nextbuf 0x414cc0, arg 0x800d647c0
  Global:.PARSEDIR = /usr/obj/usr/home/alans/freebsd/head/amd64.amd64/tests/bug/testing
  Global:.PARSEFILE = null
  ParseSetParseFile: ${.PARSEDIR} = `/usr/obj/usr/home/alans/freebsd/head/amd64.amd64/tests/bug/testing' ${.PARSEFILE} = `null'
  ParseReadLine (3): 'atf_test_case "dummy" "cleanup"'
  ParseDoDependency(atf_test_case "dummy" "cleanup")

So it looks like in every case make thinks that /dev/null is a depend file (the thing that is normally named Makefile.depend) for the target, and tries to read commands from it.  Also, for some reason it strips off the dirname.  Most of the time, the file can't be found.  But if a file named "null" exists, then make will try to read commands from it.  The target doesn't even need to be named "null".  The bug can be reproduced as long as a file named "null" is present that contains invalid make syntax.  For example:

$ cd /usr/src/tests/etc/rc.d
$ echo "a b c" > null
$ make && sudo make install
make: "/usr/src/tests/etc/rc.d/null" line 1: Need an operator
make: Fatal errors encountered -- cannot continue
make: stopped in /usr/src/tests/etc/rc.d
Comment 2 Bryan Drewery freebsd_committer 2017-12-01 18:06:17 UTC
It's an optimization to avoid reading the depend files. I'll fix this later today.
Comment 3 Bryan Drewery freebsd_committer 2017-12-02 00:07:25 UTC
So for 'make install' I am setting .MAKE.DEPENDFILE=/dev/null to avoid reading
and parsing all of the large .depend.*.o files.

There's some problems with this.

bmake's main.c:
 /* In particular suppress .depend for '-r -V .OBJDIR -f /dev/null' */
 if (!noBuiltins || !printVars) {
     makeDependfile = Var_Subst(NULL, "${.MAKE.DEPENDFILE:T}",
         VAR_CMD, VARF_WANTRES);
     doing_depend = TRUE;
     (void)ReadMakefile(makeDependfile, NULL);
     doing_depend = FALSE;
 }


It is stripping away all path information for .MAKE.DEPENDFILE even
though ReadMakefile() has explicit support for absolute pathed
files...

/* if we've chdir'd, rebuild the path name */
if (strcmp(curdir, objdir) && *fname != '/') {
...
} else {
        fd = open(fname, O_RDONLY);
        if (fd != -1)
                goto found;
}


I'm not sure why :T is used on .MAKE.DEPENDFILE there.

More importantly though I would like some value to set .MAKE.DEPENDFILE
to avoid attempting any open(2) at all.  Currently it is actually trying
to open .CURDIR/null and .OBJDIR/null when no attempts are wanted.

~/git/freebsd/bin/sh # truss env MK_TESTS=no make -n install 2>&1|grep null
openat(AT_FDCWD,"/root/git/freebsd/bin/sh/null",O_RDONLY,00) ERR#2 'No such file or directory'
openat(AT_FDCWD,"/scratch/obj/root/git/freebsd/amd64.amd64/bin/sh/null",O_RDONLY,00) = 3 (0x3)
Comment 4 Simon J. Gerraty freebsd_committer 2017-12-04 01:53:18 UTC
(In reply to Bryan Drewery from comment #3)

BTW the .MAKE.DEPENDFILE=/dev/null should be guarded by .if ${MK_DIRDEPS_BUILD} == "no"
otherwise it was causing us problems.

As to :T I vaguely recall that was done for consistency, but it was a long time ago ;-)
Comment 5 Bryan Drewery freebsd_committer 2018-01-24 18:04:28 UTC
(In reply to Simon J. Gerraty from comment #4)
> (In reply to Bryan Drewery from comment #3)
> 
> BTW the .MAKE.DEPENDFILE=/dev/null should be guarded by .if
> ${MK_DIRDEPS_BUILD} == "no"
> otherwise it was causing us problems.

I've built DIRDEPS_BUILD locally quite a few times since adding that and haven't observed any problems. What have you seen?

> 
> As to :T I vaguely recall that was done for consistency, but it was a long
> time ago ;-)

Can we change it, or give me a facility to make bmake NOT read .MAKE.DEPENDFILE if a variable is set? Like .MAKE.NOREADDEPEND