Bug 201072

Summary: /etc/mtree/BSD.include.dist and /etc/mtree/BSD.usr.dist create various atf directories that are later deleted by 'make delete-old'
Product: Base System Reporter: Kimmo Paasiala <kpaasial>
Component: miscAssignee: freebsd-testing (Nobody) <testing>
Status: Open ---    
Severity: Affects Many People CC: 000.fbsd, delphij, jmmv, koobs, ngie, philippe.michel7, re
Priority: --- Keywords: easy, needs-qa, patch, regression
Version: 10.1-STABLE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Patch for /etc/mtree/* to address the issue.
koobs: maintainer-approval-
Proposed patch none

Description Kimmo Paasiala 2015-06-23 11:29:51 UTC
System is:

FreeBSD firewall.rdnzl.info 10.2-PRERELEASE FreeBSD 10.2-PRERELEASE #10 r284708: Tue Jun 23 04:38:48 EEST 2015     root@firewall.rdnzl.info:/usr/obj/usr/src/sys/GENERIC  amd64

No /etc/src.conf and /etc/make.conf contains nothing that affects buildworld or installworld.

Every time I update the system with 'make installworld' the following directories are created and then later deleted when I run 'make delete-old':

>>> Removing old files (only deletes safe to delete libs)
>>> Old files removed
>>> Removing old directories
/usr/share/doc/atf
/usr/share/atf
/usr/include/atf-c++
/usr/include/atf-c
>>> Old directories removed
To remove old libraries run '/usr/bin/make delete-old-libs'.
>>> Removing old libraries
Please be sure no application still uses those libraries, else you
can not start such an application. Consult UPDATING for more
information regarding how to cope with the removal/revision bump
of a specific library.
>>> Old libraries removed

Shouldn't these directories taken away from the mtree(8) database since there is not even a conditional WITH/WITHOUT_ATF anymore in src.conf(5) after r261236?
Comment 1 Kimmo Paasiala 2015-06-27 10:07:08 UTC
Created attachment 158094 [details]
Patch for /etc/mtree/* to address the issue.
Comment 2 Kimmo Paasiala 2015-06-30 17:05:58 UTC
I'm not sure my patch is valid. I did some browsing of the SVN history and it turned out that the atf directories were first removed in: 

https://svnweb.freebsd.org/base?view=revision&revision=260024

The directories were later added back in:

https://svnweb.freebsd.org/base?view=revision&revision=277457

My simple patch would apparently re-break the tests stuff.

How shall we proceed with this? The issue is very minor but it would nice to have it fixed before 10.2.

-Kimmo
Comment 3 Enji Cooper freebsd_committer freebsd_triage 2015-07-01 10:08:06 UTC
Your patch would break MK_TESTS=yes. I would reject the patch in its current state.
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2015-07-01 11:12:46 UTC
@Garrett, any suggestions to help this progress?

cc re@ for input pre 10.2-RELEASE
Comment 5 Xin LI freebsd_committer freebsd_triage 2015-07-01 19:10:28 UTC
Created attachment 158235 [details]
Proposed patch

If the directories are always created unconditionally, there is no reason to remove them conditionally.

Better solution would be to split these directories in a different mtree file.
Comment 6 Julio Merino,+1 347 694 0576,New York City freebsd_committer freebsd_triage 2015-07-04 02:43:27 UTC
This is incorrect: entries should never be removed from the list of obsolete files/directories -- unless those files/directories are readded to the tree.  You do not know which revision the user is updating _from_, so the list of obsolete files need to be as exhaustive as possible and cover all previous cases.

If a directory is always being removed by "delete-old" after a fresh "make installworld", then it is most likely that the Makefiles are creating the directories by mistake.  We should track where that's happening instead and fix those instead: the directories you show should not be there.
Comment 7 Glen Barber freebsd_committer freebsd_triage 2015-07-04 02:48:42 UTC
I tend to agree with, and will defer to, jmmv@ on this.

That said, I do not consider this a blocker for 10.2-RELEASE, but it would be a nice thing to have fixed "eventually."
Comment 8 Enji Cooper freebsd_committer freebsd_triage 2015-07-04 08:03:37 UTC
I really wish there was a way to denote "severity" in this bug. It's a really trivial issue.

Net effect: there are a few extra empty test (atf, pjdfstest, etc) directories created via BSD.include.dist and BSD.usr.dist. However, this "issue" isn't isolated to just MK_TESTS=no; try running MK_CLANG=no and you'll see a similar issue with clang.

History, as to why I needed to re-"break" this, is that jmmv restructuring BSD.tests.dist to include everything test related in BSD.tests.dist in r258233 broke specifying TESTSBASE somewhere other than /usr/tests (see r275907). Isilon and potentially others depend on test programs living somewhere other than /usr/tests, due to custom/legacy partitioning schemes that can't be changed, etc.

What *could* be done is that I could create BSD.tests.include.dist and BSD.tests.usr.dist, which includes just test-related entries and put them under MK_TESTS != "no", similar to how BSD.tests.dist is currently handled. I need to know whether or not that kind of a naming scheme is acceptable.
Comment 9 Miroslav Lachman 2016-01-16 13:41:18 UTC
Is this issue fixable or will it be there forever?
It is not a major issue but it is really annoying. I did installworld (security update) three times in one week and those directories were created and deleted again and again.

FreeBSD 10.2-RELEASE-p10 amd64 GENERIC

https://lists.freebsd.org/pipermail/freebsd-stable/2016-January/083973.html