Bug 218827 - r438058 and r438901 cause trouble for PACKAGES settings with colons in them
Summary: r438058 and r438901 cause trouble for PACKAGES settings with colons in them
Status: Closed Not A Bug
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Port Management Team
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-04-23 13:37 UTC by Dimitry Andric
Modified: 2017-06-04 16:08 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dimitry Andric freebsd_committer 2017-04-23 13:37:33 UTC
On a number of FreeBSD boxes where I build ports, I have always had the PACKAGES setting point to a shared NFS directory where the packages should end up.  For example, in /etc/make.conf on a amd64-freebsd12 box:

  PACKAGES?=/share/dim/dist/FreeBSD:12:amd64/latest

On an i386-freebsd10 box:

  PACKAGES?=/share/dim/dist/FreeBSD:10:i386/latest

and so on.  I use the same version:arch convention as pkg, so I can easily specify a common url in pkg.conf files, e.g.:

  url: "file:///share/dim/dist/${ABI}/latest"

However, since r438058 and its follow-up r438901, this no longer works, and every invocation of make inside the ports tree gives a number of warnings about duplicate targets:

$ make foo
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 5130: warning: duplicate script for target "/share/dim/dist/FreeBSD" ignored
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 3294: warning: using previous script for "/share/dim/dist/FreeBSD" defined here
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 5130: warning: duplicate script for target "/share/dim/dist/FreeBSD" ignored
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 3294: warning: using previous script for "/share/dim/dist/FreeBSD" defined here
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 5130: warning: duplicate script for target "/share/dim/dist/FreeBSD" ignored
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 3294: warning: using previous script for "/share/dim/dist/FreeBSD" defined here
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 5130: warning: duplicate script for target "/share/dim/dist/FreeBSD" ignored
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 3294: warning: using previous script for "/share/dim/dist/FreeBSD" defined here
make: don't know how to make foo. Stop

This is because line 3292 of bsd.port.mk now specifies:

${PKGFILE}: ${WRKDIR_PKGFILE} ${PKGREPOSITORY}

while before that, PKGFILE has been defined as:

PKGREPOSITORY?=     ${PACKAGES}/${PKGREPOSITORYSUBDIR}
.if exists(${PACKAGES})
_HAVE_PACKAGES= yes
PKGFILE?=       ${PKGREPOSITORY}/${PKGNAME}${PKG_SUFX}

E.g. the PACKAGES setting, complete with colons, ends up in PKGFILE.

Afterwards, actually trying to build any port errors out with messages similar to:

$ make -C devel/gmake
make: "/share/dim/ports/Mk/bsd.port.mk" line 5130: warning: duplicate script for target "/share/dim/dist/FreeBSD" ignored
make: "/share/dim/ports/Mk/bsd.port.mk" line 3294: warning: using previous script for "/share/dim/dist/FreeBSD" defined here
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 5130: warning: duplicate script for target "/share/dim/dist/FreeBSD" ignored
make[1]: "/share/dim/ports/Mk/bsd.port.mk" line 3294: warning: using previous script for "/share/dim/dist/FreeBSD" defined here
===>  License GPLv3 accepted by the user
===>   gmake-4.2.1_1 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by gmake-4.2.1_1 for building
===>  Extracting for gmake-4.2.1_1
=> SHA256 Checksum OK for make-4.2.1.tar.bz2.
make[1]: don't know how to make 12:amd64/latest/All:. Stop

make[1]: stopped in /share/dim/ports/devel/gmake
*** Error code 2

Stop.
make: stopped in /share/dim/ports/devel/gmake

I can confirm that reverting both r438058 and r438901 fixes this issue.
Comment 1 Dimitry Andric freebsd_committer 2017-04-23 13:38:21 UTC
Note: ports tree is at r439221.
Comment 2 Mark Linimon freebsd_committer freebsd_triage 2017-04-25 18:07:27 UTC
438058: "Use native make(1) mechanism to create the package repository directory" bapt@
438901: "Refactor do-package aka subpackages step 2" bapt@

fyi
Comment 3 Mathieu Arnold freebsd_committer 2017-04-26 16:13:30 UTC
I am not sure how this can be fixed, but as a solution, don't use colons in your PACKAGES variables, and create symlinks to the correct directories.
Comment 4 Dimitry Andric freebsd_committer 2017-04-26 16:21:20 UTC
(In reply to Mathieu Arnold from comment #3)
> I am not sure how this can be fixed, but as a solution, don't use colons in
> your PACKAGES variables, and create symlinks to the correct directories.

My solution was to simply revert r438901 and 438058 locally.  I'm not sure if parts of make or Mk/*.mk will try to resolve symlinks?
Comment 5 Mathieu Arnold freebsd_committer 2017-04-27 10:40:40 UTC
Reverting the commits is going to break things badly when things continue to move on.

Your problem is that the directories are created using make(1) targets, and that make(1) targets are in the form of "word:" so if you have "target-with-a:in-the-middle:" it parses it as target-with-a that depends on in-the-middle:.

The targets are only there to create the directories if they do not exist, so symlinks will work as long as the target exists.
Comment 6 Baptiste Daroussin freebsd_committer 2017-04-27 11:16:14 UTC
I think surrounding the target with quotes should be enough, but I can't try it
Comment 7 Dimitry Andric freebsd_committer 2017-04-28 18:36:12 UTC
(In reply to Baptiste Daroussin from comment #6)
> I think surrounding the target with quotes should be enough, but I can't try
> it

Did you mean in bsd.port.mk, or in make.conf?  But indeed, quoting the vale of PACKAGES, or escaping the colons, in /etc/make.conf makes it work again.

E.g.:

PACKAGES?="/share/dim/dist/FreeBSD:12:i386/latest"

or:

PACKAGES?=/share/dim/dist/FreeBSD\:12\:i386/latest

both allow me to build ports, even with r438058 and r438901 applied.

Let's close this bug then, but maybe it would be nice to have some sort of warning in bsd.port.mk about using colons (or other make metacharacters) in ports-related make.conf variables?
Comment 8 Baptiste Daroussin freebsd_committer 2017-04-29 09:35:07 UTC
I was meaning in bsd.port.mk :) quoting ${PKGFILE}: should imho to the trick
Comment 9 Dimitry Andric freebsd_committer 2017-04-29 10:55:21 UTC
(In reply to Baptiste Daroussin from comment #8)
> I was meaning in bsd.port.mk :) quoting ${PKGFILE}: should imho to the trick

No, I tried that, but PKGFILE is used in a great many other macros, so it eventually ends up erroring out anyway...
Comment 10 Harald Schmalzbauer 2017-05-22 07:10:16 UTC
Reverting Mk/bsd.port.mk to r438057 is the only solution which works for me.
Since the pkg convention is to use colons for pathnames, these should also work for the upper bulding stages, in form of setting PACKAGES.
Changing PACKAGES definition is no acceptable solution, it even contradicts pkg's native convention!
Comment 11 Harald Schmalzbauer 2017-06-04 16:08:37 UTC
It seems r441712 was reviewed and accepted as fix.
Unfortunately it wasn't mentioned here, maybe I'm missing that reviews.freebsd.org is where fixes get discussed...

However, USE_PACKAGE_DEPENDS is still broken as a result of r438058, 438901 and r441712, if PACKAGES are set and contain colons, due to the same cause initially reported here by Dimitry Andric.

-harry