Bug 198045 - x11-toolkits/pango : Build tools hiding dependency issues
Summary: x11-toolkits/pango : Build tools hiding dependency issues
Status: Closed Overcome By Events
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-gnome (Nobody)
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2015-02-25 22:18 UTC by Melvyn Sopacua
Modified: 2016-12-20 14:06 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Melvyn Sopacua 2015-02-25 22:18:22 UTC
The ports system can be used using `make -C ${PORTSDIR}/category/leafport install`. This is the simplest way to use the ports system and it will compile all dependencies that aren't installed yet.

With build tools ranging from portmaster to poudriere building their own dependency tree and separating the build in it's own make process, a couple of bugs are not uncovered by maintainers, because the top-level makefile is no longer the leaf port being installed:

1. Exists checks:

.if !exists(/path/to/file/to/be/installed/by/dep)
BROKEN= Port needs foo with option bar
.endif

Case in point: x11-toolkits/pango.
Fails when: dependency is not installed
Fails because: exists check is done in top-level Makefile result not re-evaluated after dependency is installed.
Fix: use if [ ! -f ] in sh, rather then make's version.

2. Ports with different version of interpreter in upwards dependencies

DEFAULT_VERSIONS=    python=3.3
Upwards dependency forces python=2.7.

Case in point: graphics/qt4-opengl => graphics/libGL => py27-libxml:build => setuptools
Fail symptom: pkg complains about /usr/local/bin/easy_install
Fails because: not entirely sure, but I suspect that the DEFAULT_PYTHON_VERSION is being mangled downstream, causing the unversioned easy_install to be installed again.

3. Versions obtained from parsing output of != statements

BUILD_DEPENDS+= foo:${PORTSDIR}/misc/foo
FOO_VERSION!= ${LOCALBASE}/bin/foo -V
PLIST_SUB+= FOO_VERSION=${FOO_VERSON}
pkg-plist: lib/bar-foo-%%FOO_VERSION%%.so.1

Case in point: used to be in bsd.xorg.mk for Xorg Server. Don't know if this is still in a different incarnation used elsewhere
Fail symptom: package install fails with files not being present
Fail reason: FOO_VERSION is empty at evaluation time and not reevaluated when dependency is installed. Upstream build/install system does however pick up the right version at build time of leaf port resulting in packing list looking for lib/bar-foo-.so.1 and lib/bar-foo-1.0.so.1 being in STAGEDIR.

All these are hidden by recommended test tools used by maintainers because of above mentioned reasons, yet it should be the installation mechanism that should always work.

So this bugreport is two-fold:
1. Is `make -C ${PORTSDIR}/category/leafport install` still considered to be fully suported and always working?

2a. If 1) is yes, can the above be fixed and can a test tool be used that does not separate build stages?
2b. If 1) is no,  is The Project pushing binary upgrades or ports-mgmt tools to end users and as such should docs be updated: http://www.freebsd.org/doc/en/books/handbook/ports-using.html
Comment 1 Bryan Drewery freebsd_committer freebsd_triage 2015-02-26 03:41:43 UTC
Yes 'make install' is still supported, though it is very dangerous and bug-prone since it does find all of this stuff on your system that may not be expected. Yes the issues you mention are problems. They need to be fixed case-by-case.

Most of these problems are handled by building in a sandbox, which is what Poudriere does.

As for dependency tree, all of the tools use what ports give them: BUILD_DEPENDS LIB_DEPENDS RUN_DEPENDS, etc. They don't make up their own dependencies.
Comment 2 Melvyn Sopacua 2015-03-16 12:36:44 UTC
Hi Brian,

I didn't say they make new dependencies. The root cause of at least 1 and 3 and I suspect 2 as well, is that the leaf port's make doesn't re-evaluate a set value when it comes back from visiting it's children.
By closing the leaf port's make and executing the dependency builds in their own make process, these bugs are not exposed by the existing tools.

Here's the fix for pango:

diff --git a/x11-toolkits/pango/Makefile b/x11-toolkits/pango/Makefile
index 5a54d96..ce1b839e 100644
--- a/x11-toolkits/pango/Makefile
+++ b/x11-toolkits/pango/Makefile
@@ -54,10 +54,10 @@ PLIST_SUB+= X11="@comment "
 .endif

 pre-configure:
-.if !exists(${LOCALBASE}/libdata/pkgconfig/cairo-gobject.pc)
-       @${ECHO_CMD} "${PKGNAME}: Needs cairo with GLIB (GObject) support enabled."
-       @${FALSE}
-.endif
+       @if [ ! -f${LOCALBASE}/libdata/pkgconfig/cairo-gobject.pc ]; then \
+               ${ECHO_CMD} "${PKGNAME}: Needs cairo with GLIB (GObject) support enabled."; \
+               ${FALSE}; \
+       fi

 .include <bsd.port.mk>
Comment 3 Martin Wilke freebsd_committer freebsd_triage 2016-01-16 06:31:25 UTC
Over to gnome.
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2016-01-16 15:49:57 UTC
This is definitely portmgr@ territory, with gnome/xorg used as an example (So add cc)
Comment 5 Antoine Brodin freebsd_committer freebsd_triage 2016-01-16 19:24:52 UTC
This is an individual port issue for x11-toolkits/pango.
If you find problems in other ports, individual bugs should be opened.
Comment 6 John Hein 2016-01-18 15:34:58 UTC
While there was an example for x11-toolkits/pango, the original bug report was decidedly not specific to that port.  It was more general.  I'd wager that repurposing this bug to be just a pango one is not what the OP was intending.

I thing I am seeing that the response to the OP from an infrastructure point of view is that the weakness is known, probably won't be fixed more generally in ports/Mk (rather should be addressed on a port case-by-case basis) and has workarounds available via poudriere, portmaster, and the like (although the tools that use a sandbox can mask problems when the full installed ports tree is not used to build the packages).

The original "bug", however, doesn't fit well in bugzilla as opposed to a discussion forum such as ports@ - or if there is a proposed infrastructure fix perhaps at reviews.freebsd.org.

I think this bug should have been closed as "answered" or "no general fix immediately available" and new ones opened for the individual cases rather than transforming it to a bug for one of the individual cases.
Comment 7 John Hein 2016-01-18 20:40:24 UTC
If someone _does_ open a new bug for pango or fix it based on this one here, use 'if ! pkg-config --exists' rather than 'if [ ! -f xxxx ]'.  See gtk30/Makefile.
Comment 8 Melvyn Sopacua 2016-09-09 12:50:47 UTC
I can agree on a case-by-case base and the discussion about the flaws in poudriere and similar be moved to ports@.
However, using the .if exists() construct is probably something that is not understood well by maintainers. The construct can **only** be used when: The file tested for is guaranteed to be generated before this Makefile is loaded.
When the file is generated by a dependency, this condition fails for rather obvious reasons.
The only sure fire use for this construct is to test for a **system** file: if /usr/sbin/mailwrapper exists, we can assume that filling mailer.conf is a useful exercise.

The interpreter problem is definitely portmgr territory. I doubt that the WARNING issued by `Mk/Uses/python.mk` about mismatched versions ever triggers in poudriere builds. Having forgotten about this bugreport, I've even made a case for this in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212273 .

The third case, I've not encountered anymore and probably has been eliminated over time.
Comment 9 Melvyn Sopacua 2016-12-20 14:06:06 UTC
Discussions will be moved elsewhere. If I find the time, will make a doc patch for .if exists in porters handbook - cases that affected me have been fixed in the tree.
Others have not been encountered for a while. Python has a separate report.