Bug 167368 - [bsd.python.mk] [patch] Python version propagation breaks USE_PYTHON= usage for dependency builds
Summary: [bsd.python.mk] [patch] Python version propagation breaks USE_PYTHON= usage f...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Marcus von Appen
URL:
Keywords:
Depends on: 192357
Blocks:
  Show dependency treegraph
 
Reported: 2012-04-27 20:00 UTC by Marcus von Appen
Modified: 2014-08-09 15:49 UTC (History)
3 users (show)

See Also:
antoine: exp-run?


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcus von Appen freebsd_committer freebsd_triage 2012-04-27 20:00:24 UTC
        Given that port A defines PYTHON_VERSION=3.2, dependencies,
        which might define PYTHON_VERSION <> 3.2, will fail to build,
        since the version propagation to .MAKEFLAGS and DEPENDS_ARGS will
        cause a IGNORE to be triggered.

        bsd.python.mk Line 342:
        DEPENDS_ARGS+=     PYTHON_VERSION=${PYTHON_VERSION}

        bsd.python.mk Line 357f.
        # Propagate the	chosen python version to submakes.
        .MAKEFLAGS: PYTHON_VERSION=python${_PYTHON_VERSION}

        Both will cause PYTHON_VERSION to be appended to the
	command-line arguments/.MAKEFLAGS for the dependencies, causing
	the error described above.

Fix: 

The easiest would be to remove the both offending
	lines. However, those seem to be used to specify a Python
	version to (maybe) keep the installation count of different
	Python versions low. If there is no specific purpose for the
	both offending lines except from that, they should be
	removed.
How-To-Repeat:         Create two directories /usr/ports/devel/fake1 and
        /usr/ports/devel/fake2. Add the Makefiles to each from the diffs
        below.

diff -Nur /nonexistent/Makefile devel/fake1/Makefile
--- /nonexistent/Makefile	1970-01-01 01:00:00.000000000 +0100
+++ devel/fake1/Makefile	2012-04-27 20:19:53.000000000 +0200
@@ -0,0 +1,9 @@
+PORTNAME=	fake1
+PORTVERSION=	1
+CATEGORIES=	devel
+
+LIB_DEPENDS=	fake2.0:${PORTSDIR}/devel/fake2
+
+USE_PYTHON=	3.2
+
+.include <bsd.port.mk>




diff -Nur /nonexistent/Makefile devel/fake2/Makefile
--- /nonexistent/Makefile	1970-01-01 01:00:00.000000000 +0100
+++ devel/fake2/Makefile	2012-04-27 20:21:28.000000000 +0200
@@ -0,0 +1,7 @@
+PORTNAME=	fake2
+PORTVERSION=	1
+CATEGORIES=	devel
+
+USE_PYTHON=	2.5-3.1
+
+.include <bsd.port.mk>



        Afterwards run "make depends" in /usr/ports/devel/fake1:

/usr/ports/devel/fake1 # make depends
===>   fake1-1 depends on shared library: fake2.0 - not found
===>    Verifying install for fake2.0 in /usr/ports/devel/fake2
===>  fake2-1 needs Python 3.1 at most. But you specified 3.2.
*** Error code 1

Stop in /usr/ports/devel/fake2.
*** Error code 1

Stop in /usr/ports/devel/fake1.

        This approach currently break ports such as the enhancement PR
	for graphics/blender (ports/167061), in which blender pulls in
	devel/boost-python-libs as indirect dependency of
	graphics/openimageio.
Comment 1 Shane 2012-04-28 10:34:20 UTC
graphics/openimageio defines USE_PYTHON=2.6+
graphics/blender defines USE_PYTHON=3.2
devel/boost-python-libs defines USE_PYTHON=2.5-3.1

If I install openimageio by itself it installs the python lib into the 
v2.7 python dir.
If I install blender bringing in openimageio as a dependency it installs 
into v3.2 python dir, so that could be something to sort out.

My guess is the openimageio pylib would need to use the same python 
version as boost-python-libs. Is there a reason boost-python-libs stops 
at 3.1? that was just the newest version at the time? could it be 2.6+

Blender uses python pretty heavily and I am certain it needs to stay 
against a specific version of python, I recall changes happening to 
upgrade support from python 3.1 to 3.2


Either way at this point blender is the only port using openimageio and 
the pylib being installed into python 2.7 or 3.2 libdir or disabling 
python bindings doesn't effect blender's use of libOpenImageIO. I'm not 
sure that blender would use the python bindings in the future.

I am thinking this could be solved by breaking openimageio into two 
ports - one for a standard lib and another for python bindings.
Comment 2 Shane 2012-04-29 06:41:35 UTC
While this pr originated from graphics/blender (ports/167061) I will fix
that as it's own issue as I think the issue of python dependency
versions should be kept separate.

As for python versions I'm not sure if that can be improved or just
needs to be a check for port maintainers to verify that leaf ports use
compatible python versions to their dependants. Do we need to consider
ports being installed against multiple python versions?

For reference I missed this as I built openimageio with python bindings
using USE_PYTHON=2.6+ which led to the openimageio pylib and
boost-python-libs being installed in the py27 libdir. When the new
blender port was then built (using py32) it saw openimageio was
installed but done no check to verify that it was installed into the
same/compatible version of python.

This also propagated within tinderbox as I built openimageio first and
the existing packages were installed for dependants without testing
python versions used. For others testing with tinderbox, adding a step
of removing python using dependant packages before building a new port
could be useful. Maybe a new tinderbox command/option to search for
python using dependant packages for a port to be removed.
Comment 3 Marcus von Appen freebsd_committer freebsd_triage 2012-06-03 22:46:54 UTC
Responsible Changed
From-To: freebsd-ports-bugs->freebsd-python

That's a Python PR
Comment 4 Antoine Brodin freebsd_committer freebsd_triage 2014-07-25 13:07:56 UTC
Take for exp-run,  as it was requested on  https://phabric.freebsd.org/D387
Comment 5 Antoine Brodin freebsd_committer freebsd_triage 2014-07-26 08:24:56 UTC
Exp-run results:
http://package18.nyi.freebsd.org/build.html?mastername=91amd64-default-PR167368&build=2014-07-25_13h55m26s

There are 3 new failures,  but for 2 of them I believe it will fix a previously broken behaviour (egginfo with a strange path)

+ {"origin"=>"devel/py-gobject3", "pkgname"=>"py27-gobject3-3.8.1_1", "phase"=>"package", "errortype"=>"PLIST"}
+ {"origin"=>"devel/py-ice", "pkgname"=>"py27-Ice-3.5.1", "phase"=>"build", "errortype"=>"missing_header"}
+ {"origin"=>"devel/py3-gobject3", "pkgname"=>"py33-gobject3-3.8.1_1", "phase"=>"package", "errortype"=>"PLIST"}

Failures logs:
http://package18.nyi.freebsd.org/data/91amd64-default-PR167368/2014-07-25_13h55m26s/logs/errors/py27-gobject3-3.8.1_1.log
http://package18.nyi.freebsd.org/data/91amd64-default-PR167368/2014-07-25_13h55m26s/logs/errors/py27-Ice-3.5.1.log
http://package18.nyi.freebsd.org/data/91amd64-default-PR167368/2014-07-25_13h55m26s/logs/errors/py33-gobject3-3.8.1_1.log
Comment 6 Marcus von Appen freebsd_committer freebsd_triage 2014-08-03 13:43:26 UTC
devel/py-ice right now does the right thing, since PYTHON_VERSION is set in the environment. A simple MAKE_ENV+= PYTHON_VERSION=${PYTHON_VERSION} fixes the issue (as described in ${WRKSRC}/py/config/Make.rules).
Comment 7 Marcus von Appen freebsd_committer freebsd_triage 2014-08-03 13:55:01 UTC
py-gobject3 uses a wrong substitute that is easily fixed with changing the egg-info entry in pkg-plist (use %%PYTHON_VER%% instead of %%PYTHON_VERSION%%) and adding a PLIST_SUB+= PYTHON_VER=${PYTHON_VER}.
Comment 8 Marcus von Appen freebsd_committer freebsd_triage 2014-08-09 15:49:04 UTC
Committed in rev. 364450