Summary: | math/py-numpy: Builds and stages egg-info directory but silently fails to install it | ||
---|---|---|---|
Product: | Ports & Packages | Reporter: | John W. O'Brien <john> |
Component: | Individual Port(s) | Assignee: | Kubilay Kocak <koobs> |
Status: | Closed FIXED | ||
Severity: | Affects Some People | CC: | koobs, python |
Priority: | Normal | Keywords: | easy |
Version: | Latest | ||
Hardware: | amd64 | ||
OS: | Any | ||
See Also: | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192096 | ||
Bug Depends on: | |||
Bug Blocks: | 191857 | ||
Attachments: |
Description
John W. O'Brien
2014-07-14 15:52:15 UTC
An update to devel/py-setuptools to 5.4.1 is about to land shortly (pending a small QA issue). Can you take a look at the upstream changes [1] and let me know if there's anything in there that might be relevent. https://bitbucket.org/pypa/setuptools/src/b7581d108b877e360bf8311089e8dff27a53c6f0/CHANGES.txt?at=default If possible, it would also be handy to reproduce the issue on lang/python34, to isolate a root cause to a specific port Also, a directory listing of the contents of /usr/local/lib/python3.3/site-packages could prove handy (please attach it, rather than in a comment) Created attachment 144664 [details]
Recursive listing of python3.3 site-packages directory
koobs: I see nothing relevant in the upstream changelog for setuptools. I'm working on reproducing in 3.4 right now. Created attachment 144667 [details]
Test environment with DEFAULT_VERSIONS=python=3.4
To upgrade the test environment, I pkg remove'd the three py33-* packages plus python itself, updated /etc/make.conf, then built and installed lang/python3.4, devel/py-setuptools34, devel/py-nose, and math/py-numpy.
Created attachment 144668 [details]
Demonstation of problem under python3.4
This terminal capture shows the same behavior under python3.4.
Created attachment 144669 [details]
Recursive listing of python3.4 site-packages directory
See also bug 191857, comment 1, regarding a previous notice to freebsd-pkg-fallout@, which suggests that this bug affects 2.7 too, at least on 11.0-CURRENT. Oh, that's interesting and unexpected... I just reproduced this with 2.7 in one test environment, but could not in another. I wonder what's different? What was different is that on one machine, /usr/local/lib/python2.7/site-packages/numpy-1.8.1-py2.7.egg-info existed, and on the other it didn't. This is a problem because the pkg_resources.find_distributions function ends up using the pkg_resources.find_on_path finder for the site-packages directory, which only pays attention to directory entries ending in one of ['.egg-info', '.egg', '.egg-link']. When I copied the .egg-info directory from one machine to the other, pkg_resources.find_distributions found numpy. So, this is looking more like a numpy or ports infrastructure bug, not so much like a setuptools bug. The plot thickens. I've been building and rebuilding math/py-numpy, and been unable to generate a build where the .egg-info directory appears in .PLIST.mktmp, with any version of python. Of course that means that despite having .egg-info in the staging directory, it doesn't make it into the package let alone into the installed system. However, the binary package that the FreeBSD repos are shipping contains .egg-info, as does a binary package my own poudriere instance built for me of the same numpy version on or about 2014-06-03. When I rebuild with that poudriere instance from the latest rev of the ports tree, the package is missing .egg-info. math/py-numpy doesn't use PYDISTUTILS_AUTOPLIST (could it?), so while I would expect to find entries in pkg-plist for %%PYDISTUTILS_EGGINFO%% there are none. It doesn't look like math/py-numpy ever had those lines, though, so maybe that isn't the problem. Created attachment 144707 [details]
Patch math/py-numpy to install egg-info directory
From Mk/bsd.python.mk --
add-plist-egginfo:
.if !defined(PYDISTUTILS_NOEGGINFO) && \
!defined(PYDISTUTILS_AUTOPLIST) && \
defined(INSTALLS_EGGINFO) && \
defined(PYTHON_REL)
# [add .egg-info entries to plist]
math/py-numpy does not use AUTOPLIST, nor does it it inhibit with NOEGGINFO. Therefore it needs INSTALLS_EGGINFO.
Patch tested successfully with poudriere on 10.0-RELEASE against python{27,33,34}. Created attachment 144708 [details]
Patch math/py-numpy to install egg-info directory and bump PORTREVISION
I'll take it, thanks for the fantastic diagnostic & isolation effort A commit references this bug: Author: koobs Date: Wed Jul 16 13:16:18 UTC 2014 New revision: 362060 URL: http://svnweb.freebsd.org/changeset/ports/362060 Log: math/py-numpy: Use INSTALLS_EGGINFO, fixes pkg_resources detection Python packages installs .egg-info metadata directories when --single-version-externally managed is specified, as the Python ports framework does currently for all ports when USE_PYDISTUTILS is defined. In this case the py-numpy port does *not* currently define PYDISTUTILS_AUTOPLIST nor INSTALLS_EGGINFO, which precludes the contents of the add-plist-egginfo: target from being executed. This results in an .egg-info metadata directory being built, and staged, but not installed. Correctly specify INSTALLS_EGGINFO to compensate. Additionally add an "Open" Goal Wiki entry [1] to look at automatically detecting the presence of .egg-info metadata directories, since all ports leveraging USE_PYDISTUTILS also use the --record mechanism for generating installed file lists, and potentially deprecating INSTALLS_EGGINFO and related variables. [1] https://wiki.freebsd.org/Python#Goals PR: 191865 Submitted by: John W. O'Brien <john at saltant dot com> Changes: head/math/py-numpy/Makefile |