Bug 233718 - math/py-matplotlib: Flavor @py36 does not install because of wrong PYTHON*_SITELIBDIR
Summary: math/py-matplotlib: Flavor @py36 does not install because of wrong PYTHON*_SI...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-ports-bugs mailing list
URL:
Keywords: needs-qa
Depends on:
Blocks: 233726
  Show dependency treegraph
 
Reported: 2018-12-02 11:42 UTC by Rainer Hurling
Modified: 2018-12-03 14:33 UTC (History)
4 users (show)

See Also:
mainland: maintainer-feedback+
koobs: merge-quarterly?


Attachments
patch SITELIBDIR for @py36 (363 bytes, patch)
2018-12-02 11:42 UTC, Rainer Hurling
no flags Details | Diff
install py-matplotlib@py36 after building dependencies (734.29 KB, text/plain)
2018-12-03 07:03 UTC, Rainer Hurling
no flags Details
install py-matplotlib@py36 with existing dependencies (249.16 KB, text/plain)
2018-12-03 07:04 UTC, Rainer Hurling
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rainer Hurling 2018-12-02 11:42:12 UTC
Created attachment 199746 [details]
patch SITELIBDIR for @py36

In the process of preparing my new port graphics/qgis with QGIS 3, I found math/py-matplotlib as a showstopper, because it is not able to install for flavor @py36.

I think, this is because math/py-matplotlib@py36 tries to rename (and install) the Python files wrongly in PYTHON_SITELIBDIR instead of PYTHONPREFIX_SITELIBDIR.

The attached patch seems to solve the problem, now both @py27 and @py36 are installable at the same time.


Tested with Poudriere (F11.2i/a, F12.0-RC1i/a, and HEADi/a).

'portlint -AC' seems fine.
Comment 1 mainland 2018-12-02 15:06:08 UTC
Nice catch, thanks!
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2018-12-03 01:58:23 UTC
Thank you for the report Rainer. 

Could you attach a full build/error log of the failure and success cases please, ideally with the post-install: @$FIND command un-quietened (remove the prefixed @)
Comment 3 Rainer Hurling 2018-12-03 07:03:49 UTC
Created attachment 199778 [details]
install py-matplotlib@py36 after building dependencies
Comment 4 Rainer Hurling 2018-12-03 07:04:36 UTC
Created attachment 199779 [details]
install py-matplotlib@py36 with existing dependencies
Comment 5 Rainer Hurling 2018-12-03 07:05:31 UTC
(In reply to Kubilay Kocak from comment #2)

Hi Kubilay,

Thanks for taking this up.

I think your suspicion about my PR is right. Now that I try to reproduce the reported error, it is not as it seems before. Instead, it turns out, that it is not an installation problem, but most likely a problem with portmaster and its bookkeeping.

Portmaster does not create the usual directory entry 'py36-matplotlib-2.1.2_4' under /var/db/pkg, when 'portmaster math/py-matplotlib@py36' also has to build needed dependencies, like e.g. py36-cython-0.29, py36-numpy-1.15.1,1, py36-cycler-0.10.0, py36-dateutil-2.7.3, py36-pyparsing-2.3.0, and py36-tornado-5.1.1.

If the dependencies already exist, portmaster creates an entry on /var/db/pkg after successful installation of py36-matplotlib.


As requested, I created log files for the two cases, with the following command lines:

#env DISABLE_MAKE_JOBS=yes portmaster math/py-matplotlib@py36 > & build-error_deps-also-to-build.log
#env DISABLE_MAKE_JOBS=yes portmaster math/py-matplotlib@py36 > & build-error_deps-not-to-build.log


Obviously, there is no problem with math/py-matplotlib@py36 itself, so this PR should be closed. Sorry for the noise.
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2018-12-03 07:48:46 UTC
(In reply to Rainer Hurling from comment #5)

If there's a bug somewhere, whether in py-matplotlib or elsewhere (portmaster), we want to isolate and identify the root cause(s).

The only odd thing I can observe in the attached logs is the last line of the last block:

===>>> The following actions were performed:
	Installation of lang/cython@py36 (py36-cython-0.29)
	Installation of math/py-numpy@py36 (py36-numpy-1.15.1,1)
	Installation of devel/py-cycler@py36 (py36-cycler-0.10.0)
	Installation of devel/py-dateutil@py36 (py36-dateutil-2.7.3)
	Installation of devel/py-pyparsing@py36 (py36-pyparsing-2.3.0)
	Installation of www/py-tornado@py36 (py36-tornado-5.1.1)
	Installation of math/py-matplotlib@py36 (py27-matplotlib-2.1.2_4) <---

Every other reference to port origins and package names is foo@py36 and py36-foo respectively, with the only reference to py27 being the (resolved) package name in the above line, which definitely appears incorrect.

However, the logs don't seem to include the actual full logs of the py-matplotlib(@py36) process so its difficult to say at what point that issue first presents itself.

On the PYTHONPREFIX_SITELIBDIR change proposal, both the PREFIX and non-PREFIX versions of that variable include the correct Python version specific path, use of PREFIX only makes the port PREFIX-overriden safe. I think this means that its unlikely to be the cause, nor the solution.

What we need from here is:

- Full log of py-matplotlib failing. Please un-quieten the @FIND command in the port before doing this, so we can confirm the full path/filenames that get operated on.
- Test: Does the current port (without patch) fail under poudriere?
Comment 7 Rainer Hurling 2018-12-03 14:33:21 UTC
(In reply to Kubilay Kocak from comment #6)

>If there's a bug somewhere, whether in py-matplotlib or elsewhere (portmaster), 
>we want to isolate and identify the root cause(s).
I agree.

>The only odd thing I can observe in the attached logs is the last line of the last block:
>===>>> The following actions were performed:
>	Installation of lang/cython@py36 (py36-cython-0.29)
>	Installation of math/py-numpy@py36 (py36-numpy-1.15.1,1)
>	Installation of devel/py-cycler@py36 (py36-cycler-0.10.0)
>	Installation of devel/py-dateutil@py36 (py36-dateutil-2.7.3)
>	Installation of devel/py-pyparsing@py36 (py36-pyparsing-2.3.0)
>	Installation of www/py-tornado@py36 (py36-tornado-5.1.1)
>	Installation of math/py-matplotlib@py36 (py27-matplotlib-2.1.2_4) <---

>Every other reference to port origins and package names is foo@py36 and py36-foo
>respectively, with the only reference to py27 being the (resolved) package name 
>in the above line, which definitely appears incorrect.
Yup, as I tried to describe in comment #5, the math/py-matplotlib with flavor @py36, in contrary to my initial report, builds and installs fine in both cases.

'pkg info -o math/py-matplotlib' proofs this:
py27-matplotlib-2.1.2_4        math/py-matplotlib
py36-matplotlib-2.1.2_4        math/py-matplotlib

The only thing is, that there is no entry under /var/db/pkg in the first case (py-matplotlib@py36 needs to install some deps first). That's why I think, that portmaster could be the culprit.

>However, the logs don't seem to include the actual full logs of the 
>py-matplotlib(@py36) process so its difficult to say at what point that 
>issue first presents itself.
Hmm, I thought I already attached full logs and described, how they were created. If these are not full logs, which way do you create them?

>On the PYTHONPREFIX_SITELIBDIR change proposal, both the PREFIX and non-PREFIX 
>versions of that variable include the correct Python version specific path, 
>use of PREFIX only makes the port PREFIX-overriden safe. I think this means 
>that its unlikely to be the cause, nor the solution.
If I am right with the above, there is no need to change from PYTHON_SITELIBDIR to PYTHONPREFIX_SITELIBDIR. Even with PYTHONPREFIX_SITELIBDIR, the behaviour is the same.

>What we need from here is:
>- Full log of py-matplotlib failing. Please un-quieten the @FIND command 
>  in the port before doing this, so we can confirm the full path/filenames 
>  that get operated on.
>- Test: Does the current port (without patch) fail under poudriere?
In the logs, I used FIND instead of @FIND already. The relevant line should be:
/usr/bin/find /usr/ports/math/py-matplotlib/work-py36/stage/usr/local/lib/python3.6/site-packages/matplotlib -name '*.so'  | /usr/bin/xargs /usr/bin/strip

On Poudriere, math/py-matplotlib@py36 builds fine (without patch).

Do you think, the missing entry for "py36-matplotlib-2.1.2_4" under /var/db/pkg is something, the port itself (and not portmaster) is responsible for?