Bug 219641
| Summary: | devel/py-qt5-core: Fails to package/install, incorrect pk-plist entries | ||
|---|---|---|---|
| Product: | Ports & Packages | Reporter: | gdinolt |
| Component: | Individual Port(s) | Assignee: | freebsd-kde (group) <kde> |
| Status: | Closed Overcome By Events | ||
| Severity: | Affects Only Me | CC: | allcoms, antoine, cem, dmgk, freebsd-2024, freebsd, gdinolt, jcfyecrayz, john, lbartoletti, portmgr, python, rhurlin, tcberner, w.schwarzenfeld |
| Priority: | --- | Keywords: | easy, patch-ready |
| Version: | Latest | Flags: | tcberner:
maintainer-feedback+
koobs: merge-quarterly? koobs: exp-run? |
| Hardware: | Any | ||
| OS: | Any | ||
| Bug Depends on: | 232745 | ||
| Bug Blocks: | |||
| Attachments: | |||
Thank you for your report. If you could, please: 1) Test with other Python 3.x versions to determine whether the issue is reproducible in those versions. 2) Include a full build log of failures as an attachment 3) Include the proposed change (to python.mk) as an attachment rather than in-comment, in unified diff format. If indeed a python.mk bug (and modification) is implicated, an exp-run is warranted. CC'ing Antoine (antoine@) as a significant author of the section in question Created attachment 183051 [details]
py35-qt-core install failure example
This file shows the install failure messages, note that this illustrates that the .PLIST.mktmp and the stage directory do not agree on the file names.
The actual filenames in the stage directory are '*.cpython-35.opt-1.pyc
Created attachment 183052 [details]
Patch of /usr/ports/Mk/Uses/python.mk which seems to fix the problem
Without this patch, py-qt5-core does not install for python versions 3.5 and 3.6 but does work for versions less than 3.5
This file is a patch for python.mk that appears to solve the problem. I have tested it by building and installing py-qt-core for python versions 2.7, 3.4, 3.5 and 3.6.
Thank you for the additional information. To clarify, when you say: "does not install for python versions 3.5 and 3.6 but does work for versions less than 3.5" does this mean: "without this patch, 3.5 and 3.6 do not work (fail to package, as reported), but 3.4 does work (is unaffected)" ? Note this issue is/appears limited in scope to those using USE_PYTHON=py3kplist (In reply to Kubilay Kocak from comment #4) Just to be sure: Without the patch to python.mk, py-gtk5-core: builds and installs as expected using python3.4 fails to install using python3.5 and python3.6 With the patch using python3.4, python3.5 and python3.6 all build and install. Note that python3.4 uses *.cpython-34.pyo for its "optimized" compiled file while python3.5 and python3.6 use *.cpython-3[56].opt-1.pyc Hi I'm trying to install pyqt5 built against python3 under TrueOS (FreeBSD 12) but I have so far had no luck. From reading this thread I get the impression that without using the attached patch I should be able to get pyqt5 to build against python3.4 like so: cd /usr/ports/devel/py-qt5 make PYTHON_VERSION=python3.4 install That only gets so far for me before I get this error: ===> Staging for py34-qt5-5.7.1 ===> py34-qt5-5.7.1 depends on file: /usr/local/bin/python3.4 - found ===> py34-qt5-5.7.1 depends on package: py34-qt5-core>=5.7.1 - not found ===> py34-qt5-core-5.7.1 depends on file: /usr/local/bin/python3.4 - found ===> py34-qt5-core-5.7.1 depends on package: py34-sip>=4.19.2 - found ===> py34-qt5-core-5.7.1 depends on file: /usr/local/lib/qt5/bin/qmake - found ===> py34-qt5-core-5.7.1 depends on shared library: libqscintilla2-qt5.so - found (/usr/local/lib/qt5/libqscintilla2-qt5.so) ===> py34-qt5-core-5.7.1 depends on shared library: libQt5Core.so - found (/usr/local/lib/qt5/libQt5Core.so) ===> Configuring for py34-qt5-core-5.7.1 cd /usr/ports/devel/py-qt5-core/work/PyQt5_gpl-5.7.1 && /usr/bin/env QT_SELECT=qt5 PYTHON="/usr/local/bin/python3.4" XDG_DATA_HOME=/usr/ports/devel/py-qt5-core/work XDG_CONFIG_HOME=/usr/ports/devel/py-qt5-core/work HOME=/usr/ports/devel/py-qt5-core/work SHELL=/bin/sh CONFIG_SHELL=/bin/sh /usr/local/bin/python3.4 configure.py --enable QtCore --qsci-api --qsci-api-destdir=/usr/local/share/qt5/qsci -b /usr/local/bin -d /usr/local/lib/python3.4/site-packages -q /usr/local/lib/qt5/bin/qmake --confirm-license --sipdir /usr/local/share/py-sip/PyQt5 Querying qmake about your Qt installation... Determining the details of your Qt installation... This is the GPL version of PyQt 5.7.1 (licensed under the GNU General Public License) for Python 3.4.6 on freebsd12. Found the license file pyqt-gpl.sip. Error: Make sure you have a working sip on your PATH or use the --sip argument to explicitly specify a working sip. *** Error code 1 Stop. make[3]: stopped in /usr/ports/devel/py-qt5-core *** Error code 1 Stop. make[2]: stopped in /usr/ports/devel/py-qt5-core *** Error code 1 Stop. make[1]: stopped in /usr/ports/devel/py-qt5 *** Error code 1 Stop. make: stopped in /usr/ports/devel/py-qt5 devel/py-sip seems to be installed already so I'm not sure why its not finding it. I am able to build pyqt5 against python27 (the default) without problem, but I have had zero success with py3. Thanks for your help! Through having a search, it seems FreeBSD has packages for py34-sip, py36-sip etc whilst TrueOS only has py27-sip in its repos. You'll have to forgive me as I'm new to FreeBSD and even newer to using ports but does this matter ie do I need to request they add py34-sip or py36-sip to the TrueOS repo? I would've thought that building stuff from ports creates packages that are identical to what you get via pkg, only you have to compile them yourself which gives you more scope for tweaking build options etc. Is this not the case? Are ports builds treated differently by the OS compared to packages installed via pkg? I'm happy to create a new bug report for my problem if its unrelated to this one but I too was getting the misnamed pyc files when I tried building pyqt5 against python 3.6.1, but then I saw gdinolt has apparently got pyqt5 built against python 3.4 without any extra patches so I wanted to try that first, hence my previous replies. My ports knowledge is sadly lacking right now but I'm sure I'll be much more useful round here after I've found my feet. I've got building pyqt5 to progress further by linking /usr/local/bin/sip-3.4 to /usr/local/bin/sip but I expect that is a bug in that either the sip port should create such a link or the pyqt5 port should know to look for sip-x.x. Now building pyqt5 has stopped at py34-qt5-dbussupport which apparently needs at least python 3.6. When configuring pyqt5, I stuck with all the default options for all of its (sub)ports except for setting the python version to 3.4 so I presume gdinolt only built a subset of the pyqt5 packages? Yay! After a much longer than expected battle, I've finally got pyqt5 built against python 3.4. To achieve this: cd /usr/ports/devel/py-qt5 make PYTHON_VERSION=python3.4 config # Disable both dbus sub-ports and the pyqt5 demo port in the ncurses menu make PYTHON_VERSION=python3.4 install If you get the error: Error: Make sure you have a working sip on your PATH or use the --sip argument Then run: ln -s /usr/local/bin/sip-3.4 /usr/local/bin/sip Before re-running: make PYTHON_VERSION=python3.4 install Created attachment 183244 [details]
Explicitly set the sip-executable location
I can't even get py-qt5-core to BUILD with a non-default Python, because py36-sip installs only bin/sip-3.6 -- there is no bin/sip.
This additional patch is necessary in my case and useful in all cases, I think...
Created attachment 183245 [details]
Patch py-qt5-core to include QtCore.pyi
And this little patch was necessary for me to pass the check-plist in devel/py-qt5-core, when using python-3.6 -- not sure, how it would handle 3.4 or 3.5...
Created attachment 183246 [details]
Use PYTHON_VER instead PYTHON_VERSION. Duh...
*** Bug 220109 has been marked as a duplicate of this bug. *** I've tried this patch and works fine except for py-qt5-demo. (In reply to lbartoletti from comment #16) Could you, please, provide more details? Did the patch introduce _new_ breakage to py-qt5-demo, or was it simply insufficient to make that one port work? In the former case, could you provide the log showing the new error? Thank you! Sorry but this is not a Uses/python.mk bug, this is a bug in the ports... You can't mix %%PYTHON_PYOEXTENSION%% and py3kplist together. (In reply to Antoine Brodin from comment #18) > Sorry but this is not a Uses/python.mk bug, this is a bug in the ports... Do you mean, the individual py-ports, or the ports infrastructure? > You can't mix %%PYTHON_PYOEXTENSION%% and py3kplist together. Could you elaborate? None of the proposed patches mention py3kplist... Most problems should be fixed with https://svnweb.freebsd.org/changeset/ports/453212 (In reply to Mikhail T. from comment #19) The proposed Uses/python.mk patches were touching the py3kplist part. (In reply to Antoine Brodin from comment #21) For better or worse, this PR is now about more than one port... For example, my most recent patch adds explicit setting of sip to pyqt.mk -- which is crucial for people, who have more than on Python installed... (In reply to Antoine Brodin from comment #20) Hi Antoine, Thanks for https://svnweb.freebsd.org/changeset/ports/453212. One big showstopper was not solved, btw. In bug #211414, comment #8, I described a conflict with pyuic5: Installing py35-qt5-core-5.6... pkg-static: py35-qt5-core-5.6 conflicts with py27-qt5-core-5.6 (installs files into the same place). Problematic file: /usr/local/bin/pyuic5 Is there any chance to get bin/pyuic5 also moved into some 'Python version dependend' directory? Hi kde@, would you like an exp-run with:
+ --sip ${LOCALBASE}/bin/sip-${PYTHON_VER} \
added to Mk/Uses/pyqt.mk ?
Created attachment 187619 [details]
Fix py-qt5-* with python3
Comment on attachment 187619 [details] Fix py-qt5-* with python3 > +USE_PYTHON= py3kplist Why is `py3kplist' necessary, if `autoplist' was not necessary for the regular case of Python-2.7? The comment inside python.mk confuses me: # py3kplist - Automatically generates Python 3.x compatible # __pycache__ entries from a Python 2.x packaging list # when defined. Use this for ports that do *not* use # standard Python packaging mechanisms such as # distutils, and support *both* Python 2.x and 3.x. # Not needed, if USE_PYTHON=autoplist is set. Created attachment 187626 [details]
Fix py-qt5-* and py-qt4-demo with python3
A commit references this bug: Author: antoine Date: Wed Nov 1 07:32:34 UTC 2017 New revision: 453263 URL: https://svnweb.freebsd.org/changeset/ports/453263 Log: - py-qt5-*: Fix build when using non default version of python, in this case sip is installed as sip-${PYTHON_VER} - py-qt*-demo: Fix packaging with python3, those ports have a python2 pkg-plist so USE_PYTHON=py3kplist must be used to convert it PR: 219641 Changes: head/Mk/Uses/pyqt.mk head/devel/py-qt5-qscintilla2/Makefile head/misc/py-qt4-demo/Makefile head/misc/py-qt5-demo/Makefile Thanks. Comment on attachment 187626 [details]
Fix py-qt5-* and py-qt4-demo with python3
this patch was committed
(In reply to Mikhail T. from comment #13) Without this patch (attachment 183245 [details]), 'make stage check-orphans' for devel/py-qt5-core gives this error when building for python36 (e.g., DEFAULT_VERSIONS+=python=3.6 in make.conf): ===> Checking for items in STAGEDIR missing from pkg-plist Error: Orphaned: %%PYTHON_SITELIBDIR%%/PyQt5/QtCore.pyi This is different than the originally reported problem, so perhaps this merits a different bug report. Hi, I haven't tested what I'm proposing, but this conflict of pyuic5 is embarrassing for Rainer and myself for the next version of QGis 3. Since py-qt5-core has flavours, is it possible to rename the file pyuic5 according to its flavor? This should avoid installation conflicts. I think we need also to add a pkg-message, and aliases like py3uic5 for py36ui5 and py2uic5 for py27uic5. my 2c. Regards (In reply to Antoine Brodin from comment #24) I investigated a bit deeper and it seems, that there are several QT5 Python ports, which generate conflicts for py3x, if py27 is installed already: pkg-static: py36-qt5-core-5.9.2 conflicts with py27-qt5-core-5.9.2 (installs files into the same place). Problematic file: /usr/local/bin/pyuic5 pkg-static: py36-qt5-gui-5.9.2 conflicts with py27-qt5-gui-5.9.2 (installs files into the same place). Problematic file: /usr/local/share/py-sip/PyQt5/QtGui/QtGuimod.sip pkg-static: py36-qt5-widgets-5.9.2 conflicts with py27-qt5-widgets-5.9.2 (installs files into the same place). Problematic file: /usr/local/share/py-sip/PyQt5/QtWidgets/QtWidgetsmod.sip pkg-static: py36-qt5-sql-5.9.2 conflicts with py27-qt5-sql-5.9.2 (installs files into the same place). Problematic file: /usr/local/share/py-sip/PyQt5/QtSql/QtSqlmod.sip pkg-static: py36-qt5-printsupport-5.9.2 conflicts with py27-qt5-printsupport-5.9.2 (installs files into the same place). Problematic file: /usr/local/share/py-sip/PyQt5/QtPrintSupport/QtPrintSupportmod.sip pkg-static: py36-qt5-qscintilla2-2.9.1 conflicts with py27-qt5-qscintilla2-2.9.1 (installs files into the same place). Problematic file: /usr/local/share/py-sip/PyQt5/Qsci/qsciabstractapis.sip pkg-static: py36-qt5-svg-5.9.2 conflicts with py27-qt5-svg-5.9.2 (installs files into the same place). Problematic file: /usr/local/share/py-sip/PyQt5/QtSvg/QtSvgmod.sip And so on ... Obviously it is not enough to solve the problem only for the devel/py-qt5-core port. The patch suggested by Antoine Brodin in comment 24 would give us a working py-qt5-core port, for the others we probably need a change in the path of ../py-sip/.. Ting-Wei Lan submitted a patch in bug #224818, which looks interesting for these py-qt5 problems, also. He patched py-${PORTNAME} into ${PKGNAMEPREFIX}${PORTNAME}, which could be used here to change the path py-sip into py[27|3x]-sip or PyQt[45] into Py[27|3x]Qt[45] . This is not tested, especially not for any regressions. I was not able to prepare a working patch here, sorry. Hi,
I think Rainer is true. On OpenBSD, it seems that files are in py-sip/PyQt5-[2|3]
see
share/sip/PyQt5${MODPY_BIN_SUFFIX}
Link: http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/x11/py-qt5/pkg/PLIST?rev=1.1&content-type=text/x-cvsweb-markup
See also bug 225773, which includes a patch fixing the PYI file problem on devel/py-qt5-core, x11-toolkits/py-qt5-widgets, and x11-toolkits/py-qt5-gui. That patch allows me to fully package and install those three ports under all four python versions in the tree. See comment35 and https://svnweb.freebsd.org/ports?view=revision&revision=461841 This is fixed. (In reply to w.schwarzenfeld from comment #36) Hi Walter, Sorry to answer here again. But this is not solved for me. If devel/py-qt5-core@py27 is installed already and I try to install devel/py-qt5-core@py36 additionally, it conflicts: ===> Registering installation for py36-qt5-core-5.9.2_6 Installing py36-qt5-core-5.9.2_6... pkg-static: py36-qt5-core-5.9.2_6 conflicts with py27-qt5-core-5.9.2_6 (installs files into the same place). Problematic file: /usr/local/bin/pyuic5 *** Error code 70 In the bottom of comment #33 I tried to show, what works for me. Any chance to think again about the problem? Ok, reopen *** Bug 223764 has been marked as a duplicate of this bug. *** *** Bug 223409 has been marked as a duplicate of this bug. *** *** Bug 233312 has been marked as a duplicate of this bug. *** *** Bug 233311 has been marked as a duplicate of this bug. *** *** Bug 233313 has been marked as a duplicate of this bug. *** |
devel/py-qt5-core builds as expected with python3.5 but does not install because the work/.PLIST.mktmp file refers to files that don't exist in the stage directory. By default, when python3.5 compiles, for example, file Qt5.py for installation, it generates lib/python3.5/site-packages/PyQt5/__pycache__/Qt5.cpython-35.opt-1.pyc lib/python3.5/site-packages/PyQt5/__pycache__/Qt5.cpython-35.pyc in the stage directory. When the 'add-plist-python' target from /usr/ports/Mk/Uses/python.mk is called to modify pkg-plist it generates the wrong output: lib/python3.5/site-packages/PyQt5/__pycache__/Qt.cpython-35.pyc lib/python3.5/site-packages/PyQt5/__pycache__/Qt.opt-1.cpython-35.pyc Note that '.opt-1.' appears before cpython-35.pyc rather than as shown above. A fix is to modify the 'add-plist-python' target in python.mk. I updated the target to: add-plist-python: @${AWK} '\ /\.py[co]$$/ && !($$0 ~ "/" pc "/") {id = match($$0, /\/[^\/]+\.py[co]$$/); if (id != 0) {d = substr($$0, 1, RSTART - 1); dirs[d] = 1}; od = match($$0, /opt-1/);\ if ( od != 0 ) sub(/\.opt-1\./, "." mt "&"); else \ sub(/\.pyc$$/, "." mt "&"); sub(/[^\/]+\.py[co]$$/, pc "/&"); print; next} \ /^@dirrm / {d = substr($$0, 8); if (d in dirs) {print $$0 "/" pc}; print $$0; next} \ /^@dirrmtry / {d = substr($$0, 11); if (d in dirs) {print $$0 "/" pc}; print $$0; next} \ {print} \ ' \ pc="__pycache__" mt="$$(${PYMAGICTAG})" pyo="${PYTHON_PYOEXTENSION}" \ ${TMPPLIST} > ${TMPPLIST}.pyc_tmp @${MV} ${TMPPLIST}.pyc_tmp ${TMPPLIST} I have not tested with other versions of python.