Bug 213737 - [exp-run] add support for PyPy
Summary: [exp-run] add support for PyPy
Status: Closed Overcome By Events
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-10-24 05:37 UTC by David Naylor
Modified: 2018-12-11 07:03 UTC (History)
1 user (show)

See Also:


Attachments
Patch - Part 1 - v1.0.0 (57.13 KB, patch)
2016-10-24 05:37 UTC, David Naylor
no flags Details | Diff
Patch - Part 1 - v1.0.1 (56.84 KB, patch)
2016-10-24 15:14 UTC, David Naylor
no flags Details | Diff
Patch - Part 1 - v1.1.0 (92.86 KB, patch)
2016-12-22 05:14 UTC, David Naylor
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Naylor freebsd_committer freebsd_triage 2016-10-24 05:37:58 UTC
Created attachment 176092 [details]
Patch - Part 1 - v1.0.0

Part one of adding support for PyPy to USES=python is to modify USES=python such that it can switch between cPython and PyPy.  This should not change the behaviour of any of the ports that USES=python.

Please run an exp-run with the attached patch.  This patch impacts all ports that have USES=python set.
Comment 1 Antoine Brodin freebsd_committer freebsd_triage 2016-10-24 05:51:24 UTC
There are some make errors with fmake (FreeBSD 9.3) :

"/usr/ports/Mk/Uses/python.mk", line 440: "/usr/ports/Mk/Uses/python.mk", line 418: Malformed conditional (2.7 == ${_PYTHON_VERSION})
"/usr/ports/Mk/Uses/python.mk", line 418: Malformed conditional (3.5 == ${_PYTHON_VERSION})
"/usr/ports/Mk/Uses/python.mk", line 418: Malformed conditional (3.4 == ${_PYTHON_VERSION})
"/usr/ports/Mk/Uses/python.mk", line 418: Malformed conditional (3.3 == ${_PYTHON_VERSION})
Malformed conditional (!defined(_PYTHON_VERSION) &&  !(!empty(_PYTHON_VERSION_MINIMUM) && (  2.7 < ${_PYTHON_VERSION_MINIMUM})) &&  !(!empty(_PYTHON_VERSION_MAXIMUM) && (  2.7 > ${_PYTHON_VERSION_MAXIMUM})))
"/usr/ports/Mk/Uses/python.mk", line 680: "/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 2.7 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 3.5 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 3.4 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 3.3 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 446: if-less endif
"/usr/ports/Mk/Uses/python.mk", line 440: Malformed conditional (!defined(_PYTHON_VERSION) &&  !(!empty(_PYTHON_VERSION_MINIMUM) && (  2.7 < ${_PYTHON_VERSION_MINIMUM})) &&  !(!empty(_PYTHON_VERSION_MAXIMUM) && (  2.7 > ${_PYTHON_VERSION_MAXIMUM})))
"/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 2.7 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 444: if-less endif
"/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 3.5 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 444: if-less endif
"/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 3.4 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 444: if-less endif
"/usr/ports/Mk/Uses/python.mk", line 442: Malformed conditional (!defined(_PYTHON_VERSION) && 3.3 == 2.7)
"/usr/ports/Mk/Uses/python.mk", line 444: if-less endif
"/usr/ports/Mk/Uses/python.mk", line 446: if-less endif
Comment 2 David Naylor freebsd_committer freebsd_triage 2016-10-24 06:31:22 UTC
Thanks, I can reproduce to error.  fmake has issues with a loop variable coming first in a comparison.  There are quite a few occurances of this error.  I'll submit an updated patch this afternoon.
Comment 3 David Naylor freebsd_committer freebsd_triage 2016-10-24 15:14:53 UTC
Created attachment 176102 [details]
Patch - Part 1 - v1.0.1

USES=python: fixes for fmake.

 - fmake cannot compare against a constant (or for loop variable) on the left hand side.
 - fmake cannot compare against two variables on the left hand side.
Comment 4 Antoine Brodin freebsd_committer freebsd_triage 2016-10-25 12:30:46 UTC
A few comments (not exhaustive):

* audio/idjc:

this line seems no longer necessary and wrong now
 ${REINPLACE_CMD} 's|#! /usr/local/bin/python2.72.7|#\!${LOCALBASE}/bin/${PYTHON_VERSION}|'

* biology/libsbml:

when the PYTHON option is turned on,  CMAKE_ARGS looks wrong

* deskutils/calibre:
@${MKDIR} ${STAGEDIR}${PREFIX}/lib/${PYTHON_VERSION}/site-packages doesn't look good

* devel/apr2/Makefile:
the REINPLACE with PYTHON_VERSION do not look good

* devel/boost-python-libs :

the message is now wrong,
"You have built the Boost library with thread support.\n\nDon't forget to add -pthread to your linker options when\nlinking your code.\n\nYou have built the Boost.Python library. You have to add the following\noptions when building your own code:\n\nCompiler options:\n   -I/usr/local/include/python2.7 -I/usr/local/include\n\nLinker options:\n   -L/usr/local/lib/python2.7 -L/usr/local/lib -lboost_python -lcpython2.7"

* net-p2p/tribler/files/tribler.in:

will probably be wrong

* science/avogadro:
The ${PYTHON_VERSION} stuff doesn't look good

* textproc/codespell

The REINPLACE doesn't look good

* textproc/py-libxml2 and textproc/py3-libxml2:
CPPFLAGS and LDFLAGS look wrong

* x11-toolkits/py-gtk2

The CONFIGURE_ENV= PYTHON="${PYTHON_VERSION}" line seems wrong and unneeded
Comment 5 Antoine Brodin freebsd_committer freebsd_triage 2016-10-25 19:25:55 UTC
Another thing:  the ${PYTHON_VER} != ${PYTHON_DEFAULT} checks have to be changed too.
Comment 6 Antoine Brodin freebsd_committer freebsd_triage 2016-10-25 19:53:50 UTC
I did a few fixes,  here are the lines that still look suspicious:

archivers/py-attic/Makefile:35:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
archivers/py-borgbackup/Makefile:38:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
audio/lollypop/Makefile:39:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
biology/libsbml/Makefile:33:                    -DPYTHON_LIBRARY:STRING="${PYTHONBASE}/lib/lib${PYTHON_VERSION}.so" \
comms/wsjt/Makefile:106:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
comms/wspr/Makefile:66:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
deskutils/calibre/Makefile:100: @${MKDIR} ${STAGEDIR}${PREFIX}/lib/${PYTHON_VERSION}/site-packages \
devel/apr2/Makefile:113:        ${REINPLACE_CMD} -e 's|PrintPath python|PrintPath ${PYTHON_VERSION}|' \
devel/apr2/Makefile:114:                -e 's|python -c|${PYTHON_VERSION} -c|' \
devel/boost-python-libs/Makefile:17:            PYTHON_VERSION="${PYTHON_VERSION}"\
devel/boost-python-libs/Makefile:50:            -e "s|%%PYTHON_VERSION%%|${PYTHON_VERSION}|g" ${PKGMESSAGE}
devel/staf/Makefile:80:                 PYTHON_V${PYTHON_SUFFIX}_LIBS=${PYTHON_VERSION}${PYTHON_ABIVER} \
devel/boost-libs/Makefile:66:           -e "s|%%PYTHON_VERSION%%|${PYTHON_VERSION}|g" ${PKGMESSAGE}
devel/gnatcoll/Makefile:26:PYCOMPILE=   ${PREFIX}/lib/${PYTHON_VERSION}/compileall.py
devel/boost-all/pkg-message.python:8:   -L%%PYTHON_LIBDIR%% -L%%LOCALBASE%%/lib -lboost_python -l%%PYTHON_VERSION%%
devel/py-pykde4/Makefile:25:            -DPYTHON_INCLUDE_DIR:PATH=${WRKDIR}/include/${PYTHON_VERSION}
emulators/virtualbox-ose/Makefile:319:      -e 's|SUPPYTHONLIBS=.*|SUPPYTHONLIBS="${PYTHON_VERSION}${PYTHON_ABIVER}"|' \
graphics/py-graphy/Makefile:29: @${REINPLACE_CMD} -e 's|python2.4|${PYTHON_VERSION}|' \
graphics/qgis/Makefile:151:     @${REINPLACE_CMD} -e 's|env python|env ${PYTHON_VERSION}|g' \
graphics/partio/Makefile:33:            s|lib64/python$${PYTHON_VERSION}/site-packages|${PYTHONPREFIX_SITELIBDIR:S/${PREFIX}\///}|' \
mail/rss2email3/Makefile:38:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
math/plplot/Makefile:63:PYTHON_CONFIGURE_ENV=   PYTHON_VERSION="${PYTHON_VERSION}"
math/rpcalc/Makefile:45:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
math/convertall/Makefile:47:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
multimedia/miro/Makefile:60:    ${REINPLACE_CMD} -e 's|python2.4|${PYTHON_VERSION}|g' \
multimedia/mps-youtube/Makefile:50:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
multimedia/mplayer2/Makefile:219:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
net-im/poezio/Makefile:22:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
net-p2p/libtorrent-rasterbar-python/Makefile:29:                PYTHON_VERSION="${PYTHON_VERSION}"
net-p2p/tribler/files/tribler.in:11:exec %%PYTHON_VERSION%% -O Tribler/Main/tribler.py "$@" > $STATEDIR/tribler.log 2>&1
net-p2p/tribler/Makefile:44:            PYTHON_VERSION="${PYTHON_VERSION}"
net/dshell/Makefile:41: ${RM} -r ${WRKSRC}/lib/${PYTHON_VERSION}
net/tiny-network-utilities/Makefile:30:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
ports-mgmt/py-pytoport/Makefile:28:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
science/vmd/Makefile:72:                -e "s|python2.5|${PYTHON_VERSION}|" \
science/avogadro/Makefile:43:                   -DPYTHON_INCLUDE_DIR:PATH=${WRKDIR}/include/${PYTHON_VERSION}
science/avogadro/Makefile:54:   @${MKDIR} ${WRKDIR}/include/${PYTHON_VERSION}
science/avogadro/Makefile:57:           ${WRKDIR}/include/${PYTHON_VERSION}
science/avogadro/Makefile:60:           ${WRKDIR}/include/${PYTHON_VERSION}/pyport.h
security/py-gpgme/Makefile:16:  ${STRIP_CMD} ${STAGEDIR}${PREFIX}/lib/${PYTHON_VERSION}/site-packages/pyme/_gpgme.so
www/xist/Makefile:26:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
www/rtv/Makefile:27:.if defined(PACKAGE_BUILDING) && ${PYTHON_VER} != ${PYTHON_DEFAULT}
www/seahub/Makefile:83:         PYTHON=${PYTHON_VERSION}
www/seahub/Makefile:94: ${REINPLACE_CMD} -e 's|##PYTHON_VERSION##|${PYTHON_VERSION}|g' ${WRKSRC}/*.template
Comment 7 David Naylor freebsd_committer freebsd_triage 2016-10-25 19:58:15 UTC
Great, thank you for your help :-).  I'll pick this up tomorrow morning.  Could you please attach your changes, to avoid duplicate work.
Comment 8 Antoine Brodin freebsd_committer freebsd_triage 2016-10-25 20:00:28 UTC
(In reply to David Naylor from comment #7)
My changes are already committed as it was some cleanup that works with current python.mk
Comment 9 David Naylor freebsd_committer freebsd_triage 2016-10-25 20:03:51 UTC
Awesome.  I must admit that many uses of PYTHON_VERSION have been incorrect (often easily replaced with the correct variable).
Comment 10 David Naylor freebsd_committer freebsd_triage 2016-12-22 05:14:22 UTC
Created attachment 178193 [details]
Patch - Part 1 - v1.1.0

Added a new python feature: pydepends.  This will add the package guards (and is automatically detected).  

I've fixed all the ports on your list, and a few others.
Comment 11 David Naylor freebsd_committer freebsd_triage 2018-12-11 07:03:55 UTC
This needs to be reworked from scratch given the introduction of FLAVORS for USES=python