Bug 250379 - www/py-cherrypy: fails to import with lang/python38 or later
Summary: www/py-cherrypy: fails to import with lang/python38 or later
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Kai Knoblich
URL:
Keywords: regression
Depends on:
Blocks: 247597
  Show dependency treegraph
 
Reported: 2020-10-15 23:29 UTC by Jan Beich
Modified: 2021-05-07 13:34 UTC (History)
4 users (show)

See Also:
kai: merge-quarterly+


Attachments
Patch ported from upstream (2.13 KB, patch)
2020-10-16 01:55 UTC, James French
no flags Details | Diff
Poudriere build log (70.08 KB, text/plain)
2020-10-16 01:55 UTC, James French
no flags Details
Patch ported from upstream, remove contextlib2 for Python 3 (3.55 KB, patch)
2020-10-17 05:07 UTC, James French
no flags Details | Diff
Patch ported from upstream, remove contextlib2 for Python 3 (3.55 KB, patch)
2020-10-17 05:16 UTC, James French
no flags Details | Diff
Updated poudriere build log (Python 3.7) (69.86 KB, text/plain)
2020-10-17 05:16 UTC, James French
no flags Details
Poudriere build log (Python 2.7) (75.20 KB, text/plain)
2020-10-17 05:17 UTC, James French
no flags Details
py-cherrypy-fix-py38-runtime.patch (4.65 KB, patch)
2021-04-22 17:25 UTC, Kai Knoblich
kai: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Beich freebsd_committer freebsd_triage 2020-10-15 23:29:04 UTC
Can you update to >= 18.0 or apply https://github.com/cherrypy/cherrypy/commit/2fc78dc9af3e ?

$ python3.9
Python 3.9.0 (default, Oct 15 2020, 22:11:05)
[Clang 11.0.0 (git@github.com:llvm/llvm-project.git llvmorg-11.0.0-rc5-0-g60a25 on freebsd13
Type "help", "copyright", "credits" or "license" for more information.
>>> import cherrypy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/cherrypy/__init__.py", line 73, in <module>
    from ._cptools import default_toolbox as tools, Tool
  File "/usr/local/lib/python3.9/site-packages/cherrypy/_cptools.py", line 31, in <module>
    from cherrypy.lib import sessions as _sessions, xmlrpcutil as _xmlrpc
  File "/usr/local/lib/python3.9/site-packages/cherrypy/lib/sessions.py", line 112, in <module>
    import contextlib2
ModuleNotFoundError: No module named 'contextlib2'
Comment 1 James French 2020-10-16 01:02:42 UTC
Updating to a newer version is still not possible without (at least) two new ports.

Patch inbound shortly.
Comment 2 James French 2020-10-16 01:55:16 UTC
Created attachment 218781 [details]
Patch ported from upstream

Fixes the regression with upstream's patch applied to Python >= 3.8

Has been tested as far as fixing the ImportError, I don't have any applications running on the later python versions with which to test (but don't see any reason why it wouldn't work - upstream didn't make huge code changes either).
Comment 3 James French 2020-10-16 01:55:51 UTC
Created attachment 218782 [details]
Poudriere build log
Comment 4 Jan Beich freebsd_committer freebsd_triage 2020-10-17 02:36:11 UTC
Comment on attachment 218781 [details]
Patch ported from upstream

Works for me. Tested via www/py-flexget but I did't enable web_server which actually uses cherrypy.
Comment 5 Jan Beich freebsd_committer freebsd_triage 2020-10-17 04:06:41 UTC
Comment on attachment 218781 [details]
Patch ported from upstream

> +.if ${PYTHON_REL} >= 3800
[..] 
>  .if ${PYTHON_REL} < 3800
>  RUN_DEPENDS+=	${PYTHON_PKGNAMEPREFIX}contextlib2>0:devel/py-contextlib2@${PY_FLAVOR}

Why is devel/py-contextlib2 disabled for python >= 3.8? "import cherrypy" survives with external "contextlib2" under lang/python39.

Why python 3.6 and 3.7 cannot use default "contextlib" instead of "contextlib2"? "import cherrypy" survives with "contextlib" patches. https://contextlib2.readthedocs.io/en/stable/ says "contextlib2" is a backport of "contextlib" from python 3.6, so it shouldn't be necessery with lang/python36 while lang/python35 and older are no longer in the ports tree.

> +EXTRA_PATCHES+=	${FILESDIR}/extra-patch-cherrypy_lib_sessions.py
> +EXTRA_PATCHES+=	${FILESDIR}/extra-patch-setup.py

Maybe combine into one file and document the patch origin in the header.
Comment 6 James French 2020-10-17 04:47:01 UTC
(In reply to Jan Beich from comment #5)

> Why is devel/py-contextlib2 disabled for python >= 3.8?

Officially Python 3.8+ are not on the supported versions list for contextlib2. The last supported version is Python 3.7 see: https://pypi.org/project/contextlib2/

It's python though, so unless it's doing something with a dropped/modified stdlib it's not a huge surprise that it works fine.

> Why python 3.6 and 3.7 cannot use default "contextlib" instead of "contextlib2"?

After a quick code review of what that library is pulled in for (cathing an exception), they probably can. This version of CherryPy still has Python 2 support & contextlib2 is required for that. I'll apply the patch to python > 3.2 (earliest version that shipped with contextlib) and drop contextlib2 as a dependency for those versions.

> Maybe combine into one file and document the patch origin in the header.

Can do, I'll submit an updated patch shortly that does all of the above.
Comment 7 James French 2020-10-17 05:07:16 UTC
Created attachment 218825 [details]
Patch ported from upstream, remove contextlib2 for Python 3

Changes as described in comment #6

Some Makefile cleanup also included to satisfy portlint.
Comment 8 James French 2020-10-17 05:16:04 UTC
Created attachment 218826 [details]
Patch ported from upstream, remove contextlib2 for Python 3

Actually puts the dependency against Python 2 rather than Python 3, cleans up the extra .if statement.
Comment 9 James French 2020-10-17 05:16:51 UTC
Created attachment 218827 [details]
Updated poudriere build log (Python 3.7)
Comment 10 James French 2020-10-17 05:17:37 UTC
Created attachment 218828 [details]
Poudriere build log (Python 2.7)
Comment 11 Dmitry Marakasov freebsd_committer freebsd_triage 2021-03-17 22:44:01 UTC
Assign to maintainer
Comment 12 Kai Knoblich freebsd_committer freebsd_triage 2021-04-22 17:25:01 UTC
Created attachment 224358 [details]
py-cherrypy-fix-py38-runtime.patch

Attached is a revised patch as the port got some changes due dropped Python 2.7 support via the ports framework. 

It is based on James's work (comment #8) and also contains the following:

- Sort variables according to the PHB and pet portclippy.
- Remove superfluous EXAMPLESDIR as it's already set via "PYTHON=concurrent"
- Convert to newer options framework.
- Add "do-test" target to make future QA easier.

Using the test suite, I was also able to compare the results between the unpatched and patched version, which were identical in both cases.

Also taking this PR as the maintainer seems missing in action and set the maintainer-approval flag accordingly.

I'll leave the patch here for those who are interested and will commit it in the next few days if there are no objections.
Comment 13 commit-hook freebsd_committer freebsd_triage 2021-04-25 19:00:03 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=7915fe9b3fce2721c82d041ebbe47cf0a47981c1

commit 7915fe9b3fce2721c82d041ebbe47cf0a47981c1
Author:     Kai Knoblich <kai@FreeBSD.org>
AuthorDate: 2021-04-25 18:45:34 +0000
Commit:     Kai Knoblich <kai@FreeBSD.org>
CommitDate: 2021-04-25 18:45:34 +0000

    www/py-cherrypy: Fix runtime with Python 3.8 and onward

    Backport patches from the 18.x release branch that uses the Python's
    library "contextlib" instead of the external library "contextlib2".

    The "contextlib2" library is a backported version which is required for
    compatibility with Python 2.7 as the 17.x release of CherryPy is the
    last one that has compatibility for Python 2.7.

    Since the support for Python 2.7 has been removed in aa7a6c429ea6,
    "contextlib2 is no longer needed for backward compatibility.

    This is a transitional solution to fix the regression with Python 3.8
    because an update to the 18.x release needs a bit more work and also
    requires one or more new Python packages.

    While I'm here:

    * Sort variables according to the PHB and pet portclippy.
    * Remove superfluous EXAMPLESDIR.
    * Convert to newer options framework.
    * Set NO_ARCH as there are no architecture specific files.
    * Add "do-test" target to make future QA easier.

    PR:             250379
    Reported by:    jbeich
    Approved by:    maintainer timeout (6+ months)
    MFH:            2021Q2

 www/py-cherrypy/Makefile                           | 43 ++++++++++++----------
 .../files/patch-cherrypy_lib_sessions.py (new)     | 27 ++++++++++++++
 www/py-cherrypy/files/patch-setup.py (new)         | 17 +++++++++
 3 files changed, 68 insertions(+), 19 deletions(-)
Comment 14 Kai Knoblich freebsd_committer freebsd_triage 2021-04-25 19:14:25 UTC
Thank you, Jan for the report and the reviewing of the earlier versions, and to James for creating the patches!

I plan to MFH'ing it in a few days, so I leave this bug open for now.
Comment 15 commit-hook freebsd_committer freebsd_triage 2021-05-07 13:30:34 UTC
A commit in branch 2021Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=fc93af74b8da19e5742801a0aafad914d73cb9af

commit fc93af74b8da19e5742801a0aafad914d73cb9af
Author:     Kai Knoblich <kai@FreeBSD.org>
AuthorDate: 2021-04-25 18:45:34 +0000
Commit:     Kai Knoblich <kai@FreeBSD.org>
CommitDate: 2021-05-07 13:29:32 +0000

    www/py-cherrypy: Fix runtime with Python 3.8 and onward

    Backport patches from the 18.x release branch that uses the Python's
    library "contextlib" instead of the external library "contextlib2".

    The "contextlib2" library is a backported version which is required for
    compatibility with Python 2.7 as the 17.x release of CherryPy is the
    last one that has compatibility for Python 2.7.

    Since the support for Python 2.7 has been removed in aa7a6c429ea6,
    "contextlib2 is no longer needed for backward compatibility.

    This is a transitional solution to fix the regression with Python 3.8
    because an update to the 18.x release needs a bit more work and also
    requires one or more new Python packages.

    While I'm here:

    * Sort variables according to the PHB and pet portclippy.
    * Remove superfluous EXAMPLESDIR.
    * Convert to newer options framework.
    * Set NO_ARCH as there are no architecture specific files.
    * Add "do-test" target to make future QA easier.

    PR:             250379
    Reported by:    jbeich
    Approved by:    maintainer timeout (6+ months)
    MFH:            2021Q2

    (cherry picked from commit 7915fe9b3fce2721c82d041ebbe47cf0a47981c1)

 www/py-cherrypy/Makefile                           | 43 ++++++++++++----------
 .../files/patch-cherrypy_lib_sessions.py (new)     | 27 ++++++++++++++
 www/py-cherrypy/files/patch-setup.py (new)         | 17 +++++++++
 3 files changed, 68 insertions(+), 19 deletions(-)
Comment 16 Kai Knoblich freebsd_committer freebsd_triage 2021-05-07 13:34:55 UTC
MFH'ed to the 2021Q2 branch, all done!