Bug 250380
| Summary: | textproc/py-feedparser: fails to import with lang/python39 | ||
|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Jan Beich <jbeich> |
| Component: | Individual Port(s) | Assignee: | Sofian Brabez <sbz> |
| Status: | Closed FIXED | ||
| Severity: | Affects Some People | CC: | adamw, colaholiker, gregf, james, python, sbz, smalbec |
| Priority: | --- | Keywords: | needs-qa |
| Version: | Latest | Flags: | sbz:
maintainer-feedback+
sbz: maintainer-feedback+ sbz: merge-quarterly- |
| Hardware: | Any | ||
| OS: | Any | ||
| Attachments: | |||
|
Description
Jan Beich
2020-10-15 23:39:52 UTC
Created attachment 218819 [details]
Patch to 6.0.1
The attached patch should bring this up to date.
Created attachment 218820 [details]
Poudriere build log
Comment on attachment 218819 [details] Patch to 6.0.1 Appears broken: py27 build and py3? runtime. $ make install FLAVOR=py37 $ python3.7 Python 3.7.9 (default, Oct 15 2020, 12:55:07) [Clang 8.0.1 (tags/RELEASE_801/final 366581)] on freebsd12 Type "help", "copyright", "credits" or "license" for more information. >>> import feedparser Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.7/site-packages/feedparser/__init__.py", line 28, in <module> from .api import parse File "/usr/local/lib/python3.7/site-packages/feedparser/api.py", line 36, in <module> from .html import _BaseHTMLProcessor File "/usr/local/lib/python3.7/site-packages/feedparser/html.py", line 31, in <module> from .sgml import * File "/usr/local/lib/python3.7/site-packages/feedparser/sgml.py", line 30, in <module> import sgmllib ModuleNotFoundError: No module named 'sgmllib' $ make deinstall FLAVOR=py37 $ make install FLAVOR=py39 BUILD_ALL_PYTHON_FLAVORS=1 $ python3.9 Python 3.9.0 (default, Oct 13 2020, 06:49:46) [Clang 8.0.1 (tags/RELEASE_801/final 366581)] on freebsd12 Type "help", "copyright", "credits" or "license" for more information. >>> import feedparser Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.9/site-packages/feedparser/__init__.py", line 28, in <module> from .api import parse File "/usr/local/lib/python3.9/site-packages/feedparser/api.py", line 36, in <module> from .html import _BaseHTMLProcessor File "/usr/local/lib/python3.9/site-packages/feedparser/html.py", line 31, in <module> from .sgml import * File "/usr/local/lib/python3.9/site-packages/feedparser/sgml.py", line 30, in <module> import sgmllib ModuleNotFoundError: No module named 'sgmllib' $ make deinstall FLAVOR=py39 $ make install FLAVOR=py27 [...] ===> Configuring for py27-feedparser-6.0.1 Traceback (most recent call last): File "<string>", line 1, in <module> File "setup.py", line 40 name = f"{name}_{os.getenv('NAME_SUFFIX')}" ^ SyntaxError: invalid syntax *** Error code 1 Oops - I missed that new dependency, will rectify shortly. 2.7 is expected behaviour as support was dropped in 6.0.0, I'll set it to python 3.6+ in the next patch. Created attachment 218823 [details]
Patch to 6.0.1, ship sgmllib (ported from Python2)
The plot thickens somewhat. Sgmllib was in the Python 2 standard library and was dropped in Python 3. The recommended practice from Feedparser upstream is to 2to3 it and ship it until they can tidy it up (which is what I've done).
Python version set to 3.6+, License set to multi, BSD2ClAUSE (Feedparser) and PFSL (sgmllib).
Not the cleanest solution, but not sure what other options we have other than marking the port broken.
Created attachment 218824 [details]
Poudriere build log
For some reason, Sofian was never added to this PR, so it's unclear whether he is even aware of it.
Sofian: I broke sabnzbdplus in quarterly, and this patch needs to be committed to fix it. Can you look it over please? If you're unable to get to this soon then I'll back out the commit in 250411.
Also, James: ${CP} shouldn't generally be used in any of the install phases. Can you please change this to INSTALL_DATA or INSTALL_SCRIPT or whatever it needs to be?
A commit references this bug: Author: adamw Date: Tue Oct 27 23:37:38 UTC 2020 New revision: 553512 URL: https://svnweb.freebsd.org/changeset/ports/553512 Log: news/sabnzbdplus: Back out r553378 Feedburner needs to be a hard dependency, and the version currently in ports is incompatible. PR: 250411, 250380 Changes: head/news/sabnzbdplus/Makefile head/news/sabnzbdplus/distinfo head/news/sabnzbdplus/pkg-plist A commit references this bug: Author: adamw Date: Tue Oct 27 23:38:36 UTC 2020 New revision: 553513 URL: https://svnweb.freebsd.org/changeset/ports/553513 Log: MFH: r553512 Approved by: portmgr (with hat) news/sabnzbdplus: Back out r553378 Feedburner needs to be a hard dependency, and the version currently in ports is incompatible. PR: 250411, 250380 Changes: _U branches/2020Q4/ branches/2020Q4/news/sabnzbdplus/Makefile branches/2020Q4/news/sabnzbdplus/distinfo branches/2020Q4/news/sabnzbdplus/pkg-plist Created attachment 219157 [details]
Patch to 6.0.1, Ship 2to3 sgmllib (ported from Py27)
Uses INSTALL_DATA rather than CP as requested.
Created attachment 219162 [details] Patch to 6.0.1, Ship 2to3 sgmllib (ported from Py27) sgmllib.py was missing from attachment 219157 [details] Any chance on getting this committed please? I suspect this port is now entirely broken with the removal of Python 2. Sorry about the delay. I'm overwhelmed with $IRL right now. Any committer who's interested, please feel free to make this happen. FreeNAS's plugin ends up blocking on this PR too. (In reply to Adam Weinberger from comment #13) I can try to sort this for you today Adam. Can you summarise steps/whats left to do so i don't miss-interpret the history Ping on comment 14 I can get this done tomorrow afternoon if I'm clear whats left (In reply to Kubilay Kocak from comment #15) Thanks, koobs. Should be this patch, and then the patch in bug #250411. Created attachment 224091 [details]
update to 6.0.2
Patch against `main` to update to 6.0.2, switch to github and fix the python flavors issue with missing sgmllib
Hi, This bug is now on my radar, somehow it never reached me. Where we at now? Would an new update to the new release 6.0.2 available on GitHub here will solved the issue and unblock? I'm attaching a new up-to-date patch against `main` with all the changes from James, does it looks good for you? So far it looks good with my tests with py37 and py39 flavor below: - with python3.7 $ sudo make install clean FLAVOR=py37 $ python3.7 Python 3.7.10 (default, Mar 2 2021, 01:19:31) [Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611a on freebsd12 Type "help", "copyright", "credits" or "license" for more information. >>> import feedparser >>> print(feedparser.__file__) /usr/local/lib/python3.7/site-packages/feedparser/__init__.py >>> dir(feedparser) ['CharacterEncodingOverride', 'CharacterEncodingUnknown', 'FeedParserDict', 'NonXMLContentType', 'RESOLVE_RELATIVE_URIS', 'SANITIZE_HTML', 'ThingsNobodyCaresAboutButMe', 'USER_AGENT', 'UndeclaredNamespace', '__author__', '__builtins__', '__cached__', '__doc__', '__file__', '__license__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', 'api', 'datetimes', 'encodings', 'exceptions', 'html', 'http', 'mixin', 'namespaces', 'parse', 'parsers', 'registerDateHandler', 'sanitizer', 'sgml', 'urls', 'util'] >>> feedparser.parse('http://feedparser.org/docs/examples/atom10.xml')['feed'] {'html': {'lang': 'en'}, 'meta': {'name': 'viewport', 'content': 'width=device-width,initial-scale=1'}, 'links': [{'rel': 'shortcut icon', 'href': 'data:image/x-icon;,', 'type': 'image/x-icon'}], 'script': {'type': 'application/javascript'}, 'a': {'target': '_blank', 'href': 'https://www.enable-javascript.com/'}, 'summary': '<div id="contentMain"></div>'} - with python3.9 $ sudo make install FLAVOR=py39 BUILD_ALL_PYTHON_FLAVORS=1 $ python3.9 Python 3.9.2 (default, Feb 25 2021, 13:58:29) [Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611a on freebsd12 Type "help", "copyright", "credits" or "license" for more information. >>> print(feedparser.__file__) /usr/local/lib/python3.9/site-packages/feedparser/__init__.py >>> feedparser.parse('http://feedparser.org/docs/examples/atom10.xml')['feed'] {'html': {'lang': 'en'}, 'meta': {'name': 'viewport', 'content': 'width=device-width,initial-scale=1'}, 'links': [{'rel': 'shortcut icon', 'href': 'data:image/x-icon;,', 'type': 'image/x-icon'}], 'script': {'type': 'application/javascript'}, 'a': {'target': '_blank', 'href': 'https://www.enable-javascript.com/'}, 'summary': '<div id="contentMain"></div>'} Also I have updated test targets: make test ===> Testing for py37-feedparser-6.0.2 ===> py37-feedparser-6.0.2 depends on file: /usr/local/bin/python3.7 - found .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ---------------------------------------------------------------------- Ran 4334 tests in 4.560s OK [1] https://github.com/kurtmckee/feedparser/releases/tag/6.0.2 Adam, I'm taking this one, Thanks to flo@ for putting this on my radar. (In reply to Sofian Brabez from comment #19) THANK YOU! Comment on attachment 224091 [details] update to 6.0.2 > + ${INSTALL_DATA} ${FILESDIR}/sgmllib.py ${STAGEDIR}/${PYTHON_SITELIBDIR}/sgmllib.py files/sgmllib.py is missing in the patch, so it fails to build: $ make [...] install -m 0644 /usr/ports/textproc/py-feedparser/files/sgmllib.py /usr/ports/textproc/py-feedparser/work-py39/stage//usr/local/lib/python3.9/site-packages/sgmllib.py install: /usr/ports/textproc/py-feedparser/files/sgmllib.py: No such file or directory *** Error code 71 Thanks everyone! @Sofian aside from it missing sgmllib.py, the patch otherwise looks good. It also builds ok when I manually drop the missing file in (poudriere log available if required). Yes, I do have it locally: $ git diff --stat HEAD~1 textproc/py-feedparser/Makefile | 28 ++++++--- textproc/py-feedparser/distinfo | 6 +- textproc/py-feedparser/files/sgmllib.py | 547 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 570 insertions(+), 11 deletions(-) Also, I noticed there is that sgmllib3k [1] but it requires to add a new port and upstream does not plan to maintain it. Adding a patch in the port seems a better choice here. I'll commit that today. [1] https://pypi.org/project/sgmllib3k A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=d950078a6163652c8350f619bdd29e9d2d2439ca commit d950078a6163652c8350f619bdd29e9d2d2439ca Author: Sofian Brabez <sbz@FreeBSD.org> AuthorDate: 2021-04-13 21:05:53 +0000 Commit: Sofian Brabez <sbz@FreeBSD.org> CommitDate: 2021-04-14 12:14:18 +0000 textproc/py-feedparser: update to 6.0.2 This update require adding the missing sgmllib for python3 versions and makes the license dual contributed by James French. Also update the test target and switch to github. PR: 250380 Submitted by: jbeich MFH: 2021Q2 textproc/py-feedparser/Makefile | 28 +- textproc/py-feedparser/distinfo | 6 +- textproc/py-feedparser/files/sgmllib.py (new) | 547 ++++++++++++++++++++++++++ 3 files changed, 570 insertions(+), 11 deletions(-) sgmllib3k workaround is incomplete:
$ make install -C misc/pubs
$ pubs
Traceback (most recent call last):
File "/usr/local/bin/pubs", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3252, in <module>
def _initialize_master_working_set():
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 583, in _build_master
ws.require(__requires__)
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'sgmllib3k' distribution was not found and is required by feedparser
$ make install -C www/flexget
$ flexget
Traceback (most recent call last):
File "/usr/local/bin/flexget", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3252, in <module>
def _initialize_master_working_set():
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 583, in _build_master
ws.require(__requires__)
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'sgmllib3k' distribution was not found and is required by FeedParser
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=f43705587fa6644212928b7eb7464a50ac8f23dc commit f43705587fa6644212928b7eb7464a50ac8f23dc Author: Jan Beich <jbeich@FreeBSD.org> AuthorDate: 2021-04-15 00:58:16 +0000 Commit: Jan Beich <jbeich@FreeBSD.org> CommitDate: 2021-04-15 00:58:55 +0000 textproc/py-feedparser: unbreak egg d950078a6163 pkg_resources.DistributionNotFound: The 'sgmllib3k' distribution was not found and is required by feedparser PR: 250380 textproc/py-feedparser/Makefile | 4 ++++ 1 file changed, 4 insertions(+) I've applied a bandaid to avoid the regression propagating into binary packages. The egg is feedparser.egg-info/requires.txt generated by setup.py via distutils. Thanks Jan, it looks good to me, I'm removing the block to move forward on the other PR. Yet I will investigate the other strategy to create devel/py-sgmllib3k and add it to the RUN_DEPENDS. If it looks good enough with other reverse dependencies (such as misc/pubs, www/flexget, etc.) that register sgmllib3k in the egg distribution, we could get rid of this workaround on the setup.py and the sgmllib.py file. Moveover other ports might need that `sgmllib` too. I'm confident it's the right approach as other OS does the same. Created attachment 224133 [details]
use new devel/py3-sgmllib as run dependency in devel/py-feedparser
Patch without smgllib shipped.
I'm having trouble installing feedparser but I should be using python 2.7
hadoop@hadoop-VirtualBox:~/Desktop/project 2$ sudo pip install feedparser
Downloading/unpacking feedparser
Downloading feedparser-6.0.2.tar.gz (284kB): 284kB downloaded
Running setup.py (path:/tmp/pip_build_root/feedparser/setup.py) egg_info for package feedparser
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip_build_root/feedparser/setup.py", line 40
name = f"{name}_{os.getenv('NAME_SUFFIX')}"
^
SyntaxError: invalid syntax
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip_build_root/feedparser/setup.py", line 40
name = f"{name}_{os.getenv('NAME_SUFFIX')}"
SyntaxError: invalid syntax
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/feedparser
Storing debug log for failure in /home/hadoop/.pip/pip.log
Is this relevant? Also does anyone know how to fix it?
Hi, You can only use 5.2.1 version with python2.7, after feedparser is python 3 only. For you information the f-string formatting f"" is python3.6 minimum. You might be able to install the 5.2.1 version using the command below. $ sudo pip install feedparser==5.2.1 Also, you are not using the FreeBSD port or package here. FYI, python2.7 will be not supported anymore, it's time to upgrade :) A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=a08dcbae9bc945ad6e78a4e5362fd760dc0220c0 commit a08dcbae9bc945ad6e78a4e5362fd760dc0220c0 Author: Sofian Brabez <sbz@FreeBSD.org> AuthorDate: 2021-04-15 19:54:43 +0000 Commit: Sofian Brabez <sbz@FreeBSD.org> CommitDate: 2021-04-15 21:00:24 +0000 textproc/py-feedparser: use new devel/py-sgmllib3k as run dependency Do not ship sgmllib via ${FILESDIR} because it breaks the egg registration of the feedparser module when installed via distutil, instead add new created port devel/py-sgmllib3k as run dependency. This seems the right approach of solving this issue. PR: 250380 textproc/py-feedparser/Makefile | 21 +- textproc/py-feedparser/files/sgmllib.py (gone) | 547 ------------------------- 2 files changed, 5 insertions(+), 563 deletions(-) (In reply to commit-hook from comment #32) This version works fine for me. Tested for ~2 days, so flexget could process RSS feed updates. Do you plan to MFH to 2021Q2? If not reset merge-quarterly bugzilla flag. No plan to MFH Submitted reported that it was solved. We can close it now. |