Bug 250941 - www/py-requests: Update to 2.25.1
Summary: www/py-requests: Update to 2.25.1
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Danilo G. Baio
URL: https://github.com/psf/requests/blob/...
Keywords:
Depends on: 252187
Blocks: 250923 252180 254527 254784 255606
  Show dependency treegraph
 
Reported: 2020-11-08 07:53 UTC by Olivier Cochard
Modified: 2021-05-22 10:00 UTC (History)
6 users (show)

See Also:
koobs: maintainer-feedback+
dbaio: maintainer-feedback+
dbaio: merge-quarterly+


Attachments
Patch to update to 2.24.0 (3.84 KB, patch)
2020-11-08 07:53 UTC, Olivier Cochard
no flags Details | Diff
2.25.1 v1 (4.84 KB, patch)
2020-12-27 06:41 UTC, Charlie Li
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Olivier Cochard freebsd_committer 2020-11-08 07:53:44 UTC
Created attachment 219446 [details]
Patch to update to 2.24.0

net/py-GeoIP2 requires www/py-requests 2.24.0.
So here is a patch to update it.
Comment 1 Olivier Cochard freebsd_committer 2020-11-16 09:13:43 UTC
Any news about this PR ?
Comment 2 Olivier Cochard freebsd_committer 2020-12-02 21:51:02 UTC
Any news?
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-08 05:50:51 UTC
This needs comprehensive QA, in particular for dependency compatibility and reverse dependents.
Comment 4 Charlie Li freebsd_committer 2020-12-26 23:40:26 UTC
We're up to 2.25.1 now.

The current version (upstream and in ports) of devel/py-cookiecutter needs at least 2.23.0 to run, as another consumer data point.
Comment 5 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 01:22:16 UTC
(In reply to Charlie Li from comment #4)

Currently wrestling with the test suite (which is crashing). Needed to complete QA
Comment 6 Charlie Li freebsd_committer 2020-12-27 01:51:29 UTC
(In reply to Kubilay Kocak from comment #5)
Same. Noticed that it was downloading dependencies on its own using Python tooling before any tests had a chance to crash.
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 01:54:28 UTC
(In reply to Charlie Li from comment #6)

setup.py test downloads deps (via tests_require), which was very convenienent, particularly for un-ported test dependencies (eg: requests wants pytest-httpbin, very old pinned version)

setup.py test is 'deprecated' (but pypa hasn't provided a canonical replacement that does the same)

pytest et al don't install dependencies defined in tests_require, which is annoying.

Would love help diagnosing/isolating the test crash issue (related to pytest --boxed multiprocessing)
Comment 8 Charlie Li freebsd_committer 2020-12-27 02:15:36 UTC
Their CI got moved to GitHub Actions, and it appears that they install dependencies and run pytest separately. Probably best to run it from their Makefile.
Comment 9 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 02:22:44 UTC
(In reply to Charlie Li from comment #8)

They run pytest tests, whereas I was running pytest without working dir. Will try.
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 02:25:03 UTC
(In reply to Kubilay Kocak from comment #9)

This is slightly better, but need to resolve (157 instances of the error):

  E       recursive dependency involving fixture 'httpbin' detected

Found one google result on this with no indicated specific resolution
Comment 11 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 03:23:45 UTC
Yakception:

pytest-httpbin (NEW PORT)
httpbin (NEW PORT)

 9 failed, 522 passed, 12 skipped, 1 xfailed, 14 warnings in 57.28 seconds

Several "requests.exceptions.InvalidSchema: Missing dependencies for SOCKS support" to resolve

Plus a few more.
Comment 12 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 03:27:23 UTC
1 failed, 530 passed, 12 skipped, 1 xfailed, 14 warnings in 61.34 seconds

____________ TestRequests.test_https_warnings _____________________________
self = <tests.test_requests.TestRequests object at 0x804ad9490>, httpbin_secure = <function prepare_url.<locals>.inner at 0x805d6c4c0>, httpbin_ca_bundle = None

    def test_https_warnings(self, httpbin_secure, httpbin_ca_bundle):
        """warnings are emitted with requests.get"""
        if HAS_MODERN_SSL or HAS_PYOPENSSL:
            warnings_expected = ('SubjectAltNameWarning', )
        else:
            warnings_expected = ('SNIMissingWarning',
                                 'InsecurePlatformWarning',
                                 'SubjectAltNameWarning', )

        with pytest.warns(None) as warning_records:
            warnings.simplefilter('always')
            requests.get(httpbin_secure('status', '200'),
                         verify=httpbin_ca_bundle)

        warning_records = [item for item in warning_records
                           if item.category.__name__ != 'ResourceWarning']

        warnings_category = tuple(
            item.category.__name__ for item in warning_records)
>       assert warnings_category == warnings_expected
E       AssertionError: assert () == ('SubjectAltNameWarning',)
E         Right contains one more item: 'SubjectAltNameWarning'
E         Full diff:
E         - ()
E         + ('SubjectAltNameWarning',)

tests/test_requests.py:857: AssertionError
Comment 13 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 03:32:17 UTC
(In reply to Kubilay Kocak from comment #12)

https://github.com/psf/requests/issues/5530

Will exclude the test.
Comment 14 Charlie Li freebsd_committer 2020-12-27 04:27:46 UTC
There may or may not be a bug with test_requests with current versions of httpbin and pytest-httpbin. Removing the SubjectAltNameWarning from the warnings_expected tuple allows the test to pass:

--- tests/test_requests.py.orig 2020-12-16 17:43:25 UTC
+++ tests/test_requests.py
@@ -838,7 +838,7 @@ class TestRequests:
     def test_https_warnings(self, httpbin_secure, httpbin_ca_bundle):
         """warnings are emitted with requests.get"""
         if HAS_MODERN_SSL or HAS_PYOPENSSL:
-            warnings_expected = ('SubjectAltNameWarning', )
+            warnings_expected = ()
         else:
             warnings_expected = ('SNIMissingWarning',
                                  'InsecurePlatformWarning',

531 passed, 12 skipped, 1 xfailed in 63.20 seconds
Comment 15 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 06:29:43 UTC
yakception [1]. I'm currently 4 levels deep, do not know when I'll come back from this, or how deep this rabbithole goes

[1] requests -> pytest-httpbin (new port) -> httpbin (new port) -> flasgger (new port) -> apispec-webframeworks (new port) | flask-jwt (new port) ...
Comment 16 Charlie Li freebsd_committer 2020-12-27 06:36:03 UTC
Apart from httpbin (bug 251847) and pytest-httpbin (bug 252187), the other new ports are not needed to run the tests.
Comment 17 Charlie Li freebsd_committer 2020-12-27 06:41:12 UTC
Created attachment 220983 [details]
2.25.1 v1

This one has all tests passing, with the slight adjustment included. Note that httpbin and pytest-httpbin are in TEST_DEPENDS.
Comment 18 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 07:21:16 UTC
(In reply to Charlie Li from comment #16)

Thanks Charlie. I should have made it clearer that I already have all of comment 15 ported so far
Comment 19 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-27 07:22:09 UTC
(In reply to Charlie Li from comment #16)

flasgger is required for httpbin to be completed correctly, it imports it at runtime
Comment 20 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-28 03:04:13 UTC
DONE: requests -> pytest-httpbin -> httpbin -> flasgger -> pispec-webframeworks | flask-jwt

latest requests test: 531 passed, 12 skipped, 1 xfailed, 14 warnings in 61.24 seconds

Pending reverse dependents test
Comment 21 wcarson.bugzilla 2021-03-25 06:10:41 UTC
As another point, with textproc/py-chardet updating to 4.0.0 today, www/py-requests now breaks upon startup. This prevented me from building devel/cmake (not exactly sure why it's a requirement, but alas). I updated the Makefile to use 2.25.1 with no patches and everything built successfully.
Comment 22 Matthew Seaman freebsd_committer 2021-05-15 14:11:21 UTC
Any progress on this?  Ran into this again with the latest update to devel/py-cookiecutter
Comment 23 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-16 02:16:53 UTC
(In reply to Matthew Seaman from comment #22)

dbaio and/or kai will be assisting getting this landed (i'm blocked on git migration currently) per https://wiki.freebsd.org/KubilayKocak#MAINTAINER_POLICY

Last QA update/test on my end was all necessary ports required for tests we're ported, to be landed (i'll provide diffs for these), and an initial consumer QA went OK, which needs to be rerun.
Comment 24 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-16 02:17:50 UTC
Request halp from dbaio if available
Comment 25 Danilo G. Baio freebsd_committer 2021-05-16 21:07:06 UTC
(In reply to Kubilay Kocak from comment #24)

I've just started a build of all consumers (309, queued 1931 ports), it will take some time, will let you know.
Comment 26 Danilo G. Baio freebsd_committer 2021-05-17 23:20:32 UTC
(In reply to Danilo G. Baio from comment #25)

still building, this is the only issue until now:

sysutils/py-hcloud
   py38-hcloud-1.12.0 depends on package: py38-requests>=2.20,<2.23 - not found

seems easy to fix, matthew@ is maintainer, could you take a look?
https://github.com/hetznercloud/hcloud-python/blob/v1.12.0/setup.py#L17
Comment 27 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-18 05:06:48 UTC
(In reply to Danilo G. Baio from comment #26)

The <version-spec> in the ports RUN_DEPENDS appear to be incorrect (bugs). Should check the rest and make sure they match the setup.py. Those changes are:

Approved by: portmgr (blanket: fix dependencies)
MFH: 2020Q2 (blanket: fix dependencies)
Comment 28 commit-hook freebsd_committer 2021-05-18 17:37:28 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=31ac02bd2729f5701ff471453720e90c9e5d3e48

commit 31ac02bd2729f5701ff471453720e90c9e5d3e48
Author:     Matthew Seaman <matthew@FreeBSD.org>
AuthorDate: 2021-05-18 17:34:03 +0000
Commit:     Matthew Seaman <matthew@FreeBSD.org>
CommitDate: 2021-05-18 17:37:10 +0000

    sysutils/py-hcloud -- fix dependency versions

    This removes one impediment to the pending update of www/py-requests

    PR:     250941

 sysutils/py-hcloud/Makefile | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
Comment 29 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-19 01:51:06 UTC
(In reply to commit-hook from comment #28)

Thanks Matthew. Can you MFH this bug fix, we may need to MFH the update too
Comment 30 Danilo G. Baio freebsd_committer 2021-05-19 03:03:15 UTC
(In reply to Danilo G. Baio from comment #26)

All consumers building fine now.  OK
Runtime tests with several ports. OK

make test: 
1 failed, 530 passed, 12 skipped, 1 xfailed in 59.57 seconds

...
tests/test_requests.py::TestRequests::test_pyopenssl_redirect attempting to ignore so the rest of the tests can run
FAILED     [ 22%]
...
/usr/local/lib/python3.8/http/client.py:276: ProtocolError

During handling of the above exception, another exception occurred:

self = <tests.test_requests.TestRequests object at 0x805e5ca30>
httpbin_secure = <function prepare_url.<locals>.inner at 0x8054dff70>
httpbin_ca_bundle = None

   ┆def test_pyopenssl_redirect(self, httpbin_secure, httpbin_ca_bundle):
>       requests.get(httpbin_secure('status', '301'), verify=httpbin_ca_bundle)

tests/test_requests.py:816:



Downgrade pytest_httpbin to 0.3.0 didn't solve.

Did you see this before? Or is it new test issue?
Comment 31 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-19 03:06:07 UTC
(In reply to Danilo G. Baio from comment #30)

I did not, but if you don't have the whole test stack that could be why.

I'll submit a diff for those ports shortly
Comment 32 commit-hook freebsd_committer 2021-05-19 06:03:37 UTC
A commit in branch 2021Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=a52944442ca77bf18f587e70ab7b31c5daaeb1d0

commit a52944442ca77bf18f587e70ab7b31c5daaeb1d0
Author:     Matthew Seaman <matthew@FreeBSD.org>
AuthorDate: 2021-05-18 17:34:03 +0000
Commit:     Matthew Seaman <matthew@FreeBSD.org>
CommitDate: 2021-05-19 06:01:51 +0000

    sysutils/py-hcloud -- fix dependency versions

    This removes one impediment to the pending update of www/py-requests

    PR:     250941
    (cherry picked from commit 31ac02bd2729f5701ff471453720e90c9e5d3e48)

 sysutils/py-hcloud/Makefile | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
Comment 33 Danilo G. Baio freebsd_committer 2021-05-19 22:53:03 UTC
More runtime tests OK, issue just in that test, I'll push this and wait a few days before merging it to quarterly.
Comment 34 commit-hook freebsd_committer 2021-05-19 23:26:20 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=7d02ece23c1a68e64e5220bea3c28275ff5152e4

commit 7d02ece23c1a68e64e5220bea3c28275ff5152e4
Author:     Charlie Li <vishwin@FreeBSD.org>
AuthorDate: 2021-05-19 23:05:25 +0000
Commit:     Danilo G. Baio <dbaio@FreeBSD.org>
CommitDate: 2021-05-19 23:17:55 +0000

    www/py-requests: Update to 2.25.1

    - Patches removed were incorporated upstream.
    - Update WWW with current project's information.

    Changelog:  https://github.com/psf/requests/blob/v2.25.1/HISTORY.md

    PR:             250941
    Approved by:    koobs (maintainer, implicit)
    MFH:            2021Q2

    Co-authored-by: Olivier Cochard <olivier@FreeBSD.org>
    Co-authored-by: Danilo G. Baio <dbaio@FreeBSD.org>

 www/py-requests/Makefile                           | 14 +++++++----
 www/py-requests/distinfo                           |  6 ++---
 www/py-requests/files/patch-setup.py (gone)        | 29 ----------------------
 .../files/patch-tests_test__requests.py (new)      | 12 +++++++++
 .../files/patch-tests_test__utils.py (gone)        | 17 -------------
 www/py-requests/pkg-descr                          | 15 ++++++-----
 6 files changed, 33 insertions(+), 60 deletions(-)
Comment 35 commit-hook freebsd_committer 2021-05-22 09:59:08 UTC
A commit in branch 2021Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=d3a2475bbfbd9f8ff6600dafdbca745370f2f40c

commit d3a2475bbfbd9f8ff6600dafdbca745370f2f40c
Author:     Charlie Li <vishwin@FreeBSD.org>
AuthorDate: 2021-05-19 23:05:25 +0000
Commit:     Danilo G. Baio <dbaio@FreeBSD.org>
CommitDate: 2021-05-22 09:56:11 +0000

    www/py-requests: Update to 2.25.1

    - Patches removed were incorporated upstream.
    - Update WWW with current project's information.

    Changelog:  https://github.com/psf/requests/blob/v2.25.1/HISTORY.md

    PR:             250941
    Approved by:    koobs (maintainer, implicit)

    Co-authored-by: Olivier Cochard <olivier@FreeBSD.org>
    Co-authored-by: Danilo G. Baio <dbaio@FreeBSD.org>
    (cherry picked from commit 7d02ece23c1a68e64e5220bea3c28275ff5152e4)

 www/py-requests/Makefile                           | 14 +++++++----
 www/py-requests/distinfo                           |  6 ++---
 www/py-requests/files/patch-setup.py (gone)        | 29 ----------------------
 .../files/patch-tests_test__requests.py (new)      | 12 +++++++++
 .../files/patch-tests_test__utils.py (gone)        | 17 -------------
 www/py-requests/pkg-descr                          | 15 ++++++-----
 6 files changed, 33 insertions(+), 60 deletions(-)
Comment 36 Danilo G. Baio freebsd_committer 2021-05-22 10:00:42 UTC
Committed, thank you all!