Bug 220927

Summary: www/py-requests: Latest net/py-urllib3 update breaks version requirements, causing py-certbot / py-salt to fail to run
Product: Ports & Packages Reporter: P. Chevalier <chevalier>
Component: Individual Port(s)Assignee: Carlos J. Puga Medina <cpm>
Status: Closed FIXED    
Severity: Affects Many People CC: christer.edwards, cpm, jbeich, koobs, paul, python
Priority: --- Flags: koobs: maintainer-feedback+
Version: Latest   
Hardware: Any   
OS: Any   
URL: https://github.com/requests/requests/commit/8d1228cd1943bfe89ffaae32e15b2feb3f8479f5
Attachments:
Description Flags
patch
none
patch
none
patch allow urllib3 1.22
none
patch allow urllib3 1.22
koobs: maintainer-approval+
patch allow urllib3 1.22 koobs: maintainer-approval+

Description P. Chevalier 2017-07-22 22:33:16 UTC
upgrade of port py27-urllib3-1.21.1 to py27-urllib3-1.22 breaks py27-certbot-0.16.0,1

when launching the certbot utility :

Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3037, in <module>
    @_call_aside
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3021, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3050, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 657, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 670, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 855, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'urllib3<1.22,>=1.21.1' distribution was not found and is required by requests


Downgrading back to 1.21.1 makes it work again.
Comment 1 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-22 23:22:16 UTC
Created attachment 184605 [details]
patch

py-requests port needs to update py-urllib3 version required to 1.22
Comment 2 Paul Mather 2017-07-23 01:18:25 UTC
This also breaks py27-salt-2016.11.6_1:

# service salt_master start
Starting salt_master.
Traceback (most recent call last):
  File "/usr/local/bin/salt-master", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3037, in <module>
    @_call_aside
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3021, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3050, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 657, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 670, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 855, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'urllib3<1.22,>=1.21.1' distribution was not found and is required by requests
/usr/local/etc/rc.d/salt_master: WARNING: failed to start salt_master
Comment 3 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 01:22:43 UTC
(In reply to paul from comment #2)

Have you rebuilt www/py-requests?
Comment 4 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 01:30:02 UTC
Created attachment 184609 [details]
patch

I forgot to bump PORTREVISION.
Comment 5 Paul Mather 2017-07-23 03:10:52 UTC
(In reply to Carlos J. Puga Medina from comment #3)

Yes, I actually rebuilt all packages via Poudriere and force reinstalled all packages via "pkg upgrade -f".
Comment 6 Jan Beich freebsd_committer freebsd_triage 2017-07-23 05:15:20 UTC
Comment on attachment 184609 [details]
patch

This won't help. The version that ends up in egginfo comes from setup.py.
Comment 7 Jan Beich freebsd_committer freebsd_triage 2017-07-23 05:16:44 UTC
*** Bug 220934 has been marked as a duplicate of this bug. ***
Comment 8 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 07:29:39 UTC
Created attachment 184618 [details]
patch allow urllib3 1.22
Comment 9 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 07:30:06 UTC
(In reply to Jan Beich from comment #6)

Good catch, Jan ;)
Comment 10 Jan Beich freebsd_committer freebsd_triage 2017-07-23 07:38:04 UTC
Comment on attachment 184618 [details]
patch allow urllib3 1.22

> -		${PYTHON_PKGNAMEPREFIX}urllib3>=1.21.1:net/py-urllib3 \
> +		${PYTHON_PKGNAMEPREFIX}urllib3>=1.22:net/py-urllib3 \

Looking at setup.py changes they don't preclude from using the port with py-urllib3-1.22.1. Maybe drop the hunk.
Comment 11 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 07:43:54 UTC
Created attachment 184621 [details]
patch allow urllib3 1.22
Comment 12 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 07:44:38 UTC
(In reply to Jan Beich from comment #10)

Yes, now it's fixed.
Comment 13 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:02:12 UTC
@Carlos please use the comment header from https://github.com/requests/requests/commit/8d1228cd1943bfe89ffaae32e15b2feb3f8479f5.patch in the port patch
Comment 14 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:03:06 UTC
Comment on attachment 184621 [details]
patch allow urllib3 1.22

Approved (with comment added to patch)
Comment 15 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:03:54 UTC
CC py-salt maintainer
Comment 16 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 08:09:45 UTC
Created attachment 184622 [details]
patch allow urllib3 1.22
Comment 17 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:10:54 UTC
Comment on attachment 184622 [details]
patch allow urllib3 1.22

Approved. Thank you Carlos
Comment 18 Jan Beich freebsd_committer freebsd_triage 2017-07-23 08:13:25 UTC
Comment on attachment 184622 [details]
patch allow urllib3 1.22

This is confusing. files/patch-setup.py already has a header for an unrelated change. Maybe better treat backports as external patches e.g.,

PATCH_SITES=	https://github.com/${PORTNAME}/${PORTNAME}/commit/
PATCHFILES=	8d1228cd1943.patch:-p1
Comment 19 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:31:34 UTC
It also makes attachment 184622 [details] incorrect (doesn't preserve existing hunk & comment as far as i can tell)
Comment 20 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 08:37:45 UTC
I'll add a comment to clarify the change:

Backport an upstream patch to setup.py allowing urllib3 1.22
dependency.
Comment 21 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:43:20 UTC
(In reply to Jan Beich from comment #18)

I don't mind that method and it has the same issues with multiple patch-files patching common files, patch ordering, etc so no real trade-off is obvious.

The intent of my comment requirements is that patches are documented and referenced (to the extent that they can be), and reported upstream (where they can be) and so that our future selves don't need to waste time hunting for the 'why' behind the what.

Both methods (comment in patch) or PATCHFILES is approved as long as the existing patches/comments are maintained and the patches are tested
Comment 22 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:43:39 UTC
Comment on attachment 184622 [details]
patch allow urllib3 1.22

Doesn't maintain existing hunk & comment
Comment 23 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:44:39 UTC
Comment on attachment 184622 [details]
patch allow urllib3 1.22

my mistake, it is.
Comment 24 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 08:44:50 UTC
My mistake, they are maintained.
Comment 25 commit-hook freebsd_committer freebsd_triage 2017-07-23 09:20:22 UTC
A commit references this bug:

Author: cpm
Date: Sun Jul 23 09:20:08 UTC 2017
New revision: 446461
URL: https://svnweb.freebsd.org/changeset/ports/446461

Log:
  www/py-requests: Allow urllib3 1.22 dependency

  - Backport an upstream patch to setup.py allowing urllib3 1.22 dependency
  - Bump PORTREVISION

  PR:		220927
  Submitted by:	cpm
  Reported by:	P. Chevalier<chevalier@dspnet.fr>
  Reviewed by:	jbeich, koobs
  Approved by:	koobs (maintainer)
  Obtained from:	https://github.com/requests/requests/commit/8d1228cd1943bfe89ffaae32e15b2feb3f8479f5

Changes:
  head/www/py-requests/Makefile
  head/www/py-requests/files/patch-setup.py
Comment 26 Carlos J. Puga Medina freebsd_committer freebsd_triage 2017-07-23 09:22:18 UTC
Committed!

Thanks
Comment 27 Paul Mather 2017-07-23 14:37:13 UTC
(In reply to Carlos J. Puga Medina from comment #26)

This fixes py-salt for me.  Thank you.