Bug 220469 - security/py-certbot: Relax Python version (-2.7) restriction (Supports Python 3.x)
Summary: security/py-certbot: Relax Python version (-2.7) restriction (Supports Python...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Kubilay Kocak
URL:
Keywords: easy
Depends on:
Blocks:
 
Reported: 2017-07-04 15:03 UTC by Kamigishi Rei
Modified: 2017-08-20 08:11 UTC (History)
5 users (show)

See Also:
koobs: maintainer-feedback+
koobs: merge-quarterly+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kamigishi Rei 2017-07-04 15:03:36 UTC
certbot is compatible with Python 3 nowadays. The port build fails with poudriere if the default Python interpreter is set to 3.x, however replacing

USES= python:-2.7

with

USES= python

results in a successful package build (tested with Python 3.6).

The resulting package works as advertised and the script properly contacts Letsencrypt and updates certificates.

I suggest to remove this direct dependency on Python 2.7 as it does not seem like it is needed; maybe make it work with either 2.7 or 3.1+?
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-05 12:25:07 UTC
Thank you for the report Kamigishi.

This needs testing with other 3.x versions, and if broken with any, should be conditionally (.if ${PYTHON_REL = XXX}) marked as BROKEN

If the quarterly version of the port also works with Python 3.x, the change should be MFH'd.

@Carlos Do you have time to look at this?
Comment 2 Carlos J. Puga Medina freebsd_committer 2017-07-05 20:44:28 UTC
(In reply to Kubilay Kocak from comment #1)

Not all dependencies have a corresponding python 3.x version.

Don't you think we should wait until the python flavors framework been committed?

https://reviews.freebsd.org/D10327
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-07 04:14:29 UTC
(In reply to Carlos J. Puga Medina from comment #2)

Thanks for checking. How many are actually missing? If it's a relatively small number it might be worth doing as there is no concrete ETA for ports variants (flavors), and there is nothing guaranteeing that it will be immediately or completely usable in all contexts/use-cases either.

Further, py3-* port versions (and their dependencies) are only required for those ports that we want to create an official (Python 3.x) package for (on the official package building cluster)

Ports users on the other hand can just use DEFAULT_VERSIONS and the respective version of each port can and will be built with that version (if its allowed by USES=python:<version-spec>). This has been the case for a long time.
Comment 4 Kamigishi Rei 2017-07-07 10:45:24 UTC
(In reply to Kubilay Kocak from comment #1)

Support for 3.3+ was added in 0.14.0:

---
0.14.0 - 2017-05-04

Added

Python 3.3+ support for all Certbot packages. certbot-auto still currently only supports Python 2, but the acme, certbot, certbot-apache, and certbot-nginx packages on PyPI now fully support Python 2.6, 2.7, and 3.3+.
---

FreeBSD ports do not seem to rely on certbot-auto.

Sadly I am not able to test it with Python 3.3, 3.4, and 3.5 at the moment, but I'll look into it. Poudriere successfully builds packages for all dependencies with Python 3.6.
Comment 5 Carlos J. Puga Medina freebsd_committer 2017-07-07 11:26:08 UTC
(In reply to Kubilay Kocak from comment #3)

We need to create Python 3.x version of the following dependencies:

devel/py-pyrfc3339      # py-acme dependency
devel/py-configargparse # py-certbot dependency
devel/py-configobj      # py-certbot dependency
devel/py-parsedatetime  # py-certbot dependency
devel/py-zope.component # py-certbot dependency
devel/py-zope.interface # py-certbot dependency

Apart from creating the missing slave ports to support Python 3, there is no major problem :)
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-10 09:41:33 UTC
(In reply to Carlos J. Puga Medina from comment #5)

Using poudriere-devel-3.1.99.20170601_1 I see a successful build with Python 3.5 using DEFAULT_VERSIONS=python=3.5 ...

===>  Installing for py35-certbot-0.15.0_2,1
===>   py35-certbot-0.15.0_2,1 depends on package: py35-acme>=0.15.0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-configargparse>=0.9.3 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-configobj>0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-cryptography>=1.2 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-parsedatetime>=1.3 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-openssl>0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-pyrfc3339>0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-pytz>0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-setuptools>=1.0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-six>0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-zope.component>0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-zope.interface>0 - found
===>   py35-certbot-0.15.0_2,1 depends on package: py35-setuptools>0 - found
===>   py35-certbot-0.15.0_2,1 depends on file: /usr/local/bin/python3.5 - found
===>  Checking if py35-certbot already installed
===>   Registering installation for py35-certbot-0.15.0_2,1
[110amd64-koobs-py35] Installing py35-certbot-0.15.0_2,1...

This could be related to recent changes in poudriere that allow this to occur without py3- variations (Bryan cc'd). 

Can you confirm?

That aside, it has always been possible (without poudriere, using ports alone), to switch the default version of python and build any port and its dependencies with that version.

Ultimately, whether or not all dependencies are/can be satisfied, every port should as accurately as possible, declare which versions of Python is supports. From this perspective, certbot is currently incorrect/inaccurate.
Comment 7 Bryan Drewery freebsd_committer 2017-07-10 17:45:56 UTC
(In reply to Carlos J. Puga Medina from comment #2)
> (In reply to Kubilay Kocak from comment #1)
> 
> Not all dependencies have a corresponding python 3.x version.
> 
> Don't you think we should wait until the python flavors framework been
> committed?
> 
> https://reviews.freebsd.org/D10327

Poudriere-devel (with default python version to 2.x) will properly build
python ports with forced version 3 (like USES=python:3) and their dependencies
properly.  FLAVORS isn't required for it.  However this bug sounds like you are
changing the default to 3.  There should be little or no behavior change in
Poudriere-devel for that case.  It seems it works fine in comment #6.
Comment 8 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-11 05:16:19 UTC
(In reply to Bryan Drewery from comment #7)

Thanks Bryan.

This change is to relax (not change the default of) the 'ports' version requirement to USES=python, allowing all versions (specified by user), but yes, the test poudriere run in comment 6 uses -zpy36 which overrides DEFAULT_VERSIONS to python=3.5

The poudriere-devel version I'm using is 20170601, and limited pre-variants py3-* port support was added in 20170621, so that can't explain why it works.
Comment 9 Bryan Drewery freebsd_committer 2017-07-11 19:37:22 UTC
(In reply to Kubilay Kocak from comment #8)
> (In reply to Bryan Drewery from comment #7)
> 
> The poudriere-devel version I'm using is 20170601, and limited pre-variants
> py3-* port support was added in 20170621, so that can't explain why it works.

Just to be clear, the FLAVORS / DEPENDS_ARGS support in the newer Poudriere
is limited to default python=2.  If the default is changed to 3 then the logic
is disabled and it behaves just like it did before.
Comment 10 Alexander Ushakov 2017-07-24 13:13:01 UTC
I can confirm that security/py-certbot works fine with python 3.6 after removing version bind in Makefile as topic starter have suggested. Python 3.6 is currently the only python in my system.

All dependencies were resolved correctly:
py36-acme-0.16.0,1             ACME protocol implementation in Python                  
py36-certbot-0.16.0,1          Let's Encrypt client                                    
py36-certifi-2017.4.17         Mozilla SSL certificates                                
py36-cffi-1.7.0                Foreign Function Interface for Python calling C code    
py36-chardet-3.0.4             Universal encoding detector for Python 2 and 3          
py36-configargparse-0.12.0     Drop-in replacement for argparse                        
py36-configobj-5.0.6_1         Simple but powerful config file reader and writer       
py36-cryptography-1.7.2        Cryptographic recipes and primitives for Python developers                                                                                      
py36-idna-2.5                  Internationalized Domain Names in Applications (IDNA)   
py36-openssl-16.2.0            Python interface to the OpenSSL library                 
py36-parsedatetime-2.1         Python module for parsing 'human readable' date/time expressions                                                                                
py36-pyasn1-0.2.2              ASN.1 toolkit for Python                                
py36-pycparser-2.10            C parser in Python                                      
py36-pyrfc3339-1.0             Generate and parse RFC 3339 timestamps                  
py36-pysocks-1.6.7             Python SOCKS module                                     
py36-pytest-runner-2.11.1      Test support for pytest runner in setup.py              
py36-pytz-2016.10,1            World Timezone Definitions for Python                   
py36-requests-2.18.1_1         HTTP library written in Python for human beings         
py36-setuptools-36.0.1         Python packages installer                               
py36-setuptools_scm-1.15.5     Setuptools plugin to manage your versions by scm tags   
py36-six-1.10.0                Python 2 and 3 compatibility utilities                  
py36-urllib3-1.22              HTTP library with thread-safe connection pooling, file post, and more                                                                           
py36-zope.component-4.2.2      Zope Component Architecture                             
py36-zope.event-4.1.0          Very basic event publishing system                      
py36-zope.interface-4.1.3      Interfaces for Python
Comment 11 Dmitry Marakasov freebsd_committer 2017-07-31 13:16:03 UTC
I also use it in production (with python version limitation removed) on python3-only box. Let's please commit this.
Comment 12 commit-hook freebsd_committer 2017-08-06 11:29:15 UTC
A commit references this bug:

Author: koobs
Date: Sun Aug  6 11:28:15 UTC 2017
New revision: 447458
URL: https://svnweb.freebsd.org/changeset/ports/447458

Log:
  security/py-certbot: Relax Python version-spec, support Python 3.x

  certbot has supported Python 3 (3.3+) since the 0.14.0 release [1]. Update
  the USES=python:<version-spec> to match, allowing Python 3 builds. [2]

  Update acme (security/py-acme) RUN_DEPENDS entry to use PORTVERSION not
  PKGVERSION (that includes PORTEPOCH/PORTREVISION) which caused the
  dependency to be incorrectly reported as unsatisfied if either was
  defined. Further, since *_DEPENDS version-specifiers compare against
  installed package versions, only a version that includes PORTEPOCH
  and/or PORTREVISION is available, so use >= not ==. [3]

  While I'm here, enable 'concurrent' to automatically produce
  version-suffixed files.

  [1] https://github.com/certbot/certbot/issues/4507

  PR:		220469 221043 [3]
  Submitted by:	Kamigishi Rei <spambox haruhiism net> [2]
  Reported by:	amdmi3 [3]
  MFH:		2017Q3

Changes:
  head/security/py-certbot/Makefile
Comment 13 commit-hook freebsd_committer 2017-08-17 13:05:08 UTC
A commit references this bug:

Author: feld
Date: Thu Aug 17 13:04:12 UTC 2017
New revision: 448093
URL: https://svnweb.freebsd.org/changeset/ports/448093

Log:
  MFH: r445423 r447458

  security/py-{acme,certbot}: Update to 0.16.0

  Common:

  - Update PORTVERSION and distinfo checksum (0.16.0)

  Changelog: https://github.com/certbot/certbot/blob/master/CHANGELOG.md#0160---2017-07-05

  Reviewed by:	koobs (maintainer, py-certbot)
  Approved by:	koobs (maintainer, py-certbot)
  Differential Revision:	https://reviews.freebsd.org/D11517

  security/py-certbot: Relax Python version-spec, support Python 3.x

  certbot has supported Python 3 (3.3+) since the 0.14.0 release [1]. Update
  the USES=python:<version-spec> to match, allowing Python 3 builds. [2]

  Update acme (security/py-acme) RUN_DEPENDS entry to use PORTVERSION not
  PKGVERSION (that includes PORTEPOCH/PORTREVISION) which caused the
  dependency to be incorrectly reported as unsatisfied if either was
  defined. Further, since *_DEPENDS version-specifiers compare against
  installed package versions, only a version that includes PORTEPOCH
  and/or PORTREVISION is available, so use >= not ==. [3]

  While I'm here, enable 'concurrent' to automatically produce
  version-suffixed files.

  [1] https://github.com/certbot/certbot/issues/4507

  PR:		220469 221043 [3]
  Submitted by:	Kamigishi Rei <spambox haruhiism net> [2]
  Reported by:	amdmi3 [3]

  Approved by:	ports-secteam (with hat)

Changes:
_U  branches/2017Q3/
  branches/2017Q3/security/py-acme/Makefile
  branches/2017Q3/security/py-acme/distinfo
  branches/2017Q3/security/py-certbot/Makefile
  branches/2017Q3/security/py-certbot/distinfo