Bug 203405

Summary: [NEW PORT] security/py-letsencrypt: Let's Encrypt client
Product: Ports & Packages Reporter: Kurt Jaeger <pi>
Component: Individual Port(s)Assignee: Kubilay Kocak <koobs>
Status: Closed FIXED    
Severity: Affects Only Me CC: bas, brnrd, cjpm, conall, emaste, feld, jakub.warmuz+bugs.freebsd.org, koobs, nudelfabrik, pi, rigoletto, ruud, tabthorpe, tremere
Priority: --- Keywords: feature
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204303
Bug Depends on: 203819, 204303    
Bug Blocks:    
Attachments:
Description Flags
shar-v1
none
shar-v2
none
apache
none
nginx
none
letshelp
none
shar-v3
none
Shar for security/letsencrypt
none
py-letsencrypt
none
py-letsencrypt w/ pkg-message
none
py-letsencrypt w/ pkg-message v2 koobs: maintainer-approval+

Description Kurt Jaeger freebsd_committer freebsd_triage 2015-09-28 05:33:11 UTC
Created attachment 161470 [details]
shar-v1

builds on 11a, 10.2i+a, 9.3, needs py-acme.
Comment 1 Kurt Jaeger freebsd_committer freebsd_triage 2015-09-28 05:34:32 UTC
shar mostly from cpm@fbsd.es.
Comment 2 Carlos J Puga Medina 2015-10-01 10:04:30 UTC
@kuba pointed me the right direction.

https://github.com/letsencrypt/letsencrypt/issues/792#issuecomment-143864388
Comment 3 Carlos J Puga Medina 2015-10-01 10:16:32 UTC
Kurt, I tested the letsencrypt port :)

It works fine with the recently added py-acme dependency, but it shows the following message

"Configurator could not be determined"

Any thoughts?
Comment 4 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-01 11:44:31 UTC
(In reply to Carlos J Puga Medina from comment #3)
In the WRKSRC, there are

drwxrwxr-x  3 root  wheel  5 Sep 18 07:26 letsencrypt-apache
drwxrwxr-x  3 root  wheel  5 Sep 18 07:26 letsencrypt-compatibility-test
drwxrwxr-x  4 root  wheel  6 Sep 18 07:26 letsencrypt-nginx

and we probably need to install those as well ? Do we have to do those
as seperate ports, similar to py-acme ?
Comment 5 Carlos J Puga Medina 2015-10-01 11:49:20 UTC
(In reply to Kurt Jaeger from comment #4)

Yes. Those ports are necessary as well.
Comment 6 Carlos J Puga Medina 2015-10-01 11:57:03 UTC
Once created the 3 pending ports, we just have to add them in the Makefile.

E.g. using these categories

www/letsencrypt-apache
security/letsencrypt-compatibility-test
www/letsencrypt-nginx

Per documentation are only necessary letsencrypt-apache and letsencrypy-nginx 

https://github.com/letsencrypt/letsencrypt/blob/master/readthedocs.org.requirements.txt
Comment 7 Jakub Warmuz 2015-10-03 15:57:43 UTC
Hey, Let's Encrypt dev here :)

Generally, https://github.com/letsencrypt/letsencrypt/blob/master/readthedocs.org.requirements.txt is not the source of truth, and in fact it should be updated to include more packages. Things that should be packaged:
- acme
- letsencrypt
- letsencrypt-apache (should require Apache2 server)
- letsencrypt-nginx (should require Nginx server)
- letshelp-letsencrypt (this allows end-users to send debug data to the dev team)

letsencrypt-compatibility-test is only useful for the devs and it shouldn't be packaged for end users.

Lovely to see packaging efforts in FreeBSD! Let me know if you need any help.
Comment 8 Carlos J Puga Medina 2015-10-05 16:25:23 UTC
(In reply to Jakub Warmuz from comment #7)

Hi Jakub,

We are working now to have a complete letsencrypt port. Hope you can help us to solve any question related.

Only the letsencrypt port has been landed.
Comment 9 commit-hook freebsd_committer freebsd_triage 2015-10-13 19:12:27 UTC
A commit references this bug:

Author: pi
Date: Tue Oct 13 19:12:18 UTC 2015
New revision: 399211
URL: https://svnweb.freebsd.org/changeset/ports/399211

Log:
  security/py-acme: 0.0.b1 -> 0.0.0.dev20151008

  Changes: For now, see
  	https://github.com/letsencrypt/letsencrypt/commits/master

  PR:		203405

Changes:
  head/security/py-acme/Makefile
  head/security/py-acme/distinfo
Comment 10 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-13 19:43:12 UTC
Created attachment 161995 [details]
shar-v2

update for letsencrypt itself
Comment 11 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-13 19:43:46 UTC
Created attachment 161996 [details]
apache

letsencrypt-apache
Comment 12 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-13 19:44:12 UTC
Created attachment 161997 [details]
nginx

nginx
Comment 13 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-13 19:44:38 UTC
Created attachment 161998 [details]
letshelp

letshelp-letsencrypt
Comment 14 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-13 19:46:19 UTC
If someone wants to use le, le-<webserver> has to be installed. It installs
the dependency le itself, and the <webserver>, and that's it ?

Now, we need more tests on this. I'm sure it will not use the right
path to the config files right now.
Comment 15 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-13 20:06:06 UTC
even worse, the version string causes errors like this:

pkg_resources.DistributionNotFound: The 'acme==0.0.0.dev20151008' distribution was not found and is required by letsencrypt
Comment 16 Kurt Jaeger freebsd_committer freebsd_triage 2015-10-14 03:44:26 UTC
upstream issue:

https://github.com/letsencrypt/letsencrypt/issues/972
Comment 17 Ralf van der Enden 2015-10-16 14:42:08 UTC
Even though cryptography 0.8.2 is listed as the minimal version needed to run letsencrypt, this version does not support DER encoded certificates. DER support was added in version 0.9.

Therefore I just submitted PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203819 to update py-cryptography to 1.0.2 (which is the latest available version)
Comment 18 Jakub Warmuz 2015-10-17 11:53:21 UTC
Ralf, nice finding, thanks! I've filed https://github.com/letsencrypt/letsencrypt/issues/1012 upstream.
Comment 19 Carlos J Puga Medina 2015-10-19 23:03:26 UTC
Created attachment 162220 [details]
shar-v3

New release available:

https://github.com/letsencrypt/letsencrypt/releases
Comment 20 Mark Felder freebsd_committer freebsd_triage 2015-10-26 15:45:23 UTC
If we want the quarterly pkg users (FreeBSD 10.1+ by default) to have this when it lands we will need to MFH these new ports and the security/py-acme update, correct?
Comment 21 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-05 03:44:33 UTC
@Kurt I have the python client port sorted (security/py-letsencrypt) and ready to land (requires a py-acme update to match versions)

Hit me up on IRC so we can all get on the same page
Comment 22 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-05 03:47:37 UTC
(In reply to Kurt Jaeger from comment #15)

First, py-acme needs to use DISTVERSION (the current PORTVERSION is illegal)

Second, our *_DEPENDS package version matching routines don't detect that x.y.z.d<datestamp > <someversionstring>. I haven't been able to isolate it completely, but there might be a first-three-tokens-only assumption in there somewhere.

I've worked around this in the py-letsencrypt (client) port by using a file reference:

${PYTHON_SITELIBDIR}/acme/__init__.py:${PORTSDIR}/security/py-acme
Comment 23 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-05 03:48:38 UTC
(In reply to Mark Felder from comment #20)

Quarterly users don't get new ports nor there dependencies (including any updates to those deps) right?
Comment 24 Kurt Jaeger freebsd_committer freebsd_triage 2015-11-05 06:15:11 UTC
Please work on it, I'll still be busy for the next few days.
Comment 25 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-05 06:39:32 UTC
@Kurt How did I inherit the main port? :D

In case it wasn't obvious in my original reply, i have the python "client" ready, and I'd like to land that on it's own. I believe this is possible, and unrelated to these ports?

From the looks of this issue this contains many other (server, etc) components, is that correct?
Comment 26 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-05 08:57:38 UTC
Having created a port for what I originally believed was *only* the 'python client' component of letsencrypt, it turns out that the packaging approachfor letsencrypt is unsuitable.

That is to say, the current method that upstream uses to bundle all the things, bootstrap and create an isolated virtual environment for all of the python packages (letsencrypt-*) are unnecessary, and beyond that, unsuitable for packaging.

Specifically, a port depending on other python ports (themselves obtained from PyPI or elsewhere), must reference (read: import/use) those Python packages from the system Python path. More precisely, depending on python packages, and installing them anywhere other than the system python path (site-packages) means that pkg/portmaster et all cannot manage/upgrade them on an ongoing basis.

From my now more complete understanding having gone through the documentation, repositories and the install process for the standalone client (ie; without webserver plugins), I believe the way this should be approached is as follows:

security/py-letsencrypt (the "standalone" python client)
security/py-letsencrypt-* (the webserver plugins, and other components)

The standlone client, displays the following if the particular (apache,nginx, etc) plugins are not available:

Choice of server plugins for obtaining and installing cert:

  (the apache plugin is not installed)
  --standalone      Run a standalone webserver for authentication
  (nginx support is experimental, buggy, and not installed by default)

This is a good thing in terms of only supporting the commands/functions if the requisite modules are installed and importable.

Having already completed and tested successfully authenticating and obtaining a certificate from the production letsencrypt server, using the py-letsencrypt client, what's left is to package the webserver plugins separately.

These can then be added as NGINX/APACHE options to the py-letsencrypt port/package.

These patches/ports thus need to be updated to match Python port/package conventions, in particular the py- prefix, and switched to using the PyPI sdists (releases) as noted/recommended upstream.

I intend to land security/py-letsencrypt as soon as possible under my maintainership, as the port I have is fundamentally a new/different port than is currently proposed here, and much much simpler.
Comment 27 Bernard Spil freebsd_committer freebsd_triage 2015-11-05 19:44:44 UTC
Created attachment 162821 [details]
Shar for security/letsencrypt

Working on an update for the shar-file
 * Update to 0.0.0dev20151104
 * Follow rename of py-parsing to py-pyparsing
 * Fix paths for /etc and /var/lib

Seems functional here
Comment 28 Bernard Spil freebsd_committer freebsd_triage 2015-11-05 19:47:38 UTC
Well almost functional... `letsencrypt certonly` and `auth` work, letsencrypt plain fails on installer problems
"No installers are available on your OS yet; try running "letsencrypt-auto certonly" to get a cert you can install manually"

Should upstream some patch for that, needs investigation.
Comment 29 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-06 03:01:12 UTC
Created attachment 162828 [details]
py-letsencrypt
Comment 30 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-06 03:01:51 UTC
Patches for the apache, nginx letshelp components need updating
Comment 31 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-06 03:10:44 UTC
Pending approval for the py-acme update in in bug 204303. That is a blocker.
Comment 32 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-06 05:03:58 UTC
Created attachment 162830 [details]
py-letsencrypt w/ pkg-message
Comment 33 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-06 05:20:44 UTC
Created attachment 162831 [details]
py-letsencrypt w/ pkg-message v2
Comment 34 Bas Vermin 2015-11-06 07:53:16 UTC
Kubilay,

I just tested ur latest patches for py-letsencrypt and py-acme.

They build and work without any problems, was able to generate multiple certificates.
Comment 35 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-06 08:01:03 UTC
(In reply to Bas Vermin from comment #34)

Thank you for testing and for reporting back Bas! :D
Comment 36 Carlos J Puga Medina 2015-11-06 08:23:28 UTC
(In reply to Kubilay Kocak from comment #31)

Please, go ahead, Kubilay :)
Comment 37 Kubilay Kocak freebsd_committer freebsd_triage 2015-11-06 08:27:57 UTC
(In reply to Carlos J Puga Medina from comment #36)

LETS ROCK N ROLL
Comment 38 commit-hook freebsd_committer freebsd_triage 2015-11-06 08:39:19 UTC
A commit references this bug:

Author: koobs
Date: Fri Nov  6 08:38:35 UTC 2015
New revision: 400885
URL: https://svnweb.freebsd.org/changeset/ports/400885

Log:
  [NEW] security/py-letsencrypt: Welcome Let's Encrypt client!

  In short: getting and installing SSL/TLS certificates made easy.

  The Let's Encrypt Client is a tool to automatically receive and install
  X.509 certificates to enable TLS on servers. The client will
  interoperate with the Let's Encrypt CA which will be issuing
  browser-trusted certificates for free.

  It's all automated:

  The tool will prove domain control to the CA and submit a CSR
  (Certificate Signing Request).

  If domain control has been proven, a certificate will get issued and
  the tool will automatically install it.

  WWW: https://github.com/letsencrypt/letsencrypt

  PR:		203405

Changes:
  head/security/Makefile
  head/security/py-letsencrypt/
  head/security/py-letsencrypt/Makefile
  head/security/py-letsencrypt/distinfo
  head/security/py-letsencrypt/files/
  head/security/py-letsencrypt/files/patch-setup.py
  head/security/py-letsencrypt/pkg-descr
  head/security/py-letsencrypt/pkg-message
Comment 39 Kubilay Kocak freebsd_committer freebsd_triage 2015-12-26 08:10:11 UTC
This issue needs patches for the two (apache/nginx) plugins to progress
Comment 40 Kubilay Kocak freebsd_committer freebsd_triage 2015-12-26 08:11:32 UTC
Comment on attachment 162831 [details]
py-letsencrypt w/ pkg-message v2

The base client (py-letsencrypt) is DONE
Comment 41 Kubilay Kocak freebsd_committer freebsd_triage 2015-12-26 08:11:52 UTC
Re-open to the pool
Comment 42 Kurt Jaeger freebsd_committer freebsd_triage 2016-03-04 11:52:59 UTC
letsencrypt is now in the tree, so this can be closed.
Comment 43 Kubilay Kocak freebsd_committer freebsd_triage 2017-04-23 03:46:44 UTC
Update issue so that it's clear what the resolution was.

* Original submission was for letsencrypt *and* nginx, apache plugins.
* Update summary to reflect that only py-letsencrypt was committed/added. 
* Assign to myself as I committed the original security/py-letsencrypt port based on this issue.