Created attachment 195946 [details] py-certifi use CAs from ca_root_nss The proposed patch adds option to use certificate bundle from security/ca_root_nss instead of one shipped with certifi. The idea behind this patch is to add ability to trust to some extra local CAs. Such functionality is going to be added to ca_root_nss soon (I hope): https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=160387 I think it would be convenient to use trusted certificates from single source. --- QA: poudriere testport with option ON and OFF builds fine The behavior doesn't change with option OFF. With option ON the behavior is as expected: >>> import certifi >>> certifi.where() '/usr/local/etc/ssl/cert.pem'
While the functional changes itself appear OK (except for hardcoding /usr/local), given the certifi project describes itself "Certifi is a carefully curated collection of Root Certificates", and further appears to lean against the addition of addition certs [1], I'm hesitant to modify the default provided certificate bundle, for POLA and matching documentation reasons, both related to user experience. Yes, in this case the patch includes it only as an OPTION, but I think this feature may ultimately be better served as an upstream issue/pull request, similar to this request for extracting OSX trust roots [2]. There is an additional benefit here of having FreeBSD support added to an upstream project, presumably also in the documentation as such. [1] https://github.com/certifi/python-certifi/issues/72 [2] https://github.com/certifi/python-certifi/issues/25
(In reply to Kubilay Kocak from comment #1) I see your point. But the approach to use certifi as a wrapper to "system" trust store is not uncommon. E.g. OpenBSD and Debian is using it by default: http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/devel/py-certifi/patches/patch-certifi_core_py?rev=1.4&content-type=text/x-cvsweb-markup https://sources.debian.org/src/python-certifi/2018.4.16-1/debian/patches/0001-Use-Debian-provided-etc-ssl-certs-ca-certificates.cr.patch/ Is FreeBSD strictly against such approach? The main point is not to use "system" truststore, but to be able to add local trusted certificates to certifi, and certifi doesn't seem to implement it: https://github.com/certifi/python-certifi/issues/22 We could reach this goal if adding local CAs to store would be implemented in ca_root_nss and certifi just using it. Maybe we should start some discussion on maillists to hear more opinions?
Created attachment 195973 [details] py-certifi use CAs from ca_root_nss Update patch to remove hardcoded /usr/local
(In reply to Sergey Akhmatov from comment #2) I wouldn't say anyone is strictly against anything, particularly since this is a specific (third-party ecosystem) case without an obvious policy/guideline. Having said that, not being against something doesn't automatically or necessarily mean being pro/for position a change either. For what it's worth, it's good to have references to other OS's making similar changes. I think this ultimately boils down to the distinction you make in your 'main point', which I understand and agree with. It's one thing to want to extend a provided trust store (1), its another entirely to switch out a specific set with another set ((2), what is proposed here). Also, if I understand correctly, switching certifi's store out for that provided by security/ca_root_nss, would be the first step to getting the desired feature of local extensions to that store, via bug 160387. I don't think doing (2), in order to achieve (1) is the right approach. While I understand the value of the feature being described, I also believe that with the above context, the most important thing here is still user-expectation, and principle of least astonishment. Users/developers installing certifi would expect to get the certs/store/trust model the documentation of certifi stipulates, unless options provided (officially) by that package allowed otherwise. I would still recommend making the case for the added value of the "extend-certifi-store" feature to upstream.
Can this be reconsidered? There is now certctl(8) which already uses NSS bundle. What is the benefit to use yet another repackage of the NSS bundle from another party? Moreover, I need to add corporate CA certs which is impossible with certifi. I ned to pass manual verify path to py-requests which reduces portability of those scripts across OSes. I have to refactor code to provide such a path with argparse. What a pain. As a last resort, one could create py-certifi-freebsd just like py-certifi-win32, but this is really really ugly.
(In reply to Michael Osipov from comment #5) The best first course of action in my opinion would be to have this request be addressed upstream, perhaps in terms of an easy way to support extending the base set of provided root certificates. FWIW, there's also the following recently active upstream issue which while not explicitly relevant to resolving this issue, is sufficiently related that I think its worth mentioning: https://github.com/psf/requests/issues/2966
(In reply to Kubilay Kocak from comment #6) While I share your view on having this solved upstream, even if this is supported one has to maintain yet another cert store. I maintain for OpenSSL, annoyingy for Java (already initiated a change to RFC 7468, see https://bugs.openjdk.java.net/browse/JDK-8224891) and now for Python, eventhough it uses OpenSSL? This is actually a maintanence nightmare. Especially because for our entprise I need to consolidate three sources: NSS, Quo Vadis and Siemens. Consider that FreeBSD, RHEL, Windows, macOS already provide means to maintain a store. That shall be enough. (see also my issues with certctl(8)) I am also fully aware of the issue on GitHub. I have already left a few comments. Christian Heimes has also mentioned you about previous work. I'd be very helpful if you could leave a comment from your POV regarding Python on FreeBSD which can help to move this forward. Moreover, 3.0.0 may take some serious time to land. I do not really want to reinvent the wheel meantime. One would need to introduce py-certifi-unix just like py-certifi-win32 which probes for the Unix version and patches appropriate bits.
(In reply to Michael Osipov from comment #7) Thanks for the detailed feedback Michael FreeBSD UX and user-choice are important principles, and with that in mind I'm happy to reconsider the proposed option addition to the certifi port, modulo reiterating that it's also really (as) important to have things sorted out as much as possible, and deviating behaviour as little as possible from, upstream On the Python front, yep, Christian has been in touch and expressed his desire for the Python ports to use (at least as an option, and (on) by default), some root certificate bundle. I'm keeping an eye on the upstream conversation and am considering what we might be able to do downstream in the meantime as that conversation moves forward
(In reply to Kubilay Kocak from comment #8) Thank you very much taking my opinion into account, very appreciated. If you need further assistence let me know.
I am using /etc/ssl/certs with py-requests for now.
(In reply to Michael Osipov from comment #9) You're welcome Michael. What are your thoughts on a BUNDLED_CERTS or similarly named option, which uses the bundled certs when enabled (default), and ca_root_nss when disabled? My thoughts on the considerations/tradeoff space: 1) I'm not particularly a fan of 'reversed' option semantics, which can be a little confusing for users, but ... 2) The merit of having the default port/package build match upstream behaviour would seem to be nice, leaving a choice for the user to change it if they would like to, in place. 3) In this configuration, package users would (only) get the default (bundled) certs without customisation ability.
OK, let me share a bit differentiated view: * The option needs to be just like for GSS-API: GSSAPI_BASE GSSAPI_HEIMDAL GSSAPI_MIT. Converted for this we'd have: CERTS_BASE, CERTS_BUNDLED, CERTS_PORTS (ca_root_nss), CERTS_SSL (ssl.mk based) * I assume that ca_root_nss will be removed at some point in time because certctl(8) will be is available in 12.2-RELEASE (and hopefully in 11-STABLE) and having NSS certs in base and via ports looks like maintenance overhead * What should now be the default at least on 12? CERTS_BASE. Why? Because if something depends on OpenSSL from base, it should also the certs from /etc/ssl/certs. But it must obey ssl=... and point to that certs dir. If Python would have its own TLS implemenation like Java, I would be OK with having a bundled certs store. From a pkg user's POV, it should work consistently because I cannot change it, i.e., add certs or block certs to certifi while I can with certctl(8). WDYT?
Maintainer reset.