I'm trying to set up a pkg repository (with poudriere) accessible via HTTPS. However, running 'pkg update' I'm getting errors like: Certificate verification failed for /C=US/O=GeoTrust Inc./CN=GeoTrust Global CA 4286:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:997: Doing some investigation with ktrace/kdump, pkg doesn't seem to access content under /etc/ssl/certs at all and fetch only tries to read the inexistent file /etc/ssl/cert.pem. The certificates however are properly installed and 'openssl verify thecert.pem' prints "OK": # for cert in /etc/ssl/certs/*; do echo $cert; openssl x509 -noout -issuer -subject -hash < $cert; echo; done /etc/ssl/certs/2f2c2f7c.0 issuer= /C=US/O=GeoTrust Inc./CN=GeoTrust Global CA subject= /C=US/O=GeoTrust, Inc./CN=RapidSSL CA 2f2c2f7c /etc/ssl/certs/594f1775.0 issuer= /C=US/O=Equifax/OU=Equifax Secure Certificate Authority subject= /C=US/O=Equifax/OU=Equifax Secure Certificate Authority 594f1775 /etc/ssl/certs/7999be0d.0 issuer= /C=US/O=GeoTrust Inc./CN=GeoTrust Global CA subject= /C=US/O=GeoTrust Inc./CN=GeoTrust Global CA 7999be0d # openssl verify thecert.pem thecert.pem: OK Is it true that pkg or fetch do not properly support SSL?
This is still an issue on FreeBSD 10.1. I think the original posters issue would be resolved if he installed security/ca_root_nss and symlinked it to /etc/ssl/cert.pem but this remains an issue for people using their own CA's and installing their CA cert into /etc/ssl/certs. I could append my cert to ca-root-nss.crt but every time that port updates I would have an issue. As an example python's ssl code honors both /etc/ssl/cert.pem and /etc/ssl/certs/* but fetch does not.
This issue needs more eyes
The fundamental problem is that libfetch always sets the CA cert file (defaulting to /usr/local/etc/ssl/cert.pem if it exists or to /etc/ssl/cert.pem otherwise), and openssl will return from X509_STORE_load_location() upon failure to load the CAfile before trying to register the CApath. I will propose a patch shortly.
I have opened review D4771 with a proposed patch for this bug, and will commence testing. I have submitted the same patch against the pkg project on github as https://github.com/freebsd/pkg/pull/1368.
Please attach the patch to this PR.
Created attachment 165049 [details] test for /etc/ssl/cert.pem existence to avoid masking SSL_CA_CERT_PATH I have tested this and it works as intended. If you would like evidence, I would need to boil down the test results to a form suitable for sharing. In the course of testing, I realized that while the fallback to OpenSSL defaults is good, the inconsistency between the semantics of the libfetch layer of environment variables (SSL_CA_CERT_FILE, SSL_CA_CERT_PATH) and the defaults in their absence and the libcrypto layer of environment variables (SSL_CERT_FILE, SSL_CERT_DIR) and the defaults in their absence is not so good. To wit, libfetch has a default file---two, in fact---but no default path, whereas libcrypto has both, and the existence of either of the libfetch default files will prevent the fallback to the OpenSSL defaults. As I understand it, the reason that libfetch has a default to begin with, rather than always using the OpenSSL default behavior, is mainly (solely?) to allow the bundle from security/cs-nss-root to be picked up as the system default, at least for libfetch and its consumers (like pkg), merely by virtue of its installing a /usr/local/etc/ssl/cert.pem symlink, which is not a place OpenSSL looks by default. I don't have a recommendation at the moment, but when I do, it might be to add /usr/local/etc/certs as a path default for libfetch.
A commit references this bug: Author: bapt Date: Tue Jan 19 15:02:38 UTC 2016 New revision: 294326 URL: https://svnweb.freebsd.org/changeset/base/294326 Log: Test for /etc/ssl/cert.pem existence to avoid masking SSL_CA_CERT_PATH Prior to this patch, unless SSL_CA_CERT_FILE is set in the environment, libfetch will set the CA file to "/usr/local/etc/cert.pem" if it exists, and to "/etc/ssl/cert.pem" otherwise. This has the consequence of masking SSL_CA_CERT_PATH, because OpenSSL will ignore the CA path if a CA file is set but fails to load (see X509_STORE_load_locations()). While here, fall back to OpenSSL defaults if neither SSL_CA_CERT_FILE nor SSL_CA_CERT_PATH are set in the environment, and if neither of the libfetch default CA files exists. PR: 193871 Submitted by: John W. O'Brien <john@saltant.com> Approved by: des MFC after: 1 week Changes: head/lib/libfetch/common.c
A commit references this bug: Author: des Date: Sat Feb 20 13:36:25 UTC 2016 New revision: 295840 URL: https://svnweb.freebsd.org/changeset/base/295840 Log: MFH (r273114, r273124): turn SSLv3 off by default MFH (r294326): fall back to standard / configured CA store MFH (r295536): fix double-free when SSL connection fails PR: 193871 206774 Changes: _U stable/9/ _U stable/9/lib/ _U stable/9/lib/libfetch/ stable/9/lib/libfetch/common.c stable/9/lib/libfetch/fetch.3 stable/9/lib/libfetch/http.c _U stable/9/usr.bin/ _U stable/9/usr.bin/fetch/
Reopening, apparently it wasn't merged to 10.
A commit references this bug: Author: des Date: Sat Feb 20 22:58:33 UTC 2016 New revision: 295843 URL: https://svnweb.freebsd.org/changeset/base/295843 Log: MFH (r294326): fall back to standard / configured CA store PR: 193871 Approved by: re (gjb) Changes: _U stable/10/ stable/10/lib/libfetch/common.c