Bug 287250 - security/vault: certificate signed by unknown authority
Summary: security/vault: certificate signed by unknown authority
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Muhammad Moinur Rahman
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-06-03 03:08 UTC by Robert William Vesterman
Modified: 2025-06-09 21:19 UTC (History)
3 users (show)

See Also:
bugzilla: maintainer-feedback? (bofh)


Attachments
build log (28.85 KB, text/plain)
2025-06-03 20:14 UTC, Robert William Vesterman
no flags Details
patch for www/gohugo (550 bytes, patch)
2025-06-04 17:26 UTC, Geraint Edwards
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Robert William Vesterman 2025-06-03 03:08:18 UTC
Since yesterday (at least - maybe a little earlier), I've been unable to build security/vault, due to dozens of errors like this when trying to fetch distfiles:

-------

# get https://proxy.golang.org/github.com/%21azure/azure-sdk-for-go/sdk/azidentity/@v/v1.9.0.mod: Get "https://proxy.golang.org/github.com/%21azure/azure-sdk-for-go/sdk/azidentity/@v/v1.9.0.mod": tls: failed to verify certificate: x509: certificate signed by unknown authority

-------

They are all for files fetched from https://proxy.golang.org. I don't really know much about how FreeBSD and/or the ports system determines what authorities are known, but I certainly haven't knowingly changed anything about it recently.
Comment 1 Muhammad Moinur Rahman freebsd_committer freebsd_triage 2025-06-03 08:08:46 UTC
What FreeBSD version are you running? This is most probably due to an outdated root certificates at your end. Can you please provide the entire log somewhere?
Comment 2 Robert William Vesterman 2025-06-03 20:14:26 UTC
Created attachment 260949 [details]
build log

Thanks. Log is attached, and here's the version info:

# uname -a; freebsd-version -kru
FreeBSD iron.vestertopia.net 14.2-RELEASE-p1 FreeBSD 14.2-RELEASE-p1 GENERIC amd64
14.2-RELEASE-p1
14.2-RELEASE-p1
14.2-RELEASE-p3
Comment 3 Muhammad Moinur Rahman freebsd_committer freebsd_triage 2025-06-03 20:43:52 UTC
https://www.freebsd.org/security/advisories/FreeBSD-EN-25:08.caroot.asc

I think your system is not updated and/or you are missing the latest CA root which is causing this error. And you might face the same errors with other go based ports.
Comment 4 Robert William Vesterman 2025-06-03 22:07:10 UTC
(In reply to Muhammad Moinur Rahman from comment #3)

Thanks. I'm not sure I'm understanding about the "your system is not updated" idea. Are p1 (kernel) and p3 (userland) not the latest? I'm usually good about keeping point releases updated, and freebsd-update seems to think I'm set along those lines:

---
# freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 3 mirrors found.
Fetching metadata signature for 14.2-RELEASE from update1.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.
                                                                                                                        No updates needed to update system to 14.2-RELEASE-p3.
# freebsd-update install
No updates are available to install.
Run 'freebsd-update [options] fetch' first.
# 
---

I'll look into the other idea ("missing the latest CA root"), but... is this not  something that freebsd-update would detect? The update directions seem to indicate that it should have installed the latest one, at the very least.
Comment 5 Robert William Vesterman 2025-06-04 01:02:45 UTC
I'm sorry, I feel like I'm in over my head here -- I know very little about this stuff -- but I *think* this is telling me that *something* is going wrong with respect to OCSP?

First, I can get that URL just fine using curl:
---
[204059 bob@iron 1;~/bobtmp]$ curl https://proxy.golang.org/github.com/%21azure/azure-sdk-for-go/sdk/azidentity/@v/v1.9.0.mod > mod
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1143  100  1143    0     0  10194      0 --:--:-- --:--:-- --:--:-- 10297

[204111 bob@iron 1;~/bobtmp]$ cat mod
module github.com/Azure/azure-sdk-for-go/sdk/azidentity

go 1.23.0

require (
        github.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.0
        github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2
        github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.1
        github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2
        github.com/golang-jwt/jwt/v5 v5.2.2
        github.com/google/uuid v1.6.0
        github.com/redis/go-redis/v9 v9.7.3
        github.com/stretchr/testify v1.10.0
        golang.org/x/crypto v0.36.0
)

require (
        github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 // indirect
        github.com/cespare/xxhash/v2 v2.3.0 // indirect
        github.com/davecgh/go-spew v1.1.1 // indirect
        github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
        github.com/keybase/go-keychain v0.0.1 // indirect
        github.com/kylelemons/godebug v1.1.0 // indirect
        github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
        github.com/pmezard/go-difflib v1.0.0 // indirect
        golang.org/x/net v0.38.0 // indirect
        golang.org/x/sys v0.31.0 // indirect
        golang.org/x/text v0.23.0 // indirect
        gopkg.in/yaml.v3 v3.0.1 // indirect
)
---
That's running curl on the same machine as I'm trying the build on. I can also get the URL using Firefox on a different machine, and get no complaint about the certificate.

But if I go back to curl (on the original machine) and use "--cert-status":
---
[204337 bob@iron 1;~/bobtmp]$ curl --cert-status https://proxy.golang.org/github.com/%21azure/azure-sdk-for-go/sdk/azidentity/@v/v1.9.0.mod
curl: (91) No OCSP response received
---
"No OCSP response received" seems kind of vague -- like, was *ANY* response received, or no? -- so I used the directions at the following page...

https://raymii.org/s/articles/OpenSSL_Manually_Verify_a_certificate_against_an_OCSP.html

... to try to get at it via OpenSSL:
---
[205350 bob@iron 1;~/bobtmp]$ openssl s_client -connect proxy.golang.org:443 2>&1 < /dev/null | sed -n '/-----BEGIN/,/-----END/p' > pgo.pem
[205413 bob@iron 1;~/bobtmp]$ openssl x509 -noout -ocsp_uri -in pgo.pem
http://o.pki.goog/wr2
---
As I (mis-?)understand it, that means that the cert is set up to use OCSP, and moreover that that o.pki.goog URL is the appropriate thingy to ask about it. For that, you need the full chain, so:
---
[205435 bob@iron 1;~/bobtmp]$ openssl s_client -connect proxy.golang.org:443 -showcerts 2>&1 < /dev/null > chain.pem
depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R1
verify return:1
depth=1 C = US, O = Google Trust Services, CN = WR2
verify return:1
depth=0 CN = misc-sni.google.com
verify return:1
DONE
---
[205753 bob@iron 1;~/bobtmp]$ openssl ocsp -issuer chain.pem -cert pgo.pem -text -url http://o.pki.goog/wr2
OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: 5342D4848BC117F9B6144D777CFB23310F7B35CD
          Issuer Key Hash: B7C7D8D69963107DB2618E660071BEC7EB3D082C
          Serial Number: AF20FBAC920DB614105798AEC41339C2
    Request Extensions:
        OCSP Nonce:
            041000A6EEC5D517A7EABAAE5CC0D7150604
Responder Error: unauthorized (6)
---
That "Error:unauthorized (6)" is (according to that page I'm getting the directions from) what happens (at least with some servers, I guess) if "we send this request to another OCSP, one who did not issued this certificate".

Sooooooo... I'll keep poking around, but like I say, I feel like I'm over my head here.
Comment 6 Geraint Edwards 2025-06-04 17:22:39 UTC
I found this error updating another golang port (www/gohugo)
and found a fix by adding the following line to the `Makefile`

FETCH_DEPENDS=         ca_root_nss>0:security/ca_root_nss

For the record, I'm using synth (so the clean env in which it builds
must not have a suitable CA list)
Comment 7 Geraint Edwards 2025-06-04 17:26:56 UTC
Created attachment 260979 [details]
patch for www/gohugo

Patch for another port, but should be useful to put in other golang ports
Comment 8 Michael Osipov freebsd_committer freebsd_triage 2025-06-04 20:38:44 UTC
Hi folks,

I have consented the mentioned commit and expected some fallout. Let me try to reproduce first and figure out what needs to be fixed in Go rather than adding the fetch dependency.
Comment 9 Michael Osipov freebsd_committer freebsd_triage 2025-06-04 20:57:45 UTC
So I have built www/gohugo and security/vault. Both build without any TLS errors, i.e., "go get" works as expected.

Which OpenSSL version do you use? I have DEFAULT_VERSIONS+=ssl=base.
If you have OpenSSL from ports then the issue is clear and I know what to do, but let me know first.
Comment 10 Robert William Vesterman 2025-06-04 21:12:35 UTC
(In reply to Michael Osipov from comment #9)

Thanks, both of you. I'm afraid I don't know enough to feel confident in answering what you're asking, but I can tell you this:

1. The only ports I have installed that contain "ssl" (case-insensitive) in their names are p5-IO-Socket-SSL-2.089, p5-Net-SSLeay-1.9.4, and py311-openssl-25.0.0_1,1.

2. I did not explicitly install any of those; they were each dragged in by something else for reasons that are beyond my ken.

3. The only "DEFAULT_VERSIONS" change I have is "DEFAULT_VERSIONS+=java=22".

If this is not enough info to determine what you're trying to learn, could you please let me know how to get the info you want? Thanks again.
Comment 11 Michael Osipov freebsd_committer freebsd_triage 2025-06-04 21:13:29 UTC
Commit in question: 6d5b8fde29aeea4278bfc8d1f27b5016fcdf9ccb
Comment 12 Michael Osipov freebsd_committer freebsd_triage 2025-06-04 21:15:32 UTC
(In reply to Michael Osipov from comment #11)

Here are two simple commands you can run and provide the output:
root@deblndw011x:/usr/ports/security/vault
# make -V DEFAULT_VERSIONS
perl5=5.40 python=3.10 python3=3.10 ruby=3.4 ssl=base pgsql=13 php=8.3 samba=4.19 apache=2.4

# ldd $(which curl)
/usr/local/bin/curl:
        libcurl.so.4 => /usr/local/lib/libcurl.so.4 (0x745dbc2b000)
        libnghttp2.so.14 => /usr/local/lib/libnghttp2.so.14 (0x745dc823000)
        libssl.so.111 => /usr/lib/libssl.so.111 (0x745dd54e000)
        libcrypto.so.111 => /lib/libcrypto.so.111 (0x745dec0a000)
        libk5crypto.so.3.1 => /usr/local/lib/libk5crypto.so.3.1 (0x745ddfca000)
        libcom_err.so.3.0 => /usr/local/lib/libcom_err.so.3.0 (0x745df728000)
        libz.so.6 => /lib/libz.so.6 (0x745dfc2b000)
        libkrb5.so.3.3 => /usr/local/lib/libkrb5.so.3.3 (0x745e0aff000)
        libgssapi_krb5.so.2.2 => /usr/local/lib/libgssapi_krb5.so.2.2 (0x745e0ec8000)
        libthr.so.3 => /lib/libthr.so.3 (0x745e1417000)
        libc.so.7 => /lib/libc.so.7 (0x745e17a6000)
        libkrb5support.so.0.1 => /usr/local/lib/libkrb5support.so.0.1 (0x745e1c42000)
        libintl.so.8 => /usr/local/lib/libintl.so.8 (0x745e35f7000)
        [vdso] (0x7ffffffff000)
Comment 13 Robert William Vesterman 2025-06-04 21:24:28 UTC
(In reply to Michael Osipov from comment #12)

OK, I'll quote the output in a moment, but I think first I should note: Like Geraint Edwards, I use Synth, so I'm not sure if this output (which was generated outside of the Synth building process) is really relevant. I'll poke around to try to figure out how to get at the same info during the Synth build, but in the meantime here's the non-Synth output in case it is relevant:

---

[172222 root@iron 2;/stuff/packageInfo]# make -V DEFAULT_VERSIONS
java=22

[172245 root@iron 2;/stuff/packageInfo]# ldd $(which curl)                                                              /usr/local/bin/curl:                                                                                                            libcurl.so.4 => /usr/local/lib/libcurl.so.4 (0x1bf07e63a000)
        libnghttp2.so.14 => /usr/local/lib/libnghttp2.so.14 (0x1bf07ecdf000)
        libidn2.so.0 => /usr/local/lib/libidn2.so.0 (0x1bf07fc75000)
        libssh2.so.1 => /usr/local/lib/libssh2.so.1 (0x1bf080bb8000)
        libpsl.so.5 => /usr/local/lib/libpsl.so.5 (0x1bf0817f5000)
        libssl.so.30 => /usr/lib/libssl.so.30 (0x1bf081c94000)
        libheimntlm.so.11 => /usr/lib/libheimntlm.so.11 (0x1bf0821fe000)
        libhx509.so.11 => /usr/lib/libhx509.so.11 (0x1bf0839fc000)
        libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x1bf082f5f000)
        libcrypto.so.30 => /lib/libcrypto.so.30 (0x1bf084710000)
        libasn1.so.11 => /usr/lib/libasn1.so.11 (0x1bf08662f000)
        libwind.so.11 => /usr/lib/libwind.so.11 (0x1bf0858de000)
        libheimbase.so.11 => /usr/lib/libheimbase.so.11 (0x1bf087012000)
        libroken.so.11 => /usr/lib/libroken.so.11 (0x1bf0876ac000)
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x1bf08927e000)
        libzstd.so.1 => /usr/local/lib/libzstd.so.1 (0x1bf0883df000)
        libbrotlidec.so.1 => /usr/local/lib/libbrotlidec.so.1 (0x1bf089ddd000)
        libz.so.6 => /lib/libz.so.6 (0x1bf08ae2d000)
        libkrb5.so.11 => /usr/lib/libkrb5.so.11 (0x1bf08b704000)
        libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x1bf08acd1000)
        libgssapi_krb5.so.10 => /usr/lib/libgssapi_krb5.so.10 (0x1bf08c090000)
        libthr.so.3 => /lib/libthr.so.3 (0x1bf08dc21000)
        libc.so.7 => /lib/libc.so.7 (0x1bf08c93e000)
        libunistring.so.5 => /usr/local/lib/libunistring.so.5 (0x1bf08eee9000)
        libm.so.5 => /lib/libm.so.5 (0x1bf08e482000)
        libbrotlicommon.so.1 => /usr/local/lib/libbrotlicommon.so.1 (0x1bf08fc9a000)
        libprivateheimipcc.so.11 => /usr/lib/libprivateheimipcc.so.11 (0x1bf090dfd000)
        [vdso] (0x1bf07dad9000)
Comment 14 Michael Osipov freebsd_committer freebsd_triage 2025-06-04 21:29:40 UTC
(In reply to Robert William Vesterman from comment #13)

So it is OpenSSL from base. I don't use synth, but poudriere. I have tried this one as root in the terminal interactively. Can you do as well? Moreover, if can enter the synth container, please run "certctl list".
Comment 15 Robert William Vesterman 2025-06-05 02:11:00 UTC
(In reply to Michael Osipov from comment #14)

Sorry, I've been trying to figure out how to break into the Synth environment. I've done it before and vaguely remember how, but it... hasn't been going as expected.

Somehow I unintentionally got it in a state where Synth is rebuilding all my stuff (despite the fact that everything except Vault has already been built up to the latest), and that has been going for hours. I'm not sure that I'm going to have any more time this evening to dedicate to this, but I'll try to get back to it as soon as I can in any case.
Comment 16 Fernando Apesteguía freebsd_committer freebsd_triage 2025-06-05 15:27:10 UTC
I can't reproduce this with poudriere on a just updated 14.2amd64 jail. Also, builders are just fine and my host machine also builds gohugo without any issues.

Inspecting the jail, we can see that no ca_root_nss from ports is installed:

root@14_2amd64-default:/usr/ports/www/gohugo # pkg info
ccache-3.7.12_8                Tool to minimize the compile time of C/C++ programs
giflib-5.2.2                   Tools and library routines for working with GIF images
go124-1.24.3                   Go programming language
gohugo-0.147.7,1               Fast and Flexible Static Site Generator built with love in GoLang
jbigkit-2.1_3                  Lossless compression for bi-level images such as scanned pages, faxes
jpeg-turbo-3.1.0               SIMD-accelerated JPEG codec which replaces libjpeg
lerc-4.0.0                     C++ library for Limited Error Raster Compression
libdeflate-1.24                Fast, whole-buffer DEFLATE-based compression library
liblz4-1.10.0,1                LZ4 compression library, lossless and very fast
libsass-3.6.6                  C/C++ implementation of a Sass compiler
pkg-2.1.4                      Package manager
png-1.6.48                     Library for manipulating PNG images
tiff-4.7.0                     Tools and library routines for working with TIFF images
webp-1.5.0                     Google WebP image format conversion tool
zstd-1.5.7                     Fast real-time compression algorithm

In fact, what we would like now that we have certctl in all supported releases is to remove ca_root_nss from ports. See bug #284404
Comment 17 Robert William Vesterman 2025-06-05 21:21:37 UTC
I have been unable to get into the Synth build. All sorts of wackiness went on; I am not convinced that it is directly related to the issue I opened this bug report about, so I opened a thread o the general FreeBSD forums about it. But maybe it is related, or maybe they're both symptoms of the same cause, or some such thing. So, in case anyone wants to take a look:

https://forums.freebsd.org/threads/pkg-upgrade-says-abi-changed-why-would-this-happen-what-does-it-mean-what-if-anything-do-i-do.98049/
Comment 18 Robert William Vesterman 2025-06-05 21:28:23 UTC
(In reply to Fernando Apesteguía from comment #16)

Just saw this reply regarding ca_root_nss. I *do* have that installed. I did *not* install it directly - it was brought in by some other port, apparently one or more of msmtp, node22, rclone, or void-zones-tools.
Comment 19 Geraint Edwards 2025-06-08 09:42:47 UTC
I think I may have found the source of the issue

https://cgit.freebsd.org/ports/commit/Mk/Uses/go.mk?id=6d5b8fde29aeea4278bfc8d1f27b5016fcdf9ccb

which confirms my "fix" (really a workaround).
The ultimate fix may have to be inside `synth`?
Comment 20 Michael Osipov freebsd_committer freebsd_triage 2025-06-08 09:44:48 UTC
(In reply to Geraint Edwards from comment #19)

Already referenced: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=287250#c11

I assume that the system truststore is *not* present in the Synth container.
Comment 21 Geraint Edwards 2025-06-08 09:47:34 UTC
(In reply to Michael Osipov from comment #20)

Apologies for the duplicate, I didn't know where (which git repo) that hash had come from. :blush:
Comment 22 Michael Osipov freebsd_committer freebsd_triage 2025-06-08 09:49:10 UTC
(In reply to Geraint Edwards from comment #21)

All good. I still would like to know how it looks like in the Synth container otherwise we won't be able to understand the real cause at all. If fetch(1) works then everything else can work.
Comment 23 Robert William Vesterman 2025-06-08 22:36:44 UTC
(In reply to Michael Osipov from comment #22)

>I still would like to know how it looks like in the Synth container otherwise we won't be able to understand the real cause at all.

I will try to, or at least try to try to, get into Synth again, but I'm having a perfect storm of computer problems now (mostly unrelated to this), so I'm not sure when I'll be able to.

For what it's worth, last night's automated build shows the same sort of problem with www/xcaddy. So, yeah, as predicted here earlier, it's almost certainly not directly a vault problem, but ports that use Go in general.

Finally, this is all still way out of my league, so if anybody knows of, and would tell me, some relatively simple steps I could take that might clear up the immediate issue for me while the root cause is being investigated and dealt with, I would greatly appreciate it. Perhaps, for example, somehow adding some reference within the Synth config or build environment to the system root store?
Comment 24 Geraint Edwards 2025-06-09 09:05:33 UTC
I ran the following command(s) to enter the synth builder:

```shell
# ENTERAFTER=extract synth test www/gohugo
[ synth built 2 deps, then dropped to this ]


The task is complete.  Final tally:
Initial queue size: 3
    packages built: 2
           ignored: 0
           skipped: 0
            failed: 0

Duration: 00:00:52
The build logs can be found at: /var/log/synth
Starting interactive build of www/gohugo
Stand by, building up to the point requested ...
Entering interactive test mode at the builder root directory.
Type 'exit' when done exploring.
# certctl list
Listing Trusted Certificates:
#
```

It appears that the `certctl list` results are empty inside the builder.
Comment 25 Michael Osipov freebsd_committer freebsd_triage 2025-06-09 19:28:45 UTC
(In reply to Geraint Edwards from comment #24)

Well, here it is. Now someone needs to figure out why the base jail in Synth does not contain a store.
Comment 26 Robert William Vesterman 2025-06-09 21:19:47 UTC
Thank you! I have opened up a ticket on the Synth forums about this:

https://github.com/jrmarino/synth/issues/243