Bug 272674

Summary: ports-mgmt/pkg 1.20.4 fails to read from a web server using digest authentication with http
Product: Ports & Packages Reporter: Ross McKelvie <ross>
Component: Individual Port(s)Assignee: Baptiste Daroussin <bapt>
Status: Closed FIXED    
Severity: Affects Only Me CC: emaste, michael.osipov, ross
Priority: --- Flags: bugzilla: maintainer-feedback? (pkg)
Version: Latest   
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
pkg -d update output from version 1.20.4
none
always allow all supported auth
none
pkg -d update output from patched version of 1.20.4 none

Description Ross McKelvie 2023-07-23 09:16:46 UTC 
Created attachment 243563 [details]
pkg -d update output from version 1.20.4

I am running FreeBSD 12.4-RELEASE-p3 and use ports-mgmt/poudriere to create a package repository. This repository is hosted on a web server accessed by other machines and jails.  The web server uses digest authentication to provide basic privacy for the contents of that repository (aiming to hide precise software version numbers).

ports-mgmt/pkg version 1.19.2 works without issue. Version 1.20.4 fails. Running pkg -d update I can see that version 1.20.4 attempts to fetch from my web server using basic HTTP authentication but does not use digest authentication in response to the server's 401 reply.

My pkg repository configuration:
my_poudriere_repo: {
  url:            "http://redactedusername:redactedpassword@pkg/${VERSION_MAJOR}${VERSION_MINOR}${ARCH}-default"
  enabled:         true,
  signature_type: "pubkey"
  pubkey:         "/etc/ssl/keys/my_pkg_repo.pub"
  mirror_type:    "none"
}

I have attached a redacted version of the output from pkg -d update.

I note that pkg began to use libcurl in place of fetch from version 1.20.0.

Workarounds:
- Lock ports-mgmt/pkg to version 1.19.2 before updating packages by running:
# pkg lock ports-mgmt/pkg
- Remove digest authentication from webserver, switching to basic or none.
Comment 1 Ross McKelvie 2023-07-23 09:22:35 UTC 
See also the related discussion on the FreeBSD forums here: https://forums.freebsd.org/threads/problem-updating-packages.89687/

For the benefit of people searching, the output when running pkg update looks like:
Updating my_package_repository repository catalogue...
pkg: An error occured while fetching package
pkg: An error occured while fetching package
repository my_package_repository has no meta file, using default settings
pkg: An error occured while fetching package
pkg: An error occured while fetching package
Unable to update repository my_package_repository
Error updating repositories!
Comment 2 Baptiste Daroussin freebsd_committer freebsd_triage 2023-08-01 16:05:58 UTC 
Created attachment 243772 [details]
always allow all supported auth

can you try adding the patch into the files/ directory of the pkg ports and try again? it should fix your issue.
Comment 3 Ross McKelvie 2023-08-03 13:54:16 UTC 
Created attachment 243823 [details]
pkg -d update output from patched version of 1.20.4

Thank you, that is working well.

I have attached a redacted version of the output from pkg -d update, using the patched version.

I also watched the HTTP traffic using tcpdump and was happy not to see the password transmitted in the clear (as one would expect with basic HTTP authentication), appreciating that if I wanted proper security I should be using HTTP over TLS.
Comment 4 Baptiste Daroussin freebsd_committer freebsd_triage 2023-08-30 06:22:08 UTC 
Fixed in 1.20.6
Comment 5 Michael Osipov 2023-08-30 10:52:56 UTC 
FTR: Digest is a deprecated mechnism due to its complexity and use of MD5/SHA-1.