For quite some time now (since at least FreeBSD 12, if not earlier) pkg complains about "pkg: unknown format pkg, using the default" when using portupgrade on a port. This bug could also be consider a bug in portugrade, but since both are essential parts of the "core" ports system for people who build their own ports, one or the other maintainer should be able to address this. Give that .pkg seems like a reasonable suffix for ports-mgmt/pkg to support, I'm filing it against pkg. Feel free to reassign. To reproduce (on 14.2-stable), just "portupgrade -f" any port you have installed. You will see output like this: [...] ====> Compressing man pages (compress-man) ---> Backing up the old version pkg: unknown format pkg, using the default <-------------- ---> Uninstalling the old version [Reading data from pkg(8) ... - 279 packages found - done] ---> Deinstalling 'xev-1.2.4' Updating database digests format: 100% Checking integrity... done (0 conflicting) [...]
The upstream https://github.com/freebsd/portupgrade/commits/2.4.16/ shows no updates since: Mar 9, 2018 It might be out of date?
(In reply to Mark Millard from comment #1) More relevant may be: same date for https://github.com/freebsd/portupgrade/commits/master
(In reply to Mark Millard from comment #2) That should have been "same sort of date": Sep 13, 2018 It definitely is not the same date. Still: long ago.
(In reply to Mark Millard from comment #1) True. But other than making pkg complain, portupgrade still works and is documented in the Handbook. Someone could certainly float a proposal to remove it from the Handbook for FreeBSD 15 and set the portupgrade port's Makefile to reject FreeBSD versions of 15.0 or higher, but FreeBSD 14 will be supported for nearly another 4 years (EoL November 30, 2028) so we should probably do something in the meantime. I didn't actually check to see which of portupgrade or pkg was more frequently updated, but since the answer is pkg (which is what I opened this bug against), wouldn't it make sense to have pkg silently accept ".pkg" as a suffix (and convert it to its default .txz format)?
(In reply to Terry Kennedy from comment #4) You also get this message when you run: pkg create -f pkg sudo But 'file sudo-1.9.16p2_1.pkg' shows sudo-1.9.16p2_1.pkg: XZ compressed data, checksum CRC64 So everything is working, isn't it? To get rid of the message you can try: --- /usr/local/sbin/portupgrade.orig 2025-01-06 07:47:32.000000000 +0100 +++ /usr/local/sbin/portupgrade 2025-01-14 09:51:26.242888000 +0100 @@ -1077,7 +1077,7 @@ end if $pkgdb.with_pkgng? - system!(PkgDB::command(:pkg), 'create', '-o', $packages_dir, '-f', $portsdb.pkg_sufx, pkgname) + system!(PkgDB::command(:pkg), 'create', '-o', $packages_dir, pkgname) else system!(PkgDB::command(:pkg_create), '-vb', pkgname, File.join($packages_dir, pkgname + $portsdb.pkg_sufx)) @@ -2084,7 +2084,7 @@ if $pkgdb.with_pkgng? backup_pkgfile = File.join($tmpdir, pkgname + $portsdb.pkg_sufx) - backquote!(PkgDB::command(:pkg), 'create', '-o', $tmpdir, '-f', $portsdb.pkg_sufx, pkgname) + backquote!(PkgDB::command(:pkg), 'create', '-o', $tmpdir, pkgname) else if str = backquote!(PkgDB::command(:pkg_create), '-vb', pkgname, File.join($tmpdir, pkgname + $portsdb.pkg_sufx))
(In reply to Terry Kennedy from comment #4) .pkg has been the default for years. Quoting Ed Maste's commit notes from the recent removal of support for .txz in main for .pkg : QUOTE pkg: retire backwards compatibility bootstrap support pkg.pkg has been available for years and is the default in all supported releases. Retire fallback support for pkg.txz. END QUOTE See: https://lists.freebsd.org/archives/dev-commits-src-main/2025-January/029249.html
(In reply to Mark Millard from comment #6) Specifically, the FreeBSD pkg bootstrap in main switched to a default of .pkg back in 2021 when the temporary .bsd was replaced: uthor Baptiste Daroussin <bapt@FreeBSD.org> 2021-03-16 15:18:48 +0000 committer Baptiste Daroussin <bapt@FreeBSD.org> 2021-03-16 15:19:52 +0000 commit c244b1d8a38731041d0f3ff4191192a85dd8608b (patch) tree afbdd9bf2139202f5302db885ea4c465be2941d0 /usr.sbin/pkg parent 54ac6f721efccdba5a09aa9f38be0a1c4ef6cf14 (diff) pkg: settle the uniq extension to .pkg instead of .bsd Requested by: many Diffstat (limited to 'usr.sbin/pkg') -rw-r--r-- usr.sbin/pkg/pkg.c 2 1 files changed, 1 insertions, 1 deletions diff --git a/usr.sbin/pkg/pkg.c b/usr.sbin/pkg/pkg.c index 48b92049b869..04232672ac39 100644 --- a/usr.sbin/pkg/pkg.c +++ b/usr.sbin/pkg/pkg.c @@ -85,7 +85,7 @@ struct fingerprint { }; static const char *bootstrap_names [] = { - "pkg.bsd", + "pkg.pkg", "pkg.txz", NULL };
this is a bug in portupgrade, because portupgrade is unmaintained for years and never catchup with recent changes of both the ports tree. here the bug in portupgrade is probably getting the format of the package through PKG_SUFX variable from the ports, and passes it as compression format to pkg create via the -f argument. PKG_SUFX has been change to .pkg alors with pkg 1.17.0 and the right variable from the ports tree to guess the format of package expected is PKG_COMPRESSION_FORMAT. The last update on portupgrade happened in 2018, since that date a lot happen in both pkg and the ports tree that portupgrade is not able to deal with.
(In reply to Mark Millard from comment #7) Hmm. There was a much later fix (2024!) to the usage messages to track that change: diff --git a/usr.sbin/pkg/pkg.c b/usr.sbin/pkg/pkg.c index 70086886fbc5..cc23ffb8c257 100644 --- a/usr.sbin/pkg/pkg.c +++ b/usr.sbin/pkg/pkg.c @@ -895,7 +895,7 @@ static const char args_bootstrap_message[] = static const char args_add_message[] = "Too many arguments\n" -"Usage: pkg add [-f] [-y] {pkg.txz}\n"; +"Usage: pkg add [-f] [-y] {pkg.pkg}\n"; static int pkg_query_yes_no(void) @@ -1167,7 +1167,7 @@ main(int argc, char *argv[]) if (add_pkg) { if (pkgarg == NULL) { - fprintf(stderr, "Path to pkg.txz required\n"); + fprintf(stderr, "Path to pkg.pkg required\n"); exit(EXIT_FAILURE); } if (access(pkgarg, R_OK) == -1) { See: https://cgit.freebsd.org/src/commit/usr.sbin/pkg?id=f5c847ae849aab9354d0956afd683f1c90bfd91e So the bootstrap's messaging did not make discovery of the default being .pkg simple via that path of discovery.
(In reply to Baptiste Daroussin from comment #8) It does appear that: https://docs.freebsd.org/en/books/handbook/ports/ does lack any warning about the status of portupgrade .
Comments 6, 7, and 9 all note that ".pkg" is now the default suffix for packages. If that is the case, it seems reasonable that "pkg' be accepted as a package format by pkg (as it is now, just with the warning message), even though the "-f pkg" is trying to specify a format and not a file extension. I'm not suggesting that pkg accept everything that might be passed as a parameter to -f, or even document that "pkg" is an acceptable format. Just something like changing: case "txz": <do some stuff> to: case "txz": case "pkg": <do some stuff> Comment 8 reiterates that portupgrade is unmaintained. While that certainly seems to be the case, it's in the ports tree in FreeBSD 14.2 and documented in the Handbook (see my comment 4). If the team wants to decide it's deprecated and either remove it from the Handbook in FreeBSD 15 (or note that it is deprecated and going away in FreeBSD 16, which is our usual schedule for announcing deprecation and subsequent removal), that's fine. But for the nearly 4 years of remaining support for FreeBSD 14, can we please do something like the above one-line change? Further, I'm reluctant to change things in pkgconf.conf as I don't know what else relies on it.
(In reply to Terry Kennedy from comment #11) pkg-create has: QUOTE -f format, --format format Set format as the package output format. It can be one of tzst, txz, tbz, tgz or tar which are currently the only supported formats. If an invalid or no format is specified txz is assumed. END QUOTE Also: https://github.com/freebsd/pkg/blob/main/README.md reports: QUOTE The pkg package format is a tar archive that may be raw or compressed using one of the following algorithms: gz, bzip2, zstd, or xz. The default compression algorithm is zstd. END QUOTE Note that that would mean that the modern default format produced is tzst instead of txz . But all of those options produce .pkg as the file extension as far as I can tell. *.pkg gives no clue which format is in use. Nor does -f pkg do so. Making it silent would just lead to confusions when mismatches happen. Making it be txz format would be to make it not use the modern default format. Making it use tzst would not match tools that are not being updated to track. To me it seems best to have the "pay attention here" notice than to be silent.
(In reply to Mark Millard from comment #12) I'm confused. It seems you are saying that regardless of what compression format is selected, pkg outputs files with a .pkg extension which provides no clue which format is used. (And that this is confusing.) I agree that it can be confusing, but presumably all of the pkg tools can figure it out. For people wanting to know what compression format was used to create a given .pkg file, "file <somename>.pkg" should report that info. I don't see how that relates to silencing a warning for the specific case of "-f pkg" in portupgrade, since portupgrade will be around for at least nearly 4 years (remaining life of 14) and perhaps longer if it goes through the normal deprecation (15) and removal (16) cycle. Particularly since it will eliminate a warning that is also confusing 8-} You also raise a valid point that if "txz" is not the default format, then assuming txz would not be be the best case and "tzst" should be assumed instead. That just changes where the .pkg case could be added. But I should point out (as you did in comment 12) that "man pkg-create" (pkg pkg-1.21.3, the current version in the ports tree), under the description of the -f option, says "If an invalid or no format is specified txz is assumed." So we seem to also have a manpage-vs-reality discrepancy in pkg here. Regarding your comment that it would not track tools that are not being updated, I think the only thing that (incorrectly) uses "-f pkg" is portupgrade, which is how we got here. If there's no desire to change this on the pkg side, I could submit a patch for portupgrade if somebody is willing to commit it.
if you provide a patch for portupgrade, I will push it, if somehow I am not responsive fast enough don't hesitate to drop me a direct email.
(In reply to Baptiste Daroussin from comment #14) Sounds good! Look for it here in a day or two.
(In reply to Mark Millard from comment #9) > Hmm. There was a much later fix (2024!) to the usage messages to > track that change: Yes - the default changed some time ago, but a fair bit of documentation and other content wasn't updated until much more recently. bapt just merged my update to pkg-create.8 to clarify -f: https://github.com/freebsd/pkg/commit/fb47e7663ffbb4d5dd1c92f6ea9ad2f541450171 and a deprecation note for the handbook: https://reviews.freebsd.org/D48462
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/doc/commit/?id=277365f0c339e85a740bb9c0da820b82d3f634a2 commit 277365f0c339e85a740bb9c0da820b82d3f634a2 Author: Ed Maste <emaste@FreeBSD.org> AuthorDate: 2025-01-15 13:46:45 +0000 Commit: Ed Maste <emaste@FreeBSD.org> CommitDate: 2025-01-15 13:49:05 +0000 handbook: Note that portupgrade is deprecated PR: 284054 Reviewed by: bapt Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D48462 documentation/content/en/books/handbook/ports/_index.adoc | 6 ++++++ 1 file changed, 6 insertions(+)
(In reply to Terry Kennedy from comment #13) QUOTE since portupgrade will be around for at least nearly 4 years (remaining life of 14) and perhaps longer if it goes through the normal deprecation (15) and removal (16) cycle. END QUOTE You seem to be confusing procedures for: https://cgit.freebsd.org/src/ and the related branch and release handling with the procedures for: https://cgit.freebsd.org/ports/ and port depreciation. ports-mgmt/portupgrade is not part of FreeBSD and FreeBSD, of itself, makes no use of ports-mgmt/portupgrade . Similarly, the FreeBSD package builders do not use ports-mgmt/portupgrade . No FreeBSD infrastructure uses ports-mgmt/portupgrade as far as I can tell. ports-mgmt/portupgrade is not part of any FreeBSD release and not required by any either. Mentioning or describing a port's use in the handbook or elsewhere does not change that status unless explicitly indicated somewhere or the port having the status of FreeBSD or its infrastructure directly using the port.
(In reply to Ed Maste from comment #16) This doesn't seem to correct the "If an invalid or no format is specified txz is assumed." part of the manpage, if the info in comment 12 is correct and the default format is now tzst. Also, I don't think the "in the near future" deprecation notice in the handbook is reasonable. Unless there's some plan to remove portsmgmt/portupgrade from the ports tree, it is going to remain in quarterly builds for some time. db5 has been hanging around in the ports tree long past its sell-by date. I suggest two things: 1) Adding DEPRECATED= and EXPIRATION_DATE= to the port Makefile (the date should be at FreeBSD 14 EOL, IMHO). 2) Adding a section in the Handbook on "Migrating from portupgrade to portmaster"
Created attachment 256720 [details] Patch to portupgrade to eliminate warnings from pkg
(In reply to Mark Millard from comment #18) I am expecting "don't surprise the users", particularly when it is unnecessary. If someone (for example, myself) installs a port mentioned in the Handbook to manage ports on a new FreeBSD install (which would be 14.2 in this case), it is reasonable for them to expect the port to remain in the ports tree throughout the life of FreeBSD 14, unless either a) a security problem is found in the port, or b) the port no longer works with the evolving state of the FreeBSD ecosystem (for example, if pkg changes dramatically enough to break portupgrade). Note that I said "reasonable to expect". I don't want to get into an argument about what is or is not a policy that the average (but informed) user is unaware of. By your logic, the entire ports system could be abandoned because it is "not part of any FreeBSD release" (notwithstanding the fact that installing the ports tree is one of the options provided by the FreeBSD installer). I've submitted a patch to portupgrade, and made some suggestions for deprecation of portupgrade in comment 19. That will sync the port with the handbook and set expectations as to when the port will actually be removed (which I hope would be past the EoL of FreeBSD 14, otherwise "don't surprise the users" isn't going to apply).
(In reply to Terry Kennedy from comment #21) Beyond indicating that "4 years" was not required and what you referenced as the source of that figure does not apply of itself, I made no attempt to indicate when things would or should happen. It is just a less constrained context with more options (both for shorter, similar, or longer periods) and less policy. I posted a note to freebsd-ports showing the commit to the handbook for broader awareness. That may contribute to feedback that helps set the actual time frame.
(In reply to Terry Kennedy from comment #20) Patch looks good to me.
(In reply to Terry Kennedy from comment #19) > This doesn't seem to correct the "If an invalid or no format is specified txz is > assumed." part of the manpage, if the info in comment 12 is correct and the default > format is now tzst. Yep, I spotted that as well while looking over the first patch I submitted and it is now fixed in https://github.com/freebsd/pkg/commit/27b286a64cab91d3eff0e5fb6358f278ab563a30. Note that this is incorrect for FreeBSD 13 -- the default is still txz there, set at configure time for pkg itself. But at least pkg's own default is now correctly stated. > Also, I don't think the "in the near future" deprecation notice in the handbook is > reasonable. Unless there's some plan to remove portsmgmt/portupgrade from the ports > tree, it is going to remain in quarterly builds for some time. db5 has been hanging > around in the ports tree long past its sell-by date. Fair comment; I'm trying to strike a balance between "don't surprise the users" and "make the user community aware that developers already think this was deprecated years ago." I originally went with just "removed in the future" but added "near" based on a comment in the review. In any case I think the discussion that may be prompted by the more alarming notice is worthwhile, and we can change the notice to match whatever is eventually decided. > I suggest two things: > > 1) Adding DEPRECATED= and EXPIRATION_DATE= to the port Makefile (the date should be > at FreeBSD 14 EOL, IMHO). This seems reasonable to me, but we should also stop packaging it for 15.x now. > 2) Adding a section in the Handbook on "Migrating from portupgrade to portmaster" A good idea; someone who is familiar with both tools will need to write it.
This all sounds good to me. With the demise of bsdstats, we can't get a feel for how many people are still using portupgrade. While there was a very small number of sites submitting stats, at least we could point to it and go "at least N people are using it". Anyway, as soon as the portupgrade patch lands, I'll close this PR.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=22f6c414960d8b0191f39bf0a98bd43f482ac637 commit 22f6c414960d8b0191f39bf0a98bd43f482ac637 Author: Baptiste Daroussin <bapt@FreeBSD.org> AuthorDate: 2025-01-16 13:14:29 +0000 Commit: Baptiste Daroussin <bapt@FreeBSD.org> CommitDate: 2025-01-16 13:18:49 +0000 ports-mgmt/portupgrade: stop warning about wrong package format pkg expect the compression format to be set via -f option, portupgrade passes this information extracting the PKG_SUFX variable which is now invariable since pkg 1.17. the .pkg suffix is always used, but the -f option still expect the compression format to be defined in same form as before: .tar, .tbz, .tgz etc. Remove the call to the -f format to remove a useless warning. PR: 284054 Submitted by: Terry Kennedy <terry-freebsd@glaver.org> ports-mgmt/portupgrade/Makefile | 2 +- .../files/patch-pkg-compression-format (new) | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-)
pushed!