I have a current jail with the oldest snapshot from 14-ALPHA2 era. If I rollback the jail to 14-ALPHA2 and do: ABI="FreeBSD:15:amd64" pkg -r /usr/local/poudriere/jails/main/ upgrade before base 78847e1e5927 and base 4e899378bf5d, then the jail would get upgraded to 15. Then, if I upgrade to world with base 78847e1e5927 and base 4e899378bf5d, then the following would happen: [...] [11/11] Fetching FreeBSD-clibs-dev-15.snap20230916153256.pkg: 100% 18 MiB 18.9MB/s 00:01 Checking integrity... done (8 conflicting) - FreeBSD-libcompiler_rt-dev-15.snap20230916153256 [base] conflicts with FreeBSD-libcompiler_rt-15.snap20230916151719 [installed] on /usr/lib/libgcc.a - FreeBSD-clang-15.snap20230916153256 [base] conflicts with FreeBSD-utilities-15.snap20230916152506 [installed] on /usr/lib/clang/16/lib/freebsd/libclang_rt.asan-x86_64.so - FreeBSD-tests-dev-15.snap20230916153256 [base] conflicts with FreeBSD-tests-15.snap20230916152506 [installed] on /usr/lib/libxo/encoder/test.enc - FreeBSD-clibs-15.snap20230916153256 [base] conflicts with FreeBSD-runtime-15.snap20230916152506 [installed] on /usr/lib/include - FreeBSD-clibs-dev-15.snap20230916153256 [base] conflicts with FreeBSD-runtime-dev-15.snap20230916152506 [installed] on /usr/include/_ctype.h - FreeBSD-clibs-dev-15.snap20230916153256 [base] conflicts with FreeBSD-utilities-15.snap20230916152506 [installed] on /usr/lib/Scrt1.o - FreeBSD-clibs-dev-15.snap20230916153256 [base] conflicts with FreeBSD-clibs-15.snap20230916152506 [installed] on /usr/lib/libcurses.a - FreeBSD-clibs-dev-15.snap20230916153256 [base] conflicts with FreeBSD-tests-15.snap20230916152506 [installed] on /usr/tests/lib/csu/Kyuafile Cannot solve problem using SAT solver, trying another plan Checking integrity... done (0 conflicting) Conflicts with the existing packages have been found. One more solver iteration is needed to resolve them. The following 15 package(s) will be affected (of 0 checked): New packages to be INSTALLED: FreeBSD-clang-dev: 15.snap20230916153256 [base] Installed packages to be UPGRADED: FreeBSD-bootloader: 15.snap20230916151719 -> 15.snap20230916153256 [base] FreeBSD-clang: 15.snap20230916152506 -> 15.snap20230916153256 [base] FreeBSD-clibs-dev: 15.snap20230916152506 -> 15.snap20230916153256 [base] FreeBSD-tests-dev: 15.snap20230916151719 -> 15.snap20230916153256 [base] Number of packages to be installed: 1 Number of packages to be upgraded: 4 The process will require 68 MiB more space. Proceed with this action? [y/N]: y [1/15] Upgrading FreeBSD-clang from 15.snap20230916152506 to 15.snap20230916153256... pkg: FreeBSD-clang-15.snap20230916153256 conflicts with FreeBSD-utilities-15.snap20230916152506 (installs files into the same place). Problematic file: /usr/lib/clang/16/lib/freebsd/libclang_rt.asan-x86_64.so If I add -f to force upgrade after this, some packages would get upgraded, but then one package won't upgrade after the forced upgrade: [...] Checking integrity... done (1 conflicting) - FreeBSD-libcompiler_rt-dev-15.snap20230916153256 [base] conflicts with FreeBSD-libcompiler_rt-15.snap20230916151719 [installed] on /usr/lib/libgcc.a Cannot solve problem using SAT solver, trying another plan Checking integrity... done (0 conflicting) Your packages are up to date. [...] To unbork the jail, I had to revert base 4e899378bf5d, then do pkg upgrade.
This is also a WITHOUT_LIB32= yes WITH_LLVM_BINUTILS= yes WITH_LLVM_CXXFILT= yes build if this matters.
After testing the unborked jail it seems that base 78847e1e5927 also causes FreeBSD-utilities-dev to be removed after pkg upgrade https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273749#c2
I have a feeling that a work around for this would be to update the non-dev packages first which would remove the unwanted symlinks (and in a few cases files) which are moving from the main package to the dev packages. I'll set up a test tomorrow and figure out a safe way to update past this change. Apologies for the breakage :(
To upgrade past this change, you need to: # pkg upgrade -y FreeBSD-utilities # pkg upgrade -y FreeBSD-utilities-dev # pkg upgrade
how can we document all these breakages and workarounds, so that people can just do the equivalent of `do-release-upgrade`?
Traditionally, this kind of thing goes in UPDATING. I'll draft an entry which explains the problem and the workaround.
UPDATING is for people updating from source. I would expect slightly more convince when updating with pkg. I'm thinking of the recent OPIE addition to UPDATING: That is something that freebsd-update + etcupdate would trip up on. As would pkg (leaving pkgsave files) but this bug here just locks up pkg, so if we want to raise PkgBase above alpha, we need to provide a mechanism for encoding workarounds.
I don't disagree about the need to document and possibly automate these workarounds but in the meantime, I added an entry to UPDATING here: https://reviews.freebsd.org/D41892
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=4ff9d270893af757b4348398c887e630d71edf6e commit 4ff9d270893af757b4348398c887e630d71edf6e Author: Doug Rabson <dfr@FreeBSD.org> AuthorDate: 2023-09-17 10:06:01 +0000 Commit: Doug Rabson <dfr@FreeBSD.org> CommitDate: 2023-09-17 12:21:06 +0000 pkgbase: Add instructions for upgrading past 78847e1 The pkg solver gets confused by the file moves and refuses to upgrade without manually upgrading FreeBSD-utilities and FreeBSD-utilities-dev first. PR: 254173, 273859 MFC after: 1 week Reviewed by: Mina Galić <freebsd@igalic.co> Differential Revision: https://reviews.freebsd.org/D41892 UPDATING | 10 ++++++++++ 1 file changed, 10 insertions(+)
(In reply to dfr from comment #4) I have tested the following after removing local reverting commit for base 4e899378bf5d, rebuilding pkgbase, and rolling back the testing jail to 14-ALPHA2, then following the mentioned steps. The following remains when doing pkg upgrade after the mentioned steps: [...] Checking integrity... done (1 conflicting) - FreeBSD-libcompiler_rt-dev-15.snap20230918100456 [base] conflicts with FreeBSD-libcompiler_rt-14.snap20230515103602 [installed] on /usr/lib/libgcc.a Cannot solve problem using SAT solver, trying another plan Checking integrity... done (0 conflicting) Your packages are up to date. [...] + I get this after upgrading current from ~yesterday that had base 4e899378bf5d reverted by following the mentioned steps: [...] Checking integrity... done (1 conflicting) - FreeBSD-libcompiler_rt-dev-15.snap20230918100456 [base] conflicts with FreeBSD-libcompiler_rt-15.snap20230916154046 [installed] on /usr/lib/libgcc.a Cannot solve problem using SAT solver, trying another plan Checking integrity... done (0 conflicting) Conflicts with the existing packages have been found. One more solver iteration is needed to resolve them. However, my another current jail with clean src.conf (no LLVM binutils) upgrades fine following the mentioned steps, from base 8ed0ecf8024d to base b15f6400376a.
I'm not sure what happened to make pkg confused about libcompiler_rt but its ok to just remove that - literally the only thing in that package is a symlink which got moved to libcompiler_rt-dev. In my testing, with the extra steps to upgrade utilities and utilities-dev first, pkg upgrade did remove libcompiler_rt since in my latest build it did not exist and cleanly upgraded the rest.
(In reply to dfr from comment #11) Seems to work fine, although autoremove wanted to remove clang first, and there was a missing dependency warning. # pkg list FreeBSD-libcompiler_rt /usr/lib/libgcc.a # file /usr/lib/libgcc.a /usr/lib/libgcc.a: symbolic link to libcompiler_rt.a # pkg remove FreeBSD-libcompiler_rt Checking integrity... done (0 conflicting) Deinstallation has been requested for the following 5 packages (of 0 packages in the universe): Installed packages to be REMOVED: FreeBSD-clang: 15.snap20230918100456 FreeBSD-clang-dev: 15.snap20230918100456 FreeBSD-clang-man: 15.snap20230825032023 FreeBSD-libcompiler_rt: 15.snap20230918104253 FreeBSD-libcompiler_rt-dev: 15.snap20230918104253 Number of packages to be removed: 5 The operation will free 211 MiB. Proceed with deinstalling packages? [y/N]: ^C # pkg remove -f FreeBSD-libcompiler_rt Checking integrity... done (0 conflicting) Deinstallation has been requested for the following 1 packages (of 0 packages in the universe): Installed packages to be REMOVED: FreeBSD-libcompiler_rt: 15.snap20230918104253 Number of packages to be removed: 1 Proceed with deinstalling packages? [y/N]: y [1/1] Deinstalling FreeBSD-libcompiler_rt-15.snap20230918104253... [1/1] Deleting files for FreeBSD-libcompiler_rt-15.snap20230918104253: 100% # pkg upgrade Updating base repository catalogue... base repository is up to date. Updating ports repository catalogue... ports repository is up to date. All repositories are up to date. Checking for upgrades (1 candidates): 100% Processing candidates (1 candidates): 100% pkg: FreeBSD-libcompiler_rt-dev has a missing dependency: FreeBSD-libcompiler_rt The following 1 package(s) will be affected (of 0 checked): Installed packages to be UPGRADED: FreeBSD-libcompiler_rt-dev: 15.snap20230918104253 -> 15.snap20230918111945 [base] Number of packages to be upgraded: 1 185 KiB to be downloaded. Proceed with this action? [y/N]: y [1/1] Fetching FreeBSD-libcompiler_rt-dev-15.snap20230918111945.pkg: 100% 185 KiB 189.4kB/s 00:01 Checking integrity... done (0 conflicting) [1/1] Upgrading FreeBSD-libcompiler_rt-dev from 15.snap20230918104253 to 15.snap20230918111945... [1/1] Extracting FreeBSD-libcompiler_rt-dev-15.snap20230918111945: 100% # pkg upgrade Updating base repository catalogue... base repository is up to date. Updating ports repository catalogue... ports repository is up to date. All repositories are up to date. Checking for upgrades (0 candidates): 100% Processing candidates (0 candidates): 100% Checking integrity... done (0 conflicting) Your packages are up to date.
From 14-ALPHA2 to base 501edb1cc76b (WITH_LLVM_BINUTILS) without any reverts: export ABI="FreeBSD:15:amd64" pkg -r /usr/local/poudriere/jails/main upgrade -y FreeBSD-utilities pkg -r /usr/local/poudriere/jails/main upgrade -y FreeBSD-utilities-dev pkg -r /usr/local/poudriere/jails/main upgrade -y pkg -r /usr/local/poudriere/jails/main remove -f FreeBSD-libcompiler_rt pkg -r /usr/local/poudriere/jails/main upgrade -y Not sure whether to keep the bug open or not, I'll leave it as-is. Thanks for the help.
Created attachment 244991 [details] v0 Same problem here, I can't upgrade FreeBSD-libcompiler_rt. The attached patch seems to fix the problem.
I don't fully understand why we have to have a special case for -dev packages with no non-dev package but I think this can be combined into the 'libcompat-dev|liby-dev' one. Probably also need the lib32 variant as well.
Created attachment 244995 [details] v1
Created attachment 244996 [details] v2
Testing with something very similar to your latest patch (I misordered the packages because I'm bad at sorting), I was able to upgrade libcompiler_rt-dev cleanly but for the full upgrade still needed to upgrade utilities and utilities-dev before the bulk upgrade.
(In reply to dfr from comment #15) Because by default we record a dep on the non-dev package for -dev package.
(In reply to Mikael Urankar from comment #17) LGTM, go ahead and commit it.
Or in fact, let Mikael commit. LGTM from me too :)
(In reply to dfr from comment #21) I took some days off and I don't have much spare time to commit it. Can you commit it please? Thanks.
(In reply to Mikael Urankar from comment #22) I'll take care of it. Thanks for taking the time to look at the problem.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=36468371ce9565bf8989ad8e561e041bfe9cb80f commit 36468371ce9565bf8989ad8e561e041bfe9cb80f Author: Mikaël Urankar <mikael@FreeBSD.org> AuthorDate: 2023-09-22 09:41:09 +0000 Commit: Emmanuel Vadot <manu@FreeBSD.org> CommitDate: 2023-09-22 09:42:21 +0000 pkgbase: Fix ucl for libcompiler_rt It's only a -dev package now so add it to the list of -dev package that don't have non-dev package. PR: 273859 release/packages/generate-ucl.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=03ef42bbe15aed9226939daf2f3b0e10183dea7e commit 03ef42bbe15aed9226939daf2f3b0e10183dea7e Author: Mikaël Urankar <mikael@FreeBSD.org> AuthorDate: 2023-09-22 09:41:09 +0000 Commit: Emmanuel Vadot <manu@FreeBSD.org> CommitDate: 2023-10-18 14:38:10 +0000 pkgbase: Fix ucl for libcompiler_rt It's only a -dev package now so add it to the list of -dev package that don't have non-dev package. PR: 273859 (cherry picked from commit 36468371ce9565bf8989ad8e561e041bfe9cb80f) release/packages/generate-ucl.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=2eab6bc3c08f8f2238a24df336fcaa019592dca0 commit 2eab6bc3c08f8f2238a24df336fcaa019592dca0 Author: Doug Rabson <dfr@FreeBSD.org> AuthorDate: 2023-09-17 10:06:01 +0000 Commit: Doug Rabson <dfr@FreeBSD.org> CommitDate: 2023-10-20 12:16:47 +0000 pkgbase: Add instructions for upgrading past 78847e1 The pkg solver gets confused by the file moves and refuses to upgrade without manually upgrading FreeBSD-utilities and FreeBSD-utilities-dev first. PR: 254173, 273859 MFC after: 1 week Reviewed by: Mina Galić <freebsd@igalic.co> Differential Revision: https://reviews.freebsd.org/D41892 (cherry picked from commit 4ff9d270893af757b4348398c887e630d71edf6e) UPDATING | 10 ++++++++++ 1 file changed, 10 insertions(+)