Bug 254050

Summary: pkgbase: pkg autoremove is removing packages it shouldn't
Product: Base System Reporter: Mina Galić <freebsd>
Component: miscAssignee: freebsd-pkgbase (Nobody) <pkgbase>
Status: Closed FIXED    
Severity: Affects Some People CC: dfr, emaste, grahamperrin, manu, marklmi26-fbsd, pkgbase, rosenke
Priority: --- Keywords: needs-qa, pkgbase
Version: CURRENT   
Hardware: Any   
OS: Any   

Description Mina Galić freebsd_triage 2021-03-05 22:46:47 UTC
setting up a jail with pkgbase:

meena@beastix ~> sudo pkg -r /isolates/pkg-auto-remove-bug/root install -r FreeBSD-base FreeBSD-clibs \
                                              FreeBSD-clibs-dev FreeBSD-libexecinfo FreeBSD-rc \
                                              FreeBSD-runtime FreeBSD-utilities FreeBSD-vi \
                                              FreeBSD-zoneinfo
Updating FreeBSD-base repository catalogue...
Fetching meta.conf: 100%    164 B   0.2kB/s    00:01
Fetching packagesite.tzst: 100%   30 KiB  31.0kB/s    00:01
Processing entries: 100%
FreeBSD-base repository update completed. 332 packages processed.
All repositories are up to date.
The following 25 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        FreeBSD-clibs: 14.snap20210302214559 [FreeBSD-base]
        FreeBSD-clibs-dev: 14.snap20210302214559 [FreeBSD-base]
        FreeBSD-libarchive: 14.snap20210225154922 [FreeBSD-base]
        FreeBSD-libbegemot: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libbsdxml: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libbsm: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libbz2: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libdwarf: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libefivar: 14.snap20210225154922 [FreeBSD-base]
        FreeBSD-libevent1: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libexecinfo: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libldns: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-liblzma: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libmagic: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libnetmap: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libopie: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libregex: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libsmb: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libsqlite3: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-libucl: 14.snap20210220023449 [FreeBSD-base]
        FreeBSD-rc: 14.snap20210227073719 [FreeBSD-base]
        FreeBSD-runtime: 14.snap20210302214559 [FreeBSD-base]
        FreeBSD-utilities: 14.snap20210302214559 [FreeBSD-base]
        FreeBSD-vi: 14.snap20210227073719 [FreeBSD-base]
        FreeBSD-zoneinfo: 14.snap20210220023449 [FreeBSD-base]

Number of packages to be installed: 25

The process will require 263 MiB more space.
66 MiB to be downloaded.

Proceed with this action? [y/N]: y
<snip>

so, everything looks fine!
we can chroot into this jail and test basic things out:


meena@beastix ~> sudo chroot /isolates/pkg-auto-remove-bug/root
root@beastix:/ # su -
root@beastix:~ # logout
root@beastix:/ # exit

cool. sh works, su works, let's remove ""unneeded"" packages:

meena@beastix ~> sudo pkg -r /isolates/pkg-auto-remove-bug/root autoremove
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 17 packages:

Installed packages to be REMOVED:
        FreeBSD-libarchive: 14.snap20210225154922
        FreeBSD-libbegemot: 14.snap20210220023449
        FreeBSD-libbsdxml: 14.snap20210220023449
        FreeBSD-libbsm: 14.snap20210220023449
        FreeBSD-libbz2: 14.snap20210220023449
        FreeBSD-libdwarf: 14.snap20210220023449
        FreeBSD-libefivar: 14.snap20210225154922
        FreeBSD-libevent1: 14.snap20210220023449
        FreeBSD-libldns: 14.snap20210220023449
        FreeBSD-liblzma: 14.snap20210220023449
        FreeBSD-libmagic: 14.snap20210220023449
        FreeBSD-libnetmap: 14.snap20210220023449
        FreeBSD-libopie: 14.snap20210220023449
        FreeBSD-libregex: 14.snap20210220023449
        FreeBSD-libsmb: 14.snap20210220023449
        FreeBSD-libsqlite3: 14.snap20210220023449
        FreeBSD-libucl: 14.snap20210220023449

Number of packages to be removed: 17

The operation will free 11 MiB.

Proceed with deinstalling packages? [y/N]: y

and try again:

meena@beastix ~> sudo chroot /isolates/pkg-auto-remove-bug/root
root@beastix:/ # su -
ld-elf.so.1: Shared object "libbsm.so.3" not found, required by "su"
root@beastix:/ #

we just broke a jail that was working perfectly fine a second ago.
Comment 1 Mark Millard 2021-03-06 01:47:02 UTC
(In reply to Mina Galić from comment #0)

I wonder if the pkg command for automremove should
have been using -c to operate in a chroot context
instead of using -r, -c making it more likely see
a self consistent context?

sudo pkg -c /isolates/pkg-auto-remove-bug/root autoremove

For reference:

     -c ⟨chroot path⟩, --chroot ⟨chroot path⟩
             pkg will chroot in the ⟨chroot path⟩ environment.

(I had to look it up to notice the -c vs -r
distinction.)
Comment 2 Mina Galić freebsd_triage 2021-03-06 06:13:40 UTC
that's a very good point! 

i should see if pkg -c or pkg -j make any other decisions
Comment 3 Henrik Rosenke 2022-01-07 08:11:52 UTC
I got the Same Problem in 12 Stable Revision 370727. After a basic Install of ca 77 packages pkg wants to remove 68 of them.

[root@dsssrvt4j1 /usr/local/etc/rc.d]# pkg info -d FreeBSD-libarchive
FreeBSD-libarchive-12.2.s20220105232846:
[root@dsssrvt4j1 /usr/local/etc/rc.d]# pkg info -r FreeBSD-libarchive
FreeBSD-libarchive-12.2.s20220105232846:

My current workaroung when setting up jails is 
for port in $(jexec ${jailname} pkg autoremove -n | grep FreeBSD | xargs -n 1); do
    jexec ${jailname} pkg lock -y ${port}
done
Comment 4 dfr 2022-08-05 15:24:51 UTC
I'm wondering if this is related to bug 265570. Somehow, the solver is deciding that these lib packages are not needed - in the linked bug, I had problems with a rogue package in the regular ports repo which claims to provide most of the base libraries.
Comment 5 dfr 2022-08-05 15:30:46 UTC
Disabling the FreeBSD repo doesn't change the behaviour of autoremove so probably nothing to do with bug 265570
Comment 6 Emmanuel Vadot freebsd_committer freebsd_triage 2022-08-05 15:31:39 UTC
No, it's because the solver doesn't take into account the shlib_needed property of the manually installed packages.
Since there is no strict dependencies between those packages, pkg decide to remove those who were not installed directly by the user.
Comment 7 dfr 2022-08-05 15:44:21 UTC
Looking at pkg issue list, this problem has been around for a long time. Should we add explicit dependencies in FreeBSD-base? That would help defend against bad packages like gnatcross-sysroot-aarch64 as well as fixing autoremove.
Comment 8 Emmanuel Vadot freebsd_committer freebsd_triage 2022-08-05 15:47:03 UTC
(In reply to dfr from comment #7)
No, both are pkg bugs that someone (tm) needs to fix.
Comment 9 Mina Galić freebsd_triage 2022-08-05 15:52:30 UTC
perhaps we should link the appropriate pkg bugs on GitHub here and vice versa.
Comment 10 Mina Galić freebsd_triage 2022-12-16 08:03:05 UTC
(In reply to Mark Millard from comment #1)
I forgot to report back test results

-c / -r / -j seem to make no difference

on the host system, autoremove suggested I remove Kerberos, i obliged, because i don't use Kerberos, only to be reminded five minutes later that ssh links to it

i have now worked around this issue by pkg-set(8)ting affected packages as not automatic (-A 0).

This was easy for the host system, where my fetch and Kerberos were affected, but it's a lot of packages in jails, and that mystifies me.
Which packages on the host make it so that a majority of PkgBase packages are marked as actually needed?
Comment 11 Baptiste Daroussin freebsd_committer freebsd_triage 2022-12-16 08:11:11 UTC
fixed with pkg 1.19.0