With BACKUP_LIBRARIES=yes, an upgrade that bumps a shared library soname creates a *-backup-* pseudo-package as documented. The library file is written to the correct location (/usr/local/lib/compat/pkg/libfoo.so.N), but the path registered in the package database is prefixed with the process's current working directory (/root/usr/local/lib/compat/pkg/libfoo.so.N when pkg is run from /root). Two consequences follow, and the second is the damaging one: 1. "pkg check -s" reports the file missing on every run, forever. On a system where "pkg check -s" runs from the daily periodic mail, that is a permanent false alarm. 2. "pkg delete" of the backup package cannot remove the file it owns. It looks under the bogus path, warns, and completes anyway. The real library is left on disk owned by nothing, inside a directory that is in the run-time linker's search path (/usr/local/libdata/ldconfig/pkg registers /usr/local/lib/compat/pkg). So a stale library silently stays resolvable after the package that was supposed to own it is gone. The bug is invisible when pkg is run with cwd "/" (see "WHY THIS IS RARELY SEEN" below), which is why it shows up under automation and not in interactive use. ENVIRONMENT =========== - FreeBSD 15.1-RELEASE-p2 amd64, pkgbase install (FreeBSD-base repo, base_release_1); ports from quarterly. - pkg 2.7.5 - BACKUP_LIBRARIES=yes and BACKUP_LIBRARY_PATH=/usr/local/lib/compat/pkg (the latter is the default). - Upgrade driven non-interactively as root over ssh (Ansible), with the working directory set to /root; no controlling terminal; "pkg upgrade -y". WHAT HAPPENED ============= A routine "pkg upgrade -y" moved knot3 3.5.5 -> 3.5.6, plus wolfssl and xapian-core, bumping five sonames. pkg created five backup packages: knot3-backup-libdnssec.so.10-20260802031620 knot3-backup-libknot.so.16-20260802031620 knot3-backup-libzscanner.so.5-20260802031620 wolfssl-backup-libwolfssl.so.44-20260802031619 xapian-core-backup-libxapian.so.30-20260802031619 The files landed in the right place: # ls -la /usr/local/lib/compat/pkg/ -rw-r--r-- 1 root wheel 138944 Aug 2 03:16 libdnssec.so.10 -rw-r--r-- 1 root wheel 318248 Aug 2 03:16 libknot.so.16 -rw-r--r-- 1 root wheel 2117160 Aug 2 03:16 libwolfssl.so.44 -rw-r--r-- 1 root wheel 1993536 Aug 2 03:16 libxapian.so.30 -rw-r--r-- 1 root wheel 683712 Aug 2 03:16 libzscanner.so.5 But every one is registered under /root/...: # pkg list knot3-backup-libknot.so.16-20260802031620 /root/usr/local/lib/compat/pkg/libknot.so.16 so "pkg check -s" reports all five as missing: # pkg check -s knot3-backup-libdnssec.so.10-20260802031620: missing file /root/usr/local/lib/compat/pkg/libdnssec.so.10 knot3-backup-libknot.so.16-20260802031620: missing file /root/usr/local/lib/compat/pkg/libknot.so.16 knot3-backup-libzscanner.so.5-20260802031620: missing file /root/usr/local/lib/compat/pkg/libzscanner.so.5 wolfssl-backup-libwolfssl.so.44-20260802031619: missing file /root/usr/local/lib/compat/pkg/libwolfssl.so.44 xapian-core-backup-libxapian.so.30-20260802031619: missing file /root/usr/local/lib/compat/pkg/libxapian.so.30 and deleting them leaves the real files behind: # pkg delete -y knot3-backup-libzscanner.so.5-20260802031620 ... [3/5] Deleting files for knot3-backup-libzscanner.so.5-20260802031620: knot3-backup-libzscanner.so.5-20260802031620: missing file /root/usr/local/lib/compat/pkg/libzscanner.so.5 [3/5] Deleting files for knot3-backup-libzscanner.so.5-20260802031620... done # pkg which /usr/local/lib/compat/pkg/libknot.so.16 /usr/local/lib/compat/pkg/libknot.so.16 was not found in the database The five libraries then sat unowned in a directory that is on the ldconfig path. They had to be removed by hand. ROOT CAUSE ========== Confirmed by reading the 2.7.5 source. Line numbers below are from the 2.7.5 tag of https://github.com/freebsd/pkg. 1. Running a shell script permanently sets ctx.pkg_rootdir to "/" ----------------------------------------------------------------- libpkg/scripts.c:117-119, inside pkg_script_run(): setenv("PKG_NAME", pkg->name, 1); setenv("PKG_PREFIX", pkg->prefix, 1); if (ctx.pkg_rootdir == NULL) ctx.pkg_rootdir = "/"; setenv("PKG_ROOTDIR", ctx.pkg_rootdir, 1); This runs in the parent process; posix_spawn() is not called until about 60 lines later. It is a sticky mutation of global state: once any package in the transaction runs a shell script, ctx.pkg_rootdir is "/" instead of NULL for the rest of the pkg process. The equivalent code in libpkg/lua_scripts.c:88-89 sits inside the fork()ed child at line 59, so Lua scripts do not trigger this. Only shell scripts do. 2. With pkg_rootdir == "/", the backup path loses its leading slash ------------------------------------------------------------------- libpkg/backup_lib.c:35-46: static const char * backup_library_relative_path(void) { const char *path = ctx.backup_library_path; if (ctx.pkg_rootdir != NULL) { size_t rootlen = strlen(ctx.pkg_rootdir); if (strncmp(path, ctx.pkg_rootdir, rootlen) == 0) path += rootlen; } return (path); } With ctx.pkg_rootdir == "/", rootlen == 1 and the strncmp prefix test matches every absolute path, so "path += 1" strips the leading slash: /usr/local/lib/compat/pkg -> usr/local/lib/compat/pkg The return value is now a relative path. There is no component-boundary check. 3. The write path still works, which is why the file lands correctly --------------------------------------------------------------------- backup_lib.c:155-164 uses mkdirat/openat against p->rootfd via RELATIVE_PATH(), defined at libpkg/private/utils.h:22 as (&p[strspn(p, "/")]). That strips leading slashes anyway, so it behaves identically for the relative and absolute forms. backup_lib.c:178-180 builds the temp file path: rootdir = ctx.pkg_rootdir != NULL ? ctx.pkg_rootdir : ""; snprintf(tmppath, sizeof(tmppath), "%s%s/%s", rootdir, bkpath, tmpname); With rootdir == "/" and bkpath == "usr/local/lib/compat/pkg", this re-concatenates to /usr/local/lib/compat/pkg/.libfoo.so.N.XXXXXX, the correct absolute path. The stripped slash is silently restored here. 4. The registration path does not; it gets getcwd() prepended -------------------------------------------------------------- backup_lib.c:108: xasprintf(&lpath, "%s/%s", backup_library_relative_path(), libname); pkg_addfile(pkg, lpath, sum, false); lpath is "usr/local/lib/compat/pkg/libfoo.so.N", which is relative. pkg_addfile() forwards to pkg_addfile_attr(), which at libpkg/pkg.c:518 calls: path = pkg_absolutepath(path, abspath, sizeof(abspath), false); and pkg_absolutepath() at libpkg/utils.c:789-795: if (src_len != 0 && src[0] != '/') { if (fromroot) *dest = '/'; /* relative path, we use cwd */ else if (getcwd(dest, dest_size) == NULL) return (NULL); } fromroot is false, so the current working directory is prepended. Run from /root, the stored path becomes /root/usr/local/lib/compat/pkg/libfoo.so.N. That string is what pkgdb_register_pkg() writes to the database. So the write path and the registration path are computed differently: backup_library() reassembles an absolute path, while register_backup() hands a relative one to a function that resolves it against cwd. 5. Ordering: a single-package upgrade is enough ------------------------------------------------ libpkg/pkg_add.c:1427 runs the old package's shell pre-deinstall script, and libpkg/pkg_add.c:1445 calls pkg_maybe_backup_library(), eighteen lines later, in the same function, for the same package: ret = pkg_script_run(old, PKG_SCRIPT_PRE_DEINSTALL, (old != NULL), noexec); ... while (pkg_files(old, &f) == EPKG_OK) { if (!pkg_has_file(new, f->path) || ...) { pkg_maybe_backup_library(db, old, f->path); So if the package being upgraded has a shell pre-deinstall script, it poisons ctx.pkg_rootdir immediately before its own libraries are backed up. Failing that, any earlier package's pre-install or post-install script in the same transaction does it. This is why the failure looks intermittent rather than tied to a specific package. WHY THIS IS RARELY SEEN ======================= Every consumer of the stored path ("pkg check", "pkg delete") runs it through RELATIVE_PATH() before the *at() call against rootfd, and that macro strips all leading slashes via strspn(p, "/"). - cwd "/": pkg_absolutepath() yields //usr/local/lib/compat/pkg/libfoo.so.N, RELATIVE_PATH() strips both slashes, the path resolves correctly, and the bug is invisible. - cwd anything else: the extra component survives and the bug is visible. Interactive administrators upgrading from "/", and anything running with cwd "/", will never see it. Ansible, cron jobs, and CI will. CORROBORATING CHECKS ==================== If the analysis is right, these should hold on the affected system: 1. Backup packages have flatsize 0. backup_lib.c:120-124 sums sizes via fstatat(pkg->rootfd, RELATIVE_PATH(f->path), ...), which fails on the bogus path, so pkg->flatsize stays 0: pkg info -s knot3-backup-libknot.so.16-20260802031620 # expect 0B 2. Backup packages provide no shlibs. pkg_analyse_files() (libpkg/pkg_abi.c:491) cannot open the file either, so nothing is recorded: pkg info -B knot3-backup-libknot.so.16-20260802031620 # expect empty 3. The prefix tracks the working directory. Running the upgrade from a different directory should change the prefix accordingly: cd /tmp && pkg upgrade -y # expect /tmp/usr/local/lib/compat/pkg/... cd / && pkg upgrade -y # expect the bug to disappear REPRODUCTION ============ 1. On a FreeBSD 15.1 system with pkg 2.7.1-2.7.5, set BACKUP_LIBRARIES=yes in /usr/local/etc/pkg.conf. 2. As root, from a working directory other than "/" (e.g. cd /root), upgrade a package that (a) bumps a shared library soname and (b) has, or is preceded in the transaction by a package that has, a shell pre-deinstall, pre-install or post-install script. knot3 3.5.5 -> 3.5.6 did it here via libknot.so.16. 3. "pkg check -s": the backup package reports its file missing under /root/usr/local/lib/compat/pkg/. 4. "ls /usr/local/lib/compat/pkg/": the file is actually there. 5. cd / and repeat: the problem does not occur. REGRESSION RANGE AND FIX ======================== Introduced in 2.7.1 by commit dcd492f4f ("backup_libraries: now fully respectes rootdir", 2026-04-14, the 2.7-branch cherry-pick of c79dfc9e1 on main), which added backup_library_relative_path(). Fixed on main by commit c2b897612 ("backup_lib: prevent accumulating old libs and badly match them", 2026-05-01), first released in 2.8.0, which added a component-boundary check: if (strncmp(path, ctx.pkg_rootdir, rootlen) == 0 && (path[rootlen] == '/' || path[rootlen] == '\0')) path += rootlen; With ctx.pkg_rootdir == "/", path[1] is 'u', so the strip no longer happens and the path stays absolute. Two things worth flagging: - The fix looks incidental. That commit's subject and apparent intent are about backup accumulation and matching, not rootdir handling; the boundary check fixes this bug as a side effect. It would be worth an explicit regression test so it is not undone. - The 2.7 branch was never fixed. 2.7.1 through 2.7.5 are all affected, and 2.7 is what quarterly ships. A backport of just the boundary check is a two-line change. There are also two latent issues the boundary check does not address, which may be worth fixing independently: - pkg_addfile() resolving package file paths against getcwd() is a surprising default for database registration. Passing fromroot=true, or asserting the path is absolute, at backup_lib.c:109 would make this class of bug impossible rather than merely unreachable. - pkg_script_run() mutating the global ctx.pkg_rootdir as a side effect of running a script is a trap for any other code that distinguishes NULL from "/". Using a local for the setenv() value would remove it. IMPACT ====== - Permanent false positives from "pkg check -s", which is part of the stock periodic daily security/status mail. This trains operators to ignore that report, which is the real cost. - "pkg delete" silently fails to remove files it owns, leaving unowned shared libraries in an ldconfig search path. Anything that later asks for the old soname resolves to a stale library instead of failing loudly. - Because the backup packages also register no provided shlibs and zero flatsize (see CORROBORATING CHECKS), the feature does not actually deliver the compatibility guarantee it advertises on affected versions. WORKAROUNDS =========== Any one of: 1. Upgrade to pkg 2.8.0 or later. 2. Run pkg with cwd "/" (for Ansible, set "chdir: /" on the task). 3. Turn the feature off (it is not the default): # remove BACKUP_LIBRARIES=yes from /usr/local/etc/pkg.conf pkg delete -y '<pkg>-backup-*' rm -f /usr/local/lib/compat/pkg/* # pkg cannot remove these itself ldconfig -R SEPARATE DOCUMENTATION BUG ========================== pkg.conf(5) still describes the pre-2.7.1 behaviour. At docs/pkg.conf.5 in 2.7.5 it says the backup package is named after the original package "with the suffix -backup-libraries appended" and that "the package version will be bumped whenever an additional library is backed up". The implementation (backup_lib.c:64 and :112-115) instead creates one package per library, named <pkg>-backup-<libname>, with the version set to a %Y%m%d%H%M%S timestamp: (void)xasprintf(&name, "%s-backup-%s", orig->name, libname); ... strftime(buf, sizeof(buf), "%Y%m%d%H%M%S", localtime(&t)); pkg->version = xstrdup(buf); This mismatch is still present on main.