Bug 273601 - freebsd-update fills up /var/db/freebsd-update/files and never cleans them
Summary: freebsd-update fills up /var/db/freebsd-update/files and never cleans them
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Ed Maste
URL: https://reviews.freebsd.org/D42022
Keywords:
Depends on:
Blocks:
 
Reported: 2023-09-06 17:20 UTC by David Chisnall
Modified: 2023-12-27 12:44 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Chisnall freebsd_committer freebsd_triage 2023-09-06 17:20:22 UTC
There's a long forum thread about this going back almost 10 years:

https://forums.freebsd.org/threads/is-it-safe-to-clean-var-db-freebsd-update-files.45204/

FreeBSD update downloads files to /var/db/freebsd-update/files, which the man mage documents as the working directory for temporary files.  It is *probably* safe to delete them (seems to be consensus on the forum, but freebsd-update never does).  Deleting them is non-trivial because it downloads a *huge* number of tiny files here.  

I've seen UFS-based installs die because they run out of inodes for all of these files.  ZFS installs don't, but on a small VM with an 8 GiB disk (that's small now, yay for living in the future), I ran out of disk space with this.  After pruning it a few months ago, it's grown to 1.2G.

At the very lest, it would be great for someone to document *when* it is safe to delete these things.  Ideally, we'd use tmpfs for most of them (they're downloaded and then extracted immediately) and store the ones that are needed after reboot somewhere that persists and delete them after they're installed.

I believe that they might be kept for freebsd-update rollback, but since 12.something freebsd-update has created ZFS boot environments (thank you whoever added that feature!) and so the rollback feature is not necessary on ZFS systems (and has never worked for me when I've tried it, but maybe I was just unlucky).
Comment 1 Colin Percival freebsd_committer freebsd_triage 2023-09-07 01:41:48 UTC
Yes, all of those files will be used if you `freebsd-update rollback` all the way back in history.  When I first designed this code (for security updates, not upgrading between releases) the number of files was small so it was never an issue.

We should probably have a `freebsd-update purge` which make it impossible to rollback and cleans up all those files.

OTOH we're going to be moving to pkgbase any decade now and freebsd-update won't be needed any more... right...?
Comment 2 David Chisnall freebsd_committer freebsd_triage 2023-09-07 06:54:08 UTC
(In reply to Colin Percival from comment #1)

> OTOH we're going to be moving to pkgbase any decade now and freebsd-update won't be needed any more... right...?

Yup, any decade now...
Comment 3 Ed Maste freebsd_committer freebsd_triage 2023-09-19 17:33:24 UTC
(In reply to David Chisnall from comment #2)
pkgbase packages will be provided from FreeBSD 14.0 on, as a sort of technology preview. They won't yet be used by the FreeBSD installer etc., but will be available for experimentation and use by projects like dfr's cloud image creation tools.
Comment 4 commit-hook freebsd_committer freebsd_triage 2023-10-04 14:47:27 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=30beebe444f5f415a388be9df91107246bd32674

commit 30beebe444f5f415a388be9df91107246bd32674
Author:     Ed Maste <emaste@FreeBSD.org>
AuthorDate: 2023-09-29 15:47:41 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-10-04 14:45:57 +0000

    freebsd-update: add a note about when files may be deleted

    Files under /var/db/freebsd-update are required during the upgrade
    process, and to support rollback.  They may be deleted if no upgrade is
    in progress and rollback will not be required.

    PR:             273601
    Reviewed by:    bcr
    Sponsored by:   The FreeBSD Foundation
    Differential Revision: https://reviews.freebsd.org/D42022

 usr.sbin/freebsd-update/freebsd-update.8 | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Comment 5 commit-hook freebsd_committer freebsd_triage 2023-10-13 15:44:41 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=ffe4456d7d9f2c6bb14e9b747766adc3042f0b9e

commit ffe4456d7d9f2c6bb14e9b747766adc3042f0b9e
Author:     Ed Maste <emaste@FreeBSD.org>
AuthorDate: 2023-09-29 15:47:41 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-10-13 14:13:38 +0000

    freebsd-update: add a note about when files may be deleted

    Files under /var/db/freebsd-update are required during the upgrade
    process, and to support rollback.  They may be deleted if no upgrade is
    in progress and rollback will not be required.

    PR:             273601
    Reviewed by:    bcr
    Sponsored by:   The FreeBSD Foundation
    Differential Revision: https://reviews.freebsd.org/D42022

    (cherry picked from commit 30beebe444f5f415a388be9df91107246bd32674)

 usr.sbin/freebsd-update/freebsd-update.8 | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Comment 6 commit-hook freebsd_committer freebsd_triage 2023-10-13 17:29:04 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=4dd8132fc565cb08b92a918a1cae52f246417999

commit 4dd8132fc565cb08b92a918a1cae52f246417999
Author:     Ed Maste <emaste@FreeBSD.org>
AuthorDate: 2023-09-29 15:47:41 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-10-13 17:27:03 +0000

    freebsd-update: add a note about when files may be deleted

    Files under /var/db/freebsd-update are required during the upgrade
    process, and to support rollback.  They may be deleted if no upgrade is
    in progress and rollback will not be required.

    PR:             273601
    Reviewed by:    bcr
    Sponsored by:   The FreeBSD Foundation
    Differential Revision: https://reviews.freebsd.org/D42022

    (cherry picked from commit 30beebe444f5f415a388be9df91107246bd32674)
    (cherry picked from commit ffe4456d7d9f2c6bb14e9b747766adc3042f0b9e)

 usr.sbin/freebsd-update/freebsd-update.8 | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Comment 7 commit-hook freebsd_committer freebsd_triage 2023-10-13 23:36:17 UTC
A commit in branch releng/14.0 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=80b003e816849b7809e7c64e01bf3acd543e4357

commit 80b003e816849b7809e7c64e01bf3acd543e4357
Author:     Ed Maste <emaste@FreeBSD.org>
AuthorDate: 2023-09-29 15:47:41 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-10-13 23:17:39 +0000

    freebsd-update: add a note about when files may be deleted

    Files under /var/db/freebsd-update are required during the upgrade
    process, and to support rollback.  They may be deleted if no upgrade is
    in progress and rollback will not be required.

    PR:             273601
    Reviewed by:    bcr
    Sponsored by:   The FreeBSD Foundation
    Differential Revision: https://reviews.freebsd.org/D42022

    (cherry picked from commit 30beebe444f5f415a388be9df91107246bd32674)
    (cherry picked from commit ffe4456d7d9f2c6bb14e9b747766adc3042f0b9e)

    Approved by:    re (gjb)

 usr.sbin/freebsd-update/freebsd-update.8 | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Comment 8 Mark Linimon freebsd_committer freebsd_triage 2023-12-27 12:44:48 UTC
^Triage: assign to committer who made the change and MFCed.