Bug 269810 - ports-mgmt/pkg: how to restore pkg database
Summary: ports-mgmt/pkg: how to restore pkg database
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-pkg (Nobody)
URL: https://docs.freebsd.org/en/books/han...
Keywords: needs-patch
Depends on:
Blocks:
 
Reported: 2023-02-24 21:20 UTC by mfv
Modified: 2023-08-22 09:05 UTC (History)
1 user (show)

See Also:
grahamperrin: maintainer-feedback? (pkg)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description mfv 2023-02-24 21:20:57 UTC
Now that 'pkg backup' has been removed I do not know how to restore a corrupted pkg database. The FreeBSD manual at section '4.4.11. Restoring the Package Database' shows 2 methods.  The 1st methid is the now-removed 'pkg backup -r' while the 2nd method is implied by the use of the compressed file created by 'periodic daily', that is, a compressed version of a sqlite3 dump.

However there are no instructions how to create/restore a new database from the file created by 'periodic daily'.  I have uncompressed pkg.sql.xz and then read pkg.sql into a news database using sqlite3 but the new database is much smaller than the original database, local.sqlite.

In summary what are the manual commands to create/restore a new pkg database from 'periodic daily'.  Alternatively, would there be any harm to just manually make backup copies of local.sqlite, say as local.sqlite.1, local.sqlite.2, etc.

With thanks,
Marek
Comment 1 Graham Perrin freebsd_committer freebsd_triage 2023-02-25 00:38:57 UTC
I'm aware of this: 

xzcat /var/backups/pkg.sql.xz | pkg shell

pkg-shell(8)
<https://man.freebsd.org/cgi/man.cgi?query=pkg-shell&sektion=8&manpath=freebsd-ports>

> … Extreme care should be taken when using this command. …
Comment 2 mfv 2023-02-25 14:29:14 UTC
Hello Graham,

Thanks for the quick response.  I tried: 'xzcat /var/backups/pkg.sql.xz | pkg shell'

but ended up with the exact same result.  Here is a directory listing of /var/db/pkg:

-rw-r--r--  1 root  wheel         158  2023.Feb.16  19:56:30  FreeBSD.meta
-rw-r--r--  1 root  wheel  58,900,480  2023.Feb.25  08:53:06  local.sqlite
-rw-r--r--  1 root  wheel  63,741,952  2023.Feb.23  13:13:34  local.sqlite.one
-rw-r--r--  1 root  wheel  54,980,608  2023.Feb.16  19:56:30  repo-FreeBSD.sqlite
-rw-r--r--  1 root  wheel           0  2023.Feb.18  09:46:13  repo-FreeBSD.sqlite-journal
-r--r--r--  1 root  wheel   7,449,709  2023.Feb.25  07:51:28  vuln.xml
-r--r--r--  1 root  wheel   6,316,391  2020.Aug.25  16:53:50  vuln.xml.org

local.sqlite.one is the original file and local.sqlite was created by the xzcat command.  There original file is 4,841,472 bytes larger than the file created by '/usr/local/etc/periodic/daily/411.pkg-backup'.

Let me know if I should send additionl data/information.

Cheers ...
Marek
Comment 3 Graham Perrin freebsd_committer freebsd_triage 2023-02-25 18:01:16 UTC
(In reply to mfv from comment #2)

Maintainer feedback requested. 

(I took the bug from a documentation perspective, feel free to retake it.)
Comment 4 Baptiste Daroussin freebsd_committer freebsd_triage 2023-02-27 08:02:55 UTC
the fact that the new db is smaller means nothing, the original one has lived for a while and may benefit from a vaccum for example to release free blocks if needed.

xzcat /var/backups/pkg.sql.xz | pkg shell

is the right way to retore a corrupted db.
Comment 5 Baptiste Daroussin freebsd_committer freebsd_triage 2023-02-27 08:04:09 UTC
btw to not use sqlite, but use pkg shell this is important because pkg shell do set some pragma in the db.
Comment 6 Graham Perrin freebsd_committer freebsd_triage 2023-08-22 08:08:27 UTC
Thank you, Mark. 

Now, a quick test (not intending to change the assignee): 

- if change occurs, it might be a bug

- if no change, please ignore me (apologies for the noise) …
Comment 7 Graham Perrin freebsd_committer freebsd_triage 2023-08-22 08:15:03 UTC
(In reply to Graham Perrin from comment #6)

Yep, sorry. Today (and on the 20th) I forgot the limitations of auto-assignment. Re: bug 266910 comment 2, 

☑ Reset Assignee to default

– the default is not the maintainer.
Comment 8 Graham Perrin freebsd_committer freebsd_triage 2023-08-22 09:05:11 UTC
Back closer to topic, in an effort to avoid confusion: removal of <https://github.com/freebsd/freebsd-doc/pull/117> from the See Also field of this bug report was not specific to this report. This one removal was, naturally, consequent to a broader request for (indiscriminate) closure of all my PRs in that area. 

----

Back on topic, for readers who may be unfamiliar with features of GitHub: 

    .diff and .patch files do remain available, as shown below (their 
                     URLs are disallowed for See Also purposes).

<https://github.com/freebsd/freebsd-doc/pull/117.diff>

<https://github.com/freebsd/freebsd-doc/pull/117.patch>     

----

Looking ahead, more generally: Bugmeister might like to advise whether it's appropriate to have the 'needs-patch' keyword in situations such as this, where a patch file is available but not directly attached. Reports with no non-obsolete patch attached can _not_ be found by queries such as this: 

<https://bugs.freebsd.org/bugzilla/buglist.cgi?cmdtype=runnamed&namedcmd=all%20open%20reports%20with%20a%20non-obsolete%20patch%2C%20by%20assignee>

> all open reports with a non-obsolete patch, by assignee

For readers to _not_ misconstrue needs-patch as no-patch, my instinct is to misuse the deprecated 'patch' keyword ;-)

Yep, the one that I long ago batch-applied before the deprecated keyword was marked as such. Oops.

Of course, I'll not apply it :-)

This discussion can continue in private. No rush. Thanks.