Bug 258863

Summary: "freebsd-update fetch" unnecessary IO when no updates [Fix provided]
Product: Base System Reporter: tux2bsd <tux2bsd>
Component: binAssignee: freebsd-bugs (Nobody) <bugs>
Status: New ---    
Severity: Affects Many People CC: emaste, grahamperrin, pi
Priority: --- Keywords: performance
Version: Unspecified   
Hardware: Any   
OS: Any   
URL: https://reviews.freebsd.org/D32570

Description tux2bsd 2021-10-02 10:12:33 UTC 
The IO trashing has likely been present in all versions of freebsd-update.  I have a full description and code submitted the improvement against CURRENT here:

https://github.com/freebsd/freebsd-src/pull/543

Copy of the description:

When there are no updates available there is no need run through the the entirety of fetch_run ()

This introduces a simple check and its most significant benefit is avoiding the disk intensive phase of "fetch" (i.e. Inspecting system...) as there were no updates anyway.

On IO bound hardware, in my case a Raspberry Pi 3, this improvement has reduced runtime duration of freebsd-update fetch from 1m20s down to 1.5s.

Even on fast SSD hardware freebsd-update fetch currently takes ~15 seconds, this changes reduces the time to sub second.

Again for emphasis: This is for the case when there are no updates. Check early and if the version matches, its done.


I trust the github pull request has everything required, if not please let me know.
Comment 2 tux2bsd 2021-10-02 10:41:25 UTC 
That was supposed to say: "The IO thrashing..."
Comment 3 tux2bsd 2021-10-03 01:24:37 UTC 
Jose Luis Duran (jlduran) provided some valuable feedback on my patch on Github regarding style and a good question about the case if there is only a kernel update.

I came up with a more robust comparison, simply comparing tags rather than trying to ascertain version information (the more obvious approach in hindsight).

https://github.com/freebsd/freebsd-src/pull/543/commits
Comment 4 Graham Perrin freebsd_committer freebsd_triage 2021-10-03 13:29:22 UTC 
Thanks for keeping us in the loop. 

FYI <https://github.com/freebsd/freebsd-src> is a

> … (read-only mirror)
Comment 5 tux2bsd 2021-10-09 08:54:57 UTC 
Final squashed commit pushed to Github 2 days ago, I'm unsure if the "review" feature of Github is making itself known to Ed Maste.

https://github.com/freebsd/freebsd-src/pull/543
Comment 6 tux2bsd 2021-10-21 08:42:00 UTC 
https://reviews.freebsd.org/D32570
Comment 7 tux2bsd 2021-12-11 20:17:20 UTC 
The github link(s) above are no longer relevant, see Phabricator:

https://reviews.freebsd.org/D32570
Comment 9 tux2bsd 2022-03-24 10:30:27 UTC 
The previous workaround attempt was the right idea but a crude implementation.

This is much better:

https://github.com/tux2bsd/freebsd-update-probe/blob/main/freebsd-update-probe.sh
Comment 10 tux2bsd 2022-03-24 10:49:35 UTC 
root@freebsd-dns:~ # time /root/freebsd-update-probe.sh || freebsd-update fetch install
freebsd-update-probe.sh found matching tags, no update needed.
0.067u 0.141s 0:00.60 33.3%     119+187k 0+0io 0pf+0w

root@freebsd-dns:~ # time freebsd-update fetch install
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 2 mirrors found.
Fetching metadata signature for 13.0-RELEASE from update2.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

No updates needed to update system to 13.0-RELEASE-p10.
No updates are available to install.
76.380u 22.593s 1:31.64 107.9%  22+170k 0+0io 0pf+0w


On a raspberry pi 3B
Comment 11 tux2bsd 2022-03-24 10:57:22 UTC 
With regard to licensing I made it 2 clause BSD as is freebsd-update from Colin P .  A few sections of the code are from freebsd-update direct.  The intent is this is for the FreeBSD project/community, let me know if anything needs adjusting with regard to this aspect.
Comment 12 tux2bsd 2022-03-26 22:02:56 UTC 
My workaround, freebsd-update-probe.sh, provides a definitive measure with regards to room for improvement within freebsd-update.  The previous work in https://reviews.freebsd.org/D32570 shows where the fix should be applied but the intertwined nature of freebsd-update does not make this as straight forward as it should be (I bowed out at jail handling, as I do not use them).

freebsd-update-probe.sh, satisfies my needs and I have shared it for other people to use if they like.
Comment 13 tux2bsd 2022-04-01 21:37:57 UTC 
Until this bug is fixed use this, I've provided this utility:

https://github.com/tux2bsd/freebsd-update-probe