When setting compression globally, i.e. on a root dataset, one has to be careful as such datasets are not bootable. And dedup doesn't work on blocks with different compression which limits at how many cycles one can throw away to save space. This is a feature request. How-To-Repeat: $ zpool create -O compression=gzip -R /mnt/foo foo gptid/A-B-C-D-E $ qemu -nographic foo.img /boot.config: -h ZFS: unsupported compression algorithm gzip-6 ZFS: i/o error - all block copies unavailable Invalid format FreeBSD/x86 boot Default: foo:/boot/kernel/kernel boot: # in case zfsloader is compressed by off/lzjb/zle $ qemu -nographic foo.img /boot.config: -h Consoles: serial port BIOS drive C: is disk0 BIOS 637kB/130036kB available memory FreeBSD/x86 ZFS enabled bootstrap loader, Revision 1.1 (foo@bar, Tue Dec 14 00:38:42 UTC 2010) ZFS: unsupported compression algorithm gzip-6 ZFS: i/o error - all block copies unavailable start not found - ZFS: unsupported compression algorithm gzip-6 ZFS: i/o error - all block copies unavailable ZFS: unsupported compression algorithm gzip-6 ZFS: i/o error - all block copies unavailable ZFS: unsupported compression algorithm gzip-6 ZFS: i/o error - all block copies unavailable ZFS: unsupported compression algorithm gzip-6 ZFS: i/o error - all block copies unavailable can't load 'kernel' Type '?' for a list of commands, 'help' for more detailed help. OK
State Changed From-To: open->suspended We are eagerly awaiting patches for this already well-known issue. Currently only lzjb decmpression is supported in zfs boot chain.
batch change: For bugs that match the following - Status Is In progress AND - Untouched since 2018-01-01. AND - Affects Base System OR Documentation DO: Reset to open status. Note: I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.
I can confirm that this affects 13-STABLE. It would be great if this could be fixed because it would allow a 30% reduction in the size of VM root disks relative to lz4. A base system install for 13-STABLE is 547M with lz4, 377M with gzip-9. For cloud deployments, this quickly adds up.
If nobody is on the case, I can provide a solution for review in a few days.
Created attachment 231211 [details] ZFS: support booting from gzip-compressed volume Unfortunately, I'm not a committer, so here is a diff of the patch I made and tested in the UEFI environment. Also, here is a link to the Pull request in Github, if you prefer it: https://github.com/mezantrop/freebsd-src/commit/36e08c053273e0b4d8f0bbee5cd53450c3e80ea2
If the feature is not needed, perhaps the case should be closed.
zstd will provide the same compression as gzip, but faster Although I don't see any reason we would decide to exclude gzip support, other than the increased size of zfsboot/loader etc. What does Toomas thing?
This patch looks good. It leverages code we already have in the boot loader (inflate.o and friends) so the size increase should be minor (should test the before/after size). zstd is also in the boot loader (or at least compiled for the boot loader), so we could use that instead. In an ideal world, the boot loader would lose inflate.o and friends and use the functionality from zstd instead since that's supposed to be a replacement for it. Since we only use it in gzipfs and pkgfs, perhaps that wouldn't be a heavy lift and that would result in a savings of maybe 20k which I think is going to be more and more important to look for as the loader slowly bloats over time. the EFI loader is in good shape, but the BIOS/CMS loader is getting tight on space with all we've added to it over the years.
(In reply to Allan Jude from comment #7) We already do have gzip in loader (gzipfs), so this will not add up too much. I did not realize FreeBSD is not supporting gzip boot:D From other hand, it does not seem to be like vita function to have in loader, but again - as we actually do have code there already, the price is just about one wrapper. (Apparently I did add it to illumos loader in 2015).
It looks like everybody do not mind of the change in general. So, what's next?
(In reply to zmey20000 from comment #10) This is the patch attached here?
(In reply to Toomas Soome from comment #11) Yes.
(In reply to zmey20000 from comment #12) I did pick it up and made some small updates - replaced ARGSUSED with __unused, small change in README, using calloc().
(In reply to Toomas Soome from comment #13) Woops, url too: https://reviews.freebsd.org/D35320
(In reply to Toomas Soome from comment #14) Looks awesome. Thank you very much, sir!
(In reply to Toomas Soome from comment #14) Just curious as I'm not not sure about the process, does it stuck again? :)
(In reply to zmey20000 from comment #16) just pinged reviewers, will wait for the responses and then we are good to go. Will likely need to wait a bit as people in US are sleeping:)
(In reply to Toomas Soome from comment #17) Thanks a lot, sir :)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=e50e40684aa61526327be713de512e0f9934477f commit e50e40684aa61526327be713de512e0f9934477f Author: Toomas Soome <tsoome@FreeBSD.org> AuthorDate: 2022-05-25 18:53:57 +0000 Commit: Toomas Soome <tsoome@FreeBSD.org> CommitDate: 2022-06-09 17:54:30 +0000 loader: add support for gzip compression As we do have zlib code in loader, we should also support gzip compression in zfs. PR: 153173 Submitted by: Mikhail Zakharov <zmey20000@yahoo.com> Reviewed by: imp, markj, delphij Differential Revision: https://reviews.freebsd.org/D35320 MFC after: 1 month sys/cddl/boot/zfs/README | 1 + sys/cddl/boot/zfs/gzip.c (new) | 96 ++++++++++++++++++++++++++++++++++++++++++ sys/cddl/boot/zfs/zfssubr.c | 19 +++++---- 3 files changed, 107 insertions(+), 9 deletions(-)
Great! Thank you all very much!
This seems to be resolved now.