Bug 254395
| Summary: | bsdinstall: fail script install after BETA3 | ||
|---|---|---|---|
| Product: | Base System | Reporter: | Andrey Fesenko <andrey> |
| Component: | bin | Assignee: | freebsd-bugs (Nobody) <bugs> |
| Status: | Closed FIXED | ||
| Severity: | Affects Some People | CC: | current, emaste, freqlabs, imp, nwhitehorn |
| Priority: | Normal | Keywords: | regression |
| Version: | 13.0-STABLE | ||
| Hardware: | Any | ||
| OS: | Any | ||
| Bug Depends on: | |||
| Bug Blocks: | 231027 | ||
You should not include the EFI partition in the PARTITIONS variable in general, but that won't cause this issue. Does the installer actually fail, resulting in a non-bootable system? Or do you just see this message in the output? Any more detail about your install script? What are you untarring? This affects me as well. The install completely fails. I'm even doing EFI installs: PARTITIONS="zvol/pool/dataset/vol gpt" I've used this same script quite a lot and it has only started to fail recently. I noticed yesterday and started looking into it a bit before I got pulled into something else. The tarball that fails to be extracted is base.txz (it's the one that contains ./boot/efi). The output on the console shows this as well: DEBUG: dialog.subr: DEBUG_SELF_INITIALIZE=[] DEBUG: UNAME_S=[FreeBSD] UNAME_P=[amd64] UNAME_R=[14.0-CURRENT] DEBUG: common.subr: Successfully loaded. DEBUG: f_debug_init: ARGV=[mount] GETOPTS_STDARGS=[dD:] DEBUG: f_debug_init: debug=[1] debugFile=[] DEBUG: Running installation step: mount DEBUG: dialog.subr: DEBUG_SELF_INITIALIZE=[] DEBUG: UNAME_S=[FreeBSD] UNAME_P=[amd64] UNAME_R=[14.0-CURRENT] DEBUG: common.subr: Successfully loaded. DEBUG: f_debug_init: ARGV=[checksum] GETOPTS_STDARGS=[dD:] DEBUG: f_debug_init: debug=[1] debugFile=[] DEBUG: Running installation step: checksum DEBUG: Extracting /storage/bhyve/14.0-CURRENT/a771bf748f9/kernel.txz DEBUG: Extracting /storage/bhyve/14.0-CURRENT/a771bf748f9/kernel-dbg.txz DEBUG: Extracting /storage/bhyve/14.0-CURRENT/a771bf748f9/base.txz ERROR: bsdinstall script failed At this point the script exits, leaving the filesystems mounted even: /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep2 on /mnt (ufs, local, journaled soft-updates) /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep1 on /mnt/boot/efi (msdosfs, local) devfs on /mnt/dev (devfs) # cat /tmp/bsdinstall-tmp-fstab /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep1 /mnt/boot/efi msdosfs rw 2 2 /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep2 /mnt ufs rw 1 1 # cat /tmp/bsdinstall_etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep1 /boot/efi msdosfs rw 2 2 /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep2 / ufs rw 1 1 /dev/zvol/storage/bhyve/14.0-CURRENT/a771bf748f9/templatep3 none swap sw 0 0 Well, I found the issue and there's even a nice comment in the relevant code that an older, smarter version of me put in in 2018 describing exactly why this is going to break:
--------
# Unpack distributions
bsdinstall checksum
for set in $DISTRIBUTIONS; do
f_dprintf "Extracting $BSDINSTALL_DISTDIR/$set"
# XXX: this will fail if any mountpoints are FAT, due to inability to
# set ctime/mtime on the root of FAT partitions. tar has no option to
# ignore this. We probably need to switch back to distextract here
# to properly support EFI.
tar -xf "$BSDINSTALL_DISTDIR/$set" -C $BSDINSTALL_CHROOT
done
--------
I'll try to get a patch in today. Apologies for the breakage, and thanks for the report.
(In reply to Nathan Whitehorn from comment #1) Installer fail, if set PARTITIONS="$DISKSLICE GPT { auto freebsd-ufs / }" fail too https://64.media.tumblr.com/2c95d2ddeed5cc2bd7ffb8fc68f34436/f8abb8c6d81ea645-e4/s1280x1920/6c4753c8bfdd3af75f597371189c4dcb1cbf0880.png build packer "qemu_binary": "/usr/libexec/qemu-kvm", "-bios", "/usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd" Check virtualbox builder BETA4 installer ok. RC2 fail, similar error (In reply to Nathan Whitehorn from comment #3) If possible, it is also worth mentioning in the release notes that you no longer need to specify the efi and boot partitions, but is it possible to make the efi partition smaller, since 260M is a lot for a virtual machine, where there will be only 1 system Thanks for the suggestion about the documentation -- I've updated the man page. The core problem here is that our tar can't extract archives to FAT32 with default options, since it treats inability to set modification time as a fatal error and FAT32 doesn't let you do that on the root directory. As such, any file in the release tarballs can't be extracted to FAT32. For interactive installations, the bsdinstall distextract tool, a CURSES-y frontend to libarchive, solves this by ignoring ctime/mtime errors. Some extra commentary on solutions, so it can be in one place. Possibilities are: 1. We drop /boot/efi from mtree. That will result in it not existing in base.txz, solving this issue, but will result in it not being in mtree. It will also leave in place an identical bug that will break scripted installation on bare-metal POWER8 and POWER9 systems, although that is a tier-2 platform. 2. We add an option to tar to ignore failure in setting ctime/mtime, like the interactive installer uses. This has the difficulty that the patch is hacky and would have to go through upstream. 3. We go back to using distextract for scripted installations as well as interactive ones, reverting d7640440fb644fde697f62fdff0b55aa3a4d5ef7. This fixes this issue but will result in installation failures for scripted installs without a controlling tty. (It will also add nice progress bars to scripted installs). 4. We do --exclude /boot/efi when running tar, then mkdir -p it by hand afterward. This is incredibly hacky and otherwise essentially functionally equivalent to #1. Like #1, it will fix this issue and has no obvious functional downside, but leaves scripted installs bare-metal POWER8 and POWER9 broken. 5. We patch the file system driver to (pretend to) allow setting times on the mount point. I don't want to do this, since I don't want to solve this in the kernel at RC3 and I don't like it pretending to do things it can't do. -------- Of these, 1, 3, and 4 are quite easy to implement, but all have some downside. My temptation is to do 4 for 13.0, since it will definitely work but is just lame, then either do #2 or a variant on #3 where distextract notices there is no tty and doesn't try to set up a dialog as a longer-term fix in HEAD. Any thoughts? #4 sounds good for 13.0. Of the hacks, it's the most localized and least hacky, imho. I'd prefer #5, honestly, longer term but I've not thought through all the implictions. We should loop in cem@ since he's been touching that code most recently. Or delphij@ as he has too... Would it be possible to defer mounting the ESP until after the tarballs have been extracted? We don't actually have anything in /boot/efi in the base archive. My understanding is that it's just there to ensure the mountpoint exists. (In reply to Ryan Moeller from comment #8) We could delay it, but it's harder and less-localized than the other solutions. It also completely breaks PowerPC and other systems with analagous but non-ESP boot partions. (In reply to Ryan Moeller from comment #8) Oh! I like this idea better, I think, but I don't know how hard it is to do. (In reply to Warner Losh from comment #10) It's pretty tricky, since it touches code in a lot of places, has to be conditional on platform, and runs a risk of breaking interactive installs just because it is pretty invasive. Related things: https://reviews.freebsd.org/D29380 <- patch in review here https://github.com/libarchive/libarchive/issues/1516 <- libarchive issue here A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=c2f16c595eb51c6e0cb6ece3f6f078d738019059 commit c2f16c595eb51c6e0cb6ece3f6f078d738019059 Author: Nathan Whitehorn <nwhitehorn@FreeBSD.org> AuthorDate: 2021-03-23 13:19:42 +0000 Commit: Nathan Whitehorn <nwhitehorn@FreeBSD.org> CommitDate: 2021-03-23 13:29:54 +0000 Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Reviewed by: gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 usr.sbin/bsdinstall/partedit/partedit.c | 39 +++++++++++++++++++++++++++++++++ usr.sbin/bsdinstall/scripts/script | 30 ++++++++++++++++++------- 2 files changed, 61 insertions(+), 8 deletions(-) A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=4601382e1362352f17a33e4ed38db5dcfe3f6be5 commit 4601382e1362352f17a33e4ed38db5dcfe3f6be5 Author: Nathan Whitehorn <nwhitehorn@FreeBSD.org> AuthorDate: 2021-03-23 13:19:42 +0000 Commit: Nathan Whitehorn <nwhitehorn@FreeBSD.org> CommitDate: 2021-03-23 19:21:33 +0000 Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Approved by: re (gjb) Reviewed by: gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 (cherry picked from commit c2f16c595eb51c6e0cb6ece3f6f078d738019059) usr.sbin/bsdinstall/partedit/partedit.c | 39 +++++++++++++++++++++++++++++++++ usr.sbin/bsdinstall/scripts/script | 30 ++++++++++++++++++------- 2 files changed, 61 insertions(+), 8 deletions(-) A commit in branch releng/13.0 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3ef46634cd5a6ba0e156d5da095bbee5f63ec1cd commit 3ef46634cd5a6ba0e156d5da095bbee5f63ec1cd Author: Nathan Whitehorn <nwhitehorn@FreeBSD.org> AuthorDate: 2021-03-23 13:19:42 +0000 Commit: Nathan Whitehorn <nwhitehorn@FreeBSD.org> CommitDate: 2021-03-23 19:22:20 +0000 Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Approved by: re (gjb) Reviewed by: gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 (cherry picked from commit c2f16c595eb51c6e0cb6ece3f6f078d738019059) (cherry picked from commit 4601382e1362352f17a33e4ed38db5dcfe3f6be5) usr.sbin/bsdinstall/partedit/partedit.c | 39 +++++++++++++++++++++++++++++++++ usr.sbin/bsdinstall/scripts/script | 30 ++++++++++++++++++------- 2 files changed, 61 insertions(+), 8 deletions(-) |
After BETA3 or RC1 (CURRENT add efi partition /boot/efi mount) script install UEFI system fail checksum/extract base.txz Old working `PARTITIONS="$DISKSLICE GPT { 260M efi , auto freebsd-ufs / }"` if edit `PARTITIONS="$DISKSLICE GPT { 260M efi /boot/efi, auto freebsd-ufs / }"` not fix root /mnt and efi /mnt/boot/efi mount succesfull bsdinstall_log * DEBUG: f_variable_set_defaults: Initializing defaults... * DEBUG: f_getvar: var=[debug] value=[1] r=0 * DEBUG: f_getvar: var=[editor] value=[/usr/bin/ee] r=0 * DEBUG: f_getvar: var=[ftpState] value=[auto] r=0 * DEBUG: f_getvar: var=[ftpUser] value=[ftp] r=0 * DEBUG: f_getvar: var=[hostname] value=[] r=0 * DEBUG: f_getvar: var=[MEDIA_TIMEOUT] value=[300] r=0 * DEBUG: f_getvar: var=[nfs_reserved_port_only] value=[NO] r=0 * DEBUG: f_getvar: var=[nfs_use_tcp] value=[NO] r=0 * DEBUG: f_getvar: var=[nfs_use_v3] value=[YES] r=0 * DEBUG: f_getvar: var=[PKG_TMPDIR] value=[/var/tmp] r=0 * DEBUG: f_getvar: var=[releaseName] value=[13.0-RC2] r=0 * DEBUG: f_variable_set_defaults: Defaults initialized. * DEBUG: variable.subr: Successfully loaded. * DEBUG: f_include_lang: file=[/usr/libexec/bsdconfig/include/messages.subr] lang=[] * DEBUG: dialog.subr: DIALOG_SELF_INITIALIZE=[1] * DEBUG: f_dialog_init: ARGV=[/tmp/installerconfig] GETOPTS_STDARGS=[dD:SX] * DEBUG: f_dialog_init: SECURE=[] USE_XDIALOG=[] * DEBUG: f_dialog_init: dialog(1) API initialized. * DEBUG: dialog.subr: Successfully loaded. * DEBUG: f_include: file=[/usr/share/bsdconfig/variable.subr] * DEBUG: Began Installation at Fri Mar 19 02:37:31 UTC 2021 * ./boot/efi/: Can't restore time * tar: Error exit delayed from previous errors.