Bug 277828

Summary:	15-CURRENT installation fails when /usr is on seperate UFS volume
Product:	Base System	Reporter:	Michael Osipov <michaelo>
Component:	bin	Assignee:	Dag-Erling Smørgrav <des>
Status:	Closed FIXED
Severity:	Affects Some People	CC:	d8zNeCFG, des, michaelo, mp
Priority:	---
Version:	15.0-CURRENT
Hardware:	Any
OS:	Any
URL:	https://reviews.freebsd.org/D44496

Description Michael Osipov freebsd_committer

2024-03-20 07:54:43 UTC

I have created a UEFI guest with vm-bhyve on 13.3-STABLE from 77814c959. 50 GB disk space (virtio-blk) with FreeBSD-15.0-CURRENT-amd64-20240314-220ee18f1964-268793-disc1.iso.

Selected the following (simplified) disk layout:
>             ┌─────────────────┤Partition Editor├──────────────────┐
>             │ Please review the disk setup. When complete, press  │
>             │ the Finish button.                                  │
>             │                                                     │
>             │                                                     │
>             │ ┌─────────────────────────────────────────────────┐ │
>             │ │vtbd0       50 GB     GPT                        │ │
>             │ │  vtbd0p1   260 MB    efi             /boot/efi  │ │
>             │ │  vtbd0p2   4.0 GB    freebsd-ufs     /          │ │
>             │ │  vtbd0p3   8.0 GB    freebsd-swap    none       │ │
>             │ │  vtbd0p4   38 GB     freebsd-ufs     /usr       │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ └─────────────────────────────────────────────────┘ │
>             ├─────────────────────────────────────────────────────┤
>             │[Create] [Delete] [Modify] [Revert] [ Auto ] [Finish]│
>             └─────────────────────────────────────────────────────┘

continue with the installation and you will receive:
>                    ┌─────────┤Archive Extraction├─────────┐
>                   ┌─────────────┤Extract Error├────────────┐
>                   │ Error while extracting base.txz: Can't │
>                   │ create 'etc/ssl/untrusted/c47d9980.0'  │
>                   │                                        │
>                   ├────────────────────────────────────────┤
>                   │                [  OK  ]                │
>                   └────────────────────────────────────────┘
>                    │
>                    └──────────────────────────────────────┘

as soon as /usr is not on a separate partition the installation succeeds.

I have also tried the following sizes:
>             ┌─────────────────┤Partition Editor├──────────────────┐
>             │ Please review the disk setup. When complete, press  │
>             │ the Finish button.                                  │
>             │                                                     │
>             │                                                     │
>             │ ┌─────────────────────────────────────────────────┐ │
>             │ │vtbd0       50 GB     GPT                        │ │
>             │ │  vtbd0p1   260 MB    efi             /boot/efi  │ │
>             │ │  vtbd0p2   12 GB     freebsd-ufs     /          │ │
>             │ │  vtbd0p3   4.0 GB    freebsd-swap    none       │ │
>             │ │  vtbd0p4   12 GB     freebsd-ufs     /usr       │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ │                                                 │ │
>             │ └─────────────────────────────────────────────────┘ │
>             ├─────────────────────────────────────────────────────┤
>             │[Create] [Delete] [Modify] [Revert] [ Auto ] [Finish]│
>             └─────────────────────────────────────────────────────┘

Same result. It does not seems to be a size issue, but some other problem.

I am trying to replicate a VM for Bug 277804 with the same UFS setup. I would expect it to work as it did when I installed the original machine with 11/12 five years so.

Comment 1 Michael Osipov freebsd_committer

2024-03-20 07:57:20 UTC

FWIW: I have tried some older ISO as well: same result.

Comment 2 Mark Peek freebsd_committer

2024-03-24 23:28:15 UTC

The issue is base.txz having hard links across filesystems (/ for /etc and /usr). This looks to have been introduced via the below commit where the install flags were changed from "-lrs" to "-lm". As "make distributeworld" creates the base directory on a single filesystem when certctl is run, "-lm" creates hard links which requires /etc and /usr/ to be on the same filesystem.

https://github.com/freebsd/freebsd-src/commit/a401c8cb26b22688087ad7c5ee527718459df15a

This patch should fix this specific issue.

diff --git a/usr.sbin/certctl/certctl.sh b/usr.sbin/certctl/certctl.sh
index 997a7d835d53..f95b4561d852 100755
--- a/usr.sbin/certctl/certctl.sh
+++ b/usr.sbin/certctl/certctl.sh
@@ -110,7 +110,7 @@ create_trusted()
 {
        local hash certhash otherfile otherhash
        local suffix
-       local link=${2:+-lm}
+       local link=${2:+-lrs}

        hash=$(do_hash "$1") || return
        certhash=$(openssl x509 -sha1 -in "$1" -noout -fingerprint)
@@ -159,7 +159,7 @@ resolve_certname()
 create_untrusted()
 {
        local srcfile filename
-       local link=${2:+-lm}
+       local link=${2:+-lrs}

        set -- $(resolve_certname "$1")
        srcfile=$1

Comment 3 commit-hook freebsd_committer

2024-03-25 16:00:25 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3fed4f0db53a66a0135bea5c491431eb3348cf51

commit 3fed4f0db53a66a0135bea5c491431eb3348cf51
Author:     Mark Peek <mp@FreeBSD.org>
AuthorDate: 2024-03-25 15:58:46 +0000
Commit:     Dag-Erling Smørgrav <des@FreeBSD.org>
CommitDate: 2024-03-25 15:59:02 +0000

    certctl: Revert to symlinks.

    Unfortunately tar will not be able to extract base.txz to a system where
    /etc and /usr are not on the same filesystem if the certificates are
    hard links.

    PR:             277828
    Reviewed by:    mp
    Differential Revision:  https://reviews.freebsd.org/D44496

 usr.sbin/certctl/certctl.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comment 4 Martin Birgmeier 2024-03-27 18:28:51 UTC

See also bug 275449

Comment 5 Michael Osipov freebsd_committer

2024-03-28 09:18:19 UTC

Thanks guys for the quick fix. Quite surprised that such a simple change can have such an impact.