Bug 273723 - bsdinstall breaking auto install after f66a8328c
Summary: bsdinstall breaking auto install after f66a8328c
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 14.0-STABLE
Hardware: Any Any
: --- Affects Some People
Assignee: John Baldwin
URL:
Keywords: install, regression
: 274223 (view as bug list)
Depends on:
Blocks: 14.0r
  Show dependency treegraph
 
Reported: 2023-09-11 16:34 UTC by Andrey Fesenko
Modified: 2024-01-05 00:23 UTC (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Fesenko 2023-09-11 16:34:22 UTC
Long time work scenario like packer https://github.com/chef/bento/blob/main/os_pkrvars/freebsd/freebsd-13-x86_64.pkrvars.hcl

1. boot single user
2. mount /mnt and /tmp in md 100mb
3. dhcp
4. bsdinstall script ...
5. sucess

After commit https://cgit.freebsd.org/src/commit/usr.sbin/bsdinstall?h=stable/14&id=f66a8328c3effcb4fbd7807b798d0288b865421d (last working commit fe06db1817e0af6cbfa963598e249810773c115c)

Now, 14.0-ALPHA1 include install breaking not mount and format ufs and efi. /mnt is full.

screens https://www.tumblr.com/f-andrey/728183482719862784/freebsd-13-and-14-previous?source=share
Comment 1 John Baldwin freebsd_committer freebsd_triage 2023-09-17 11:33:46 UTC
Can you provide the installerconfig file?
Comment 2 Andrey Fesenko 2023-09-17 16:25:28 UTC
(In reply to John Baldwin from comment #1)
may use simpliest

```
export nonInteractive="YES"

HOSTNAME="freebsd"

DISTRIBUTIONS="base.txz kernel.txz"

PARTITIONS="vtbd0 GPT { auto freebsd-ufs / }"

#!/bin/sh -x

gpart show
mount
```
Comment 3 Andrey Fesenko 2023-09-17 16:54:22 UTC
Interesting, zfs install work https://raw.githubusercontent.com/chef/bento/main/packer_templates/http/freebsd/installerconfig
Comment 4 Michal Nowak 2023-10-16 15:56:56 UTC
We likely have the same issue with FreeBSD 14.0 RC1 at https://gitlab.isc.org/isc-projects/images/-/merge_requests/279.

The installer config is:
```
PARTITIONS="vtbd0"

#!/bin/sh -e

echo "vagrant" | pw usermod root -h 0
chsh -s /bin/sh root
sed -i -e "s|^#\(PermitRootLogin\) .*|\1 yes|" /etc/ssh/sshd_config

sysrc -f /boot/loader.conf autoboot_delay="-1"
sysrc -f /boot/loader.conf console=comconsole
sysrc hostname="freebsd.fqdn"
sysrc ifconfig_DEFAULT="DHCP"
sysrc sshd_enable="YES"

reboot
```

FreeBSD 13.2 and 12.4 install fine.
Comment 5 John Baldwin freebsd_committer freebsd_triage 2023-10-16 19:53:36 UTC
Humm, I am not able to reproduce, at least with invoking bsdinstall directly.

I first wrote a test program with both parsers:

```
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

static int
parse_disk_config(const char *token)
{
	printf("token: \"%s\"\n", token);
	return (0);
}

static int
old_editor(int argc, const char **argv)
{
	char *token;
	int i, error = 0, len = 0;
 
	for (i = 1; i < argc; i++)
		len += strlen(argv[i]) + 1;
	char inputbuf[len], *input = inputbuf;
	strcpy(input, argv[1]);
	for (i = 2; i < argc; i++) {
		strcat(input, " ");
		strcat(input, argv[i]);
	}
 
	while ((token = strsep(&input, ";")) != NULL) {
		error = parse_disk_config(token);
		if (error != 0)
			return (error);
	}
 
	return (0);

}

static int
new_editor(int argc, const char **argv)
{
	FILE *fp;
	char *input, *token;
	size_t len;
	int i, error = 0;
 
	fp = open_memstream(&input, &len);
	fputs(argv[1], fp);
	for (i = 2; i < argc; i++) {
		fprintf(fp, " %s", argv[i]);
	}
	fclose(fp);
 
	while ((token = strsep(&input, ";")) != NULL) {
		error = parse_disk_config(token);
		if (error != 0) {
			free(input);
			return (error);
		}
	}
	free(input);
 
	return (0);

}

int
main(int ac, const char **av)
{
	printf("old:\n");
	old_editor(ac, av);
	printf("new:\n");
	new_editor(ac, av);
	return (0);
}
```

And it invokes parse_disk_config with the same string in both cases:

```
> ./test "vtbd0 GPT { auto freebsd-ufs / }"
old:
token: "vtbd0 GPT { auto freebsd-ufs / }"
new:
token: "vtbd0 GPT { auto freebsd-ufs / }"
> ./test vtbd0 GPT { auto freebsd-ufs / }
old:
token: "vtbd0 GPT { auto freebsd-ufs / }"
new:
token: "vtbd0 GPT { auto freebsd-ufs / }"
```

I then ran this in a VM:

```
root@head:~ # mdconfig -a -t malloc -s 1g
md0
root@head:~ # /usr/libexec/bsdinstall/scriptedpart md0 GPT { auto freebsd-ufs / }
root@head:~ # gpart show md0
=>     40  2097072  md0  GPT  (1.0G)
       40     1024    1  freebsd-boot  (512K)
     1064  2096048    2  freebsd-ufs  (1.0G)
```

I also tried running `bsdinstall scriptedpart md0 GPT { auto freebsd-ufs / }` and it worked just as well.

Re: ZFS, ZFS doesn't use "scriptedpart":

From usr.sbin/bsdinstall/scripts/script:

```
# Make partitions
rm -f $PATH_FSTAB
touch $PATH_FSTAB
if [ "$ZFSBOOT_DISKS" ]; then
	bsdinstall zfsboot
else
	bsdinstall scriptedpart "$PARTITIONS"
fi
bsdinstall mount
```

I also tried this simple config file:

```
export nonInteractive="YES"

PARTITIONS="md0"

#!/bin/sh

gpart show
mount
```

with `bsdinstall script ./testinstall`

and aborted it when it asked for a mirror to download the distributions from.  Afterwards, from gpart show (vtbd0 in this case is the real root of the VM):

```
root@head:~ # gpart show
=>      34  75497398  vtbd0  GPT  (36G)
        34       128      1  freebsd-boot  (64K)
       162  72560446      2  freebsd-ufs  (35G)
  72560608   2936816      3  freebsd-swap  (1.4G)
  75497424         8         - free -  (4.0K)

=>     63  2097089  md0  MBR  (1.0G)
       63        1       - free -  (512B)
       64  2097088    1  freebsd  (1.0G)

=>      0  2097088  md0s1  BSD  (1.0G)
        0  1990656      1  freebsd-ufs  (972M)
  1990656   104448      2  freebsd-swap  (51M)
  2095104     1984         - free -  (992K)

```

This was all tested on

commit edd2a9b887864d07ac5af480b4b8f35cb76443f6 (HEAD -> main, origin/main)
Author: John Baldwin <jhb@FreeBSD.org>
Date:   Fri Oct 13 12:26:58 2023 -0700

    bhyve ahci: Replace WPRINTF with EPRINTLN
    
    Reviewed by:    markj
    Differential Revision:  https://reviews.freebsd.org/D42181

I'll see if I can't reproduce with an install image.
Comment 6 John Baldwin freebsd_committer freebsd_triage 2023-10-16 21:16:09 UTC
So the bug is not in the commit in question, rather the bug is that /etc/fstab isn't getting updated because of a later commit I made.   Specifically, I got the sense of a check wrong here:

https://github.com/bsdjhb/freebsd/commit/23099099196548550461ba427dcf09dcfb01878d#diff-d15da43b9961bf230557a7ae9bab338b880f066ac3db7728498feff3eb64973aR220

As a result, when geom_getmesh() succeeded, we weren't seeing error != 0 and thus didn't call apply_changes.
Comment 7 John Baldwin freebsd_committer freebsd_triage 2023-10-16 21:24:57 UTC
Fix at https://reviews.freebsd.org/D42236

With this I now get a non-empty fstab after running `bsdinstall scriptedpart` manually:

```
root@head:/ # bsdinstall scriptedpart md0 GPT { auto freebsd-ufs / }
...
root@head:/ # cat /tmp/bsdinstall_etc/fstab 
# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/md0p2      /               ufs     rw      1       1
```
Comment 8 commit-hook freebsd_committer freebsd_triage 2023-10-16 22:15:36 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=5307bbcc038f878b4b3714f03a2c824a0caeba4f

commit 5307bbcc038f878b4b3714f03a2c824a0caeba4f
Author:     John Baldwin <jhb@FreeBSD.org>
AuthorDate: 2023-10-16 22:13:31 +0000
Commit:     John Baldwin <jhb@FreeBSD.org>
CommitDate: 2023-10-16 22:13:31 +0000

    bsdinstall partedit: Apply changes from scripted installs

    I got a check inverted in a previous cleanup commit and as a result
    partedit was only applying GEOM changes (and generating an /etc/fstab)
    if it got an error reading the current GEOM mesh.  Instead, it needed
    to do those actions if it succeeded in reading the mesh.

    The lack of /etc/fstab meant that bsdinstall mount didn't mount
    anything in a scripted install.

    PR:             273723
    Reported by:    Andrey Fesenko <andrey@bsdnir.info>
    Reported by:    Michal Nowak <mnowak@startmail.com>
    Reviewed by:    cognet, brooks
    Fixes:          230990991965 bsdinstall: Handle errors from geom_gettree.
    MFC after:      3 days
    Differential Revision:  https://reviews.freebsd.org/D42236

 usr.sbin/bsdinstall/partedit/partedit.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 9 John Baldwin freebsd_committer freebsd_triage 2023-10-17 17:07:12 UTC
*** Bug 274223 has been marked as a duplicate of this bug. ***
Comment 10 commit-hook freebsd_committer freebsd_triage 2023-10-19 19:47:54 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=bfbb28f3b1b659805364eea580033b4b3afcc0eb

commit bfbb28f3b1b659805364eea580033b4b3afcc0eb
Author:     John Baldwin <jhb@FreeBSD.org>
AuthorDate: 2023-10-16 22:13:31 +0000
Commit:     John Baldwin <jhb@FreeBSD.org>
CommitDate: 2023-10-19 18:59:57 +0000

    bsdinstall partedit: Apply changes from scripted installs

    I got a check inverted in a previous cleanup commit and as a result
    partedit was only applying GEOM changes (and generating an /etc/fstab)
    if it got an error reading the current GEOM mesh.  Instead, it needed
    to do those actions if it succeeded in reading the mesh.

    The lack of /etc/fstab meant that bsdinstall mount didn't mount
    anything in a scripted install.

    PR:             273723
    Reported by:    Andrey Fesenko <andrey@bsdnir.info>
    Reported by:    Michal Nowak <mnowak@startmail.com>
    Reviewed by:    cognet, brooks
    Fixes:          230990991965 bsdinstall: Handle errors from geom_gettree.
    MFC after:      3 days
    Differential Revision:  https://reviews.freebsd.org/D42236

    (cherry picked from commit 5307bbcc038f878b4b3714f03a2c824a0caeba4f)

 usr.sbin/bsdinstall/partedit/partedit.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 11 commit-hook freebsd_committer freebsd_triage 2023-10-19 20:44:05 UTC
A commit in branch releng/14.0 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=929e4723c3f5e66dd98fe3f342c6ec393c8eb9a5

commit 929e4723c3f5e66dd98fe3f342c6ec393c8eb9a5
Author:     John Baldwin <jhb@FreeBSD.org>
AuthorDate: 2023-10-16 22:13:31 +0000
Commit:     John Baldwin <jhb@FreeBSD.org>
CommitDate: 2023-10-19 20:42:35 +0000

    bsdinstall partedit: Apply changes from scripted installs

    I got a check inverted in a previous cleanup commit and as a result
    partedit was only applying GEOM changes (and generating an /etc/fstab)
    if it got an error reading the current GEOM mesh.  Instead, it needed
    to do those actions if it succeeded in reading the mesh.

    The lack of /etc/fstab meant that bsdinstall mount didn't mount
    anything in a scripted install.

    PR:             273723
    Reported by:    Andrey Fesenko <andrey@bsdnir.info>
    Reported by:    Michal Nowak <mnowak@startmail.com>
    Reviewed by:    cognet, brooks
    Fixes:          230990991965 bsdinstall: Handle errors from geom_gettree.
    MFC after:      3 days
    Differential Revision:  https://reviews.freebsd.org/D42236

    (cherry picked from commit 5307bbcc038f878b4b3714f03a2c824a0caeba4f)
    (cherry picked from commit bfbb28f3b1b659805364eea580033b4b3afcc0eb)

    Approved by:    re (gjb)

 usr.sbin/bsdinstall/partedit/partedit.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 12 commit-hook freebsd_committer freebsd_triage 2024-01-05 00:23:32 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=c81bdb4998683aa6b795b6e1cb294be6f493329d

commit c81bdb4998683aa6b795b6e1cb294be6f493329d
Author:     John Baldwin <jhb@FreeBSD.org>
AuthorDate: 2023-10-16 22:13:31 +0000
Commit:     John Baldwin <jhb@FreeBSD.org>
CommitDate: 2024-01-04 23:43:33 +0000

    bsdinstall partedit: Apply changes from scripted installs

    I got a check inverted in a previous cleanup commit and as a result
    partedit was only applying GEOM changes (and generating an /etc/fstab)
    if it got an error reading the current GEOM mesh.  Instead, it needed
    to do those actions if it succeeded in reading the mesh.

    The lack of /etc/fstab meant that bsdinstall mount didn't mount
    anything in a scripted install.

    PR:             273723
    Reported by:    Andrey Fesenko <andrey@bsdnir.info>
    Reported by:    Michal Nowak <mnowak@startmail.com>
    Reviewed by:    cognet, brooks
    Fixes:          230990991965 bsdinstall: Handle errors from geom_gettree.
    MFC after:      3 days
    Differential Revision:  https://reviews.freebsd.org/D42236

    (cherry picked from commit 5307bbcc038f878b4b3714f03a2c824a0caeba4f)

 usr.sbin/bsdinstall/partedit/partedit.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)