Bug 262554 - [armv7][GENERICSD][patch] release.sh fails due to wrong argument to gpart add
Summary: [armv7][GENERICSD][patch] release.sh fails due to wrong argument to gpart add
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: arm (show other bugs)
Version: CURRENT
Hardware: arm Any
: --- Affects Only Me
Assignee: Alexander Motin
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2022-03-14 15:37 UTC by Matteo Riondato
Modified: 2022-05-11 01:44 UTC (History)
3 users (show)

See Also:


Attachments
Patch that fixes the issue (580 bytes, patch)
2022-03-14 22:52 UTC, Matteo Riondato
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Matteo Riondato freebsd_committer freebsd_triage 2022-03-14 15:37:46 UTC
I am trying to build a vanilla -CURRENT image for my beaglebone enhanced. 

In release/arm/GENERICSD.conf, I added info about CHROOT_DIR, MAKE_CONF, SRC_CONF (both with harmless contents, imho), and changed GENERIC to GENERIC-NODEBUG. 

I run sh ./release.sh -c arm/GENERICSD.conf and got the following:

--------------------------------------------------------------
>>> Kernel(s)  GENERIC-NODEBUG built in 94 seconds, ncpu: 40, make -j20
--------------------------------------------------------------
md0 created
md0s1 added
active set on md0s1
/dev/md0s1: 102264 sectors in 12783 FAT16 clusters (4096 bytes/cluster)
BytesPerSec=512 SecPerClust=8 ResSectors=1 FATs=2 RootDirEnts=512 Media=0xf0
FATsecs=50 SecPerTrack=63 Heads=255 HiddenSecs=0 HugeSectors=102400
md0s2 added
md0s2 created
gpart: No partitioning scheme found on geom /dev/md0s2. Create one first using
'gpart create'.

The error is raised on line 80 of tools/arm.subr:

chroot ${CHROOTDIR} gpart add -t freebsd-ufs -a 64k /dev/${mddev}s2

It seems quite weird that it fails on this line, because there is a "gpart create" command on line 79. The whole sequence of commands here looks correct to me. Could it be a timing issue? 

The building host is an amd64 running -CURRENT (from yesterday)

By the way, if I change PART_SCHEME="GPT" and change the arm_install_uboot_* functions in GENERICSD.conf,  to account for the different names of partitions in GPT vs MBR, I can successfully build a GPT image (I haven't tried booting it yet though).
Comment 1 Mark Millard 2022-03-14 20:32:38 UTC
(In reply to Matteo Riondato from comment #0)

You did not list the sequence of gpart commands so I can not
tell the details. So I provide a more generic example:

I assume you are attempting to create something analogous to
the following MBR style of media, with its "slices" (s1, s2):

# gpart show 
=>       63  244277185  mmcsd0  MBR  (116G)
         63      32705          - free -  (16M)
      32768     102312       1  fat32lba  [active]  (50M)
     135080      28760          - free -  (14M)
     163840  241172480       2  freebsd  (115G)
  241336320    2940928          - free -  (1.4G)

=>        0  241172480  mmcsd0s2  BSD  (115G)
          0  230686720         1  freebsd-ufs  (110G)
  230686720   10485760            - free -  (5.0G)

(I do not have an ada0 around to match the "man gpart" MBR example, so
for the below mmcsd0 <=> ada0 notationally and do not expect sizes and
such below to match the above details.)

Adjusting man gpart's example commands for MBR partitioning
to produce similar to the above:

           /sbin/gpart create -s MBR mmcsd0
(The "gpart man" MBR example does not have a msdosfs slice, so I added a command.)
           /sbin/gpart add -t fat32lba -s 50M mmcsd0
           /sbin/gpart add -t freebsd -s 64G mmcsd0
           /sbin/gpart set -a active -i 1 mmcsd0
           (IGNORED for arm: /sbin/gpart bootcode -b /boot/boot0 ada0)
           /sbin/gpart create -s BSD -n 20 mmcsd0s2
           /sbin/gpart add -t freebsd-ufs -s 8G mmcsd0s2
(The above gpart output does not show a swap partition.)
           /sbin/gpart add -t freebsd-swap -s 4G mmcsd0s2
           (IGNORED for arm: /sbin/gpart bootcode -b /boot/boot ada0s1)

(Creating file systems are separate steps.)

Note the 2nd "gpart create". the one with "-s BSD". Is that what you had?
I can not tell from what you wrote.

Note: Your msdosfs slice might need to be a different variant than fat32lba.

For reference, in partition style notation:

# gpart show -p
=>       63  244277185    mmcsd0  MBR  (116G)
         63      32705            - free -  (16M)
      32768     102312  mmcsd0s1  fat32lba  [active]  (50M)
     135080      28760            - free -  (14M)
     163840  241172480  mmcsd0s2  freebsd  (115G)
  241336320    2940928            - free -  (1.4G)

=>        0  241172480   mmcsd0s2  BSD  (115G)
          0  230686720  mmcsd0s2a  freebsd-ufs  (110G)
  230686720   10485760             - free -  (5.0G)

Note the "a" in mmcsd0s2a . There would be a mmcsd0s2b if there was
also a freebsd-swap present.


As far as I can tell, this submittal should have been a question on the
lists or forums instead of a bug report: It is more of a "what am I missing?"
sort of question than anything else. If true, you should close this submittal
as not-a-bug.
Comment 2 Matteo Riondato freebsd_committer freebsd_triage 2022-03-14 21:36:17 UTC
Mark,

Thanks for your reply. 

As I mention in my report, I'm using the release.sh script from /usr/src/release. I even mention where the script fails, with line numbers, so you can see the gpart commands. Perhaps you should re-read my message?

This is not a report about a home-made set of scripts  or sequence of commands: I'm not missing anything, as everything should be in the release.sh script, which is a script included in the src tree, and the documented way to create images for this architecture (https://wiki.freebsd.org/arm/Build_image_using_release_building_infrastructure).

This seems like a bug to me.
Comment 3 Matteo Riondato freebsd_committer freebsd_triage 2022-03-14 22:27:05 UTC
To collect more infos, I manually created a file-backed md device like it is done in release/release.sh, lines 396-398:

# truncate -s 3072M test
# mdconfig -f test -x 63 -y 255

This created the /dev/md1 backed by the test file.

Then I continued as in release/tools/arm.subr, line 65 (because this is the function that is called next by release/release.sh after the lines above):

# gpart create -s MBR md1
# gpart add -t '!12' -a 512k -s 50m -b 1m md1
# gpart set -a active -i 1 md1
# newfs_msdos -L msdosboot -F 16 /dev/md1s1
# gpart add -t freebsd md1
# gpart create -s bsd md1s2

At this point, we are just before line 80 of release/tools/arm.subr, which is the line where the script will fail. Let's look at the situation of the md1 device:

# gpart show -p
[snip other devices]
 =>     63  6291393    md1  MBR  (3.0G)
       63     1985         - free -  (993K)
     2048   102400  md1s1  fat32lba  [active]  (50M)
   104448  6187008  md1s2  freebsd  (3.0G)

=>      0  6187008  md1s2  BSD  (3.0G)
        0  6187008         - free -  (3.0G)

Everything actually looks correct (but see footnote 1 below), as in: the BSD partition scheme is there.

At this point though, if we try line 80 of release/tools/arm.subr, we get the error message:

# gpart add -t freebsd-ufs -a 64k /dev/md1s2
gpart: No partitioning scheme found on geom /dev/md1s2. Create one first using 'gpart create'.
#

And here we see what the bug is: in the call to gpart add..., it should not be /dev/md1s2, it should be md1s2:

# gpart add -t freebsd-ufs -a 64k /dev/md1s2
md1s2a added
#

I'll come up with a patch (which really, just involves removing /dev/)

Footnote 1: Everything looks almost correct: there is the weirdness that the s1 partition is created with a partition type of '!12', which is fat32lba, but then the filesystem is created with '-F 16', which specify a FAT16. Perhaps '!6' should be used in the first call to gpart add, but this seems unlikely to be the problem.
Comment 4 Matteo Riondato freebsd_committer freebsd_triage 2022-03-14 22:52:23 UTC
Created attachment 232462 [details]
Patch that fixes the issue

arm.subr: fix call to "gpart add" when using MBR.

The geom(4) provider should be passed without leading /dev/.
Comment 5 Mark Millard 2022-03-14 23:10:23 UTC
(In reply to Matteo Riondato from comment #2)

Nothing says that the version of the release.sh in my
environment will match yours. I'd have to do the work
to be sure I had a matching version before even looking
at the details.

Where do you list the commit that your environment is
based on? Based on the submit ("Version: CURRENT")
it is some commit in the main branch, but which?

I just recommend avoiding folks needing to do research
or other work to avoid just guessing.

Copying the line(s) like you did the one chroot line
avoids the whole issue of being sure of the context.
Comment 6 Mark Millard 2022-03-14 23:29:44 UTC
(In reply to Matteo Riondato from comment #3)

Thanks for providing the context.

This might be specific to mdN usage or MBR usage:
as I remember, gpart normally allows /dev/ prefixes,
probably stripping them off at some internal stage.
May be that stage is more "type" specific than I
thought.

An example from something I once did that I recorded:

# gpart destroy -F /dev/da0
# gpart create -s gpt /dev/da0
# gpart add -a 4k -tefi -s532480 /dev/da0
# gpart add -tfreebsd-swap -a1m -s16g /dev/da0
# gpart add -tfreebsd-swap -s16g /dev/da0
# gpart add -tfreebsd-swap -s32g /dev/da0
# gpart resize -i2 -s14g /dev/da0
# gpart add -tfreebsd-zfs /dev/da0
# gpart show

Unfortunately I've not recorded a prior MBR/BSD
sequence to show.

While the patch to release.sh may be a valid fix
for the script, it might be that gpart is actually
supposed to allow and strip /dev/ prefixes more
generally and might need to be fixed to fix the
issue more generally. (I do not know the intent.
I've not found documentation on the issue.)
Comment 7 Matteo Riondato freebsd_committer freebsd_triage 2022-03-15 00:45:20 UTC
The patch fixes the issue and makes the use of gpart consistent in this part of the arm.subr script: the previous calls to "gpart add" does not use /dev/, the corresponding call to "gpart add -t freebsd-ufs ..." in the GPT case also does not use /dev), thus, it is applicable no matter what the "intended" behavior of gpart should be.
Comment 8 Mark Millard 2022-03-15 01:10:26 UTC
(In reply to Matteo Riondato from comment #7)

Yep: valid patch.

But, I'm unsure if this submittal should cover the gpart
question as part of the closure ciriteria vs. if a
different submittal just for gpart would be appropriate.

I'll leave it to you for what you want here.

By the way: Nice find.
Comment 9 Matteo Riondato freebsd_committer freebsd_triage 2022-03-15 12:53:18 UTC
(In reply to Mark Millard from comment #8)

I don't have commit bits, so if by any chance you do, I would appreciate if you could please commit the patch. =)

I suspect that the deeper gpart issue may be that sys/geom/part/g_part.c uses an homegrown function (g_part_parm_geom, and functions called within) to determine the geom provider from its name (i.e., from a char*), rather than using the geom-wide helper g_provider_by_name from sys/geom/g_subr.c. The code is a bit too complex for me to know for sure, but this commit (https://cgit.freebsd.org/src/commit/sys/geom?id=fcf69f3dbce6b3b6187ff584031e690fbe2479d2) shows that similar conversions were done for other geom classes.
Comment 10 Mark Millard 2022-03-15 22:29:11 UTC
(In reply to Matteo Riondato from comment #9)

I'm not a committer. Nor do I have a FreeBSD.org E-mail address.
Comment 11 commit-hook freebsd_committer freebsd_triage 2022-03-16 04:07:19 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=bd0f3d34fa20686bac9a418d3a37a3f59c25ab19

commit bd0f3d34fa20686bac9a418d3a37a3f59c25ab19
Author:     Alexander Motin <mav@FreeBSD.org>
AuthorDate: 2022-03-16 03:58:09 +0000
Commit:     Alexander Motin <mav@FreeBSD.org>
CommitDate: 2022-03-16 04:06:49 +0000

    GEOM: Fix regression after 7f16b501e25.

    find_geom() in some classes trim leading "/dev/" from geom names.
    Lack of that in geom_gettree_geom() broke some existing scripts.

    PR:             262554
    MFC after:      2 months

 lib/libgeom/geom_xml2tree.c | 3 +++
 1 file changed, 3 insertions(+)
Comment 12 Alexander Motin freebsd_committer freebsd_triage 2022-03-16 04:12:45 UTC
I think the proposed patch is right.  But since it was my regression that triggered the problem, I've restored the previous behavior also.
Comment 13 Alexander Motin freebsd_committer freebsd_triage 2022-03-16 04:15:06 UTC
This is the other patch:  https://cgit.freebsd.org/src/commit/?id=ed5d6089113cac259202a28f2e1f89727c4d6ea3
Comment 14 Matteo Riondato freebsd_committer freebsd_triage 2022-03-16 12:28:22 UTC
Thank you @mav!
Comment 15 commit-hook freebsd_committer freebsd_triage 2022-05-11 01:44:54 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=217a0796fb0b8f43efef5a371e98aab36e3a68f7

commit 217a0796fb0b8f43efef5a371e98aab36e3a68f7
Author:     Alexander Motin <mav@FreeBSD.org>
AuthorDate: 2022-03-16 03:58:09 +0000
Commit:     Alexander Motin <mav@FreeBSD.org>
CommitDate: 2022-05-11 01:20:25 +0000

    GEOM: Fix regression after 7f16b501e25.

    find_geom() in some classes trim leading "/dev/" from geom names.
    Lack of that in geom_gettree_geom() broke some existing scripts.

    PR:             262554
    MFC after:      2 months

    (cherry picked from commit bd0f3d34fa20686bac9a418d3a37a3f59c25ab19)

 lib/libgeom/geom_xml2tree.c | 3 +++
 1 file changed, 3 insertions(+)