Bug 230246

Summary:

GPT label mishandling

Product:

Base System

Reporter:

Mason Loring Bliss <mason>

Component:

kern

Assignee:

freebsd-geom (Nobody) <geom>

Status:

New ---

Severity:

Affects Only Me

CC:

ae, andrew, chris, markj

Priority:

---

Version:

12.0-RELEASE

Hardware:

Any

OS:

Any

Attachments:

Description	Flags
Midway through...	none
confdot from previous comment, as file to avoid munging	none

Description Mason Loring Bliss freebsd_triage

2018-07-31 18:38:32 UTC

Observed:

root@:~ # gmirror label -v efi gpt/freebsd-efi0 gpt/freebsd-efi1
Metadata value stored on gpt/freebsd-efi0.
Metadata value stored on gpt/freebsd-efi1.
Done.
root@:~ # ls /dev/gpt
ls: /dev/gpt: No such file or directory
root@:~ # gmirror destroy efi
root@:~ # ls /dev/gpt
freebsd-boot0	freebsd-efi1	freebsd-swap0	gentoo-efi1
freebsd-boot1	freebsd-root0	freebsd-swap1	gentoo-root0
freebsd-efi0	freebsd-root1	gentoo-efi0	gentoo-root1
root@:~ # gmirror label -v efi gpt/freebsd-efi0 gpt/freebsd-efi1
Metadata value stored on gpt/freebsd-efi0.
Metadata value stored on gpt/freebsd-efi1.
Done.
root@:~ # ls /dev/gpt
ls: /dev/gpt: No such file or directory
root@:~ #

with kern.geom.mirror.debug=1:

GEOM_MIRROR: Device mirror/efi launched (2/2).
GEOM_MIRROR: Device efi: provider destroyed.
GEOM_MIRROR: Device efi destroyed.
GEOM_MIRROR: Device mirror/efi launched (2/2).
GEOM_MIRROR: Device efi: provider destroyed.
GEOM_MIRROR: Device efi destroyed.
GEOM_MIRROR: Device mirror/efi launched (2/2).
GEOM_MIRROR: Device efi: provider destroyed.
GEOM_MIRROR: Device efi destroyed.
GEOM_MIRROR: Device mirror/efi launched (2/2).
GEOM_MIRROR: Device efi: provider destroyed.
GEOM_MIRROR: Device efi destroyed.
GEOM_MIRROR: Device mirror/efi launched (2/2).
GEOM_MIRROR: Device efi: provider destroyed.
GEOM_MIRROR: Device efi destroyed.
GEOM_MIRROR[1]: Creating device efi (id=426299166).
GEOM_MIRROR[1]: Device efi created (2 components, id=426299166).
GEOM_MIRROR[1]: root_mount_hold 0xfffff80008aa8540
GEOM_MIRROR[1]: Adding disk diskid/DISK-CVDA5066005L1207GNp1 to efi.
GEOM_MIRROR[1]: Disk diskid/DISK-CVDA5066005L1207GNp1 state changed from NONE to NEW (device efi).
GEOM_MIRROR[1]: Device efi: provider diskid/DISK-CVDA5066005L1207GNp1 detected.
GEOM_MIRROR[1]: Adding disk diskid/DISK-PNY20162178980100273p1 to efi.
GEOM_MIRROR[1]: Disk diskid/DISK-PNY20162178980100273p1 state changed from NONE to NEW (device efi).
GEOM_MIRROR[1]: Device efi: provider diskid/DISK-PNY20162178980100273p1 detected.
GEOM_MIRROR[1]: Device efi state changed from STARTING to RUNNING.
GEOM_MIRROR[1]: Disk diskid/DISK-PNY20162178980100273p1 state changed from NEW to ACTIVE (device efi).
GEOM_MIRROR[1]: Device efi: provider diskid/DISK-PNY20162178980100273p1 activated.
GEOM_MIRROR[1]: Disk diskid/DISK-CVDA5066005L1207GNp1 state changed from NEW to ACTIVE (device efi).
GEOM_MIRROR[1]: Device efi: provider diskid/DISK-CVDA5066005L1207GNp1 activated.
GEOM_MIRROR[0]: Device mirror/efi launched (2/2).
GEOM_MIRROR[1]: root_mount_rel[2570] 0xfffff80008aa8540

markj on efnet suggested adding the partitions without referencing gpt labels, and on doing that I could make the mirror without losing /dev/gpt.

Comment 1 Mason Loring Bliss freebsd_triage

2018-08-26 19:37:50 UTC

Something I've observed is that I don't see this on fresh disks. I only see this when I'm installing to a disk that formerly had gmirrors on it.

Comment 2 Andrey V. Elsukov freebsd_committer

2018-09-11 16:18:21 UTC

For me all looks like I expected. What do you expected to see after these commands?

Comment 3 Mason Loring Bliss freebsd_triage

2019-07-27 00:55:51 UTC

Persists on 12. Typical encounter for me is a server that was running FreeBSD and
subsequently moved to GNU/Linux. Redeploying FreeBSD, evidently spoor from the
old install remains, and when I kldload geom_mirror, a degraded mirror is raised
from the dead. Evidently there's some sort of in-memory corruption after this as removing the remnants gives me assertion errors, and I'm not able to proceed
until I clear memory with a reboot.

I haven't read or traced the code yet, but at a guess something's seeing enough
indication of the old mirror to assume there's a mirror component there, and
then trusting that additional on-disk data is valid when it's not. This would
be pretty consistent with an assertion firing.

If life permits, I'll learn enough to track down just what's happening.

Comment 4 Mason Loring Bliss freebsd_triage

2019-09-03 16:06:28 UTC

Still here today, still biting me:

# ls /dev/gpt
efi0    efi1    swap0   swap1   tank0   tank1
# gmirror label -v -b prefer efi gpt/efi0 gpt/efi1
Metadata value stored on gpt/efi0.
Metadata value stored on gpt/efi1.
Done.
# ls /dev/gpt
tank0   tank1
# gmirror destroy efi
# ls /dev/gpt
efi0    efi1    swap0   swap1   tank0   tank1

Comment 5 Andrew "RhodiumToad" Gierth 2019-09-05 11:38:07 UTC

Can you attach the sysctl kern.geom.confdot output at all three stages? also the output of geom part show  on all relevant physical disks

Comment 6 Mason Loring Bliss freebsd_triage

2019-09-05 14:56:09 UTC

I'll gather this, but it'll take some churn, as I work past it when I hit it by
dd'ing zeroes onto the disks (or, the effected partitions, anyway) and
restarting. So I'll have to get the system back into the right state.

Amusingly, I thought maybe I could save time and reproduce this on a VM, and that
provided a quick reminder that I need to open a bug about the graphical console
going away with the 12.0 install media. (It looks like a freeze until you check
the serial console, and see that it's switched over to it. This appears under
libvirt.)

I'll try to have the requested debugging data by this evening, EST.

Comment 7 Mason Loring Bliss freebsd_triage

2019-09-06 02:02:43 UTC

Created attachment 207220 [details]
Midway through...

Midway through...

Comment 8 Mason Loring Bliss freebsd_triage

2019-09-06 02:08:08 UTC

And then:

# sysctl kern.geom.confdot
kern.geom.confdot: digraph geom {
z0xfffff80008ee6d00 [shape=box,label="MIRROR\nswap.sync\nr#1"];
z0xfffff80003635c00 [shape=box,label="MIRROR\nswap\nr#4"];
z0xfffff80003653580 [label="r1w1e1"];
z0xfffff80003653580 -> z0xfffff80008ee5100;
z0xfffff80003635c00 -> z0xfffff80003653580;
z0xfffff80008c63300 [shape=hexagon,label="mirror/swap\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008c63300 -> z0xfffff80003635c00;
z0xfffff80008c60000 [shape=box,label="PART\ndiskid/DISK-PNY20162178980100273\nr#3"];
z0xfffff800088d4c00 [label="r1w1e2"];
z0xfffff800088d4c00 -> z0xfffff8000336c400;
z0xfffff80008c60000 -> z0xfffff800088d4c00;
z0xfffff80008ee4e00 [shape=hexagon,label="diskid/DISK-PNY20162178980100273p3\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008ee4e00 -> z0xfffff80008c60000;
z0xfffff80008ee5100 [shape=hexagon,label="diskid/DISK-PNY20162178980100273p2\nr1w1e1\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008ee5100 -> z0xfffff80008c60000;
z0xfffff80008c62c00 [shape=hexagon,label="diskid/DISK-PNY20162178980100273p1\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008c62c00 -> z0xfffff80008c60000;
z0xfffff8005704e900 [shape=box,label="PART\ndiskid/DISK-CVDA5066005L1207GN\nr#3"];
z0xfffff8005752f100 [label="r0w0e0"];
z0xfffff8005752f100 -> z0xfffff800087f3c00;
z0xfffff8005704e900 -> z0xfffff8005752f100;
z0xfffff80008886700 [shape=hexagon,label="diskid/DISK-CVDA5066005L1207GNp3\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff80008886700 -> z0xfffff8005704e900;
z0xfffff800088e3b00 [shape=hexagon,label="diskid/DISK-CVDA5066005L1207GNp2\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff800088e3b00 -> z0xfffff8005704e900;
z0xfffff8000336c800 [shape=hexagon,label="diskid/DISK-CVDA5066005L1207GNp1\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff8000336c800 -> z0xfffff8005704e900;
z0xfffff80008886000 [shape=box,label="PART\nada0\nr#2"];
z0xfffff80008827c00 [label="r0w0e0"];
z0xfffff80008827c00 -> z0xfffff8000336c200;
z0xfffff80008886000 -> z0xfffff80008827c00;
z0xfffff800087f3900 [shape=hexagon,label="ada0p3\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff800087f3900 -> z0xfffff80008886000;
z0xfffff80008886500 [shape=hexagon,label="ada0p2\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff80008886500 -> z0xfffff80008886000;
z0xfffff8000889b900 [shape=hexagon,label="ada0p1\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff8000889b900 -> z0xfffff80008886000;
z0xfffff800088e5800 [shape=box,label="PART\ndiskid/DISK-110074973765s2\nr#4"];
z0xfffff80003654880 [label="r0w0e0"];
z0xfffff80003654880 -> z0xfffff800088e4000;
z0xfffff800088e5800 -> z0xfffff80003654880;
z0xfffff8000888a100 [shape=hexagon,label="diskid/DISK-110074973765s2a\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff8000888a100 -> z0xfffff800088e5800;
z0xfffff800088e6100 [shape=box,label="PART\ndiskid/DISK-110074973765\nr#3"];
z0xfffff800088d7980 [label="r0w0e0"];
z0xfffff800088d7980 -> z0xfffff800088e4200;
z0xfffff800088e6100 -> z0xfffff800088d7980;
z0xfffff800088e4000 [shape=hexagon,label="diskid/DISK-110074973765s2\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff800088e4000 -> z0xfffff800088e6100;
z0xfffff800088e5e00 [shape=hexagon,label="diskid/DISK-110074973765s1\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff800088e5e00 -> z0xfffff800088e6100;
z0xfffff800087f4800 [shape=box,label="PART\nda0s2\nr#3"];
z0xfffff80003363d80 [label="r1w0e1"];
z0xfffff80003363d80 -> z0xfffff800035ac500;
z0xfffff800087f4800 -> z0xfffff80003363d80;
z0xfffff80008898c00 [shape=hexagon,label="da0s2a\nr1w0e1\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008898c00 -> z0xfffff800087f4800;
z0xfffff800088d2e00 [shape=box,label="PART\nda0\nr#2"];
z0xfffff8000335f700 [label="r1w0e1"];
z0xfffff8000335f700 -> z0xfffff80008899a00;
z0xfffff800088d2e00 -> z0xfffff8000335f700;
z0xfffff800035ac500 [shape=hexagon,label="da0s2\nr1w0e1\nerr#0\nsector=512\nstripe=0"];
z0xfffff800035ac500 -> z0xfffff800088d2e00;
z0xfffff80008899700 [shape=hexagon,label="da0s1\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008899700 -> z0xfffff800088d2e00;
z0xfffff80057036100 [shape=box,label="LABEL\nada1\nr#2"];
z0xfffff800088d4e00 [label="r1w1e3"];
z0xfffff800088d4e00 -> z0xfffff80008886900;
z0xfffff80057036100 -> z0xfffff800088d4e00;
z0xfffff8000336c400 [shape=hexagon,label="diskid/DISK-PNY20162178980100273\nr1w1e2\nerr#0\nsector=512\nstripe=0"];
z0xfffff8000336c400 -> z0xfffff80057036100;
z0xfffff80057036700 [shape=box,label="LABEL\nada0\nr#2"];
z0xfffff800088d5380 [label="r0w0e0"];
z0xfffff800088d5380 -> z0xfffff8000336c200;
z0xfffff80057036700 -> z0xfffff800088d5380;
z0xfffff800087f3c00 [shape=hexagon,label="diskid/DISK-CVDA5066005L1207GN\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff800087f3c00 -> z0xfffff80057036700;
z0xfffff8005704eb00 [shape=box,label="LABEL\nada0p3\nr#3"];
z0xfffff80008828200 [label="r0w0e0"];
z0xfffff80008828200 -> z0xfffff800087f3900;
z0xfffff8005704eb00 -> z0xfffff80008828200;
z0xfffff8005704ea00 [shape=hexagon,label="gptid/3f9109f8-d049-11e9-8cd7-0021ccb4f63d\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff8005704ea00 -> z0xfffff8005704eb00;
z0xfffff8005704ee00 [shape=box,label="LABEL\nada0p3\nr#3"];
z0xfffff800088d6000 [label="r0w0e0"];
z0xfffff800088d6000 -> z0xfffff800087f3900;
z0xfffff8005704ee00 -> z0xfffff800088d6000;
z0xfffff8005704ed00 [shape=hexagon,label="gpt/tank0\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff8005704ed00 -> z0xfffff8005704ee00;
z0xfffff80057045800 [shape=box,label="LABEL\nada0p2\nr#3"];
z0xfffff800088d6a80 [label="r0w0e0"];
z0xfffff800088d6a80 -> z0xfffff80008886500;
z0xfffff80057045800 -> z0xfffff800088d6a80;
z0xfffff80057045700 [shape=hexagon,label="gptid/3f882ec4-d049-11e9-8cd7-0021ccb4f63d\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff80057045700 -> z0xfffff80057045800;
z0xfffff80057045b00 [shape=box,label="LABEL\nada0p2\nr#3"];
z0xfffff80003647f00 [label="r0w0e0"];
z0xfffff80003647f00 -> z0xfffff80008886500;
z0xfffff80057045b00 -> z0xfffff80003647f00;
z0xfffff80057045a00 [shape=hexagon,label="gpt/swap0\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff80057045a00 -> z0xfffff80057045b00;
z0xfffff80008c61a00 [shape=box,label="LABEL\nada0p1\nr#3"];
z0xfffff800088d6a00 [label="r0w0e0"];
z0xfffff800088d6a00 -> z0xfffff8000889b900;
z0xfffff80008c61a00 -> z0xfffff800088d6a00;
z0xfffff80057036800 [shape=hexagon,label="gptid/3f64646b-d049-11e9-8cd7-0021ccb4f63d\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff80057036800 -> z0xfffff80008c61a00;
z0xfffff80008c60b00 [shape=box,label="LABEL\nada0p1\nr#3"];
z0xfffff800088d6980 [label="r0w0e0"];
z0xfffff800088d6980 -> z0xfffff8000889b900;
z0xfffff80008c60b00 -> z0xfffff800088d6980;
z0xfffff80008c60200 [shape=hexagon,label="gpt/efi0\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff80008c60200 -> z0xfffff80008c60b00;
z0xfffff8000888a600 [shape=box,label="LABEL\ndiskid/DISK-110074973765s2a\nr#5"];
z0xfffff800088d7800 [label="r0w0e0"];
z0xfffff800088d7800 -> z0xfffff8000888a100;
z0xfffff8000888a600 -> z0xfffff800088d7800;
z0xfffff8000330ec00 [shape=hexagon,label="ufsid/5c0a029708f9d844\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff8000330ec00 -> z0xfffff8000888a600;
z0xfffff80008899400 [shape=box,label="LABEL\nda0\nr#2"];
z0xfffff8000335c880 [label="r0w0e0"];
z0xfffff8000335c880 -> z0xfffff80008899a00;
z0xfffff80008899400 -> z0xfffff8000335c880;
z0xfffff800088e4200 [shape=hexagon,label="diskid/DISK-110074973765\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff800088e4200 -> z0xfffff80008899400;
z0xfffff8000888a400 [shape=box,label="LABEL\nda0s2a\nr#4"];
z0xfffff80003654a80 [label="r1w0e1"];
z0xfffff80003654a80 -> z0xfffff80008898c00;
z0xfffff8000888a400 -> z0xfffff80003654a80;
z0xfffff8000888a300 [shape=hexagon,label="ufs/FreeBSD_Install\nr1w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff8000888a300 -> z0xfffff8000888a400;
z0xfffff80008895700 [shape=box,label="LABEL\nda0s1\nr#3"];
z0xfffff80003654b80 [label="r0w0e0"];
z0xfffff80003654b80 -> z0xfffff80008899700;
z0xfffff80008895700 -> z0xfffff80003654b80;
z0xfffff80008895100 [shape=hexagon,label="msdosfs/EFISYS\nr0w0e0\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008895100 -> z0xfffff80008895700;
z0xfffff800088e5000 [shape=box,label="VFS\nffs.ufs/FreeBSD_Install\nr#5"];
z0xfffff80003363e00 [label="r1w0e0"];
z0xfffff80003363e00 -> z0xfffff8000888a300;
z0xfffff800088e5000 -> z0xfffff80003363e00;
z0xfffff80008899b00 [shape=box,label="DISK\nda0\nr#1"];
z0xfffff80008899a00 [shape=hexagon,label="da0\nr1w0e1\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008899a00 -> z0xfffff80008899b00;
z0xfffff80008886a00 [shape=box,label="DISK\nada1\nr#1"];
z0xfffff80008886900 [shape=hexagon,label="ada1\nr1w1e3\nerr#0\nsector=512\nstripe=0"];
z0xfffff80008886900 -> z0xfffff80008886a00;
z0xfffff8000336c300 [shape=box,label="DISK\nada0\nr#1"];
z0xfffff8000336c200 [shape=hexagon,label="ada0\nr0w0e0\nerr#0\nsector=512\nstripe=4096"];
z0xfffff8000336c200 -> z0xfffff8000336c300;
z0xfffff80057036600 [shape=box,label="DEV\nmirror/swap\nr#5"];
z0xfffff8005752f380 [label="r0w0e0"];
z0xfffff8005752f380 -> z0xfffff80008c63300;
z0xfffff80057036600 -> z0xfffff8005752f380;
z0xfffff80008ee5000 [shape=box,label="DEV\ndiskid/DISK-PNY20162178980100273p3\nr#4"];
z0xfffff8005752e600 [label="r0w0e0"];
z0xfffff8005752e600 -> z0xfffff80008ee4e00;
z0xfffff80008ee5000 -> z0xfffff8005752e600;
z0xfffff80008896000 [shape=box,label="DEV\ndiskid/DISK-PNY20162178980100273p2\nr#4"];
z0xfffff800088d4980 [label="r0w0e0"];
z0xfffff800088d4980 -> z0xfffff80008ee5100;
z0xfffff80008896000 -> z0xfffff800088d4980;
z0xfffff80008ee5300 [shape=box,label="DEV\ndiskid/DISK-PNY20162178980100273p1\nr#4"];
z0xfffff800088d4b00 [label="r0w0e0"];
z0xfffff800088d4b00 -> z0xfffff80008c62c00;
z0xfffff80008ee5300 -> z0xfffff800088d4b00;
z0xfffff80008ee4c00 [shape=box,label="DEV\ndiskid/DISK-PNY20162178980100273\nr#3"];
z0xfffff8005752e900 [label="r0w0e0"];
z0xfffff8005752e900 -> z0xfffff8000336c400;
z0xfffff80008ee4c00 -> z0xfffff8005752e900;
z0xfffff80008898700 [shape=box,label="DEV\ndiskid/DISK-CVDA5066005L1207GNp3\nr#4"];
z0xfffff800088d5100 [label="r0w0e0"];
z0xfffff800088d5100 -> z0xfffff80008886700;
z0xfffff80008898700 -> z0xfffff800088d5100;
z0xfffff8000889a600 [shape=box,label="DEV\ndiskid/DISK-CVDA5066005L1207GNp2\nr#4"];
z0xfffff8005752ec00 [label="r0w0e0"];
z0xfffff8005752ec00 -> z0xfffff800088e3b00;
z0xfffff8000889a600 -> z0xfffff8005752ec00;
z0xfffff80008c62e00 [shape=box,label="DEV\ndiskid/DISK-CVDA5066005L1207GNp1\nr#4"];
z0xfffff8005752ee00 [label="r0w0e0"];
z0xfffff8005752ee00 -> z0xfffff8000336c800;
z0xfffff80008c62e00 -> z0xfffff8005752ee00;
z0xfffff80008ee5200 [shape=box,label="DEV\ndiskid/DISK-CVDA5066005L1207GN\nr#3"];
z0xfffff8005752f000 [label="r0w0e0"];
z0xfffff8005752f000 -> z0xfffff800087f3c00;
z0xfffff80008ee5200 -> z0xfffff8005752f000;
z0xfffff8005704ec00 [shape=box,label="DEV\ngptid/3f9109f8-d049-11e9-8cd7-0021ccb4f63d\nr#4"];
z0xfffff800088d5300 [label="r0w0e0"];
z0xfffff800088d5300 -> z0xfffff8005704ea00;
z0xfffff8005704ec00 -> z0xfffff800088d5300;
z0xfffff800088e3d00 [shape=box,label="DEV\ngpt/tank0\nr#4"];
z0xfffff80003647880 [label="r0w0e0"];
z0xfffff80003647880 -> z0xfffff8005704ed00;
z0xfffff800088e3d00 -> z0xfffff80003647880;
z0xfffff800035acc00 [shape=box,label="DEV\nada0p3\nr#3"];
z0xfffff800088d5b80 [label="r0w0e0"];
z0xfffff800088d5b80 -> z0xfffff800087f3900;
z0xfffff800035acc00 -> z0xfffff800088d5b80;
z0xfffff80057045900 [shape=box,label="DEV\ngptid/3f882ec4-d049-11e9-8cd7-0021ccb4f63d\nr#4"];
z0xfffff80003361680 [label="r0w0e0"];
z0xfffff80003361680 -> z0xfffff80057045700;
z0xfffff80057045900 -> z0xfffff80003361680;
z0xfffff80003635b00 [shape=box,label="DEV\ngpt/swap0\nr#4"];
z0xfffff80003653180 [label="r0w0e0"];
z0xfffff80003653180 -> z0xfffff80057045a00;
z0xfffff80003635b00 -> z0xfffff80003653180;
z0xfffff8000336a600 [shape=box,label="DEV\nada0p2\nr#3"];
z0xfffff80003653380 [label="r0w0e0"];
z0xfffff80003653380 -> z0xfffff80008886500;
z0xfffff8000336a600 -> z0xfffff80003653380;
z0xfffff80008c60400 [shape=box,label="DEV\ngptid/3f64646b-d049-11e9-8cd7-0021ccb4f63d\nr#4"];
z0xfffff800088d6500 [label="r0w0e0"];
z0xfffff800088d6500 -> z0xfffff80057036800;
z0xfffff80008c60400 -> z0xfffff800088d6500;
z0xfffff8000336a900 [shape=box,label="DEV\ngpt/efi0\nr#4"];
z0xfffff800088d6780 [label="r0w0e0"];
z0xfffff800088d6780 -> z0xfffff80008c60200;
z0xfffff8000336a900 -> z0xfffff800088d6780;
z0xfffff800087f4000 [shape=box,label="DEV\nada0p1\nr#3"];
z0xfffff800088d6800 [label="r0w0e0"];
z0xfffff800088d6800 -> z0xfffff8000889b900;
z0xfffff800087f4000 -> z0xfffff800088d6800;
z0xfffff80008898500 [shape=box,label="DEV\nufsid/5c0a029708f9d844\nr#6"];
z0xfffff80003654780 [label="r0w0e0"];
z0xfffff80003654780 -> z0xfffff8000330ec00;
z0xfffff80008898500 -> z0xfffff80003654780;
z0xfffff800088e5d00 [shape=box,label="DEV\ndiskid/DISK-110074973765s2a\nr#5"];
z0xfffff800088d7880 [label="r0w0e0"];
z0xfffff800088d7880 -> z0xfffff8000888a100;
z0xfffff800088e5d00 -> z0xfffff800088d7880;
z0xfffff800088e6000 [shape=box,label="DEV\ndiskid/DISK-110074973765s2\nr#4"];
z0xfffff8000335c580 [label="r0w0e0"];
z0xfffff8000335c580 -> z0xfffff800088e4000;
z0xfffff800088e6000 -> z0xfffff8000335c580;
z0xfffff800088e4100 [shape=box,label="DEV\ndiskid/DISK-110074973765s1\nr#4"];
z0xfffff80003654980 [label="r0w0e0"];
z0xfffff80003654980 -> z0xfffff800088e5e00;
z0xfffff800088e4100 -> z0xfffff80003654980;
z0xfffff800088e6200 [shape=box,label="DEV\ndiskid/DISK-110074973765\nr#3"];
z0xfffff80003363580 [label="r0w0e0"];
z0xfffff80003363580 -> z0xfffff800088e4200;
z0xfffff800088e6200 -> z0xfffff80003363580;
z0xfffff8000888a500 [shape=box,label="DEV\nufs/FreeBSD_Install\nr#5"];
z0xfffff8000335cf00 [label="r0w0e0"];
z0xfffff8000335cf00 -> z0xfffff8000888a300;
z0xfffff8000888a500 -> z0xfffff8000335cf00;
z0xfffff80008898d00 [shape=box,label="DEV\nda0s2a\nr#4"];
z0xfffff8000335f200 [label="r0w0e0"];
z0xfffff8000335f200 -> z0xfffff80008898c00;
z0xfffff80008898d00 -> z0xfffff8000335f200;
z0xfffff8000330eb00 [shape=box,label="DEV\nmsdosfs/EFISYS\nr#4"];
z0xfffff80003363b80 [label="r0w0e0"];
z0xfffff80003363b80 -> z0xfffff80008895100;
z0xfffff8000330eb00 -> z0xfffff80003363b80;
z0xfffff80008898a00 [shape=box,label="DEV\nda0s2\nr#3"];
z0xfffff8000335f480 [label="r0w0e0"];
z0xfffff8000335f480 -> z0xfffff800035ac500;
z0xfffff80008898a00 -> z0xfffff8000335f480;
z0xfffff80008899600 [shape=box,label="DEV\nda0s1\nr#3"];
z0xfffff8000335f600 [label="r0w0e0"];
z0xfffff8000335f600 -> z0xfffff80008899700;
z0xfffff80008899600 -> z0xfffff8000335f600;
z0xfffff80008899200 [shape=box,label="DEV\nda0\nr#2"];
z0xfffff8000335f680 [label="r0w0e0"];
z0xfffff8000335f680 -> z0xfffff80008899a00;
z0xfffff80008899200 -> z0xfffff8000335f680;
z0xfffff80008897e00 [shape=box,label="DEV\nada1\nr#2"];
z0xfffff80003655500 [label="r0w0e0"];
z0xfffff80003655500 -> z0xfffff80008886900;
z0xfffff80008897e00 -> z0xfffff80003655500;
z0xfffff800087f3d00 [shape=box,label="DEV\nada0\nr#2"];
z0xfffff80008828500 [label="r0w0e0"];
z0xfffff80008828500 -> z0xfffff8000336c200;
z0xfffff800087f3d00 -> z0xfffff80008828500;
}

Comment 9 Mason Loring Bliss freebsd_triage

2019-09-06 02:09:27 UTC

And at this point, the labels for ada1 are missing:

# ls /dev/gpt
efi0	swap0	tank0

Comment 10 Mason Loring Bliss freebsd_triage

2019-09-06 02:10:46 UTC

Created attachment 207221 [details]
confdot from previous comment, as file to avoid munging

confdot from previous comment, as file to avoid munging

Comment 11 Mason Loring Bliss freebsd_triage

2019-09-06 02:17:56 UTC

# ls /dev/ada
/dev/ada0   /dev/ada0p1 /dev/ada0p2 /dev/ada0p3 /dev/ada1  
# gpart show ada1
gpart: No such geom: ada1.
# gpart show /dev/ada1
gpart: No such geom: /dev/ada1.

Comment 12 Mason Loring Bliss freebsd_triage

2019-09-06 02:18:33 UTC

# gmirror destroy swap
# ls /dev/ada*
/dev/ada0	/dev/ada0p2	/dev/ada1	/dev/ada1p2
/dev/ada0p1	/dev/ada0p3	/dev/ada1p1	/dev/ada1p3

Comment 13 Mason Loring Bliss freebsd_triage

2019-09-06 02:38:57 UTC

# gmirror list
# ls /dev/gpt
efi0	efi1	swap0	swap1	tank0	tank1
# dd if=/dev/zero of=/dev/gpt/efi0 bs=1m
dd: /dev/gpt/efi0: end of device
2+0 records in
1+0 records out
1048576 bytes transferred in 0.171360 secs (6119140 bytes/sec)
# dd if=/dev/zero of=/dev/gpt/efi1 bs=1m
dd: /dev/gpt/efi1: end of device
2+0 records in
1+0 records out
1048576 bytes transferred in 0.012050 secs (87022279 bytes/sec)
# dd if=/dev/zero of=/dev/gpt/swap0 bs=1m
dd: /dev/gpt/swap0: Operation not supported

Comment 14 Mason Loring Bliss freebsd_triage

2019-09-06 02:40:56 UTC

More interesting:

# gmirror list
# ls /dev/gpt
efi0	efi1	swap0	swap1	tank0	tank1
# dd if=/dev/zero of=/dev/gpt/efi0 bs=1m
dd: /dev/gpt/efi0: end of device
2+0 records in
1+0 records out
1048576 bytes transferred in 0.171360 secs (6119140 bytes/sec)
# dd if=/dev/zero of=/dev/gpt/efi1 bs=1m
dd: /dev/gpt/efi1: end of device
2+0 records in
1+0 records out
1048576 bytes transferred in 0.012050 secs (87022279 bytes/sec)
# dd if=/dev/zero of=/dev/gpt/swap0 bs=1m
dd: /dev/gpt/swap0: Operation not supported
# gmirror list
Geom name: swap
State: DEGRADED
Components: 2
Balance: load
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 3028984319
Type: AUTOMATIC
Providers:
1. Name: mirror/swap
   Mediasize: 8589934080 (8.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
Consumers:
1. Name: diskid/DISK-CVDA5066005L1207GNp2
   Mediasize: 8589934592 (8.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: (null)
   GenID: 0
   SyncID: 1
   ID: 1191147779

#

Comment 15 Andrew "RhodiumToad" Gierth 2019-09-06 20:30:50 UTC

Just to wrap this up for the record (I explained all this to the OP in another place):

There's no bug in the code as such, the odd behavior is just an artifact of how geom tasting works, and the fact that it works rather unpredictably when geoms are being created and removed while the system is up (rather than at boot time).

Specifically, the sort of thing that happens is that a partition might become visible via many different provider paths, such as ada0p1, diskid/DISK-NNNNp1, gpt/partname, gptid/someuuid, and then if there is existing metadata at the partition's last sector, in this case for gmirror, the mirror geom might attach to any of the providers. If it attaches (say) diskid/DISK-NNNNp1, then the exclusive open of the underlying disk causes the gpart instances providing ada0p*, gpt/*, etc. to detach, so those labels go away.

This shows up as more of an issue with GPT because the combination of GPT and glabel produces many providers, often inconveniently many. (For example you can end up with "gmirror stop" failing to work, because as soon as the mirror closes, it becomes visible via another provider and immediately starts again.)

So this is ultimately, in my view, a "user experience" issue.

Comment 16 Mason Loring Bliss freebsd_triage

2019-09-22 19:00:48 UTC

I'm not decided on whether we should close this, or turn it into a feature
request for something to let us globally forget what we've discovered about
particular devices. I can see it going either way, so I won't close it for 
now, but I'll understand if it's something to put off for the time being.