Bug 207188

Summary: gptzfsboot may choose wrong pool
Product: Base System Reporter: Matthias Pfaller <matthias.pfaller>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Open ---    
Severity: Affects Some People CC: grahamperrin
Priority: ---    
Version: 10.1-RELEASE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
output of zpool and gpart commands none

Description Matthias Pfaller 2016-02-14 13:07:59 UTC
Created attachment 166981 [details]
output of zpool and gpart commands

I'm using five disks in my system (ada0, ada1 and ada4 are SSDs, ada2 and ada3 conventional disks). 

ada2, ada3 and ada4 hold one freebsd-zfs partition each. ada2 and ada3 contain the data and ada4 is used for L2ARC.

ada0 and ada1 hold two freebsd-zfs partitions each. Partition one on each disk is used for a mirrored intent log for the data pool. Partition two holds the mirrored root pool.

I have set the bootfs property on the pool root but not on the pool data. gptzfsboot tries to load data/boot/gptzfsloader (and fails). It fails even when there you put zfsloader into data/boot.

After reordering the GPT entries the system boots fine. I think this behaviour should be documented in gptzfsboot(8).

The attached was done with the reordered gpt.
Comment 1 Andriy Gapon freebsd_committer freebsd_triage 2016-02-14 13:42:49 UTC
Could you please point out which part of gptzfsboot(8) has mislead you?
Comment 2 Matthias Pfaller 2016-02-14 14:22:19 UTC
(In reply to Andriy Gapon from comment #1)
there is no gptzfsboot(8) on my 10.1 system...

But I just looked at the 10.2 documentation and you are correct, the behaviour is documented. So instead of complaining I should have just looked harder for documentation :-)

But it might be nicer to skip pools without bootfs (imho).

The other thing I don't like, is that gptzfsboot doesn't seem to be able to boot from a pool when there is only the zil on the boot device. It fails with "all block copies unavailable". I think it should just skip a pool it can't use anyway.
Comment 3 Andriy Gapon freebsd_committer freebsd_triage 2016-02-17 12:15:30 UTC
(In reply to Matthias Pfaller from comment #2)
> But it might be nicer to skip pools without bootfs (imho).

I think that that would not be unreasonable.
But it certainly requires a wider discussion first, because there could be people who depend on the current behaviour in one way or another,

>The other thing I don't like, is that gptzfsboot doesn't seem to be able to boot from a pool when there is only the zil on the boot device. It fails with "all block copies unavailable". I think it should just skip a pool it can't use anyway.

Hmm, this is unexpected if other devices comprising the pool are fully visible to the firmware (BIOS).  Could you please provide some more details?
Comment 4 Matthias Pfaller 2016-02-17 13:41:26 UTC
(In reply to Andriy Gapon from comment #3)
>> But it might be nicer to skip pools without bootfs (imho).
> I think that that would not be unreasonable.
> But it certainly requires a wider discussion first, because there could be
> people who depend on the current behaviour in one way or another,

Maybe one should fall back to the old behaviour when no pool is marked with bootfs.

>> The other thing I don't like, is that gptzfsboot doesn't seem to be able to
>> boot from a pool when there is only the zil on the boot device. It fails
>> with "all block copies unavailable". I think it should just skip a pool it
>> can't use anyway.
> Hmm, this is unexpected if other devices comprising the pool are fully  
> visible to the firmware (BIOS).  Could you please provide some more details?

All five disks (and the embedded micro-sdcard) are visible to the bios. 
zfsloader will list them as c:, d:, e:, f:, g: and h:.

But when gptzfsboot decides to boot from ada0:data (instead of ada0:root) it will complain about "all block copies unavailable". 

Typing "status" at the gptzfsboot prompt will crash...

Please note that I had to patch zfsloader as described in https://wiki.freebsd.org/HPBIOS3TBHDD. Maybe gptzfsboot needs the patch as well?