Bug 160706 - [zfs] zfs bootloader fails when a non-root vdev exists on a slice before the root slice
Summary: [zfs] zfs bootloader fails when a non-root vdev exists on a slice before the ...
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 1.0-CURRENT
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-bugs mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-09-13 15:00 UTC by Peter Maloney
Modified: 2019-07-18 11:30 UTC (History)
0 users

See Also:


Attachments
FreeBSD_zfs_bootfail_rc_loader_conf.png (19.39 KB, image/png)
2011-09-13 15:19 UTC, Peter Maloney
no flags Details
FreeBSD_zfs_bootfail_slices.png (25.09 KB, image/png)
2011-09-13 15:19 UTC, Peter Maloney
no flags Details
FreeBSD_zfs_bootfail_tank_pool.png (20.51 KB, image/png)
2011-09-13 15:19 UTC, Peter Maloney
no flags Details
FreeBSD_zfs_bootfail_zroot_pool.png (18.77 KB, image/png)
2011-09-13 15:19 UTC, Peter Maloney
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Maloney 2011-09-13 15:00:20 UTC
zfs bootloader fails when a non-root vdev exists on a slice before the root slice

Fix: 

Fix unknown... 

Probably a source code change is needed somewhere in sys/boot/zfs.
Possibilities could be:
- retry the same loop/scan again skipping failed zfs pools
- skip zfs vdevs/pools that have no /boot directory
- skip log and cache vdevs (which I guess would be only a workaround and specific to my case)

Workaround: Make sure your bootable system is the first zfs slice on your boot disk. 

eg.

    gpart add -b 34 -s 64k -t freebsd-boot da0
    gpart add -s 512M -t freebsd-swap -l swap0 da0
    gpart add -s 80G -t freebsd-zfs -l root0 da0
    gpart add -s 4G -t freebsd-zfs -l log0 da0
    gpart add -t freebsd-zfs -l cache0 da0


(where root0 is the zfs root slice)
How-To-Repeat: gpart create -s gpt da0

gpart add -b 34 -s 64k -t freebsd-boot da0
gpart add -s 512M -t freebsd-swap -l swap0 da0
gpart add -s 4G -t freebsd-zfs -l log0 da0
gpart add -s 170G -t freebsd-zfs -l cache0 da0
gpart add -t freebsd-zfs -l root0 da0

gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 da0

zpool create zroot mirror gpt/root0 gpt/root1

glabel label da1 tank1d1
..
glabel label da<x> tank<y>d<z>

zpool create tank \
    raidz2 label/tank1d1 label/tank1d2 ... \ 
    raidz2 ...

At this point the system works fine. And survives rebooting.

zpool add tank cache gpt/cache0 gpt/cache1
zpool add tank log mirror gpt/log0 gpt/log1

Now if you boot the system, you get an screen looking like:
=========================
-
FreeBSD/x86 boot
Default: tank:/boot/kernel/kernel
boot:
|
FreeBSD/x86 boot
Default: tank:/boot/kernel/kernel
boot: _








=========================

I have screenshots (which I plan to attach to this report if such a button appears after submitting it). And a note about the screenshots, the shell looks broken as a side effect of editing the installer iso file. I only added scripts and the mps driver to it. And the same problem happens without my changes in FreeBSD-8.2-RELEASE and FreeBSD-8.2-STABLE-201105.


FYI: 
I used mirrors, and installed FreeBSD using this guide, and added things such as the cache and log slices.

http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror
Comment 1 Peter Maloney 2011-09-13 15:19:00 UTC
Screenshots attached
Comment 2 Mark Linimon freebsd_committer freebsd_triage 2011-09-19 06:17:48 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-fs

reclassify.
Comment 3 Andriy Gapon freebsd_committer 2011-09-19 09:05:53 UTC
I think that this is a WONTFIX bug.

It is well know, if less documented, that FreeBSD (gpt)zfsboot boot program
tries to use a pool that contains a very first vdev seen by (gpt)zfsboot to load
zfsloader or kernel.  You just have to take this into account.
This behavior makes sense.  Trying to boot all pools may create more problems
than it solves, e.g. if you have more than one pool that can be booted.

You may try to work-around your problem using boot.config file.
Place it in a root dataset of a pool that gets used by (gpt)zfsboo (tank, I
presume) with the following contents:
zroot:/boot/zfsloader

P.S. Your example is incomplete, it doesn't show where root1, cache1, log1 come
from :-)
-- 
Andriy Gapon
Comment 4 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:58:42 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped
Comment 5 Andriy Gapon freebsd_committer 2019-07-18 11:30:24 UTC
Lots of changes to ZFS boot code, assume fixed (or not a bug).