Bug 192184 - [uefi] fresh install of 2014-07-14 11.0-CURRENT amd64 snapshot doesn't boot
Summary: [uefi] fresh install of 2014-07-14 11.0-CURRENT amd64 snapshot doesn't boot
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-bugs mailing list
URL:
Keywords: uefi
Depends on:
Blocks:
 
Reported: 2014-07-28 09:41 UTC by Dave Cottlehuber
Modified: 2014-10-10 07:26 UTC (History)
3 users (show)

See Also:


Attachments
dmesg (28.93 KB, text/plain)
2014-07-28 09:41 UTC, Dave Cottlehuber
no flags Details
dmesg (11.13 KB, text/plain)
2014-07-30 17:04 UTC, Dave Cottlehuber
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dave Cottlehuber freebsd_committer 2014-07-28 09:41:33 UTC
Created attachment 145067 [details]
dmesg

# Objective

install FreeBSD 11.0-CURRENT to 80Gb external USB drive

# Expected Result

After standard UFS install, system boots into FreeBSD.

previous snapshot http://ftp.at.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/11.0/FreeBSD-11.0-CURRENT-amd64-VT-20140614-r267488-memstick-uefi.img.xz works as expected.

dmesg from working boot attached, see https://gist.github.com/dch/22c28211c539a6d612a4 for further info x11 related info being discussed in x11 issue https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191206

# Actual Result

Installing via burned ISO from current snapshot http://ftp.at.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/11.0/FreeBSD-11.0-CURRENT-amd64-20140714-r268622-disc1.iso.xz completes installation successfully but is unable to boot the kernel from the resulting system.

It is possible to get to the loader, e.g. and run:

    mode 2
    set boot_verbose="YES"

However no additional output is received with this flag on.

Last successful message is:

Booting… Start @ 0xffffffff802db000 …

The previous snapshot's next message starts with "EFI Framebuffer", hopefully that helps.

It's not just a video / display issue, the network port / IP doesn't come alive either.
Comment 1 Dave Cottlehuber freebsd_committer 2014-07-28 09:44:11 UTC
hardware is early 2011 15" macbook pro, further info at https://gist.github.com/dch/22c28211c539a6d612a4 incl dmesg, hardware specs and file-devinfo-vr-log etc.
Comment 2 Dave Cottlehuber freebsd_committer 2014-07-28 16:52:20 UTC
Further testing shows that any non-CD boot method (e.g. dd of FreeBSD-11.0-CURRENT-amd64-20140714-r268622-mini-memstick.img or similar) to a spare USB disk or SDXC card) freezes at exactly the same place, i.e. nothing to do with installer itself, but the boot process.
Comment 3 Glen Barber freebsd_committer 2014-07-28 17:08:12 UTC
At the loader(8) prompt, can you do 'set kern.vty=vt' and see if that changes the behavior?
Comment 4 Dave Cottlehuber freebsd_committer 2014-07-28 19:30:38 UTC
yup, adding the following:

    set kern.vty=vt
    boot_verbose="YES"

gets us all the way to mountroot>

mountroot is failing as it always is trying to load ufs:/dev/ufs/FreeBSD_Install and returns error 19. Looks like the installer CD/image presumably.

trying again with:

    set kern.vty=vt
    boot_verbose="YES"
    load zfs
    vfs.root.mountfrom="zfs:zroot"
    vfs.root.zfs.debug="1"

produces a fair bit more output but fails to find/load init.
Comment 5 Dave Cottlehuber freebsd_committer 2014-07-28 19:55:04 UTC
thanks for vt protip from @gjb!

This now boots using a more common sense path:

vfs.root.mountfrom="zfs:zroot/ROOT/default"

If you point me in the right direction, I'll add a documentation note for these 2 tweaks.
Comment 6 Ed Maste freebsd_committer 2014-07-30 13:37:27 UTC
Would you mind attaching the dmesg from your (working) verbose boot?

Something rather strange is going on here -- the "Start @ 0xffffffff802db000" message indicates a (U)EFI boot, that should automatically select the vt(4) console and kern.vty=vt should not be necessary.
Comment 7 Dave Cottlehuber freebsd_committer 2014-07-30 17:04:00 UTC
Created attachment 145160 [details]
dmesg

dmesg from uefi boot to zfs. some notes:

zroot is on its own drive, on a zfsosx-created partition (so OSX doesn't get surly with the partition types) and the original FreeBSD-created zpool snapshotted and recursive send/received.

the system boots via refind http://www.rodsbooks.com/refind/ locating the EFI FAT partition on the other drive. The only change to the Apple EFI partition is to copy in ./EFI/freebsd/boot1.efi

booting requires 3 changes:

    set kern.vty="vt"
    load zfs
    set vfs.root.mountfrom="zfs:zroot/ROOT/default"

Setting these in loader.conf (either in ./EFI/freebsd/ or on zfs) has no effect.

BTW I'd love to know how to set `mode 2` in loader.conf as well instead of the small default quadrant in the middle of the EFI console.
Comment 8 Dave Cottlehuber freebsd_committer 2014-07-31 21:06:49 UTC
PEBKAC.

An older FreeBSD boot .img had been dd'd to another disk partition. It turns out that UEFI code will pick up the first available image it recognises and boots from that, and not the loader.efi and loader.conf etc in the same EFI location at boot1.efi or similar. This would be worth addressing.
Comment 9 Ed Maste freebsd_committer 2014-07-31 21:10:00 UTC
> An older FreeBSD boot .img had been dd'd to another disk partition.
> It turns out that UEFI code will pick up the first available image
> it recognises and boots from that, and not the loader.efi and
> loader.conf etc in the same EFI location at boot1.efi or similar.
> This would be worth addressing.

Ahh, yes.  Sorry this tripped you up.

boot1.efi finds the first available UFS filesystem and loads loader.efi from there.  You can skip boot1.efi altogether if you like, and just but loader.efi and the .4th and config files in your EFI system partition, but you'll then need to explicitly set currdev to load the kernel.
Comment 10 Dave Cottlehuber freebsd_committer 2014-07-31 21:14:45 UTC
np, thank-you *so* much for writing this. I've been waiting 2 years to install FreeBSD on this laptop.

I'll submit a doc patch with some more info from what I learned/uncovered, should this go to the handbook or the wiki?
Comment 11 Hiren Panchasara freebsd_committer 2014-08-02 00:43:21 UTC
(In reply to Dave Cottlehuber from comment #4)
> yup, adding the following:
> 
>     set kern.vty=vt
>     boot_verbose="YES"
> 
> gets us all the way to mountroot>

This did not work for me. On minnowboard max, I am trying FreeBSD-11.0-CURRENT-amd64-20140714-r268622-mini-memstick.img via microSD card and console o/p looks like this:

>>> FreeBSD EFI boot block
   Loader path: /boot/loader.efi
Consoles: EFI console
Image base: 0x79bab000
EFI version: 2.40
EFI Firmware: EDK II (rev 1.00)

FreeBSD/amd64 EFI loader, Revision 1.1
(root@grind.freebsd.org, Mon Jul 14 18:35:42 UTC 2014)
Loading /boot/defaults/loader.conf
/boot/kernel/kernel text=0xfb6e98 data=0x1296b0+0x4e96e0
syms=[0x8+0x140e38+0x8+0x15ba73]

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel] in 3 seconds...

Type '?' for a list of commands, 'help' for more detailed help.
OK boot -sv
?[37;44mBooting...?[0m
Start @ 0xffffffff802db000 ...
/

(and sits there forever)

I also tried following combination without success (and same failure as above):
'set kern.vty=vt'
'set hw.vga.textmode=1' and 'set kern.vty=vt'
'set hw.vga.textmode=1' and 'set kern.vty=syscons'

I haven't attached hdmi to this yet so no clue whats going on there.

This is how I got the image on sdcard:
# dd if=/home/hirenp/Downloads/FreeBSD-11.0-CURRENT-amd64-20140714-r268622-mini-memstick.img of=/dev/mmcsd0 bs=1m conv=sync
232+1 records in
233+0 records out
244318208 bytes transferred in 24.974703 secs (9782627 bytes/sec)

 # gpart show mmcsd0
=>     3  475616  mmcsd0  GPT  (1.8G) [CORRUPT]
       3    1600       1  efi  (800K)
    1603      32       2  freebsd-boot  (16K)
    1635  471936       3  freebsd-ufs  (230M)
  473571    2048       4  freebsd-swap  (1.0M)

I've been told that [CURRUPT] in the o/p above is "okay" and not the real problem.
Comment 12 Hiren Panchasara freebsd_committer 2014-08-02 01:15:25 UTC
(In reply to Hiren Panchasara from comment #11)
> (In reply to Dave Cottlehuber from comment #4)
> > yup, adding the following:
> > 
> >     set kern.vty=vt
> >     boot_verbose="YES"
> > 
> > gets us all the way to mountroot>
> 
> This did not work for me. On minnowboard max, I am trying
> FreeBSD-11.0-CURRENT-amd64-20140714-r268622-mini-memstick.img via microSD
> card and console o/p looks like this:
> 
> >>> FreeBSD EFI boot block
>    Loader path: /boot/loader.efi
> Consoles: EFI console
> Image base: 0x79bab000
> EFI version: 2.40
> EFI Firmware: EDK II (rev 1.00)
> 
> FreeBSD/amd64 EFI loader, Revision 1.1
> (root@grind.freebsd.org, Mon Jul 14 18:35:42 UTC 2014)
> Loading /boot/defaults/loader.conf
> /boot/kernel/kernel text=0xfb6e98 data=0x1296b0+0x4e96e0
> syms=[0x8+0x140e38+0x8+0x15ba73]
> 
> Hit [Enter] to boot immediately, or any other key for command prompt.
> Booting [/boot/kernel/kernel] in 3 seconds...
> 
> Type '?' for a list of commands, 'help' for more detailed help.
> OK boot -sv
> ?[37;44mBooting...?[0m
> Start @ 0xffffffff802db000 ...
> /
> 
> (and sits there forever)
> 
> I also tried following combination without success (and same failure as
> above):
> 'set kern.vty=vt'
> 'set hw.vga.textmode=1' and 'set kern.vty=vt'
> 'set hw.vga.textmode=1' and 'set kern.vty=syscons'
> 
> I haven't attached hdmi to this yet so no clue whats going on there.
> 
> This is how I got the image on sdcard:
> # dd
> if=/home/hirenp/Downloads/FreeBSD-11.0-CURRENT-amd64-20140714-r268622-mini-
> memstick.img of=/dev/mmcsd0 bs=1m conv=sync
> 232+1 records in
> 233+0 records out
> 244318208 bytes transferred in 24.974703 secs (9782627 bytes/sec)
> 
>  # gpart show mmcsd0
> =>     3  475616  mmcsd0  GPT  (1.8G) [CORRUPT]
>        3    1600       1  efi  (800K)
>     1603      32       2  freebsd-boot  (16K)
>     1635  471936       3  freebsd-ufs  (230M)
>   473571    2048       4  freebsd-swap  (1.0M)
> 
> I've been told that [CURRUPT] in the o/p above is "okay" and not the real
> problem.

This looks like a separate issue. Closing this bug and opened a new one: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192324
Comment 13 Hiren Panchasara freebsd_committer 2014-08-02 18:26:10 UTC
(In reply to Dave Cottlehuber from comment #10)
> np, thank-you *so* much for writing this. I've been waiting 2 years to
> install FreeBSD on this laptop.
> 
> I'll submit a doc patch with some more info from what I learned/uncovered,
> should this go to the handbook or the wiki?

Documenting it on https://wiki.freebsd.org/UEFI would be a good start, I think.
Comment 14 Pablo Olmos de Aguilera C. 2014-10-09 20:52:59 UTC
>> An older FreeBSD boot .img had been dd'd to another disk partition.
>> It turns out that UEFI code will pick up the first available image
>> it recognises and boots from that, and not the loader.efi and
>> loader.conf etc in the same EFI location at boot1.efi or similar.
>> This would be worth addressing.

> Ahh, yes.  Sorry this tripped you up.
>
> boot1.efi finds the first available UFS filesystem and loads loader.efi from 
> here.  You can skip boot1.efi altogether if you like, and just but loader.efi
> and the .4th and config files in your EFI system partition, but you'll then 
> need to explicitly set currdev to load the kernel.

I'm finding a similar issue, but loading boot1.efi just get me:

    >> FreeBSD EFI boot block
       Loader path: /boot/loader.efi
    panic: No bootable partition found

I guess that's because my system doesn't have any UFS partition, because I have a freebsd-boot partition, and my root is on zfs. I tried to load.efi, but it says the kernel is wrong and drop me into a loader console that works very weird (characters get added, if I write "show", sometimes say "sh" not found, or inserts random characters "shkokw". I tried to copy loader.4th over there, but still nothing happens... even worse, it seems that after adding .4th not even the `set` command is being recognized.
Comment 15 Dave Cottlehuber freebsd_committer 2014-10-10 07:26:47 UTC
@pablo I'm successfully booting direct from ZFS, here's what I learned:

1. it must be a UEFI-enabled boot image (10.1 pre-releases are good)

2. zfs must have a dataset with mountpoint=/ and your usual /boot/* stuff in here. Note this is not actually where the kernel will be loaded from, but it's where I keep the master copy so it's safely versioned on zfs.

3. the zpool* must have property bootfs set to point to that dataset, for example my default bootfs is:

    bootfs=zroot/ROOT/default

4. you must have a FAT-style EFI partition, with the kernel modules & all boot loader stuff in it. I am not exactly sure if there's a way to trim this down and have a smaller set. You *must* copy this across from within FreeBSD, I was unable to boot FreeBSD if I used either a linux system to transfer, or OSX -- the FAT 8.3 path mangling screwed things up.

I keep the master of /boot in ZFS but rsync it across to the EFI partition.  My /boot/loader.conf is:

# cat /efi/boot/loader.conf

# master in zfs:/boot/loader.conf but
# actually loaded from /EFI FAT partition
# remember to keep settings mirrored
# mkdir -m 0700 /efi
# mount -f msdosfs /dev/diskid/DISK-Z1F21XGNp1 /efi
# rsync -harv /boot /efi/boot --exclude=\*.symbols --del --partial --inplace
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
kern.vty="vt"
zfs_load="YES"
vfs.root.mountfrom="zfs:zroot/ROOT/default"

my EFI partition is (directories only):

# tree -qcd
.
├── boot
│   ├── defaults
│   ├── kernel
│   ├── modules
│   ├── zfs
│   └── firmware
└── EFI
    ├── APPLE
    │   ├── EXTENSIONS
    │   ├── FIRMWARE
    │   └── CACHES
    │       └── CAFEBEEF
    └── refind
        ├── icons
        ├── drivers_x64
        └── tools_x64


As this is still a mac there is apple cruft in here. Working on getting rid of that. This might be better discussed on one of the mailing lists BTW rather than in a resolved ticket ;-).