Bug 207269

Summary: boot1.efi loading wrong system
Product: Base System Reporter: eu9gu4
Component: kernAssignee: Steven Hartland <smh>
Status: Closed Not A Bug    
Severity: Affects Only Me CC: amd64, emaste, smh
Priority: ---    
Version: 10.2-STABLE   
Hardware: amd64   
OS: Any   

Description eu9gu4 2016-02-17 02:56:00 UTC
Actually this is about FreeBSD 10.3-beta2, but it was not option to chose from when reporting the bug.

I think /boot/boot1.efi boots the first FreeBSD bootable partition that it finds, instead of looking further and trying to boot the system on its own partition.
This is wrong in case there are two FreeBSD systems installed, as in my case:

My machine has a HDD connected to 1st SATA port, a SSD connected to 2nd SATA port and a few other hard disks used for data and not interacting yet with FreeBSD.

The HDD is ada4 (sda in Linux) and the SDD is ada5 (sdb in Linux).

The HDD has a first partition set as EFI, from where GRUB boots all my operating systems: FreeBSD, Linux and Windows.

Everything was fine with the first FreeBSD installed on HDD.
After installing the second FreeBSD on SSD, I can only boot the first one installed on HDD, regardless of the option I select in GRUB, which has two separate entries for FreeBSD:

..
chainloader (hd0,gpt9)/boot/boot1.efi (for HDD)

and

..
chainloader (hd3,gpt5)/boot/boot1.efi (for SSD)

During setup I skipped the creation of another EFI partition as proposed by installer, because I already had one and I knew that /EFI/BOOT/BOOTX64.EFI is a simple copy of boot1.efi.

My choice of this specific configuration is coming from a recently failed SSD, I just wanted to have an immediate option available if my SSD fails again.

Can boot1.efi by modified in a way that allows booting multiple systems if they exist on different partitions?

Thanks.
Comment 1 Steven Hartland freebsd_committer freebsd_triage 2016-02-17 16:24:53 UTC
The boot1.efi load does have a preferred boot device, which is the device that boot1.efi was loaded from, assuming that device is a valid boot device i.e. its UFS or ZFS with a /boot/loader.efi present.

If this fails it will search the other devices present.

Do you see * or + during boot1.efi processing?

* represents a preferred device and + represents a non-preferred device.

If its not booting from the device boot1.efi and it is a valid device then please recompile and install boot1.efi with -DEFI_DEBUG set e.g.
cd /usr/src
make buildenv -DEFI_DEBUG
cd sys/boot/efi/boot1
make && make install

When you boot with this it provide significant information on the state it sees, which should identify the issue.
Comment 2 eu9gu4 2016-02-17 23:03:05 UTC
(In reply to Steven Hartland from comment #1)

Thanks for your suggestion.

As explained, the two systems are just the result of installer, did not rebuilt anything yet. Each partition has its own /boot/boot1.efi,loader.efi and is UFS, so what you explained is not happening.

I don't want to use -CURRENT, had too many problems last time I tried, so I will build my system as you indicated when the subversion branch for 10.3 is created on Feb 26 and will let you know the results.
Comment 3 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 00:23:56 UTC
If we're going to fix it for 10.3 I'm going to need those details ASAP.

You only need an updated boot1.efi so I've built one for you here:
http://blog.multiplay.co.uk/dropzone/freebsd/r295732-boot1.efi

If you have a serial console that will be useful for capturing the debug, otherwise it does run slowly so screenshots can be used as a fallback.
Comment 4 eu9gu4 2016-02-18 01:40:05 UTC
(In reply to Steven Hartland from comment #3)

Copied your boot1.efi on SSD.
Unfortunately I don't know anything about serial consoles and taking proper screen shots of the console while booting was almost impossible... so I recorded a movie that shows perfectly the whole boot process.

It is about 200MB and I uploaded it to my Google Drive.
I need your an email address of yours so I can share it with you.
Comment 5 eu9gu4 2016-02-18 01:46:58 UTC
Ok, I see it's smh@FreeBSD.org... can I use this for sharing that movie with you?
Comment 6 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 02:19:22 UTC
This looks to be doing the right thing to me:

boot1 imagepath is on:
pciroot(0x0):pci(0x1f,0x02):sata(0x0,0xfff,0x0):hd(1)

It then picks and boots from:
pciroot(0x0):pci(0x1f,0x02):sata(0x0,0xfff,0x0):hd(9)

Which is correctly on the same device as the boot1, according efi.

If you choose your other device you should preferred tags against the other device and finall "Load Complete" reported for it too.

What makes you think boot1 is doing the wrong thing?
Comment 7 eu9gu4 2016-02-18 02:29:36 UTC
(In reply to Steven Hartland from comment #6)

Simple:

boot1.efi on ada4p9 (sda9/Linux) should boot that system while boot1.efi on ada5p5 (sdb5/Linux) should boot its own system.

What happens in the recorded movie is that ada4p9 gets booted with both options.
Don't understand what is wrong.
Comment 8 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 02:34:00 UTC
The video only shows one boot grub option BSD Vector.

Is there meant to be two videos?
Comment 9 eu9gu4 2016-02-18 02:41:29 UTC
(In reply to Steven Hartland from comment #8)

Just the one you mention.

Looking at your comment #6

boot1 imagepath is on:
pciroot(0x0):pci(0x1f,0x02):sata(0x0,0xfff,0x0):hd(1) - something is wrong here... why hd(1)? it should be hd(5), there is no boot1.efi on hd(1), which I assume is the EFI partition.
Comment 10 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 02:45:43 UTC
(In reply to eu9gu4 from comment #9)
hd(1) is the partition not the device, the device is:
pciroot(0x0):pci(0x1f,0x02):sata(0x0,0xfff,0x0)
Comment 11 eu9gu4 2016-02-18 02:53:29 UTC
(In reply to Steven Hartland from comment #10)

Noticed that
pciroot(0x0):pci(0x1f,0x02):sata(0x0,0xfff,0x0):hd(9) is * preferred

while
pciroot(0x0):pci(0x1f,0x02):sata(0x1,0xfff,0x0):hd(5) is just + supported.

Probably that's why this happens... it should also be preferred, in reference to the drives layout that I explained before.
Comment 12 eu9gu4 2016-02-18 02:54:56 UTC
Realized that my comment explaining my drives layout got lost. Would it be useful to repeat it?
Comment 13 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 03:02:08 UTC
(In reply to eu9gu4 from comment #11)
No that's not the case as hd(5) is on a different physical device:
sata(0x1,0xfff,0x0)

boot1.efi was loaded from:
sata(0x0,0xfff,0x0):hd(1)

Hence why the sata(0x0,0xfff,0x0):hd(1) is preferred.

If you want to boot from:
sata(0x0,0xfff,0x0):hd(5)

You would need to chain load a boot1.efi on:
sata(0x1,0xfff,0x0)
Comment 14 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 03:02:55 UTC
(In reply to eu9gu4 from comment #12)
It might.
Comment 15 eu9gu4 2016-02-18 03:10:44 UTC
(In reply to Steven Hartland from comment #13)

What is "imagepath"?
Could that mean that the booting process is initiated by GRUB which of course is on partition 1 on HDD (first disk)?

My chain loading for Vector is:

chainloader (hd3,gpt5)/boot/boot1.efi

I went in command line in GRUB and verified that hd3 means actually the SSD for GRUB, it is my only drive that has 5 partitions.
Comment 16 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 03:22:08 UTC
imagepath is where EFI reports that the currently running image was loaded from, so in this case boot1.efi
Comment 17 eu9gu4 2016-02-18 03:25:28 UTC
(In reply to Steven Hartland from comment #14)

Ok see below:

Z87 controller

SATA6G_1 sda WD Blue 1
SATA6G_2 sdb OCZ Vector
SATA6G_3 sdc Samsung EVO
SATA6G_4 sdd WD Velociraptor 1
SATA6G_5 sde Toshiba 1
SATA6G_6     ASUS BW-16D1HT

1st ASM1061 controller

SATA6G_E1 sdh Toshiba 2
SATA6G_E2 sdg WD Blue 2

2nd ASM1061 controller

SATA6G_E3 sdf WD Velociraptor 2
SATA6G_E4 sdi WD Blue 3

WD Blue, 1000 GB

sda1	EFI system partition, fat32
sda2	gentoo-alt,	btrfs
sda3	transfer, ntfs
sda4	bsd-var, ufs
sda5	bsd-root, ufs
sda6	bsd-src, ufs
sda7	bsd-ports, ufs
sda8	bsd-distfiles, ufs
sda9	bsd-alt, ufs

sdg, sdi btrfs pool, whole disk, mirror


OCZ Vector 180, 480 GB

sdb1	windows, ntfs
sdb2	windows-diag, ntfs
sdb3	Microsoft reserved partition, -
sdb4	gentoo-main, btrfs
sdb5	bsd-main, ufs
-	free space (provisioning)


Samsung EVO, 120 GB

sdc1	slog, zfs
sdc2	l2arc, zfs


WD Velociraptor, 1000 GB

sdd, sdf  zfs pool, whole disk, mirror (zpool actually created 2 partitions on
each disk, numbered 1 and 9, don't know why it did that)


Toshiba, 3001 GB

sde, sdh  btrfs	pool, whole disk, mirror
Comment 18 eu9gu4 2016-02-18 03:34:40 UTC
(In reply to Steven Hartland from comment #16)

Confused why the image was loaded from hd(1) on SSD and checked: there was an older /EFI/BOOT/BOOTX64.EFI, which may have been be a remnant copy of boot1.efi from about a year ago when I was running 10.1.

I deleted it and checked no other bootx64.efi remains on that partition.
Booted again and recorded another video. It still shows imagepath on hd(1) instead of hd(5).

Anyway, even in the first case, why would imagepath would locate a boot1.efi on hd(1) when I clearly point it to hd(5) ?
Comment 19 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 03:36:08 UTC
I only see one EFI partition there, so where is your other boot1.efi?
Comment 20 eu9gu4 2016-02-18 03:40:46 UTC
(In reply to Steven Hartland from comment #19)

One EFI partition with GRUB which chain loads either hd(9) on the same disk or hd(5) on the second disk.

What I'm starting to think is that it may be GRUB chain loading's fault.
I may try tomorrow to reinstall on SSD, but this time allowing to create a second EFI partition there and then chain loading bootx64.efi on EFI instead of boot1.efi on ufs.
Comment 21 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 03:52:04 UTC
(In reply to eu9gu4 from comment #20)
Ahh that won't work boot1.efi must be loaded from device you wish to boot from, that's how it determines the "preferred" device.
Comment 22 eu9gu4 2016-02-18 04:15:04 UTC
9(In reply to Steven Hartland from comment #21)

Ok, reinstalled and allowed creation of own EFI partition. Current situation is:

- Disc 1, partition 1 (EFI): GRUB and no boot1.efi or any other FreeBSD boot loader on it
- Disc 1, partition 9 (UFS): my alternate FreebBSD which includes /boot/boot1.efi, which is chain loaded in GRUB (FreeBSD Blue option)
- Disc 2, partition 5 (EFI): created by FreBSD install; it has /EFI/BOOT/BOOTX64.EFI as a copy of the boot1.efi you provided, which is chain loaded in GRUB (FreeBSD Vector option)
- Disc 2, partition 6 (UFS): my main FreebBSD which includes /boot/boot1.efi

FreeBSD Blue option works fine.
FreeBSD Vector option still shows imagepath as disc 1 hd(1) instead of disc 2 hd(5). This is in my opinion what's wrong, but I don't know if it's the fault of boot1.efi or GRUB.
Comment 23 eu9gu4 2016-02-18 04:18:26 UTC
Maybe a fix would be to add another option to the beastie screen that detects all boot1.efi on all disks and partitions and allows to choose which one gets booted.
Comment 24 Steven Hartland freebsd_committer freebsd_triage 2016-02-18 04:20:31 UTC
What happens if you boot to a UEFI shell and then run boot.efi from the two different locations?

I'm wondering if grub is doing something strange which makes efi report the wrong location for the loaded image.
Comment 25 eu9gu4 2016-02-18 04:31:34 UTC
(In reply to Steven Hartland from comment #24)

Ok Steven, case closed: your boot1.efi is fine, it was all GRUB's fault, or maybe my fault.

I added this line

set root=(hd3,gpt5)

before the chain loading and it booted the correct system.