Created attachment 220575 [details] trigger bug After debugging, if you modify EFI_STAGING_SIZE to be greater than 64, vmware cannot start the operating system. #ifndef EFI_STAGING_SIZE #if defined(__amd64__) #define EFI_STAGING_SIZE 100 #elif defined(__arm__) #define EFI_STAGING_SIZE 32 #else #define EFI_STAGING_SIZE 64 #endif #endif Must be reduced to 64 to ensure that vmware can boot correctly.
The file is located in freebsd-src/stand/efi/loader/copy.c
It seems that efi_check_space also has a problem, it will reallocate staging_base, which may cause unknown problems.
I can confirm. Using VMware Workstation 16, and a FreeBSD 12.2-RELEASE amd64 iso, if I install it using UEFI, it fails to boot to completion, and VMware shows: [The firmware encountered an unexpected exception. The virtual machine cannot boot.] Last few lines in vmware.log for this VM are: 2020-12-15T20:08:49.485+01:00| vcpu-0| I005: SCSI0: RESET BUS 2020-12-15T20:08:49.485+01:00| vcpu-0| I005: SCSI0: RESET BUS 2020-12-15T20:08:49.486+01:00| vcpu-0| I005: DISKUTIL: scsi0:0 : capacity=209715200 logical sector size=512 2020-12-15T20:08:49.505+01:00| vcpu-0| I005: Guest: Status upon boot failure: No Media 2020-12-15T20:08:49.506+01:00| vcpu-0| I005: DISKUTIL: scsi0:1 : capacity=20971520 logical sector size=512 2020-12-15T20:08:49.508+01:00| vcpu-0| I005: Guest: Status upon boot failure: No Media 2020-12-15T20:08:49.745+01:00| vcpu-0| I005: Guest: About to do EFI boot: EFI VMware Virtual IDE CDROM Drive (IDE 1:0) 2020-12-15T20:08:58.704+01:00| vcpu-0| I005: UHCI: HCReset 2020-12-15T20:08:58.716+01:00| vcpu-1| I005: CPU reset: soft (mode Interp) 2020-12-15T20:08:58.716+01:00| vcpu-2| I005: CPU reset: soft (mode Interp) 2020-12-15T20:08:58.716+01:00| vcpu-3| I005: CPU reset: soft (mode Interp) 2020-12-15T20:08:58.716+01:00| vcpu-0| I005: Guest: Firmware has transitioned to runtime. 2020-12-15T20:08:59.140+01:00| vcpu-0| I005: Msg_Post: Error 2020-12-15T20:08:59.140+01:00| vcpu-0| I005: [msg.efi.exception] The firmware encountered an unexpected exception. The virtual machine cannot boot. 2020-12-15T20:08:59.140+01:00| vcpu-0| I005: ----------------------------------------
(In reply to YUAN RUI from comment #0) > Must be reduced to 64 to ensure that vmware can boot correctly. I can at least confirm that building /usr/src/stand/ (from the latest stable/12 svn checkout) with EFI_STAGING_SIZE set to 64, then installing loader_lua.efi to /boot/loader.efi appears to work.
Reverting EFI_STAGING_SIZE back to 64 allows built "cdrom" image to successfully boot on 64GB VM on ESXi 7. Thanks a lot for finding what broke it!
What do you get from the memmap loader command? I'm guessing there is reserved memory that conflicts.
Note that this is with a loader.efi rebuilt with EFI_STAGING_SIZE=64: OK memmap Type Physical Virtual #Pages Attr ACPIMemoryNVS 000000000000 000000000000 00000001 UC WC WT WB ConventionalMemory 000000001000 000000000000 0000009f UC WC WT WB ConventionalMemory 000000100000 000000000000 000046de UC WC WT WB LoaderData 0000047de000 000000000000 00008000 UC WC WT WB LoaderCode 00000c7de000 000000000000 00000081 UC WC WT WB LoaderData 00000c85f000 000000000000 0000217d UC WC WT WB LoaderCode 00000e9dc000 000000000000 0000001a UC WC WT WB BootServicesCode 00000e9f6000 000000000000 000001ba UC WC WT WB BootServicesData 00000ebb0000 000000000000 00000301 UC WC WT WB BootServicesCode 00000eeb1000 000000000000 00000040 UC WC WT WB RuntimeServicesCode 00000eef1000 000000000000 00000009 UC WC WT WB BootServicesCode 00000eefa000 000000000000 00000013 UC WC WT WB RuntimeServicesCode 00000ef0d000 000000000000 00000005 UC WC WT WB BootServicesCode 00000ef12000 000000000000 0000001a UC WC WT WB RuntimeServicesCode 00000ef2c000 000000000000 00000005 UC WC WT WB BootServicesCode 00000ef31000 000000000000 00000037 UC WC WT WB BootServicesData 00000ef68000 000000000000 0000048f UC WC WT WB ConventionalMemory 00000f3f7000 000000000000 0000006a UC WC WT WB BootServicesData 00000f461000 000000000000 000007a0 UC WC WT WB ConventionalMemory 00000fc01000 000000000000 00000002 UC WC WT WB BootServicesData 00000fc03000 000000000000 000000fa UC WC WT WB ConventionalMemory 00000fcfd000 000000000000 0000000c UC WC WT WB BootServicesData 00000fd09000 000000000000 000000ee UC WC WT WB ConventionalMemory 00000fdf7000 000000000000 00000001 UC WC WT WB BootServicesCode 00000fdf8000 000000000000 000000ef UC WC WT WB RuntimeServicesCode 00000fee7000 000000000000 00000020 UC WC WT WB RuntimeServicesData 00000ff07000 000000000000 00000050 UC WC WT WB ACPIReclaimMemory 00000ff57000 000000000000 0000001c UC WC WT WB ACPIMemoryNVS 00000ff73000 000000000000 00000004 UC WC WT WB BootServicesData 00000ff77000 000000000000 00000048 UC WC WT WB BootServicesCode 00000ffbf000 000000000000 00000041 UC WC WT WB ConventionalMemory 000010000000 000000000000 000b0000 UC WC WT WB ConventionalMemory 000100000000 000000000000 00040000 UC WC WT WB Reserved 0000000c0000 000000000000 00000040 WP RP MemoryMappedIO 0000ffc00000 000000000000 0000002a UC
It looks like the issue is the first LoaderData memory. Can you try commenting out the if(running_on_hyperv()) check in efi_copy_init in
(In reply to Andrew Turner from comment #8) It looks like the issue is the first LoaderData memory. Can you try commenting out the if(running_on_hyperv()) check in efi_copy_init in copy.c so efi_verify_staging_size always runs. It should limit the memory to just the number of pages available.
If (running_on_hyperv()) is commented out, this problem still occurs.
Did it print the line starting "Staging area's size is reduced:" and if so what did it print?
Would it print "Staging area's size is reduced:" before or after the initial messages? If it prints it very early, I cannot see it because the output scrolls off the screen, and the loader's prompt does not seem to support any form of scrollback. (I tried scroll lock, and shift-pageup/down, but neither worked.) In any case, commenting out the if, but not the call to efi_verify_staging_size(), and going with the default EFI_STAGING_SIZE of 100 still does not work: OK boot -s Start @ 0xffffffff8036b000 ... EFI framebuffer information: addr, size 0xf0000000, 0x300000 dimensions 1024 x 768 stride 1024 masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000 !!!! X64 Exception Type - 0E(#PF - Page-Fault) CPU Apic ID - 00000000 !!!! ExceptionData - 0000000000000000 I:0 R:0 U:0 W:0 P:0 PK:0 S:0 RIP - FFFFFFFF81885F90, CS - 0000000000000018, RFLAGS - 0000000000010046 RAX - 00000000087DF000, RCX - 0000000006600000, RDX - 00000000087DF000 RBX - 00000000023DEFF8, RSP - 00000000023DEFF8, RBP - 000B0001000B8781 RSI - 0000000000000000, RDI - 00000000023DEFF8 R8 - 00000000023DB000, R9 - FFFFFFFF8036B000, R10 - 000000000A9706D0 R11 - 00000000000005D0, R12 - 0000000002655000, R13 - 000000000264B000 R14 - 00000000023DB000, R15 - FFFFFFFF8036B000 DS - 0000000000000008, ES - 0000000000000008, FS - 0000000000000008 GS - 0000000000000008, SS - 0000000000000008 CR0 - 0000000080010033, CR2 - FFFFFFFF81885F90, CR3 - 000000000FF78000 CR4 - 0000000000000668, CR8 - 0000000000000000 DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000 DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400 GDTR - 00000000FFFFFEA0 000000000000002F, LDTR - 0000000000000000 IDTR - 000000000FEBF3E0 0000000000000FFF, TR - 0000000000000000 FXSAVE_STATE - 00000000023DEC50 !!!! Can't find image information. !!!! and then VMware shows its "fatal exception" dialog.
It's early in the boot process. It looks like you can use /efi/freebsd/loader.env on the same fs as loader.efi (probably the FAT fs) to set early variables, e.g. to use the serial console.
A commit references this bug: Author: imp Date: Thu Dec 17 17:02:09 UTC 2020 New revision: 368721 URL: https://svnweb.freebsd.org/changeset/base/368721 Log: Drop EFI_STAGING_SIZE back down to 64M vmware can't cope with anything larger than 64MB. Drop this back to 64MB everywhere but arm. PR: 251866 MFC After: 1 week Changes: head/stand/efi/loader/copy.c
It seems to increase the size for compatibility with the nvidia driver, but if it is reduced back, the nvidia driver cannot be loaded?
*** Bug 244906 has been marked as a duplicate of this bug. ***
Thank you. I should have the opportunity to test a 13.0-CURRENT snapshot on affected hardware – an HP ProBook 440 G7 – on, or very soon after, Monday 2021-01-04.
^Triage: Assign to committer resolving @Warner Is an EN and/or a RELEASE/images re-roll possible and warranted for this?
(In reply to Kubilay Kocak from comment #18) I can answer the second part - a re-roll is not possible for release builds at the moment.
(In reply to Glen Barber from comment #19) But is it possible to remake the installation image (iso)?
(In reply to commit-hook from comment #14) It seems that efi_check_space will dynamically expand this size. Does this cause unknown problems?
(In reply to YUAN RUI from comment #20) Unfortunately, no. There is no precident for doing so, however this is something that has been on my list of things to look into for quite some time.
Thank you, (In reply to Glen Barber from comment #22) Reading this alongside (duplicate) bug 244906 comment 2 Where 12.2 can not be installed on a physical or virtual computer: * should we advise installation of 12.1 followed by an upgrade to 12.2-RELEASE-p2 or greater? <https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.1/> <https://download.freebsd.org/ftp/releases/VM-IMAGES/12.1-RELEASE/>
(In reply to YUAN RUI from comment #21) I would like to add a maximum size for this memory to grow on x86 to fix this. I think the code in efi_verify_staging_size could be used to check this, but will need to get an x86 UEFI test environment set up first.
FreeBSD-13.0-CURRENT-amd64-20201224-3cc0c0d66a0-255241-memstick.img > still not working on HP EliteBook 830 G7. – <https://lists.freebsd.org/pipermail/freebsd-current/2020-December/077970.html>
Any overlap with some of the more recent comments under bug 209821?
(In reply to Graham Perrin from comment #17) > HP ProBook 440 G7 – confirmed, this type of new hardware can not boot from a USB flash drive that was written from: FreeBSD-14.0-CURRENT-amd64-20210304-483c6da3a20-257149-disc1.iso ---- If I install EOL FreeBSD 12.1-RELEASE, then upgrade (to 13.0-RC1† for example): * where the .iso files for 12.2-RELEASE (and greater) will not boot, might I find that an _installed_ system will boot? If I recall correctly, the answer is "No" … ---- <http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/old-releases/amd64/ISO-IMAGES/> – the .iso files for 12.1-RELEASE are not there. Please, where are they? † https://gist.github.com/grahamperrin/6ae91f3cb559c7cd65f04cee1be13159
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b304cd9789ca7ff3df629af42a976450e8660a11 commit b304cd9789ca7ff3df629af42a976450e8660a11 Author: Warner Losh <imp@FreeBSD.org> AuthorDate: 2020-12-17 17:02:09 +0000 Commit: Ryan Moeller <freqlabs@FreeBSD.org> CommitDate: 2021-03-22 20:30:23 +0000 Drop EFI_STAGING_SIZE back down to 64M vmware can't cope with anything larger than 64MB. Drop this back to 64MB everywhere but arm. PR: 251866 MFC After: 1 week (cherry picked from commit 4d6047edb675e52b8fad57135ab3ded8e66d0dac) stand/efi/loader/copy.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
Created attachment 224114 [details] 13.0-RELEASE not booting – UEFI – HP ProBook 440 G7 Comparative test results: <https://gist.github.com/grahamperrin/5eca8231fa7e6a94a1f55991bcd7f3c4>
(In reply to Graham Perrin from comment #29) There's been a lot of chance since 12.x. this may be unrelated to this size change and should have a new bug assigned to it, I think, so we don't conflate the two issues if they are indeed separate.
Thank you, (In reply to Warner Losh from comment #30) continuation here was at the suggestion of Toomas Soome (tsoome in IRC). The hardware cases are now spun off: * bug 255072 for legacy boot * bug 255073 for UEFI boot.
(In reply to Graham Perrin from comment #31) thanks!
I'm not actively working on a fix, so move this back to 'open'
Is this still a problem on 13 or 14?
(In reply to Warner Losh from comment #34) VMware says no: https://kb.vmware.com/s/article/94528 Symptoms Virtual Machine (VM) fails to reboot into the installed guest OS after installing FreeBSD 12.x in VM with Extensible Firmware Interface (EFI). Cause FreeBSD 12.x does not create any EFI Boot Option during installation which causes the bootup to fail after rebooting. Resolution The issue has been fixed in VMware vSphere ESXi 8.0 version. Workaround To workaround the issue, please install FreeBSD 13 instead of FreeBSD 12 in a VM running on ESXi 7.0U3 and earlier versions.
To be honest 12 has become too hard to support. I don't have the gear and we don't have the manpower. So my position is the same as VMware. I'm sorry I have yo do that.
(In reply to Christian Ullrich from comment #35) This totally reminds me of Bug 247845.
<https://www.freebsd.org/releases/12.2R/errata/#open-issues> will benefit from a belated link to this bug 251866, or, at least, an outline of the loader.efi issue. Keyword: * needs patch (In reply to Warner Losh from comment #36) I hesitate before closing this report only because of the regression keyword. Given: a) 13.0-RELEASE <https://www.freebsd.org/releases/13.0R/> announced more than two years ago b) 12.4-RELEASE no longer recommended for new deployments c) stable/12 reaching end of life in less than three months d) the available resolution for users of supported versions of FreeBSD 12.⋯ (upgrade to VMware vSphere ESXi 8.0 or greater) – I do think that documentation and then closure will be reasonable.
Comment on attachment 224114 [details] 13.0-RELEASE not booting – UEFI – HP ProBook 440 G7 Obsolete (irrelevant) in the context of comment 30 (2021-04-14). What's observed in photographs such as this was fixed in FreeBSD 13.1-RELEASE (2022-05-16). From <https://www.freebsd.org/releases/13.1R/relnotes/#boot-loader>: > UEFI boot is improved for amd64. ⋯ (Sponsored by The FreeBSD Foundation)
Not a standards issue.
This appears to have been fixed in 13.1 by kib's work to allow loading the kernel anywhere.