After upgrading U-Boot from 2022.10 to 2023.01, my RPi4/8G doesn't boot from MMC any longer: U-Boot 2023.01 (Jan 25 2023 - 09:38:54 +0100) DRAM: 948 MiB (effective 7.9 GiB) RPI 4 Model B (0xd03114) Core: 203 devices, 16 uclasses, devicetree: board MMC: mmc@7e300000: 3, emmc2@7e340000: 0 Loading Environment from FAT... ** Bad device specification mmc 1 ** In: serial Out: vidconsole Err: vidconsole Net: eth0: ethernet@7d580000 PCIe BRCM: link up, 5.0 Gbps x1 (SSC) starting USB... Bus xhci_pci: Register 5000420 NbrPorts 5 Starting the controller USB XHCI 1.00 scanning bus xhci_pci for devices... 2 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... No EFI system partition BootOrder not defined EFI boot manager: Cannot load any image Found EFI removable media binary efi/boot/bootaa64.efi ** Reading file would overwrite reserved memory ** Failed to load 'efi/boot/bootaa64.efi' No UEFI binary known at 0x00080000 The delta against the 2022.10 boot log starts with the "** Reading file would overwrite reserved memory **" line. Reverting back to 2022.10 u-boot.bin and all is fine again. This is the config.txt (actually it's just sysutils/rpi-firmware/files/config_arm64.txt): [all] arm_64bit=1 dtparam=audio=on,i2c_arm=on,spi=on dtoverlay=mmc dtoverlay=disable-bt device_tree_address=0x4000 kernel=u-boot.bin [pi4] hdmi_safe=1 armstub=armstub8-gic.bin
Just a possibility for possible further investigation . . . I did some experimenting and discovered that u-boot.bin and bootaa64.efi are being loaded to the same starting location. Probably U-Boot had been relocated before that point --and might be known relocate far enough way to not be a problem? I'll note that u-boot.bin is smaller than bootaa64.efi (at least for the ones I looked at). (RPi* firmware debug output to start with . . .) MESS:00:00:07.664888:0: Loaded 'u-boot.bin.2023.01.arm64' to 0x80000 size 0x93208 MESS:00:00:07.672108:0: Device tree loaded to 0x4000 (size 0xd3b9) (NOTE above [vs. later]: 0x80000 for where u-boot.bin was loaded.) . . . U-Boot> printenv . . . boot_efi_binary=load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi; if fdt addr -q ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r};else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi . . . kernel_addr_r=0x00080000 . . . U-Boot> env run usb_boot . . . Found EFI removable media binary efi/boot/bootaa64.efi ** Reading file would overwrite reserved memory ** Failed to load 'efi/boot/bootaa64.efi' No UEFI binary known at 0x00080000 EFI LOAD FAILED: continuing... (NOTE vs. earlier: 0x80000 for where u-boot.bin was loaded and for where efi/boot/bootaa64.efi was to be loaded.) . . . U-Boot> echo load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi load usb :1 0x00080000 efi/boot/bootaa64.efi . . . U-Boot> load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi ** Reading file would overwrite reserved memory ** Failed to load 'efi/boot/bootaa64.efi' NOTE: I see the same double load (u-boot.bin and bootaa64.efi) to 0x00080000 for the prior 2020.10 of U-Boot. (May be it just does not check for such and it happened to work but was not guaranteed to? May be it is supposed to be okay? Unsure.)
(In reply to Mark Millard from comment #1) Well, that was not the problem. And the problem is not specific to RPi*'s at all, through they are examples of hitting the issue. I give details of the actual problem later below, but the issue can be worked around by providing a couple of CONFIG_LMB_*'s in the fragments. Using the RPi*'s as examples: # git -C /usr/ports diff sysutils/u-boot-rpi*/files/*fragment | more diff --git a/sysutils/u-boot-rpi-0-w/files/rpi_0_w_fragment b/sysutils/u-boot-rpi-0-w/files/rpi_0_w_fragment index 56ca46fe5f31..7274e7a5f219 100644 --- a/sysutils/u-boot-rpi-0-w/files/rpi_0_w_fragment +++ b/sysutils/u-boot-rpi-0-w/files/rpi_0_w_fragment @@ -1,3 +1,5 @@ CONFIG_OF_EMBED=n CONFIG_OF_BOARD=y CONFIG_API=y +CONFIG_LMB_USE_MAX_REGIONS=y +CONFIG_LMB_MAX_REGIONS=64 diff --git a/sysutils/u-boot-rpi-arm64/files/rpi_arm64_fragment b/sysutils/u-boot-rpi-arm64/files/rpi_arm64_fragment index 7a61bd18844a..8cc45d6fd3bc 100644 --- a/sysutils/u-boot-rpi-arm64/files/rpi_arm64_fragment +++ b/sysutils/u-boot-rpi-arm64/files/rpi_arm64_fragment @@ -1,2 +1,4 @@ CONFIG_ENV_FAT_DEVICE_AND_PART="1:1" CONFIG_RPI_EFI_NR_SPIN_PAGES=2 +CONFIG_LMB_USE_MAX_REGIONS=y +CONFIG_LMB_MAX_REGIONS=64 diff --git a/sysutils/u-boot-rpi/files/rpi_fragment b/sysutils/u-boot-rpi/files/rpi_fragment index 56ca46fe5f31..7274e7a5f219 100644 --- a/sysutils/u-boot-rpi/files/rpi_fragment +++ b/sysutils/u-boot-rpi/files/rpi_fragment @@ -1,3 +1,5 @@ CONFIG_OF_EMBED=n CONFIG_OF_BOARD=y CONFIG_API=y +CONFIG_LMB_USE_MAX_REGIONS=y +CONFIG_LMB_MAX_REGIONS=64 diff --git a/sysutils/u-boot-rpi2/files/rpi2_fragment b/sysutils/u-boot-rpi2/files/rpi2_fragment index e5849534639e..05ca226cd650 100644 --- a/sysutils/u-boot-rpi2/files/rpi2_fragment +++ b/sysutils/u-boot-rpi2/files/rpi2_fragment @@ -2,3 +2,5 @@ CONFIG_OF_EMBED=n CONFIG_OF_BOARD=y CONFIG_ARMV7_NONSEC=n CONFIG_EFI_GRUB_ARM32_WORKAROUND=y +CONFIG_LMB_USE_MAX_REGIONS=y +CONFIG_LMB_MAX_REGIONS=64 diff --git a/sysutils/u-boot-rpi3/files/rpi3_fragment b/sysutils/u-boot-rpi3/files/rpi3_fragment index 1dbf430f1931..dd8033f49d4e 100644 --- a/sysutils/u-boot-rpi3/files/rpi3_fragment +++ b/sysutils/u-boot-rpi3/files/rpi3_fragment @@ -2,3 +2,5 @@ CONFIG_OF_EMBED=n CONFIG_OF_BOARD=y CONFIG_ENV_FAT_DEVICE_AND_PART="1:1" CONFIG_RPI_EFI_NR_SPIN_PAGES=2 +CONFIG_LMB_USE_MAX_REGIONS=y +CONFIG_LMB_MAX_REGIONS=64 diff --git a/sysutils/u-boot-rpi4/files/rpi4_fragment b/sysutils/u-boot-rpi4/files/rpi4_fragment index 7a61bd18844a..8cc45d6fd3bc 100644 --- a/sysutils/u-boot-rpi4/files/rpi4_fragment +++ b/sysutils/u-boot-rpi4/files/rpi4_fragment @@ -1,2 +1,4 @@ CONFIG_ENV_FAT_DEVICE_AND_PART="1:1" CONFIG_RPI_EFI_NR_SPIN_PAGES=2 +CONFIG_LMB_USE_MAX_REGIONS=y +CONFIG_LMB_MAX_REGIONS=64 What this does is avoid a case of lack of initialization tied to lack of either of 2 types of CONFIG_LMB_* use. The details are as follows. lib/lmb.c has: /* This routine called with relocation disabled. */ static long lmb_add_region_flags(struct lmb_region *rgn, phys_addr_t base, phys_size_t size, enum lmb_flags flags) { . . . if (rgn->cnt >= rgn->max) return -1; /* Couldn't coalesce the LMB, so add it to the sorted table. */ . . . This code expects rgn->max to always be well defined. But note the lack of initialization of the 2 .max fields for the implicit "#else" case in: void lmb_init(struct lmb *lmb) { #if IS_ENABLED(CONFIG_LMB_USE_MAX_REGIONS) lmb->memory.max = CONFIG_LMB_MAX_REGIONS; lmb->reserved.max = CONFIG_LMB_MAX_REGIONS; #elif defined(CONFIG_LMB_MEMORY_REGIONS) lmb->memory.max = CONFIG_LMB_MEMORY_REGIONS; lmb->reserved.max = CONFIG_LMB_RESERVED_REGIONS; lmb->memory.region = lmb->memory_regions; lmb->reserved.region = lmb->reserved_regions; #endif lmb->memory.cnt = 0; lmb->reserved.cnt = 0; } I've not found other initialization of the 2 .max fields. Presuming I got that right, then for the combination: !IS_ENABLED(CONFIG_LMB_USE_MAX_REGIONS) && !defined(CONFIG_LMB_MEMORY_REGIONS) neither .max is initialized. Another possibility is that there should be a #else leading to rejection of the compile (combination invalid): #else #error . . . (In that case CONFIG_LMB_* additions would need to have landed in all the official U-Boot configs/*_defconfig files at some point. Only a small subset of such in 2023.01 of U-Boot.) There was a time when there was: void lmb_init(struct lmb *lmb) { #if IS_ENABLED(CONFIG_LMB_USE_MAX_REGIONS) lmb->memory.max = CONFIG_LMB_MAX_REGIONS; lmb->reserved.max = CONFIG_LMB_MAX_REGIONS; #else lmb->memory.max = CONFIG_LMB_MEMORY_REGIONS; lmb->reserved.max = CONFIG_LMB_RESERVED_REGIONS; lmb->memory.region = lmb->memory_regions; lmb->reserved.region = lmb->reserved_regions; #endif lmb->memory.cnt = 0; lmb->reserved.cnt = 0; } That always provided an initialization of the 2 .max fields if the code compiled. (In this form the #error mentioned earlier would not be needed in order to catch errors. But, as stands, most configs/*_defconfig files would lead to build failure for CONFIG_LMB_MEMORY_REGIONS and CONFIG_LMB_RESERVED_REGIONS being undefined.)
(In reply to Mark Millard from comment #2) I'll note that github reports that the first tagged release with the missing .max initialization case via the preprocessor directive conditional structure was: v2022.10-rc2 So it looks like it was luck that the problem was not noticed with 2022.10.
FYI . . . I've tried some official main [so: 14] snapshots released today on the few types of systems that I have access to that I boot via U-Boot. These snapshots are using 2023.01 of U-Boot. The following booted, no "would overwrite reserved memory" report from U-Boot involved: RPi2B v1.1 (so: armv7) Used: microsd card media RPi3B (so: aarch64) Used: USB3 boot media + microsd card bootcode.bin (But see later note.) B0T 4 GiByte RPi4B Rev 1.1 Used: same USB3 boot media, no microsd card 4 GiByte Rock64 Used: microsd card media Note: When the USB3 media a was used in the RPi3B, usb reset would not find the drive unless I did something like "setenv usb_gpood_delay 2000" first and did a "usb reset" after. This is using media I've not historically needed to do such for. Using the same media in the RPi4B's, I did not have this issue. I doubt that this is usefully related to what this bugzilla is about. The following had the "would overwrite reserved memory" report and failed to load the FreeBSD EFI loader because of what it was reporting: B0T 8 GiByte RPi4B Rev 1.4 Used: same USB3 boot media, no microsd card C0T 8 GiByte RPi4B Rev 1.5 Used: same USB3 boot media, no microsd card
Use of CONFIG_LMB_MAX_REGIONS=64 with CONFIG_LMB_USE_MAX_REGIONS=y was not such a great idea for my test. So I explored more reasonable numbers and discovered: Exploring (decreasing decimal number order, not time sequence): 12 worked 10 worked (hmm: 10 == 3+7, matching memory.cnt+reserved.cnt below) 9 failed (less than memory.cnt+reserved.cnt below) 8 failed When I looked at the bdinfo just before attempting a boot, what I got was (8 GiByte RPi4B): Hit any key to stop autoboot: U-Boot> bdinfo . . . lmb_dump_all: memory.cnt = 0x3 memory[0] [0x0-0x3b2fffff], 0x3b300000 bytes flags: 0 memory[1] [0x40000000-0xfbffffff], 0xbc000000 bytes flags: 0 memory[2] [0x100000000-0x1ffffffff], 0x100000000 bytes flags: 0 reserved.cnt = 0x7 reserved[0] [0x0-0x3fff], 0x00004000 bytes flags: 0 reserved[1] [0x39c2b000-0x3b2fffff], 0x016d5000 bytes flags: 0 reserved[2] [0x3ac3b4a0-0x3b0fffff], 0x004c4b60 bytes flags: 0 reserved[3] [0x3ee62be0-0x3ee62fdf], 0x00000400 bytes flags: 4 reserved[4] [0x3ee63020-0x3ee63055], 0x00000036 bytes flags: 4 reserved[5] [0x40000000-0xfbffffff], 0xbc000000 bytes flags: 0 reserved[6] [0x100000000-0x1ffffffff], 0x100000000 bytes flags: 0 devicetree = board arch_number = 0x0000000000000000 TLB addr = 0x000000003b0f0000 irq_sp = 0x000000003ac3f940 sp start = 0x000000003ac3f940 Early malloc usage: 878 / 2000 (I've not yet seen in the code why memory and reserved are not independent, each with CONFIG_LMB_MAX_REGIONS space available.) Overall, this tends to indicate that the "implicit #else" case's lack of initialization of the 2 max fields is not directly what has been run into here. Note: The "[0x0-0x3fff], 0x00004000" is because I have U-boot reserve more than the first page for armstub8*.bin and related run-time data (e.g., its stack). The standard port just reserves the first page, not covering everything.
(In reply to Mark Millard from comment #5) I think that the 10 == 3+7 matchup may just be an unhappy accident. So far I have found that the attempt to load the FreeBSD EFI loader will attempt to add a reserved range, taking 7->8 (at least). May be there are 2 more reservations involved that I've yet to identify. In any case, it would appear that using a figure somewhat more than the minimum that currently works would be appropriate: it could cut down on the frequency of having to adjust the figure if/as more reservations are added via either the .dtb content indicating such or U-Boot adding some more of its own.
Thanks for the investigative journey so far! Just do add, I just pulled https://download.freebsd.org/snapshots/ISO-IMAGES/13.2/FreeBSD-13.2-PRERELEASE-arm64-aarch64-RPI-20230204-d07eb716f35d-254363.img.xz (which comes with 2023.01 U-Boot) and the boot log is identical to mine above. Which means that the snapshot is broken.
This patch works for my 8G Pi4 CMs: https://patchwork.ozlabs.org/project/uboot/patch/20230125230823.1567778-1-trini@konsulko.com/
(In reply to Kaveman from comment #8) Looks like that patch is already not the way they are going: QUOTE (Feb. 8, 2023, 3:16 p.m. UTC) . . . Ugh, you're right and I missed what this ended up doing, thanks for explaining it. I'll go revert my commit shortly and pick up one of the ones that raised the default. END QUOTE At some point I expect U-Boot will settle on what future updates will be like in this area. Until such an update all the alternatives are somewhat messy instead of being stable over time.
The upstream fixes to U-Boot 2023.01 for what broke things for 8 GiByte RPi4B/CM4 (and other things) look to have settled on the following 2 changes . . . lmb: Treat a region which is a subset as equal : https://source.denx.de/u-boot/u-boot/-/commit/0d91c88230fe8bd9f8d39ca2ab69cd6282e9f949 Bump LMB_MAX_REGIONS default to 16 : https://source.denx.de/u-boot/u-boot/-/commit/2dc16a2c1f924985216b3f1d6710f96d6c4fb1ab
This is fixed with u-boot-rpi-arm64-2023.07.02.