Bug 253983
| Summary: | RPI4 boot fails when geekworm x829 is attached over USB | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | ekoort <eimar.koort> | ||||||||
| Component: | arm | Assignee: | freebsd-arm (Nobody) <freebsd-arm> | ||||||||
| Status: | Closed FIXED | ||||||||||
| Severity: | Affects Only Me | CC: | hselasky, maciphone2, marklmi26-fbsd | ||||||||
| Priority: | --- | ||||||||||
| Version: | 13.0-STABLE | ||||||||||
| Hardware: | arm64 | ||||||||||
| OS: | Any | ||||||||||
| See Also: | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256441 | ||||||||||
| Attachments: |
|
||||||||||
|
Description
ekoort
2021-03-03 16:10:08 UTC
(In reply to eimar.koort from comment #0) I see "cannot initialize UEFI-subsystem".... do you boot from rpi4UEFIdev(not directly supported by FreeBSD)? if you're booting from u-boot(what I assume), please take a look : https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253981 , you can upgrade the msdos-partition-files(starup4.dat etc.) and directly boot the root-filesystem from USB, so you won't need an uSD-card. "unrecognized filesystem type" in your screenshot normally happens with incompatible u-boot-versions . there is also a u-boot-file in the source forge-link in show_bug.cgi?id=253981 , `hope that helps for you. by the way: instead of screenshots you can post dmesg(ideally from the UART-console) to: https://dmesgd.nycbug.org Hi, Dumb as i am, i don't know how to save console output from u-boot prompt. I'm not able to boot up os to provide failed boot dmesg. Sorry about that. And i almost don't undestand what do You mean by "do you boot from rpi4UEFIdev". What i did is i took sd card, did a 'dd' RPI Beta4 image to it, inserted to RPI and booted up. So yes, i'm using u-boot. That is coming with image. Now i removed USB bridge, booted FBSD, replaced files (start4.elf, fixup4.dat and bcm2711-rpi-4-b.dtb) mentioned in "show_bug.cgi?id=253981". I'm unable to find u-boot.bin file in sourceforge page or link to it. And the problem still persists - when USB bridge is connected, then it skips sd-card and eventually tries to boot from network until it ends on u-boot prompt. Also i'd like to boot from sd-card instead of usb because it makes me easier to handle this machine in case of failure. I'm not sure rpi is able to boot from mirrored usb disk(s). "When USB bridge is present during boot, then it skips SD card for unknown reason": See the following for controlling which order boot devices are tried in for RPi4B's and such. This is not under FreeBSD's control so the references are to www.raspberrypi.org pages and that is where questions about the specific type of issue should go. https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/bootflow_2711.md and: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2711_bootloader_config.md The status of this bug report should be changed to indicate that this is not a FreeBSD bug. But I can not do that for something that you submit. Hi, I did that couple a days ago with linux and set eeprom values especially to boot from SD card. Here they are: root@rpi4:~# rpi-eeprom-config BOOT_UART=0 WAKE_ON_GPIO=1 POWER_OFF_ON_HALT=0 FREEZE_VERSION=0 [all] BOOT_ORDER=0xf1 SD_BOOT_MAX_RETRIES=-1 Also it is worth to mention that linux (rasberry pi os) is able to boot up from SD card when USB bridge is connected. If eeprom values are somehow messed up then shouldn't linux boot fail too? And why did 13-CURRENT snapshot version from october 2020 boot fine? (In reply to eimar.koort from comment #4) Intersting . . . Your pictures do not cover enough material to see the full sequence. But it sounds like you can: A) Boot without the USB drives B) connect the USB drives C) use gpart show and report what is listed D) mount file systems on the various drives and see what is listed in ones possibly involved in early booting E) report on what the various file systems contain As stands, to avoid your error messages about UEFI not being able to initialize and the like, I'd be looking for whatever might try to set up and start an UEFI style boot. Unless you have deliberately tried to set up such a boot, no such attempt should be made by the FreeBSD boot sequecne on a RPi4B. Something extra or non-standard has to be present to cause the activity. Note: efi/boot/bootaa64.efi does not start a UEFI boot, it is something an already-started uefi style boot tries to find and then use. Some background notes . . . The evidence in your screen image is that the USB drive(s) are set up to boot in a UEFI style where something found a: efi/boot/bootaa64.efi (It might have found a EFI/BOOT/BOOTaa64.efi or other capitalization variation.) Currently a modern RPI image such as: FreeBSD-13.0-BETA4-arm64-aarch64-RPI.img (listed only because I've previously checked the contents of this particular example image) does contain such a file. I can even use the microsd card produced from the image to boot systems that are designed/configured for UEFI booting, such as an OverDrive 1000, via it using that file. The files in the image that are RPi* specific are simply ignored when booting a OverDrive 1000 or other such. But, despite containing the file, FreeBSD is not acively supporting booting RPi4B's via a method that uses that file (i.e., via UEFI style booting). It is possible to set up UEFI booting from software from outside the FreeBSD project. I'm only familiar with one way but there might be others. I have no direct clue how your context got a configuration that would attempt a UEFI boot. (In reply to eimar.koort from comment #4) (In reply to eimar.koort from comment #4) <<And why did 13-CURRENT snapshot version from october 2020 boot fine?>> because of the firmware. RPI4 4GB-model, right? so now you have the easiest bugfix ever seen :-) : simply overwrite the whole October 2020-msdos-partition to your current disk if you felt comfortable with it. (In reply to Klaus Küchemann from comment #7) Not likely to work without also deleting/disabling whatever is providing the UEFI-style-boot infrastructure that is looking for efi/boot/bootaa64.efi . Given the BOOT_ORDER and such, I expect the UEFI infrastructure is on the microsd card in a way that the RPi4B firmware finds, loads, and executes it. This means that looking at config.txt content is part of what should be examined. It likely has to be involved in making teh firmware load and start the UEFI infrastrcture. It appears that the UEFI infrastructure is looking at the USB drives after the infrastructure is started. But the infrastructure should not be there in the first place based on what the desired behavior is reported to be. FreeBSD does not implement or provide such infrastrcture, just the efi/boot/bootaa64.efi that the infrastrcture looks for. Similarly The RPi4B firmware does not implement a UEFI infrastructure. The UEFI infrastructure is from outside both of those places ( but might involve material from the RPi4B firmware, such as config.txt ). (In reply to Mark Millard from comment #8) <<Not likely to work without also deleting/disabling whatever is providing the UEFI-style-boot...> eimar.koort@tutamail.com swore to us :-) that he doesn't even know uefiDev. So we can assume that the crap is in the new firmware files. By overwriting the whole msdos partition with October 2020 release, the issue should be fixed for him. (In reply to Klaus Küchemann from comment #9) I am using the newest published firmware files and do not end up with an UEFI-based boot. The RPi* folks have not published a UEFI implementation in their firmware. I do not claim that there is only one UEFI implementation around for the RPi4B. (Although, that is a possibility.) The context needs to be examined independent of any claims by any of us. Things are too odd to be running on unchecked assumptions, at least if understanding is intended. (In reply to Mark Millard from comment #10) << Things are too odd to be running on unchecked assumptions, at least if understanding is intended......>> you're right, really odd.... do you remember : "GUESSING" ,:-) , my GUESS No.1 is that October 2020-release will fix it for him, if not, something odd happens in eeprom... or in the src-tree?? let's wait if 2020-release will fix it or not for eimar.koort@tutamail.com (In reply to eimar.koort from comment #0) ah, looking closer here I see two mentioned versions from you: BETA1 / BETA4 , perhaps you could try 14-current, there were issues mentioned in the ML with some 13-beta-versions.... (In reply to Klaus Küchemann from comment #11) If the context is cleanedout without looking for the UEFI implementation and finding it first, we will never know what was actually going on. Clearly it is not for me to specify what is done but I hope for having the implementation found and reported so we know what to look for in the future. But this is likely not the fastest route to eimar having things operational. Isolating what is happening when odd things are going on tends to be time consuming. So eimar could naturally decide to avoid the activity, such as by a clean-out and start over sequence. (In reply to Mark Millard from comment #13) yes, I'm also interested in finding the cause of course... for that we have to know the exact version ( Beta1, Beta4 or whatever)... in other context there were bugs reported in the ML which appeared in Beta1 and were fixed in Beta4. to be honest: I`m always on current(14 here) and didn't track all beta-versions. if 14-current works for Eimar we could ignore the cause(because it's fixed then) Thank You for responces! FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-20210225-bf667f282a7-256946.img has same behavior as 13-Betas (boots up w/o usb bridge and is not able to boot with bridge). I've been looking around trying to find 13-current snapshot image without any luck. Also copied fixup4.dat, start4.elf, bcm2711-rpi-4-b.dtb from linux msdos partition. Tried to different older available u-boot.bin versions. Even tried whole 12.2 release msdos content also without luck. One thing is: if i rename EFI directory to EFI_orig or whatever, then boot also fails and RPI tries to boot from network (with or w/o usb bridge does not matter). So i start to thinking, that the culprit is "bootaa64.efi". "config.txt" is unchanged from what comes from *.img. [all] arm_64bit=1 dtparam=audio=on,i2c_arm=on,spi=on dtoverlay=mmc dtoverlay=disable-bt device_tree_address=0x4000 kernel=u-boot.bin [pi4] hdmi_safe=1 armstub=armstub8-gic.bin Disk layout is also unchanged. USB disks are empty: dd if=/dev/zero of=/dev/da[0|1] ...etc. One thing i can try out is to remove hard disks completely and see what happends. Is there a way to get msdos content from now disappeared 13-current snapshot releases? Hi, Removed both hard disks, connected bridge and it boots up just fine. Inserted one disk x829 module (does not matter is hdd1 slot or hdd2 slot) - it boots up from sd card. Both disks added and it's not able to boot up. (In reply to eimar.koort from comment #16) ah, now we have an idea where the UEFI-message comes from: there seems to be a chip on the X829 which is activated if 2 disks are connected. that chip is probably detected by ACPI/UEFI from the OS... is the UEFI-message gone when only 1 disk is connected? I would suggest to boot from USB-disk(or are that SATA-disks on the X829)? since we don't have the X829 available here `guessing`is the only thing we can do(although don't like guessing:-).. you can send screenshots of course if you don't have a UART-console. ASMT USB bridge chipset, ASMedia ASM235CM-Chipset.... should be supported(since it is detected on boot)... hmmm..guessing ;-).. is the disk in one-disk-connection detected by FreeBSD? if you can boot up and login: you can type: 'dmesg' and attach dmesg here ...also 'gpart show' if you're logged in... Created attachment 222971 [details]
Freebsd 14-current 20210225-bf667f282a7-256946.img dmesg
Created attachment 222972 [details]
gpart show
umass1: <SupTronics X829, class 0/0, rev 3.00/1.00, addr 6> on usbus0 umass1: SCSI over Bulk-Only; quirks = 0x0100 umass1:1:1: Attached to scbus1 da1 at umass-sim1 bus 1 scbus1 target 0 lun 0 da1: <ASMT 2115 0> Fixed Direct Access SPC-4 SCSI device da1: Serial Number 20201000000A da1: 400.000MB/s transfers da1: 953869MB (1953525168 512 byte sectors) da1: quirks=0x2<NO_6_BYTE> ---- detected but not available in gpart, normally I would say : 'gpart create -s GPT da1`. .... ' gpart add -t freebsd-ufs da1 ...'newfs -U /dev/da1p1`(or da1s1 or whatever shows up in `ls /dev`)... but that doesn't answer the 2-disk-setup-issue for now... perhaps 1st format 1 disk, then connect&format the 2nd? .. hard to remote-debug by guessing... but all in all good news that the phy is detected... well... in Germany we say: "I eat a broom" if the UEFI-message reappears when you connect the 2nd data-disk now.. this could perhaps be an fbsd-bug since you boot from fdt... there is a specialist for that , called HPS@, I'll add him to this bug-report. I removed disk from second bay, booted up, gpart create -s GPT da1, gpart add -t freebsd-ufs da1; newfs -U /dev/da1p1.
# camcontrol devlist
<ASMT 2115 0> at scbus0 target 0 lun 0 (da0,pass0)
<ASMT 2115 0> at scbus1 target 0 lun 0 (da1,pass1)
# ls -la /dev/da*
crw-r----- 1 root operator 0x73 Feb 25 09:13 /dev/da0
crw-r----- 1 root operator 0x74 Feb 25 09:13 /dev/da0p1
crw-r----- 1 root operator 0x81 Feb 25 09:13 /dev/da1
# gpart show
=> 63 62333889 mmcsd0 MBR (30G)
63 2016 - free - (1.0M)
2079 102312 1 fat32lba [active] (50M)
104391 62228537 2 freebsd (30G)
62332928 1024 - free - (512K)
=> 0 62228537 mmcsd0s2 BSD (30G)
0 57 - free - (29K)
57 62228480 1 freebsd-ufs (30G)
=> 40 1953525088 da0 GPT (932G)
40 1953525088 1 freebsd-ufs (932G)
=> 40 1953525088 diskid/DISK-202010000009 GPT (932G)
40 1953525088 1 freebsd-ufs (932G)
Power down. Added second disk to hdd2 bay. Poweron and unable to boot (as it can be seen from very first attacment/screenshot).
does this mean I have to eat a broom? :-) Ha Ha .. O.K., clearly a thing for HPS@, if he doesn't know, we all do not know what happens with your chipset on 2-disk-boot-setup..at least not by "remote-guessing".. although unprofessional one last guess: do the same thing with the second disk(gpart) and try again... **unrecognized filesystem** on your screenshot made me think of this... and one very last guess: only 10 percent that this will fix it.. but who knows... otherwise you can hope hope for an HPS-idea... Hi,
Now both HDD-s are gparted/ufs'ed. It's able to boot up with only one hdd disk connected.
Current layout is like that (Booted up without usb bridge. When up, added bridge and did "gpart show"):
=> 63 62333889 mmcsd0 MBR (30G)
63 2016 - free - (1.0M)
2079 102312 1 fat32lba [active] (50M)
104391 62228537 2 freebsd (30G)
62332928 1024 - free - (512K)
=> 0 62228537 mmcsd0s2 BSD (30G)
0 57 - free - (29K)
57 62228480 1 freebsd-ufs (30G)
=> 40 1953525088 da0 GPT (932G)
40 1953525088 1 freebsd-ufs (932G)
=> 40 1953525088 diskid/DISK-202010000009 GPT (932G)
40 1953525088 1 freebsd-ufs (932G)
=> 40 1953525088 da1 GPT (932G)
40 1953525088 1 freebsd-ufs (932G)
=> 40 1953525088 diskid/DISK-20201000000A GPT (932G)
40 1953525088 1 freebsd-ufs (932G)
And just for clarification: disks are regular 2.5" sata hdd-s, which are connected over usb bridge to RPI.
Regards,
yeah, that was expected... well, I think again of your screenshot.... to force the boot from uSD you can interrupt to the u-boot-prompt... commands like : setenv devnum 1 (Enter) 'run mmc_boot' ( or simply : 'boot') (Enter) .. depending on the devnum of the uSD... saveenv often fails to write uSD but if you're lucky, 'saveenv' would make that setting persistent... hope that will help... and.... : according to your screenshot the issue consists of the bootdevice (which is one the sata-disks)... I would consider to prepare one sata-disk as the boot disk... which should fix the bootup if you can't 'setenv devnum xy' for the uSD... Hi, "Screenshot": U-Boot> setenv devnum 0 U-Boot> run mmc_boot switch to partition #0, OK mmc0 is current device Scanning mmc 0:1 ... Found EFI removable media binary efi/boot/bootaa64.efi libfdt fdt_check_header(): FDT_ERR_BADMAGIC Scanning disk mmc@7e300000.blk... Disk disk mmc@7e300000.blk not ready. Scanning emmc2@7e340000.blk... ** Unrecognized filesystem type ** Scanning disk usb_mass_storage.lun0... ** Unrecognized filesystem type ** Scanning disk usb_mass_storage.lun0... ERROR: failure to add disk device usb_mass_storage.lun0, r = 20 Eeror: Cannot initialize UEFI sub-system, r = 20 1258028 bytes read in 72 ms (16.7 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC Error: Cannot initialize UEFI sub-system, r = 20 EFI LOAD FAILED: continuing... U-Boot> U-Boot> setenv devnum 1 ends with booting from network yeah, of course the controller hangs on FreeBSD-boot in bootaa64.efi, not in u-boot...but was worth a try... thanks for testing everything by "remote-guessing" ;-) ... out of ideas for now ... HPS has developed a tool called hw.usb.template... where you can set e.g. the voltage( if the second e.g disk causes undervoltage).. perhaps he'll find the time to take a look into this bug here... Regards K. (In reply to eimar.koort from comment #30) #30 answered the question of where UEFI came from: It is an implementation in the u-boot that is in use. I've still not clue why u-boot engages its UEFI implementation or how to disable it --other than building and installing a u-boot that does not include a UEFI implementation. But the direction forward seems to be finding a way to not have a u-boot based UEFI-style-boot involved since FreeBSD does not actively support such on the RPi4B. Note that UEFI use of itself does not indicate if ACPI is in use vs. a Device Tree. The way to boot with UEFI that I know of is just ACPI based in the only mode that I've tested. (It is not a u-boot based UEFI implementation.) One possibility is seeing if UEFI with ACPI can be enabled in u-boot (and the Device Tree disabled?). (In reply to Mark Millard from comment #32) I did some reading and I got this wrong: apparently, for armv8, u-boot is always set up and used as a UEFI implementation (generally not ACPI style, so EBBR UEFI or a useful subset). That appears to mean that what we have run into is a UEFI implementation limitation of some kind in the u-boot, one tied to handling 2 drives instead of one in the context, resulting in the explicit ERROR/Error/FAILED messages listed in the below sequence, for example: Scanning disk usb_mass_storage.lun0... ERROR: failure to add disk device usb_mass_storage.lun0, r = 20 Error: Cannot initialize UEFI sub-system, r = 20 1258028 bytes read in 72 ms (16.7 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC Error: Cannot initialize UEFI sub-system, r = 20 EFI LOAD FAILED: continuing... So it looks like the issue would require a u-boot change/fix, not a RPi* firwmare or FreeBSD fix. In other words, u-boot needs to be changed to manage to initialize its "UEFI sub-system" successfully for the type of context. This would mean that the proper place for a bug report is upstream for u-boot and possibly one in the FreeBSD port indicating to pick up the fix if one is made. Relative to the FreeBSD system it is "not a bug" because FreeBSD is not the upstream for the u-boot FreeBSD is trying to use. (In reply to Mark Millard from comment #33) if u-boot were the cause that would probably mean that every Linux using u-boot (except Raspberry Pi OS(comment #4)) would also fail to boot in 2-disk-setup. I normally really don't like to refer to Linux in FreeBSD-topics, but if it were mine controller, I would test e.g. ubuntu with u-boot. Then we could also be sure it's not a fbsd-bug and the issue could be reported to e.g. the manufacturer of the controller. (In reply to Klaus Küchemann from comment #34) That presumes FreeBSD and Linux build the exact same u-boot sources with the exact same options and the like. I do not expect that such is the case. It might be how u-boot was built or some such, but the UEFI implementation is contained in u-boot, not elsewhere. u-boot does not make use of FreeBSD when initializing its UEFI implementation as far as I can tell. u-boot produced the error messages, not FreeBSD. But I'm not aware of a good substitution test sequence (no alternatives to substitute). May be something that would execute far enough to tell if the UEFI initialization failed or not but that might not be able to deal with later activities? Debug builds of u-boot might report something about why its UEFI initialization fails. (In reply to Mark Millard from comment #35) FYI: # pwd /wrkdirs/usr/ports/sysutils/u-boot-rpi4/work/u-boot-2020.04 # grep -r 'UEFI sub-system' ./ ./cmd/bootefi.c: printf("Error: Cannot initialize UEFI sub-system, r = %lu\n", ./cmd/efidebug.c: printf("Error: Cannot initialize UEFI sub-system, r = %lu\n", ./cmd/nvedit_efi.c: printf("Error: Cannot initialize UEFI sub-system, r = %lu\n", ./cmd/nvedit_efi.c: printf("Error: Cannot initialize UEFI sub-system, r = %lu\n", ./common/bootm_os.c: printf("## Failed to initialize UEFI sub-system: r = %lu\n", ./doc/api/efi.rst:Initialization of the UEFI sub-system ./test/py/tests/test_efi_selftest.py: """Test the device tree support in the UEFI sub-system Each instance of the Error message code looks like: /* Initialize EFI drivers */ ret = efi_init_obj_list(); if (ret != EFI_SUCCESS) { printf("Error: Cannot initialize UEFI sub-system, r = %lu\n", ret & ~EFI_ERROR_MASK); return CMD_RET_FAILURE; } (In reply to Mark Millard from comment #36) (I should note that I'm not looking at the exact u-boot sources but the 2020.04 sources I had around from a prior investigation of some behavior.) The earlier ERROR message is from: efi_status_t efi_disk_register(void) { . . . for (uclass_first_device_check(UCLASS_BLK, &dev); dev; uclass_next_device_check(&dev)) { struct blk_desc *desc = dev_get_uclass_platdata(dev); const char *if_typename = blk_get_if_type_name(desc->if_type); /* Add block device for the full device */ printf("Scanning disk %s...\n", dev->name); ret = efi_disk_add_dev(NULL, NULL, if_typename, desc, desc->devnum, 0, 0, &disk); if (ret == EFI_NOT_READY) { printf("Disk %s not ready\n", dev->name); continue; } if (ret) { printf("ERROR: failure to add disk device %s, r = %lu\n", dev->name, ret & ~EFI_ERROR_MASK); return ret; } . . . } (The "Scanning disk" message matches the above.) There is seaprate code in the routine based on CONFIG_BLK being defined (above case) vs. not (below case) that looks like: for (if_type = 0; if_type < IF_TYPE_COUNT; if_type++) { const struct blk_driver *cur_drvr; const char *if_typename; cur_drvr = blk_driver_lookup_type(if_type); if (!cur_drvr) continue; if_typename = cur_drvr->if_typename; printf("Scanning disks on %s...\n", if_typename); for (i = 0; i < 4; i++) { struct blk_desc *desc; char devname[32] = { 0 }; /* dp->str is u16[32] long */ desc = blk_get_devnum_by_type(if_type, i); if (!desc) continue; if (desc->type == DEV_TYPE_UNKNOWN) continue; snprintf(devname, sizeof(devname), "%s%d", if_typename, i); /* Add block device for the full device */ ret = efi_disk_add_dev(NULL, NULL, if_typename, desc, i, 0, 0, &disk); if (ret == EFI_NOT_READY) { printf("Disk %s not ready\n", devname); continue; } if (ret) { printf("ERROR: failure to add disk device %s, r = %lu\n", devname, ret & ~EFI_ERROR_MASK); return ret; } . . . The distinction might get into if more than one "LUN" is not handled vs. handled. (Not clear to me at this point.) Separately, it looks to me like the "20" figure from the ERROR and Error messages are from: #define EFI_ALREADY_STARTED (EFI_ERROR_MASK | 20) Based on the lack of another type of message it appears that the code that returned that value was: static efi_status_t efi_protocol_open( struct efi_handler *handler, void **protocol_interface, void *agent_handle, void *controller_handle, uint32_t attributes) { struct efi_open_protocol_info_item *item; struct efi_open_protocol_info_entry *match = NULL; bool opened_by_driver = false; bool opened_exclusive = false; /* If there is no agent, only return the interface */ if (!agent_handle) goto out; /* For TEST_PROTOCOL ignore interface attribute */ if (attributes != EFI_OPEN_PROTOCOL_TEST_PROTOCOL) *protocol_interface = NULL; /* * Check if the protocol is already opened by a driver with the same * attributes or opened exclusively */ list_for_each_entry(item, &handler->open_infos, link) { if (item->info.agent_handle == agent_handle) { if ((attributes & EFI_OPEN_PROTOCOL_BY_DRIVER) && (item->info.attributes == attributes)) return EFI_ALREADY_STARTED; } else { . . . Unfortunately, I do nto have the background to readily/rapidly put all these pieces together into an explanation or an idea of what would need to change. (In reply to Mark Millard from comment #35) yes, u-boot produces the error-message , so it's the trigger, but is it the cause?? I doubt that. while of course possible, that there's a patch in uboot-linux-versions that isn't connected to a FreeBSD-driver(or voltage-setting or bootefi or whatever unknown)(what would mean that the cause is in FreeBSD-boot-mechanism)... but that's all wild guessing without real knowledge about what odd thing happens here... if possible I would test a working linux-u-boot with FreeBSD(if exists and if possible) to prove there is no special-u-boot-implementation by a company which didn't made it to u-boot-upstream. (In reply to Mark Millard from comment #37) <<Unfortunately, I do not have the background to....>> -> HPS (In reply to Mark Millard from comment #37) I got the lack-of-message evidence wrong: the message would only show up for debug output enabled contexts. So here is the other place where EFI_ALREADY_STARTED ends up being introduced: efi_status_t EFIAPI efi_install_multiple_protocol_interfaces (efi_handle_t *handle, ...) { EFI_ENTRY("%p", handle); efi_va_list argptr; const efi_guid_t *protocol; void *protocol_interface; efi_handle_t old_handle; efi_status_t r = EFI_SUCCESS; int i = 0; if (!handle) return EFI_EXIT(EFI_INVALID_PARAMETER); efi_va_start(argptr, handle); for (;;) { protocol = efi_va_arg(argptr, efi_guid_t*); if (!protocol) break; protocol_interface = efi_va_arg(argptr, void*); /* Check that a device path has not been installed before */ if (!guidcmp(protocol, &efi_guid_device_path)) { struct efi_device_path *dp = protocol_interface; r = EFI_CALL(efi_locate_device_path(protocol, &dp, &old_handle)); if (r == EFI_SUCCESS && dp->type == DEVICE_PATH_TYPE_END) { EFI_PRINT("Path %pD already installed\n", protocol_interface); r = EFI_ALREADY_STARTED; break; } } (In reply to Klaus Küchemann from comment #38) There is another test: eliminate FreeBSD from a copy of the media, including not having a copy of FreeBSD's loader.efi. U-boot should still start and if this activity is before loading FreeBSD's loader.efi copy (bootaa64.efi) then likely still reports the same error for the failed drive configuration. Of course, this media would not complete a boot, but we would only be checking if: ERROR: failure to add disk device usb_mass_storage.lun0, r = 20 Eeror: Cannot initialize UEFI sub-system, r = 20 1258028 bytes read in 72 ms (16.7 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC Error: Cannot initialize UEFI sub-system, r = 20 EFI LOAD FAILED: continuing... happens or not or if it does not even get that far in the message sequence. If the messages still occur, FreeBSD would then have been confirmed as not involved. (In reply to Mark Millard from comment #40) <<There is another test:..........If the messages still occur, FreeBSD would then have been confirmed as not involved.>> while I like the idea to make such a test : that wouldn't be a real confirmation because: we would know that it fails with&without a current unchanged fbsd-ufs-partition, what we wouldn't know is: would a change in fbsd-boot-mechanism(or driver or elsewhere) fix the issue? ;-) -- @eimar.koort@tutamail.com : there's a command in u-boot prompt: 'usb start' as far as I remember ... won`t probably fix the issue but perhaps puts out information... I don't think we can remote-debug this issue without having the hardware, except a hardcore-specialist can enlighten us. (In reply to Klaus Küchemann from comment #41) I've replicated the failure using a reader for multimedia cards that has 2 cards plugged in (and there is the boot media in the normal microsd card slot): Scanning usb 0:1... Found EFI removable media binary efi/boot/bootaa64.efi libfdt fdt_check_header(): FDT_ERR_BADMAGIC Error: Cannot initialize UEFI sub-system, r = 20 695648 bytes read in 22 ms (30.2 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC Error: Cannot initialize UEFI sub-system, r = 20 EFI LOAD FAILED: continuing... So I can now experiment. The above and the original report seem to both be examples of not handling multiple LUNs in a single USB device. I will note that the messages following the above are: BOOTP broadcast 1 BOOTP broadcast 2 DHCP client bound to address 192.168.1.161 (435 ms) *** ERROR: `serverip' not set Cannot autoload with TFTPGET missing environment variable: pxeuuid missing environment variable: bootfile Retrieving file: pxelinux.cfg/01-dc-a6-32-b2-e2-aa *** ERROR: `serverip' not set missing environment variable: bootfile Retrieving file: pxelinux.cfg/C0A801A1 . . . If I gather right, that is not from u-boot any more but is from the RPi4B firmware: it appears that u-boot quit when it could not initialize UEFI. If so, nothing else that requires more u-boot activity than that will happen. (In reply to Mark Millard from comment #42) Note: The experiment was with 13.0-BETA4 without removing anything. The RPi4B has the default firmware ordering. I should have looked to the end of the message sequence (it is still in u-boot): BOOTP broadcast 1 BOOTP broadcast 2 DHCP client bound to address 192.168.1.161 (435 ms) *** ERROR: `serverip' not set Cannot autoload with TFTPGET missing environment variable: pxeuuid missing environment variable: bootfile Retrieving file: pxelinux.cfg/01-dc-a6-32-b2-e2-aa *** ERROR: `serverip' not set missing environment variable: bootfile Retrieving file: pxelinux.cfg/C0A801A1 . . . *** ERROR: `serverip' not set missing environment variable: bootfile Retrieving file: pxelinux.cfg/default *** ERROR: `serverip' not set Config file not found BOOTP broadcast 1 DHCP client bound to address 192.168.1.161 (8 ms) *** ERROR: `serverip' not set Cannot autoload with TFTPGET BOOTP broadcast 1 DHCP client bound to address 192.168.1.161 (9 ms) *** ERROR: `serverip' not set Cannot autoload with TFTPGET U-Boot> So it is still in u-boot but it automatically did not try any more than the above, just going to its prompt after the network alternative did not make progress. (In reply to Mark Millard from comment #42) (In reply to Mark Millard from comment #42) good work to replicate the issue ! while off-topic : the BOOTP/DHCP-messages are from u-boot. it switches to try to netboot after failing disk-detection. when it says : ``serverip' not set" that means that you can e.g. : `setenv serverip 10.0.0.23` tftp file.xy boot (presumed you have a pxe-setup @youserverip) (In reply to Mark Millard from comment #42) <<<<So I can now experiment. The above and the original report seem to both be examples of not handling multiple LUNs in a single USB device.>>>> that was a real fu**ing good idea ! :-) since we are on rpi4 and have a special dedicated pcie/xhci-driver , I would be interested in a test with e.g. amd64-driver or RPI3-USB or similar to get closer to the root cause(if it's in FreeBSD instead of u-boot)... but I will not test tonight, working on other things... (In reply to Mark Millard from comment #43) All the experiments so far have the same result: Both of the following lead to the u-boot attempting a network boot instead of other alternatives in my configuration: A) There is not EFI/BOOT/bootaa64.efi or the like to find on a/the microsd card or via a USB device/LUN. B) There is one or more EFI/BOOT/bootaa64.efi to find but there are 2 LUNs on one device. Which device(s) have a EFI/BOOT/bootaa64.efi does not seem to matter. So u-boot gives up and tries a network boot, giving no opportunity to do more from teh microsd card media or USB media in both types of context. Note: I do not have a network boot environment set up, never have. But I've not done anything to force the RPi4B from attempting such booting. The network boot attempts are just default behavior, not something I've explicitly configured. Unless u-boot is changed so (B) does not lead to a UEFI initialization error, it appears that network booting would need to be used or some other form of u-boot configuration control would need to be used. I've no evidence that u-boot can be told to retry UEFI initialization after its first failure. But I'm no u-boot expert. (In reply to Klaus Küchemann from comment #45) u-boot never starts FreeBSD or its loader.efi copy in the failing examples (no output from the bootaa64.efi at all). FreeBSD is not involved yet at the time of u-boot switching to attempting a network boot: u-boot is what is still active. (In reply to Mark Millard from comment #46) <<<that network booting would need to be used or some other form of u-boot configuration control would need to be used.>>>> if you are willing to test whether your genius 2-disk-setup-replication is detected on a network boot I have something for you so that you can boot to pxe-server without knowing/typing any u-boot- or fbsd-prompt-command : https://sourceforge.net/projects/freebsdonlyrpi3and4netbootimg/ as said: awesome idea you had ! (In reply to Mark Millard from comment #46) <<<I've no evidence that u-boot can be told to retry UEFI initialization after its first failure. But I'm no u-boot expert.>>> hint: at first trial you can interrupt to the u-boot-prompt and type : `usb start` give a question mark ( ? ) for further command options... ... 'printenv' shows current setup... 'setenv xyz' to overwrite settings.... just for a first hint... .. ah, forgot to say : `mmcinfo` should be an interesting request to u-boot-prompt for the first trials.. mmc devices are written as e.g : `mmc 0:1` sorry for triple posting... very tired and multiple things tonight here ;-).... e.g. 'setenv devnum 1` ... for the case that you can detect any uSD by 'usb start` before... `boot` or `run mmc_boot`after 'setenv devnum xy'. I doubt that it will succeed but perhaps gives information... good night (In reply to Klaus Küchemann from comment #48) I do not expect going this direction would help identify a fix to the problem eimar ran into. (No indication of a desire for network booting.) And I've no interest in setting up a network boot environment for myself at this time. Network configuration is not something I'm knowledgable about. I provide no outward bound services and have a mostly default setup for very basic operation. But, looking, I've no clue how to address the lack of handling of multiple populated LUNs in a USB device for u-boot. Identifying the multiple-populated-LUNs context looks to be about my limit for eimar's problem. It may well be that a u-boot build configuration could be specified that would work already but FreeBSD's port just does not do such. I've no clue if this is just a configuration problem for building u-boot vs. more being involved. A commit references this bug: Author: manu Date: Fri Mar 5 10:32:15 UTC 2021 New revision: 567376 URL: https://svnweb.freebsd.org/changeset/ports/567376 Log: sysutils/rpi-firmware: Update to 20210303 Fix USB on RPI4 PR: 252971, 253981, 253983 MFH: 2021Q1 Changes: head/sysutils/rpi-firmware/Makefile head/sysutils/rpi-firmware/distinfo head/sysutils/rpi-firmware/pkg-plist A commit references this bug: Author: manu Date: Fri Mar 5 10:33:54 UTC 2021 New revision: 567377 URL: https://svnweb.freebsd.org/changeset/ports/567377 Log: MFH: r567376 sysutils/rpi-firmware: Update to 20210303 Fix USB on RPI4 PR: 252971, 253981, 253983 Changes: _U branches/2021Q1/ branches/2021Q1/sysutils/rpi-firmware/Makefile branches/2021Q1/sysutils/rpi-firmware/distinfo branches/2021Q1/sysutils/rpi-firmware/pkg-plist So far I've not found any u-boot-2020.10 from
the port's code supporting UEFI's :
#define MSG_DEVICE_LOGICAL_UNIT_DP 0x11
typedef struct {
EFI_DEVICE_PATH_PROTOCOL Header;
///
/// Logical Unit Number for the interface.
///
UINT8 Lun;
} DEVICE_LOGICAL_UNIT_DEVICE_PATH;
It appears that for USB the above is used in
UEFI for LUNs. (Some other things like SCSI
have LUN fields built in instead of in a separate.)
[Clearly: I grabbed some notation from something
else that does support such, in this case from
a EDK2 file.]
I only see in u-boot-2020.10/include/efi_api.h :
#define DEVICE_PATH_TYPE_MESSAGING_DEVICE 0x03
# define DEVICE_PATH_SUB_TYPE_MSG_ATAPI 0x01
# define DEVICE_PATH_SUB_TYPE_MSG_SCSI 0x02
# define DEVICE_PATH_SUB_TYPE_MSG_USB 0x05
# define DEVICE_PATH_SUB_TYPE_MSG_MAC_ADDR 0x0b
# define DEVICE_PATH_SUB_TYPE_MSG_USB_CLASS 0x0f
# define DEVICE_PATH_SUB_TYPE_MSG_SATA 0x12
# define DEVICE_PATH_SUB_TYPE_MSG_NVME 0x17
# define DEVICE_PATH_SUB_TYPE_MSG_SD 0x1a
# define DEVICE_PATH_SUB_TYPE_MSG_MMC 0x1d
with matching struct types. Nothing for 0x11.
May be some u-boot's have been modified to ignore
USB devices with multiple populated storage LUNs
as far as UEFI support is concerned --instead of
rejecting the whole UEFI initialization. But I see
no evidence of even a start at handling an
equivalent of DEVICE_LOGICAL_UNIT_DEVICE_PATH for
USB.
That results in the conflicting context (via an
incomplete path specification).
Of course, ignoring the USB LUNs would not make
the multi-drive context work, though it could
let the microsd card boot occur for the example
context that cause the bugzilla entry.
It looks like providing USB storage multi-LUN
support would be a fairly significant upstream
effort for active u-boot developers that are
familiar with working on the code.
I have not checked u-boot code from after what
the sysutils/u-boot-rpi* get.
(In reply to commit-hook from comment #54) I have meanwhile replicated & successfully booted your genius multi-LUN-2-disk-replication-setup :-) Ha Ha great catch you did, to say that again ! I`m in the middle of compiling a patch into u-boot and will upload that u-boot-compilation if succeeds... if I can't patch it tonight I will give some hints for eimar.koort@tutamail.com how he can boot his setup... no warranties that I can patch it tonight.. (In reply to Mark Millard from comment #55) In: QUOTE Of course, ignoring the USB LUNs would not make the multi-drive context work, though it could let the microsd card boot occur for the example context that cause the bugzilla entry. END QUOTE "work" was meant to be "as the boot drive". I was not trying to imply anything about FreeBSD being unable to use the drives. Ignoring the multiple-storage-LUNs on a a device likely would make things operate if those LUNs were never to be boot media. (In reply to Klaus Küchemann from comment #56) Note: I added a second multimedia reader with media on it as well to make a more complete test case. All the media are simply formatted as one msdsofs on the MBR media as stands, no files or directories added. On the readers I used, the LUNs that I populated are 1 and 3 on each. (There are a bunch more LUNs for various kinds of media. I doubt the readers would work if all were populated at once.) As far as I can tell, an existing workaround is to connect the device(s) that have multiple storage LUNs only after booting if once's context allows for such a manual operation. (Presumes not booting via one of those LUNs.) Have fun getting it going in whatever way you are working on. (In reply to Mark Millard from comment #58) first u-boot-source-patch trials (of course) failed for me tonight, no real surprise, :-) Ha Ha O.K., here`s a workaround I hope that will work for your test-setup and the bug-reporter`s geekworm-board: interrupt to get into the u-boot-prompt(e.g. by holding the TAB-key). -- type exactly following one-liner : usb stop; boot (then press Enter) ..... this dirty workaround is ONLY useful for this exact multi-lun-usb-setup when booting from the uSD-slot... it stops the failed controller in u-boot and Rob Crowston`s xhci-driver will restart USB later through another xhci-reset (In reply to Mark Millard from comment #58) FYI https://github.com/u-boot/u-boot/blob/6a026e5649f00c0b157a935279dfd625889db675/common/usb_storage.c#L1468-L1476 https://github.com/u-boot/u-boot/blob/6a026e5649f00c0b157a935279dfd625889db675/drivers/usb/host/xhci-ring.c#L484-L492 (In reply to Klaus Küchemann from comment #59) I use a serial console but others may not. "usb stop" might stop USB keyboard input and such until something happens to start up usb again. I'm not sure when that would be in various cases. RPi4B's 4GB and less (v1.1, v1.2 at least): I've never checked if "Rob Crowston`s xhci-driver will restart USB later through another xhci-reset" or not. Nor do I have a means of checking models newer than the RPi4B 8GB model. In fact, no means of checking any v1.2 RPi4B either. (In reply to Mark Millard from comment #61) <<<I use a serial console but others may not. "usb stop" might stop USB keyboard input>>>>> you can do it without serial console directly in the glass-console ;-)(HDMI) if you want : the exact one-liner : usb stop; boot (then press Enter) I would be interested if it works for you (because than it will probably work for the geek worm-board as a workaround until we have an idea how to patch in u-boot) (In reply to Klaus Küchemann from comment #60) I'm not sure what the point of the two references is but I'll note that I've had no u-boot command problems looking at and dealing with the multi-LUN-storage devices: that context is not UEFI handling of it at all. The only type of context for which I know of multi-LUN storage devices having problems is for UEFI support, where u-boot's USB handling is incomplete for the issue and ends up generating UEFI references that do not include an LUN identification at all. I probably do not need to know what the two references are for. (This bugzilla is already large just from trying to get very basic information about the problem.) (In reply to Klaus Küchemann from comment #62) I'll have to get HDMI set up first. My context is normally headless. (In reply to Mark Millard from comment #63) <<This bugzilla is already large just from trying to get very basic information about the problem.>> yeah, We always do that way until nobody knows what happens :-) Ha Ha, ... but as long as it leads to a bugfix or workaround .. and you were right: it's a u-boot bug . (In reply to Mark Millard from comment #64) I am able to login at the login prompt via the USB keyboard afterwards on the RPi4B 8GB. But I'm unable do any loader input. So, for example, no ability to do a "boot -s" via loader input: no control of the boot at that stage (beyond cutting power or the like). Similarly for a RPi4B 4GB v1.1 that I tried. I could imagine a HDMI that was slow displaying from powerup, leading to need to type to get into u-boot before being able to see the prompt. (For the most part, the RPi4B*'s that I have access to have BOOT_UART=1 in the eeprom and config.txt material to enable debug output for that stage as well. This makes the early activity take a little longer, giving a little more time before u-boot shows up compared to otherwise.) (In reply to Mark Millard from comment #66) <<no ability to do a "boot -s" via loader input:>> yes , of course, because xhci will be reset(reinitialized) in later boot-stage by Rob`s driver . professionals like you have anyway headless-context or both serial&glass-console ;-) so you can interrupt to get to loader-input by serial-console even if the useless glass is connected :-) Ha Ha so what we have reached: a workaround for this issue , so that eimar.koort@tutamail.com hopefully doesn't have to plug/unplug his Geekworm-board 20 times a day.. so again for readers : one-liner @uboot-prompt : usb stop; boot (then press Enter) `guess we can close this issue because detected by you as u-boot-bug.. thanks for testing and your great replication-idea ! Hi, Installed 13-RC1 and it workaround (usb stop; boot) works for now. Thank you all! (In reply to eimar.koort from comment #68) glad that it works ! ... all credits to Mark ... wouldn't have been a solution to unplug/replug the Geekworm-board if power is on... some boards are extremely sensitive.. 2 weeks ago I toasted my RockPro64 only by one-touch-hotplug(although I guess the RPI4 is a bit less sensitive in that discipline). (In reply to eimar.koort from comment #68) I'll note that 13.0-RC1 does not have the recent sysutils/rpi-firwmare update that fixes some USB related issues, among other things. It will be 13.0-RC2 before 13.0 gets such. You might want to update once you get ahold of an updated sysutils/rpi-firwmare . (None of this deals with the u-boot multi-storage-LUN device handling issue(s).) Future: Also, I'm not sure that snapshot builds and release builds automatically track sysutils/u-boot-rpi* updates if any happen. Even now, I think the official builds might not be using the latest sysutils/u-boot-rpi-arm64. (I got some observable differences in my test builds that I would not otherwise expect.) Hi, Since there is some clarity in this, i'm closing this PR. Thanks again! FYI, from a fedora f33 context with the bootable media on unit 1 and the empty file system on unit 3 of the multi-media card reader, nothing in the RPi4B's microsd card slot: Scanning disk usb_mass_storage.lun1... ** Unrecognized filesystem type ** ** Unrecognized filesystem type ** Scanning disk usb_mass_storage.lun3... ERROR: failure to add disk device usb_mass_storage.lun3, r = 20 Error: Cannot initialize UEFI sub-system, r = 20 2676208 bytes read in 41 ms (62.2 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC Error: Cannot initialize UEFI sub-system, r = 20 EFI LOAD FAILED: continuing... BOOTP broadcast 1 DHCP client bound to address 192.168.1.171 (121 ms) *** ERROR: `serverip' not set If the bootable media was on unit 3 and the empty file system on unit 1, then it directly started attempting a network boot, not trying unit 3 at all, just unit 1. So FreeBSD's u-boot-rpi4-arm64 configuration's properties for such multi-storage-LUN USB devices need not be unusual. (In reply to Mark Millard from comment #72) See: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256441 for other examples of getting the: ERROR: failure to add disk device usb_mass_storage.lun0 . . . type of error messages. An example was via a powered hub and the 2 storage devices being plugged into the powered hub instead of directly pluggged into the RPi4B. This is sort of analogous to a single device with multiple storage LUNs populated. I'm unsure at this point if the submitter's context is analogous or not. So there may be more ways to get this. (In reply to Mark Millard from comment #73) Turns out part of my initial testing context was getting some behavior from the RPi4B not being able to well power both USB3 SSDs that I was using in the tests. So I switched to a matched pair of lower-power thumb drives. In that context, direct connection to RPi4B USB3 ports and use via a powered USB3 hub got the same U-Boot failures, even for the boot media being a microsd card. Getting: ERROR: failure to add disk device usb_mass_storage.lun0 in U-Boot and the consequence behavior is not as special-case as we thought earlier: having more than one storage device/LUN on the same RPi4B port is not required to see the problem in the U-Boot behavior. I will note that when I had a mix of one thumb drive and one USB3 SSD, things worked, both directly connected and via the powered hub. Somehow having sufficiently well matched device types looks to be involved in getting the problem. (This context looked to be well powered, all devices showing up at the expected stages and the like.) (In reply to Mark Millard from comment #74) I have replicated the problem with Fedora 34 Server on a RPi4B. Fedora 34 also uses U-Boot. I used the powered hub context to attach the thumb drives because I do not have microsd card media around for Fedora 34, just a USB3 SSD. So both RPi4B USB3 ports were in use, one for the USB3 SSD boot media and one for the powered hub with the 2 thumb drives. U-Boot 2021.04-rc3 (Mar 13 2021 - 00:00:00 +0000) DRAM: 7.9 GiB RPI 4 Model B (0xd03114) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 Loading Environment from FAT... Card did not respond to voltage select! : -110 In: serial Out: serial Err: serial Net: eth0: ethernet@7d580000 PCIe BRCM: link up, 5.0 Gbps x1 (SSC) starting USB... Bus xhci_pci: Register 5000420 NbrPorts 5 Starting the controller USB XHCI 1.00 scanning bus xhci_pci for devices... 7 USB Device(s) found scanning usb for storage devices... 3 Storage Device(s) found Hit any key to stop autoboot: 0 Card did not respond to voltage select! : -110 Card did not respond to voltage select! : -110 Device 0: Vendor: OWC Rev: 0 Prod: Envoy Pro mini Type: Hard Disk Capacity: 228936.5 MB = 223.5 GB (468862128 x 512) ... is now current device Scanning usb 0:1... Found EFI removable media binary efi/fedora/grubaa64.efi ** Unrecognized filesystem type ** ** Unrecognized filesystem type ** ** Unrecognized filesystem type ** ** Unrecognized filesystem type ** ** Unrecognized filesystem type ** ** Unrecognized filesystem type ** 2692592 bytes read in 11 ms (233.4 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC Card did not respond to voltage select! : -110 Scanning disk mmcnr@7e300000.blk... Disk mmcnr@7e300000.blk not ready Card did not respond to voltage select! : -110 Scanning disk emmc2@7e340000.blk... Disk emmc2@7e340000.blk not ready Scanning disk usb_mass_storage.lun0... ** Unrecognized filesystem type ** ** Unrecognized filesystem type ** Scanning disk usb_mass_storage.lun0... ** Unrecognized filesystem type ** Scanning disk usb_mass_storage.lun0... ERROR: failure to add disk device usb_mass_storage.lun0, r = 20 Error: Cannot initialize UEFI sub-system, r = 20 EFI LOAD FAILED: continuing... BOOTP broadcast 1 DHCP client bound to address 192.168.1.191 (64 ms) *** ERROR: `serverip' not set Cannot autoload with TFTPGET missing environment variable: pxeuuid missing environment variable: bootfile Retrieving file: . . . |
