Created attachment 219835 [details] screenshot of the step where booting hangs With 12.1-RELEASE, 12.2-RELEASE, and 13.0-CURRENT's /boot/loader, my server prints "BIOS drive X: is diskXX" lines, spins the cursor for a bit, and hangs indefinitely. Removing all non-OS drives does not resolve the issue. I can replace /boot/loader with the version from 12.0-RELEASE and boot fine, so this seems like a regression. This is likely hardware-specific; the server is a Supermicro SuperStorage 6049P-E1CR36L with a mirrored zpool as the root. It was initially installed with 12.0-RELEASE, freebsd-update'd to 12.1-RELEASE, where this issue first occurred (was worked around via UEFI, see below), then freebsd-update'd to 12.2-RELEASE, which didn't resolve the issue. I then tested 13.0-CURRENT's loader and reverted to 12.0-RELEASE's loader. UEFI boot has other problems - it hangs at the "Consoles: EFI" line and will only boot normally if two drives are removed (all drive bays are usually populated). The behavior is the same with 12.0-RELEASE's /boot/loader.efi, so while it's an issue, it's probably unrelated and not a regression from 12.0-RELEASE. This is potentially related to bug #241118, but the fix in base r356003 released in 12.2-RELEASE/13.0-CURRENT has not resolved the problem.
(In reply to ncollins from comment #0) I have 2 servers SuperStorage 6029P-E1CR12L with same motherboard as in your SuperStorage 6049P-E1CR36L: Super X11DPH-T. https://www.supermicro.com/en/products/system/4U/6049/SSG-6049P-E1CR36L.cfm https://www.supermicro.com/en/products/system/2U/6029/SSG-6029P-E1CR12L.cfm Firmware Revision: 01.71.11 Firmware Build Time: 10/25/2019 BIOS Version: 3.3 BIOS Build Time: 02/24/2020 Redfish Version: 1.0.1 CPLD Version: 02.b6.01 Never had any issues with boot 11.x, 12.0 and 12.1. Didn't updated to 12.2 yet. What versions of BIOS and Firmware do you have?
Firmware Revision : 01.24 Firmware Build Time : 06/15/2017 BIOS Version: 2.1 BIOS Build Time: 06/15/2018 Redfish Version : 1.0.1 CPLD Version: 02.b6.01 Looks like I'm more out of date than I thought, I can try upgrading it. You're using BIOS boot rather than UEFI?
(In reply to ncollins from comment #2) Can't remember. Do I need reboot to BIOS for check this? I'm using zfs mirror of partitions for root. Loader is gptzfsboot. BTW, current firmware zip file is broken: > Bundled Software File Name: X11DPH-I,T,Tq_3_4_AS17305_SUM250.zip > Size (KB): 83,949 > BIOS Revision: 3.4 But real file size after download is 79,105,435. $ unzip -t X11DPH-I,T,Tq_3_4_AS17305_SUM250.zip Archive: X11DPH-I,T,Tq_3_4_AS17305_SUM250.zip testing: X11DPH-I,T,Tq_3.4_AS17305_SUM250/SMT_X11AST2500_173_05.zip OK testing: X11DPH-I,T,Tq_3.4_AS17305_SUM250/sum_2.5.0_BSD_x86_64_20200722.tar.gz OK testing: X11DPH-I,T,Tq_3.4_AS17305_SUM250/sum_2.5.0_Linux_x86_64_20200722.tar.gz OK testing: X11DPH-I,T,Tq_3.4_AS17305_SUM250/sum_2.5.0_Win_x86_64_20200722.zip OK testing: X11DPH-I,T,Tq_3.4_AS17305_SUM250/X11DPH-I,T,Tq_Software_Package_Readme.txt OK testing: X11DPH-I,T,Tq_3.4_AS17305_SUM250/X11DPH0.B03.zip ZIP decompression failed (-5) unzip: (null) You can check self: https://www.supermicro.com/support/resources/bios_ipmi.php?type=BMC_Firmware I have previous version: X11DPH-I,T,Tq_3_3_AS17111_SUM240.zip 78,815,592.
(In reply to VVD from comment #3) I found previous version at: https://drunkencat.net/misc/SupermicroBIOS.html https://www.supermicro.com/support/resources/getfile.php?SoftwareItemID=11392
(In reply to VVD from comment #3) They fixed zip-file with FW: https://www.supermicro.com/Bios/softfiles/12546/X11DPH-I,T,Tq_3_4_AS17305_SUM250.zip
(In reply to ncollins from comment #2) > You're using BIOS boot rather than UEFI? I think I'm using "Legacy BIOS Boot", not UEFI. On one server just updated FW and FreeBSD to 12.2: Firmware Revision: 01.73.05 Firmware Build Time: 09/15/2020 BIOS Version: 3.4 BIOS Build Time: 11/03/2020 Redfish Version: 1.8.0 CPLD Version: 02.b6.01 Boot and work fine. Will update 2nd in few days.
Updated 2nd server (FW + FreeBSD 12.2) - all work fine.
Thanks, I'll plan on updating the BIOS soon.