I work on SBSA Reference Platform (sbsa-ref in QEMU) and decided to check how *BSD systems work on it. NetBSD 10 boots, OpenBSD 7.3 boots, FreeBSD 14/15 boot fine. FreeBSD 13.2 hangs on initializing XHCI controller: OK set hw.usb.debug=255 OK set hw.usb.xhci.debug=255 OK boot Loading kernel... /boot/kernel/kernel text=0x2a8 text=0x8d5250 text=0x212d5c data=0x1ae510 data=0x 0+0x2a9000 0x8+0x1295d0+0x8+0x14e631- Loading configured modules... can't find '/boot/entropy' can't find '/etc/hostid' No valid device tree blob found! WARNING! Trying to fire up the kernel, but no device tree blob found! EFI framebuffer information: addr, size 0x80000000, 0x3e8000 dimensions 1280 x 800 stride 1280 masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000 ---<<BOOT>>--- Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC arm64 FreeBSD clang version 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386ae247c) VT(efifb): resolution 1280x800 module firmware already present! real memory = 4294967296 (4096 MB) avail memory = 4160184320 (3967 MB) Starting CPU 1 (1) FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled. random: entropy device external interface MAP 100fbdf0000 mode 2 pages 128 MAP 100fbe70000 mode 2 pages 160 MAP 100fbf10000 mode 2 pages 80 MAP 100fbfb0000 mode 2 pages 80 MAP 100ff500000 mode 2 pages 400 MAP 100ff690000 mode 2 pages 592 MAP 10000000 mode 0 pages 1728 MAP 60010000 mode 0 pages 1 kbd0 at kbdmux0 acpi0: <LINARO SBSAQEMU> acpi0: Power Button (fixed) acpi0: Sleep Button (fixed) acpi0: Could not update all GPEs: AE_NOT_CONFIGURED psci0: <ARM Power State Co-ordination Interface Driver> on acpi0 gic0: <ARM Generic Interrupt Controller v3.0> iomem 0x40060000-0x4007ffff,0x40080000-0x4407ffff on acpi0 its0: <ARM GIC Interrupt Translation Service> mem 0x44081000-0x440a0fff on gic0 generic_timer0: <ARM Generic Timer> irq 3,4,5 on acpi0 Timecounter "ARM MPCore Timecounter" frequency 62500000 Hz quality 1000 Event timer "ARM MPCore Eventtimer" frequency 62500000 Hz quality 1000 efirtc0: <EFI Realtime Clock> efirtc0: registered as a time-of-day clock, resolution 1.000000s uart0: <PrimeCell UART (PL011)> iomem 0x60000000-0x60000fff irq 0 on acpi0 uart0: console (115200,n,8,1) ahci0: <AHCI SATA controller> iomem 0x60100000-0x6010ffff irq 1 on acpi0 ahci0: AHCI v1.00 with 6 1.5Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 xhci0: <Generic USB 3.0 controller> iomem 0x60110000-0x6011ffff irq 2 on acpi0 xhci_init: xhci_init: CAPLENGTH=0x40 xhci_init: RUNTIMEOFFSET=0x1000 xhci_init: DOOROFFSET=0x2000 xhci_init: xHCI version = 0x0000 xhci_init: HCS0 = 0x00087001 xhci0: 32 bytes context size, 32-bit DMA xhci_init: Max slots: 64 xhci_init: HCS2=0x0000000f xhci_init: Max scratch: 0 FreeBSD 14 it works: xhci0: <Generic USB 3.0 controller> iomem 0x60110000-0x6011ffff irq 2 on acpi0 xhci_init: xhci_init: CAPLENGTH=0x40 xhci_init: RUNTIMEOFFSET=0x1000 xhci_init: DOOROFFSET=0x2000 xhci_init: xHCI version = 0x0000 xhci_init: HCS0 = 0x00087001 xhci0: 32 bytes context size, 64-bit DMA xhci_init: Max slots: 64 xhci_init: HCS2=0x0000000f xhci_init: Max scratch: 0 xhci_start_controller: xhci_reset_controller: xhci_start_controller: CONFIG=0x00000000 -> 0x00000040 xhci_start_controller: ERSTSZ=0x00000000 -> 0x00000001 xhci_start_controller: ERDP(0)=0x00000100fb109080 xhci_start_controller: ERSTBA(0)=0x00000100fb109000 xhci_start_controller: CRCR=0x00000100fb109f00 usbus0 on xhci0 Platform is QEMU HEAD version 8.1.50 (v8.1.0-1146-g50d0bfd0ed) QEMU command line arguments: -drive file=disks/FreeBSD-13.2-RELEASE-arm64-aarch64-bootonly.iso,media=cdrom -machine sbsa-ref -m 4096 -smp 2 -cpu neoverse-v1 -drive file=fat:rw:/home/marcin/devel/linaro/sbsa-qemu/sbsa-ref-status/disks/virtual/,format=raw -drive format=raw,file=/home/marcin/devel/linaro/sbsa-qemu/sbsa-ref-status/disks/full-debian.hddimg -device usb-kbd -device usb-tablet -watchdog-action none -no-reboot -monitor telnet::45454,server,nowait -serial stdio -nographic -drive if=pflash,file=firmware/SBSA_FLASH0.fd,format=raw -drive if=pflash,file=firmware/SBSA_FLASH1.fd,format=raw Firmware files are available: https://fedora.juszkiewicz.com.pl/sbsa-ref/
I'm pretty sure this is a problem we were discussing the other day, though I don't remember if Andy noted SBSA as affected. The short version is that we think the xhci controller can only do 32-bit DMA (which may be true -- does xhci actually work on 14 where it seems to think that it does 64-bit DMA?), so busdma will try to allocate memory for it below 4G but the request can't possibly be satisfied and we get stuck in the VM layer hopelessly looping. There's presumably an IOMMU that we should be using but IOMMU isn't enabled in aarch64 GENERIC. It would also be nice if we could identify this and perhaps kick back an ENOMEM instead so that we can still move on with broken xhci.
USB keyboard connected to that XHCI controller works fine with both 14-BETA and 15-CURRENT.
(In reply to Marcin Juszkiewicz from comment #2) Isn't the usb keyboard connected on a ohci controller ??
There are no other USB controllers on this system.
It occurred to me later that I read the versions backwards; the problem I complained about earlier isn't an issue for SBSA as of 332af8c25dfc[0], where we stop forcing 32-bit DMA on generic xhci controllers. I don't see any reason we couldn't backport this to stable/13 at least to solve the immediate problem... [0] https://cgit.freebsd.org/src/commit/?id=332af8c25dfc
Thanks ;)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=05a0e28167d5ab68762ad0a56d6dc9af4ed48d55 commit 05a0e28167d5ab68762ad0a56d6dc9af4ed48d55 Author: Stephen J. Kiernan <stevek@FreeBSD.org> AuthorDate: 2023-08-11 16:06:02 +0000 Commit: Kyle Evans <kevans@FreeBSD.org> CommitDate: 2023-10-04 01:42:34 +0000 xhci: Add support for 64-bit DMA in generic XHCI controller driver The XHCI controller on 64-bit SoCs need to use 64-bit DMA. Add a quirk to tell the generic XHCI driver that 32-bit DMA needs to be used, if there are any that may need to use 32-bit DMA only. PR: 274237 Reviewed by: andrew Obtained from: Juniper Networks, Inc. (cherry picked from commit 332af8c25dfc9857e997817281d7b7fa406783ef) sys/dev/usb/controller/generic_xhci.c | 9 ++++++++- sys/dev/usb/controller/generic_xhci_fdt.c | 20 +++++++++++++++----- sys/dev/usb/controller/xhci.h | 1 + 3 files changed, 24 insertions(+), 6 deletions(-)
Commit in question already exists in stable/14 + releng/14.0, backported to stable/13 and no intention of backporting it to stable/12. Thanks for the report!
Thanks for such quick fixing!