Bug 285051 - [bhyve] ampere emag freezes after kldload vmm, reboot needed not even a panic
Summary: [bhyve] ampere emag freezes after kldload vmm, reboot needed not even a panic
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: bhyve (show other bugs)
Version: 15.0-CURRENT
Hardware: arm64 Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-02-26 11:55 UTC by Dave Cottlehuber
Modified: 2025-04-10 13:34 UTC (History)
3 users (show)

See Also:


Attachments
ampere emag dmesg (50.25 KB, text/plain)
2025-02-26 11:55 UTC, Dave Cottlehuber
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dave Cottlehuber freebsd_committer freebsd_triage 2025-02-26 11:55:43 UTC
Created attachment 257995 [details]
ampere emag dmesg

CURRENT, 3b60c193fc8e (upstream/main) RELNOTES: Add 02d4eeabfd73 note

running `kldload vmm` over ssh results in machine freezing.

# console

```
FreeBSD/arm64 (straylight.skunkwerks.at) (ttyu0)

login: << now we run kldload vmm in ssh session >>
vgic0: <Virtual GIC v3> on gic0
```

The machine needs a power cycle to continue, no /var/crash left either.
Comment 1 Michael Tuexen freebsd_committer freebsd_triage 2025-02-26 13:46:55 UTC
Same for me on my system. Please note that this happens for a long time.
Everytime I tried
kldload vmm
the system freezes reproducibly (100% of all cases):

tuexen@ampere32:~ % dmesg
---<<BOOT>>---
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.0-CURRENT #7 main-n275324-2be8b121bf82: Sat Feb  8 15:05:38 CET 2025
    root@ampere32.nplab.de:/usr/obj/usr/home/tuexen/freebsd-src/arm64.aarch64/sys/TCP arm64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
WARNING: WITNESS option enabled, expect reduced performance.
VT(efifb): resolution 800x600
real memory  = 137170145280 (130815 MB)
avail memory = 133673795584 (127481 MB)
Starting CPU 1 (1)
Starting CPU 2 (100)
Starting CPU 3 (101)
Starting CPU 4 (200)
Starting CPU 5 (201)
Starting CPU 6 (300)
Starting CPU 7 (301)
Starting CPU 8 (400)
Starting CPU 9 (401)
Starting CPU 10 (500)
Starting CPU 11 (501)
Starting CPU 12 (600)
Starting CPU 13 (601)
Starting CPU 14 (700)
Starting CPU 15 (701)
Starting CPU 16 (800)
Starting CPU 17 (801)
Starting CPU 18 (900)
Starting CPU 19 (901)
Starting CPU 20 (a00)
Starting CPU 21 (a01)
Starting CPU 22 (b00)
Starting CPU 23 (b01)
Starting CPU 24 (c00)
Starting CPU 25 (c01)
Starting CPU 26 (d00)
Starting CPU 27 (d01)
Starting CPU 28 (e00)
Starting CPU 29 (e01)
Starting CPU 30 (f00)
Starting CPU 31 (f01)
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
random: unblocking device.
TCP Hpts created 32 swi interrupt threads and bound 0 to cpus
random: entropy device external interface
kbd0 at kbdmux0
acpi0: <ALASKA A M I >
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
acpi0: Could not update all GPEs: AE_NOT_CONFIGURED
psci0: <ARM Power State Co-ordination Interface Driver> on acpi0
smccc0: <ARM SMCCC v1.0> on psci0
gic0: <ARM Generic Interrupt Controller v3.0> iomem 0x78000000-0x7800ffff,0x78400000-0x787fffff on acpi0
its0: <ARM GIC Interrupt Translation Service> mem 0x78020000-0x7803ffff on gic0
generic_timer0: <ARM Generic Timer> irq 11,12,13,14 on acpi0
Timecounter "ARM MPCore Timecounter" frequency 40000000 Hz quality 1000
Event timer "ARM MPCore Eventtimer" frequency 40000000 Hz quality 1000
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s
smbios0: <System Management BIOS>
smbios0: Version: 3.2
pmu0: <Performance Monitoring Unit> on acpi0
ahci0: <AHCI SATA controller> iomem 0x1c000000-0x1c000fff irq 3 on acpi0
ahci0: AHCI v1.31 with 2 6Gbps ports, Port Multiplier not supported with FBS
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahci1: <AHCI SATA controller> iomem 0x1c100000-0x1c100fff irq 4 on acpi0
ahci1: AHCI v1.31 with 2 6Gbps ports, Port Multiplier not supported with FBS
ahcich2: <AHCI channel> at channel 0 on ahci1
ahcich3: <AHCI channel> at channel 1 on ahci1
snps_dwc3_acpi0: <Synopsys Designware DWC3> iomem 0x13800000-0x138fffff irq 5 on acpi0
snps_dwc3_acpi0: 64 bytes context size, 32-bit DMA
usbus0: trying to attach
usbus0 on snps_dwc3_acpi0
snps_dwc3_acpi1: <Synopsys Designware DWC3> iomem 0x13900000-0x139fffff irq 6 on acpi0
snps_dwc3_acpi1: 64 bytes context size, 32-bit DMA
usbus1: trying to attach
usbus1 on snps_dwc3_acpi1
acpi_button0: <Power Button> on acpi0
acpi_ged0: <Generic Event Device> irq 7 on acpi0
acpi_ged0: Raw IRQ 84
acpi_ged1: <Generic Event Device> irq 8 on acpi0
acpi_ged1: Raw IRQ 72
apei0: <ACPI Platform Error Interface> on acpi0
pcib0: <Generic PCI host controller> on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <PCI-PCI bridge> at device 0.0 on pci0
pci1: <PCI bus> on pcib1
pci1: <network, ethernet> at device 0.0 (no driver attached)
pci1: <network, ethernet> at device 0.1 (no driver attached)
pcib2: <Generic PCI host controller> on acpi0
pci2: <ACPI PCI bus> on pcib2
pcib3: <PCI-PCI bridge> at device 0.0 on pci2
pci3: <PCI bus> on pcib3
pcib4: <Generic PCI host controller> on acpi0
pci4: <ACPI PCI bus> on pcib4
pcib5: <PCI-PCI bridge> at device 0.0 on pci4
pci5: <PCI bus> on pcib5
igb0: <Intel(R) I210 (Copper)> mem 0x30100000-0x301fffff,0x30200000-0x30203fff at device 0.0 on pci5
igb0: EEPROM V3.16-0 eTrack 0x8000068f
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 4 RX queues 4 TX queues
igb0: Using MSI-X interrupts with 5 vectors
igb0: Ethernet address: 68:05:ca:92:c5:41
igb0: netmap queues/slots: TX 4/1024, RX 4/1024
pcib6: <Generic PCI host controller> on acpi0
pci6: <ACPI PCI bus> on pcib6
pcib7: <PCI-PCI bridge> at device 0.0 on pci6
pci7: <PCI bus> on pcib7
pcib8: <Generic PCI host controller> on acpi0
pci8: <ACPI PCI bus> on pcib8
pcib9: <PCI-PCI bridge> at device 0.0 on pci8
pci9: <PCI bus> on pcib9
pcib10: <Generic PCI host controller> on acpi0
pci10: <ACPI PCI bus> on pcib10
pcib11: <PCI-PCI bridge> at device 0.0 on pci10
pci11: <PCI bus> on pcib11
pcib12: <Generic PCI host controller> on acpi0
pci12: <ACPI PCI bus> on pcib12
pcib13: <PCI-PCI bridge> at device 0.0 on pci12
pci13: <PCI bus> on pcib13
pcib14: <Generic PCI host controller> on acpi0
pci14: <ACPI PCI bus> on pcib14
pcib15: <PCI-PCI bridge> at device 0.0 on pci14
pcib14: pci_host_generic_core_alloc_resource FAIL: type=4, rid=28, start=0000000010000000, end=0000000010000fff, count=0000000000001000, flags=102
pcib15: failed to allocate initial I/O port window: 0x10000000-0x10000fff
pci15: <PCI bus> on pcib15
pcib16: <PCI-PCI bridge> at device 0.0 on pci15
pcib14: pci_host_generic_core_alloc_resource FAIL: type=4, rid=28, start=0000000010000000, end=0000000010000fff, count=0000000000001000, flags=3102
pcib16: failed to allocate initial I/O port window: 0x10000000-0x10000fff
pci16: <PCI bus> on pcib16
pcib14: pci_host_generic_core_alloc_resource FAIL: type=4, rid=28, start=0000000010000000, end=0000000010000fff, count=0000000000001000, flags=3102
vgapci0: <VGA-compatible display> port 0-0x7f mem 0x30000000-0x30ffffff,0x31040000-0x3105ffff at device 0.0 on pci16
cpu0: <ACPI CPU> on acpi0
uart0: <PrimeCell UART (PL011)> iomem 0x12600000-0x12600fff irq 1 on acpi0
uart0: console (115200,n,8,1)
uart1: <PrimeCell UART (PL011)> iomem 0x12610000-0x12610fff irq 2 on acpi0
armv8crypto0: <AES-CBC,AES-XTS,AES-GCM>
Timecounters tick every 1.000 msec
Attempting to load tcp_bbr
tcp_bbr is now available
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to accept, logging disabled
usbus0: 5.0Gbps Super Speed USB v3.0
usbus1: 5.0Gbps Super Speed USB v3.0
mlx5_core0: <mlx5_core> mem 0x14082000000-0x14083ffffff at device 0.0 on pci1
mlx5: Mellanox Core driver 3.7.1 (November 2021)ugen1.1: <Synopsys XHCI root HUB> at usbus1
uhub0 on usbus1
uhub0: <Synopsys XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
ugen0.1: <Synopsys XHCI root HUB> at usbus0
uhub1 on usbus0
uhub1: <Synopsys XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <Samsung SSD 860 EVO 500GB RVT02B6Q> ACS-4 ATA SATA 3.x device
ada0: Serial Number S3Z2NB0M352023L
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 476940MB (976773168 512 byte sectors)
ada0: quirks=0x3<4K,NCQ_TRIM_BROKEN>
uhub0: 1 port with 1 removable, self powered
uhub1: 1 port with 1 removable, self powered
ugen1.2: <vendor 0x04b4 product 0x6560> at usbus1
uhub2 on uhub0
uhub2: <vendor 0x04b4 product 0x6560, class 9/0, rev 2.00/90.15, addr 1> on usbus1
ugen0.2: <American Megatrends Inc. Virtual Hub> at usbus0
uhub3 on uhub1
uhub3: <American Megatrends Inc. Virtual Hub, class 9/0, rev 2.00/1.00, addr 1> on usbus0
mlx5_core0: INFO: mlx5_port_module_event:720:(pid 12): Module 0, status: plugged and enabled
uhub2: 4 ports with 4 removable, self powered
uhub3: 5 ports with 5 removable, self powered
ugen1.3: <shengxiangzuo LANBEKVM47> at usbus1
ukbd0 on uhub2
ukbd0: <shengxiangzuo LANBEKVM47, class 0/0, rev 2.00/1.00, addr 2> on usbus1
kbd1 at ukbd0
usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device American Megatrends Inc. Virtual Cdrom Device (0x046b:0xff20)
ugen0.3: <American Megatrends Inc. Virtual Cdrom Device> at usbus0
umass0 on uhub3
umass0: <American Megatrends Inc. Virtual Cdrom Device, class 0/0, rev 2.00/1.00, addr 3> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x100<NO_GETMAXLUN>
umass0:4:0: Attached to scbus4
cd0 at umass-sim0 bus 0 scbus4 target 0 lun 0
cd0: <AMI Virtual CDROM0 1.00> Removable CD-ROM SCSI device
cd0: Serial Number AAAABBBBCCCC1
cd0: 40.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
cd0: quirks=0x10<10_BYTE_ONLY>
mlx5_core: INFO: (mlx5_core0): E-Switch: Total vports 9, l2 table size(65536), per vport: max uc(1024) max mc(16384)
mlx5_core1: <mlx5_core> mem 0x14080000000-0x14081ffffff at device 0.1 on pci1
usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device American Megatrends Inc. Virtual HardDisk Device (0x046b:0xff31)
usb_msc_auto_quirk: UQ_MSC_NO_TEST_UNIT_READY set for USB mass storage device American Megatrends Inc. Virtual HardDisk Device (0x046b:0xff31)
usb_msc_auto_quirk: UQ_MSC_NO_START_STOP set for USB mass storage device American Megatrends Inc. Virtual HardDisk Device (0x046b:0xff31)
ugen0.4: <American Megatrends Inc. Virtual HardDisk Device> at usbus0
umass1 on uhub3
umass1: <American Megatrends Inc. Virtual HardDisk Device, class 0/0, rev 2.00/1.00, addr 5> on usbus0
umass1:  SCSI over Bulk-Only; quirks = 0x105<NO_TEST_UNIT_READY,NO_START_STOP,NO_GETMAXLUN>
umass1:5:1: Attached to scbus5
da0 at umass-sim1 bus 1 scbus5 target 0 lun 0
da0: <AMI Virtual HDisk0 1.00> Removable Direct Access SCSI device
da0: Serial Number AAAABBBBCCCC3
da0: 40.000MB/s transfers
da0: Attempt to query device size failed: NOT READY, Medium not present
da0: quirks=0x2<NO_6_BYTE>
mlx5_core1: INFO: mlx5_port_module_event:720:(pid 12): Module 1, status: plugged and enabled
ugen0.5: <American Megatrends Inc. Virtual Keyboard and Mouse> at usbus0
ukbd1 on uhub3
ukbd1: <American Megatrends Inc. Virtual Keyboard and Mouse, class 0/0, rev 1.10/1.00, addr 6> on usbus0
kbd2 at ukbd1
mlx5_core: INFO: (mlx5_core1): E-Switch: Total vports 9, l2 table size(65536), per vport: max uc(1024) max mc(16384)
mce0: Ethernet address: 98:03:9b:7d:2d:da
mce0: link state changed to DOWN
mce1: Ethernet address: 98:03:9b:7d:2d:db
mce1: link state changed to DOWN
CPU  0: APM eMAG 8180 r3p2 affinity:  0  0
                   Cache Type = <64 byte CWG,64 byte ERG,64 byte D-cacheline,PIPT I-cache,64 byte I-cacheline>
 Instruction Set Attributes 0 = <CRC32,SHA2,SHA1,AES+PMULL>
 Instruction Set Attributes 1 = <>
 Instruction Set Attributes 2 = <>
         Processor Features 0 = <GIC,AdvSIMD,FP,EL3,EL2,EL1 32,EL0 32>
         Processor Features 1 = <>
         Processor Features 2 = <>
      Memory Model Features 0 = <TGran4,TGran64,TGran16,SNSMem,BigEnd,16bit ASID,4TB PA>
Trying to mount root from ufs:/dev/ada0p2 [rw]...
      Memory Model Features 1 = <8bit VMID>
      Memory Model Features 2 = <32bit CCIDX,48bit VA>
      Memory Model Features 3 = <>
      Memory Model Features 4 = <>
             Debug Features 0 = <DoubleLock,2 CTX BKPTs,4 Watchpoints,6 Breakpoints,PMUv3,Debugv8>
             Debug Features 1 = <>
         Auxiliary Features 0 = <>
         Auxiliary Features 1 = <>
AArch32 Instruction Set Attributes 5 = <CRC32,SHA2,SHA1,AES+VMULL,SEVL>
AArch32 Media and VFP Features 0 = <FPRound,FPSqrt,FPDivide,DP VFPv3+v4,SP VFPv3+v4,AdvSIMD>
AArch32 Media and VFP Features 1 = <SIMDFMAC,FPHP DP Conv,SIMDHP SP Conv,SIMDSP,SIMDInt,SIMDLS,FPDNaN,FPFtZ>
CPU  1: APM eMAG 8180 r3p2 affinity:  0  1
CPU  2: APM eMAG 8180 r3p2 affinity:  1  0
CPU  3: APM eMAG 8180 r3p2 affinity:  1  1
CPU  4: APM eMAG 8180 r3p2 affinity:  2  0
CPU  5: APM eMAG 8180 r3p2 affinity:  2  1
CPU  6: APM eMAG 8180 r3p2 affinity:  3  0
CPU  7: APM eMAG 8180 r3p2 affinity:  3  1
CPU  8: APM eMAG 8180 r3p2 affinity:  4  0
CPU  9: APM eMAG 8180 r3p2 affinity:  4  1
CPU 10: APM eMAG 8180 r3p2 affinity:  5  0
CPU 11: APM eMAG 8180 r3p2 affinity:  5  1
CPU 12: APM eMAG 8180 r3p2 affinity:  6  0
CPU 13: APM eMAG 8180 r3p2 affinity:  6  1
CPU 14: APM eMAG 8180 r3p2 affinity:  7  0
CPU 15: APM eMAG 8180 r3p2 affinity:  7  1
CPU 16: APM eMAG 8180 r3p2 affinity:  8  0
CPU 17: APM eMAG 8180 r3p2 affinity:  8  1
CPU 18: APM eMAG 8180 r3p2 affinity:  9  0
CPU 19: APM eMAG 8180 r3p2 affinity:  9  1
CPU 20: APM eMAG 8180 r3p2 affinity: 10  0
CPU 21: APM eMAG 8180 r3p2 affinity: 10  1
CPU 22: APM eMAG 8180 r3p2 affinity: 11  0
CPU 23: APM eMAG 8180 r3p2 affinity: 11  1
CPU 24: APM eMAG 8180 r3p2 affinity: 12  0
CPU 25: APM eMAG 8180 r3p2 affinity: 12  1
CPU 26: APM eMAG 8180 r3p2 affinity: 13  0
CPU 27: APM eMAG 8180 r3p2 affinity: 13  1
CPU 28: APM eMAG 8180 r3p2 affinity: 14  0
CPU 29: APM eMAG 8180 r3p2 affinity: 14  1
CPU 30: APM eMAG 8180 r3p2 affinity: 15  0
CPU 31: APM eMAG 8180 r3p2 affinity: 15  1
gic0: using for IPIs
Release APs...done
WARNING: / was not properly dismounted
TCP_ratelimit: Is now initialized
WARNING: WITNESS option enabled, expect reduced performance.
Dual Console: Serial Primary, Video Secondary
ig4iic0: <Designware I2C Controller> iomem 0x126b0000-0x126b0fff irq 0 on acpi0
iicbus0: <Philips I2C bus (ACPI-hinted)> on ig4iic0
iicbus0: <unknown card> at addr 0x10
iic0: <I2C generic I/O> on iicbus0
igb0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb0 (ifp 0xffffa00806fbd800), ignoring.
lo0: link state changed to UP
igb0: link state changed to DOWN
igb0: link state changed to UP
ums0 on uhub3
ums0: <American Megatrends Inc. Virtual Keyboard and Mouse, class 0/0, rev 1.10/1.00, addr 6> on usbus0
ums1 on uhub2
ums1: <shengxiangzuo LANBEKVM47, class 0/0, rev 2.00/1.00, addr 2> on usbus1
ums0: 3 buttons and [Z] coordinates ID=0
ums1: 3 buttons and [XYZ] coordinates ID=0
tuexen@ampere32:~ %
Comment 2 commit-hook freebsd_committer freebsd_triage 2025-03-27 13:48:34 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=872fc1b8de1ac2a10465a315457cd69b4268cbe9

commit 872fc1b8de1ac2a10465a315457cd69b4268cbe9
Author:     Andrew Turner <andrew@FreeBSD.org>
AuthorDate: 2025-03-27 13:17:30 +0000
Commit:     Andrew Turner <andrew@FreeBSD.org>
CommitDate: 2025-03-27 13:47:37 +0000

    arm64/vmm: Don't load on Ampere eMAG

    Loading vmm.ko on Ampere eMAG locks up the system. Block loading for
    now until it can be fixed.

    PR:             285051
    Reviewed by:    tuexen
    Sponsored by:   Arm Ltd
    Differential Revision:  https://reviews.freebsd.org/D49155

 sys/arm64/vmm/vmm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
Comment 3 John F. Carr 2025-03-28 13:48:20 UTC
See also bug 277559.  The vmm module works reliably on some systems, hangs some of the time on my RockPro64, and always hangs on Ampere eMAG.

I have physical access to an Ampere eMAG server.  If there is a JTAG or similar debug tool I can plug in I will give it a try.
Comment 4 Dave Cottlehuber freebsd_committer freebsd_triage 2025-04-10 13:34:03 UTC
we decided not to fix this issue, and dropped vmm support for this specific mobo/cpu combo.