Created attachment 267719 [details] Text dump of panic Target hardware: Minisforum MS-R1 and Orange Pi 6 with the Cortex-A720 CPU Tested host versions: 15.0-STABLE from 2026-01 and 16.0-CURRENT 20260202 Expected behavior: 15.0-RELEASE VM-IMAGE boot per Ampere, ThunderX Surprise behavior: Whole system freezes requiring reset, partial freeze, network reset, console corruption, panic. Prior to the 20260202 16.0-CURRENT snapshot, the results were very unpredictable and never panicking. Pings in one console would stop upon VM boot but but be resumable. top(1) would continue running. 'dwatch execve' would only show the bhyve launch string. One console would expand 'top' to 'toppppppppppppppp'. See the attached text dump. I can add photos of the hardware console panic if it will help and am happy to retrieve other requested information.
Created attachment 267720 [details] Truss output of bhyve command
Created attachment 267724 [details] Minisforum MS-R1 CIX/A720 dmesg under 16.0-CURRENT
Created attachment 267725 [details] Minisforum MS-R1 CIX/A720 pciconf -lv under 16.0-CURRENT
(In reply to Michael Dexter from comment #0) The text dump shows that there're lots of `160456931108421470xx` from output of `vmstat -m`. That is 16045693110842147038 ( or 0xdeadc0dedeadc0de ) + some offset. It appears the kernel is accessing from / writing to un-allocated pages, and results in corrupted memory. > Whole system freezes requiring reset, partial freeze, network reset, console corruption, panic. To be clear, is that the host but not the VM freezes ? Does older 15.0-STABLE or 16.0-CURRENT work good ?
I read that only recent 15.0-STABLE is supported but I can give 15.0-RELEASE a try. Unfortunately, the snapshots are not retained for long.
Clarification: The host freezes, both from the hardware console and Xorg/XFCE.
Can you also share your loaded kernel modules ?
Fresh boot prior to loading vmm.ko: Id Refs Address Size Name 1 50 0xffff000000000000 15513d0 kernel 2 1 0xffff000001554000 4c8f18 zfs.ko 3 1 0xffff0001a3600000 22000 hcons.ko 4 2 0xffff0001a3622000 24000 hidmap.ko 5 1 0xffff0001a3646000 22000 hms.ko 6 1 0xffff0001a3668000 55000 if_rge.ko 7 1 0xffff0001a36bd000 29000 ng_ubt.ko 8 7 0xffff0001a36e6000 31000 netgraph.ko 9 3 0xffff0001a3717000 2c000 ng_hci.ko 10 4 0xffff0001a3743000 22000 ng_bluetooth.ko 11 1 0xffff0001a3765000 30000 ng_l2cap.ko 12 1 0xffff0001a3795000 3f000 ng_btsocket.ko 13 1 0xffff0001a37d4000 25000 ng_socket.ko 14 1 0xffff0001a4c00000 22000 mac_ntpd.ko
From rc.conf (Not sure why netgraph.ko is loaded): ifconfig_ue0="DHCP" ifconfig_ue0_ipv6="inet6 accept_rtadv" sshd_enable="YES" ntpd_enable="YES" ntpd_sync_on_start="YES" moused_enable="YES" dumpdev="AUTO" zfs_enable="YES"
(In reply to Michael Dexter from comment #9) > From rc.conf (Not sure why netgraph.ko is loaded): Some loaded netgraph modules, ng_ubt.ko, ng_hci.ko, ng_bluetooth.ko etc require netgraph.ko.
Giving this a try on the 2026-02-05 weekly snapshot: FreeBSD 16.0-CURRENT main-n283667-39148cbead21 A VM freezes at: Loading kernel...seconds. [Space] to pause /boot/kernel/kernel text=0x318 text=0xa62b58 The host freezes, rather than panicking, though this may relate to the randomness of the memory corruption that Zhenlei mentioned.
(In reply to Michael Dexter from comment #11) > Giving this a try on the 2026-02-05 weekly snapshot: > FreeBSD 16.0-CURRENT main-n283667-39148cbead21 > > A VM freezes at: > > Loading kernel...seconds. [Space] to pause > /boot/kernel/kernel text=0x318 text=0xa62b58 > > The host freezes, rather than panicking, though this may relate to the randomness > of the memory corruption that Zhenlei mentioned. Does the host persist without loading vmm(4) ? I did a quick look through recent commits ( from about 26/01/6 to 26/02/3 ) in main branch related to your loaded modules ( except for vmm ) but did not find any suspicious commit.
The host is stable with or without loading vmm.ko but booting a VM brings immediate or briefly-delayed host freeze or panic. Your observation that memory gets corrupted sounds plausible.
The chip appears to have a hardware flaw and a workaround is available for Linux: https://github.com/ErcinDedeoglu/orangepi-6plus-cix-sky1-smmu-fix
(In reply to Michael Dexter from comment #14) > The chip appears to have a hardware flaw and a workaround is available for Linux: > https://github.com/ErcinDedeoglu/orangepi-6plus-cix-sky1-smmu-fix Thanks for sharing that !
Giving that fix a try: devmem2 will build on FreeBSD, only throwing a few errors. This dd(1) command should be equivalent to the read operation of devmem2 as called by smmu-evtq-fix.sh: dd if=/dev/mem bs=1 skip=$((0x0b010020)) count=4 | hexdump -C However, the read causes a panic on 16.0-CURRENT on a MS-R1 and the Orange Pi 6+, which share the same CPU: panic: pmap_map_io_transient: TODO: Map out of DMAP data cpuid = 9 time = 1778010919 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x38 vpanic() at vpanic+0x1b8 panic() at panic+0x48 pmap_map_io_transient() at pmap_map_io_transient+0x184 uiomove_fromphys() at uiomove_fromphys+0x158 memrw() at memrw+0x1a8 devfs_read_f() at devfs_read_f+0xf8 dofileread() at dofileread+0x74 kern_readv() at kern_readv+0x4c sys_read() at sys_read+0xc0 do_el0_sync() at do_el0_sync+0x600 handle_el0_sync() at handle_el0_sync+0x4c --- exception, esr 0x56000000 KDB: enter: panic This happens with or without vmm.ko loaded.