Bug 203643 - [bhyve] NetBSD causes bhyve to exit on AMD processors
Summary: [bhyve] NetBSD causes bhyve to exit on AMD processors
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.2-STABLE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-10-08 18:19 UTC by Jimmy Olgeni
Modified: 2025-04-19 04:58 UTC (History)
10 users (show)

See Also:


Attachments
bhyve exception log (3.37 KB, text/plain)
2015-10-08 18:19 UTC, Jimmy Olgeni
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jimmy Olgeni freebsd_committer freebsd_triage 2015-10-08 18:19:09 UTC
Created attachment 161834 [details]
bhyve exception log

NetBSD 7.0 causes bhyve to exit with an exception, but only on AMD processors. Intel works fine.

uname -a:

FreeBSD olgeni 10.2-STABLE FreeBSD 10.2-STABLE #5 r288925: Tue Oct  6 20:48:48 CEST 2015     root@olgeni:/usr/obj/usr/src/sys/KERNEL  amd64

Processor:

CPU: AMD Phenom(tm) II X4 965 Processor (3411.31-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x100f43  Family=0x10  Model=0x4  Stepping=3
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
  SVM: NP,NRIP,NAsids=64
  TSC: P-state invariant

How to repeat - download NetBSD-7.0-amd64.iso, then try this:

======================================================================
truncate -s8G netbsd.img

cat > netbsd.map << EOF
(hd0) netbsd.img
(cd0) NetBSD-7.0-amd64.iso
EOF

cat > netbsd.grub << EOF
knetbsd -h -r cd0a /netbsd
boot
EOF

sudo grub-bhyve -r cd0 -m netbsd.map -M 1024 netbsd < netbsd.grub

sudo bhyve -c 1 -m 1024M -H -P -A -W -w \
    -l com1,stdio \
    -s 0:0,hostbridge \
    -s 1:0,lpc \
    -s 2:0,virtio-net,tap0 \
    -s 3,ahci-cd,NetBSD-7.0-amd64.iso \
    -s 4,virtio-blk,netbsd.img netbsd
======================================================================

You'll get something similar to this:

root on cd0a dumps on cd0b
root file system type: cd9660
warning: no /dev/console
vm exit[0]
        reason          SVM
        rip             0xffffffff80229f5f
        inst_length     2
        exitcode        0x7b
        exitinfo1       0x3f8021c
        exitinfo2       0xffffffff80229f61
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2015-10-20 23:49:38 UTC
Canonicalize assignment.
Comment 2 Jimmy Olgeni freebsd_committer freebsd_triage 2016-09-12 08:22:49 UTC
Tested again. 11.0-RC2 with NetBSD 7.0.1:

cd0 at atapibus0 drive 0: <BHYVE SATA DVD ROM, BHYVE-445C-CB8B-8A6, 001> cdrom removable
pad0: outputs: 44100Hz, 16-bit, stereo
audio0 at pad0: half duplex, playback, capture
boot device: cd0
cd0(ahcisata0:0:0):  Check Condition on CDB: 0x46 00 00 00 00 00 00 00 08 00
    SENSE KEY:  Illegal Request
     ASC/ASCQ:  Invalid Command Operation Code

root on cd0a dumps on cd0b
root file system type: cd9660
warning: no /dev/console
Abort trap

From the vm-bhyve log:

Sep 12 10:20:16: initialising
Sep 12 10:20:16:  [loader: grub]
Sep 12 10:20:16:  [uefi: no]
Sep 12 10:20:16:  [cpu: 1]
Sep 12 10:20:16:  [memory: 512M]
Sep 12 10:20:16:  [hostbridge: standard]
Sep 12 10:20:16:  [com ports: com1]
Sep 12 10:20:16:  [uuid: 9a8e5695-78c1-11e6-a88c-bcaec55be5e5]
Sep 12 10:20:16:  [utctime: no]
Sep 12 10:20:16:  [debug mode: no]
Sep 12 10:20:16:  [primary disk: disk0.img]
Sep 12 10:20:16:  [primary disk dev: file]
Sep 12 10:20:16: generated static mac 58:9c:fc:0b:b5:ee (based on 'netbsd:0:1473668416:0')
Sep 12 10:20:16: initialising network device tap0
Sep 12 10:20:16: adding tap0 -> bridge1 (public)
Sep 12 10:20:16: booting
Sep 12 10:20:16: create file /storage/vm-bhyve/netbsd/device.map
Sep 12 10:20:16:  -> (hd0) /storage/vm-bhyve/netbsd/disk0.img
Sep 12 10:20:16:  -> (cd0) /storage/vm-bhyve/.iso/NetBSD-7.0.1-amd64.iso
Sep 12 10:20:16: create file /storage/vm-bhyve/netbsd/grub.cfg
Sep 12 10:20:16:  -> timeout=3
Sep 12 10:20:16:  -> menuentry 'netbsd (bhyve install)' {
Sep 12 10:20:16:  ->     root=cd0
Sep 12 10:20:16:  ->     knetbsd -h -r cd0a /netbsd
Sep 12 10:20:16:  -> }
Sep 12 10:20:16: /usr/local/sbin/grub-bhyve -m /storage/vm-bhyve/netbsd/device.map -M 512M -r host -d /storage/vm-bhyve/netbsd netbsd
Sep 12 10:20:20:  [bhyve options: -c 1 -m 512M -AHP -U 9a8e5695-78c1-11e6-a88c-bcaec55be5e5]
Sep 12 10:20:20:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/storage/vm-bhyve/netbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:0b:b5:ee]
Sep 12 10:20:20:  [bhyve console: -l com1,stdio]
Sep 12 10:20:20:  [bhyve iso device: -s 3:0,ahci-cd,/storage/vm-bhyve/.iso/NetBSD-7.0.1-amd64.iso]
Sep 12 10:20:20: starting bhyve (run 1)
Sep 12 10:20:27: bhyve exited with status 134
Sep 12 10:20:27: destroying network device tap0
Sep 12 10:20:27: stopped
Comment 3 HrstBW 2017-04-08 22:04:45 UTC
I can confirm this with identical symptoms, on different AMD hardware, with more recent versions:

uname -a
FreeBSD bhyvehost 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64


dmesg
[...]
CPU: AMD E-350 Processor (1596.63-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x500f10  Family=0x14  Model=0x1  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802209<SSE3,MON,SSSE3,CX16,POPCNT>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,IBS,SKINIT,WDT>
[...]


Linux installations and live CDs are working fine.

Steps:

zfs create -s -V 5G bhyveguests/netbsd

==> install.map <==
(hd0) /dev/zvol/bhyveguests/netbsd
(cd0) /scratch/NetBSD-7.1/amd64/boot-com.iso

==> install.sh <==
grub-bhyve -m install.map -r cd0 -M 512 netbsdvm << EOF
knetbsd -h -r cd0a (cd0)/netbsd
boot
EOF
bhyve -c 1 -m 512M -H -P -A \
    -l com1,stdio \
    -s 0:0,hostbridge \
    -s 1:0,lpc -s 2:0,virtio-net,tap0 \
    -s 3,ahci-cd,/scratch/NetBSD-7.1/amd64/boot-com.iso \
    -s 4,virtio-blk,/dev/zvol/bhyveguests/netbsd \
    netbsdvm

./install.sh dies with:


[...]
cd0 at atapibus0 drive 0: <BHYVE SATA DVD ROM, BHYVE-5932-F7DC-D40, 001> cdrom removable
pad0: outputs: 44100Hz, 16-bit, stereo
audio0 at pad0: half duplex, playback, capture
boot device: cd0
cd0(ahcisata0:0:0):  Check Condition on CDB: 0x46 00 00 00 00 00 00 00 08 00
    SENSE KEY:  Illegal Request
     ASC/ASCQ:  Invalid Command Operation Code

root on cd0a dumps on cd0b
root file system type: cd9660
warning: no /dev/console
vm exit[0]
                reason          SVM
                                        rip             0xffffffff8022a72f
                                                                                inst_length     2
                                                                                                        exitcode        0x7b
                                                                                                                                exitinfo1       0x3f8021c  
        exitinfo2       0xffffffff8022a731
Comment 4 Peter Grehan freebsd_committer freebsd_triage 2017-04-23 19:09:35 UTC
I was able to get an install/boot working fine on a r317273 CURRENT Ryzen system with 1 and 4 vCPUs using the NetBSD 7.1 boot-com ISO.
Comment 5 Jimmy Olgeni freebsd_committer freebsd_triage 2024-12-15 22:03:05 UTC
Quick update - did not notice anything like this in a long time.
Comment 6 Michael Dexter freebsd_triage 2024-12-17 04:26:41 UTC
Closing per reporter's no longer experiencing the issue.
Comment 7 NAKAJI Hiroyuki 2025-03-10 05:06:21 UTC
Can someone reopen this bug?

I saw this issue, too, after 9+ years.

Host:
FreeBSD/amd64 14.2-RELEASE-p2 (GENERIC)
FUJITSU PRIMERGY MX130S2
CPU: AMD Sempron(tm) 145 Processor (2800.14-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x100f63  Family=0x10  Model=0x6  Stepping=3
  Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x837fd<LAHF,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,NodeId>
  SVM: NP,NRIP,NAsids=64
  TSC: P-state invariant

dmesg: https://github.com/n12i/bsd/blob/main/dmesg.boot-mx130s2.txt

Setup:
# pkg install sysutils/vm-bhyve
# pkg install sysutils/grub2-bhyve
$ grep '^vm' /etc/rc.conf.local
vm_enable="YES"
vm_dir="zfs:zroot/bhyve"

Results:
# vm create netbsd
# vm install -f netbsd NetBSD-10.1-amd64.iso
...
[   2.6641566] swwdog0: software watchdog initialized
[   2.6641566] boot device: cd0
[   2.6641566] cd0(ahcisata0:0:0):  Check Condition on CDB: 0x46 00 00 00 00 00 00 00 08 00
[   2.6727995]     SENSE KEY:  Illegal Request
[   2.6727995]      ASC/ASCQ:  Invalid Command Operation Code
[   2.6727995] root on cd0a dumps on cd0b
[   2.6727995] cd0(ahcisata0:0:0):  Check Condition on CDB: 0x46 00 00 00 00 00 00 00 08 00
[   2.6827311]     SENSE KEY:  Illegal Request
[   2.6827311]      ASC/ASCQ:  Invalid Command Operation Code
[   2.6827311] root file system type: cd9660
[   2.6827311] kern.module.path=/stand/amd64/10.1/modules
[   2.7624147] entropy: best effort
Abort trap
Comment 8 Michael Osipov freebsd_committer freebsd_triage 2025-03-10 12:37:40 UTC
Reopening as per request
Comment 9 Mark Peek freebsd_committer freebsd_triage 2025-03-12 18:41:16 UTC
Booting and installing NetBSD on an AMD Ryzen 7 7840U worked fine.

Looking at the error being returned:
exitcode 0x7b shows it was an IO exit

exitinfo1 can be decoded from the AMD Programmers Manual Volume 2 Chapter 15.10.2
0x3f8021c shows REP OUTS to port 0x3f8 (COM1 serial port)

exitinfo2 is the address of the instruction following the OUTS

The IO is handled in svm_handle_io()
https://github.com/freebsd/freebsd-src/blob/main/sys/amd64/vmm/amd/svm.c#L785

There is a check for the SVM DecodeAssist feature here:
https://github.com/freebsd/freebsd-src/blob/main/sys/amd64/vmm/amd/svm.c#L808

The system information listed in this bug report do not have DecodeAssist listed.
The 7840U I used shows these SVM features:
SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=32768

By stubbing that call to decode_assist() to return 0, I was able to get the same error to occur on my hardware.

Looking back in the history, the check svm_handlt_io() was put in via this commit and commit message:
https://github.com/freebsd/freebsd-src/commit/bbadcde418c29fe312a1d9d39b768735f285efbd

"Don't attempt to handle INS/OUTS VM-exits unless the DecodeAssist capabilityis available. The effective segment field in EXITINFO1 is not valid withoutthis capability."

You could try modifying the code to not do the decode_assist() check and see if it works on your system. Or try turning on graphics for the VM to see if it will boot using a graphics console.
Comment 10 NAKAJI Hiroyuki 2025-03-13 02:55:46 UTC
I modified sys/amd64/vmm/amd/svm.c, but got same "Abort trap".

--- a/sys/amd64/vmm/amd/svm.c
+++ b/sys/amd64/vmm/amd/svm.c
@@ -142,8 +142,8 @@ flush_by_asid(void)
 static __inline int
 decode_assist(void)
 {
-
-       return (svm_feature & AMD_CPUID_SVM_DECODE_ASSIST);
+       return 1;
+       /* return (svm_feature & AMD_CPUID_SVM_DECODE_ASSIST); */
 }
 
 static void

The command lines:
# make -C /usr/src/sys/modules/vmm obj depend all install
# kldunload vmm
# kldload vmm
# vm install -f netbsd-10 NetBSD-10.99.12-amd64-dvd.iso
# cat netbsd-10/bhyve.log
vm exit[0]
        reason          SVM
        rip             0xffffffff80234d9f
        inst_length     2
        exitcode        0x7b
        exitinfo1       0x3f8021c
        exitinfo2       0xffffffff80234da1

I'll try graphics="yes" later. Thanks.
Comment 11 Mark Peek freebsd_committer freebsd_triage 2025-03-14 21:20:24 UTC
(In reply to NAKAJI Hiroyuki from comment #10)

I would not expect that code change to give the exact same error. Given your build steps, vmm.ko was likely installed into /boot/modules. The kldload would likely look in from /boot/kernel first (look at sysctl kern.module_path) loading the existing one. You should kldload it with a fully qualified path to make sure you are loading the new one.
Comment 12 NAKAJI Hiroyuki 2025-03-17 02:34:39 UTC
(In reply to Mark Peek from comment #11)

Exactly, I had wrong vmm.ko loaded. Thanks for finding my mistake.

$ kldstat -v | grep vmm
26    1 0xffffffff83400000   33e438 vmm.ko (/boot/kernel/vmm.ko)

I reloaded vmm.ko and got the message of NetBSD's installer!!

$ sudo kldunload vmm
$ sudo kldload /boot/modules/vmm.ko
$ kldstat -v | grep vmm
26    1 0xffffffff83400000   33c0a8 vmm.ko (/boot/modules/vmm.ko)
$ sudo vm install -f netbsd-10 NetBSD-10.99.22-amd64.iso
...
[   3.1743834] entropy: best effort
Created tmpfs /dev (1835008 byte, 3552 inodes)


You are using a serial console, we do not know your terminal emulation.
Please select one, typical values are:

        vt100
        ansi
        xterm

Terminal type (just hit ENTER for 'vt220'):

My code change was so ad-hoc, and I'm not sure about the better way.
Comment 13 Keith White 2025-04-08 16:51:09 UTC
A "me too" on this.

I'm using a quite old Lenovo Ideapad Z575 and vm-bhyve.  The SVM error is slightly different, it looks like EDK2 is trying to access 0x511 (Firmware port?). I used a different workaround: update bhyve(8) to log and ignore SVM errors if environment variable "_BHYVE_AMD_A8_ENABLE_" is set.

==== head /var/run/dmesg.boot =====
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.0-CURRENT #4 main-n276261-297473069610-dirty: Mon Apr  7 12:16:25 EDT 2025
    root@dell7390:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
VT(vga): resolution 640x480
CPU: AMD A8-3500M APU with Radeon(tm) HD Graphics (1497.33-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x300f10  Family=0x12  Model=0x1  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
  SVM: NP,NRIP,NAsids=64
  TSC: P-state invariant, performance statistics

===== bhyve.log =====
vm exit[0]
        reason          SVM
        rip             0x000000007fa6e480
        inst_length     2
        exitcode        0x7b
        exitinfo1       0x511021d
        exitinfo2       0x7fa6e482


==== usr.sbin/bhyve/amd64/vmexit.c patch ====
diff --git a/usr.sbin/bhyve/amd64/vmexit.c b/usr.sbin/bhyve/amd64/vmexit.c
index 944f5de34645..94dde19eb931 100644
--- a/usr.sbin/bhyve/amd64/vmexit.c
+++ b/usr.sbin/bhyve/amd64/vmexit.c
@@ -288,6 +288,10 @@ vmexit_svm(struct vmctx *ctx __unused, struct vcpu *vcpu, struct vm_run *vmrun)
        EPRINTLN("\texitcode\t%#lx", vme->u.svm.exitcode);
        EPRINTLN("\texitinfo1\t%#lx", vme->u.svm.exitinfo1);
        EPRINTLN("\texitinfo2\t%#lx", vme->u.svm.exitinfo2);
+       if (getenv("_BHYVE_AMD_A8_ENABLE_")) { /* workaround for Lenovo Ideapad Z575 */
+               EPRINTLN("\tignoring\t_BHYVE_AMD_A8_ENABLE_ defined");
+               return (VMEXIT_CONTINUE);
+       }
        return (VMEXIT_ABORT);
 }
Comment 14 Mark Peek freebsd_committer freebsd_triage 2025-04-08 23:26:16 UTC
(In reply to Keith White from comment #13)

I'd be curious to know if the workaround in #10 would work for you as well. Did you happen to try that change at all?
Comment 15 Keith White 2025-04-09 00:48:10 UTC
No, I had not tried #10.  I still wanted to see any errors and to be able to turn any workaround on/off.

Thanks for the suggestion, though!  I tried a modified version of #10 that provides a hw.vmm.svm.disable_decode_assist sysctl that optionally disables the decode_assist() check.  This too works for me.

A long-winded way of saying "yes" #10 also works for me.

diff --git a/sys/amd64/vmm/amd/svm.c b/sys/amd64/vmm/amd/svm.c
index 6c16daaa47c2..63f559037687 100644
--- a/sys/amd64/vmm/amd/svm.c
+++ b/sys/amd64/vmm/amd/svm.c
@@ -122,6 +122,11 @@ static uint32_t nasid;
 SYSCTL_UINT(_hw_vmm_svm, OID_AUTO, num_asids, CTLFLAG_RDTUN, &nasid, 0,
     "Number of ASIDs supported by this processor");
 
+/* AMD A8 has no decode_assist, allow disabling the check */
+static int disable_decode_assist;
+SYSCTL_INT(_hw_vmm_svm, OID_AUTO, disable_decode_assist, CTLFLAG_RWTUN,
+    &disable_decode_assist, 0, "Set to 1 to disable the decode_assist check");
+
 /* Current ASID generation for each host cpu */
 static struct asid asid[MAXCPU];
 
@@ -146,7 +151,7 @@ static __inline int
 decode_assist(void)
 {
 
-       return (svm_feature & AMD_CPUID_SVM_DECODE_ASSIST);
+       return ((svm_feature & AMD_CPUID_SVM_DECODE_ASSIST) || disable_decode_assist);
 }
 
 static void
Comment 16 NAKAJI Hiroyuki 2025-04-09 12:19:42 UTC
(In reply to Keith White from comment #15)

This change worked for me. Thanks!
Comment 17 Mark Peek freebsd_committer freebsd_triage 2025-04-09 23:46:34 UTC
Thank you for the confirmations and the sysctl patch. I'll see about getting it out for a code review before committing it.