Bug 282852 - bhyve: Failed to emulate instruction sequence
Summary: bhyve: Failed to emulate instruction sequence
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bhyve (show other bugs)
Version: 14.2-STABLE
Hardware: Any Any
: --- Affects Only Me
Assignee: Mark Peek
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-11-18 18:03 UTC by Sean Farley
Modified: 2025-11-17 16:01 UTC (History)
7 users (show)

See Also:
jlduran: mfc-stable14+
jlduran: mfc-stable13+


Attachments
Add F6 test emulation (1.27 KB, patch)
2024-11-19 23:56 UTC, Mark Peek
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sean Farley freebsd_committer freebsd_triage 2024-11-18 18:03:46 UTC
After an update and reboot to a VM I have, bhyve now returns an error:

Failed to emulate instruction sequence [ 41f646040874064c017d88eb144c89 ] at 0xbea5fcf9

Apparently, this occurs when a UEFI variables file is updated.  The claim is that bhyve did not support emulation of "testb imm8,r/m8".  I can confirm that replacing the UEFI variables file with a previous copy does allow the VM to boot.

There is an issue that fixes this for illumos (https://www.illumos.org/issues/14483) with the fix on GitHub (https://github.com/illumos/illumos-gate/commit/e1ded6bd708926c1adf348bccd10d6df6a12eedb).  The fix makes a call to vie_mmio_read() which is only in illumos.

I had tried that patch with using memread() without success, but my bhyve/vmm-foo is lacking.  :)
Comment 1 Mark Peek freebsd_committer freebsd_triage 2024-11-19 23:56:40 UTC
Created attachment 255307 [details]
Add F6 test emulation

Likely your change looked similar to this patch. You need to:

1. Apply the patch
2. Rebuild vmm.ko
3. Rebuild bhyve

I'm guessing you did #1 and #2. The instruction emulation runs in the bhyve user process which needs #3.

Note: I tested this with a fabricated test in a FreeBSD VM with a user program and it seemed to work. I'd be curious if this fixes your uefi issue.
Comment 2 Sean Farley freebsd_committer freebsd_triage 2024-11-20 10:32:39 UTC
(In reply to Mark Peek from comment #1)
Yes, you are exactly right.  Are you psychic?  :)  I did not rebuild bhyve.  I just tested, and a rebuilt bhyve works now with the patch from GitHub.

The patch is very similar except in the linked patch:
    .op_flags = VIE_OP_F_IMM8
vs your patch:
    .op_flags = VIE_OP_F_IMM

Is there a particular reason to choose the *_IMM flag?
Comment 3 mario felicioni 2024-11-20 10:58:36 UTC
(In reply to Mark Peek from comment #1)

If I can suggest : 

1. Apply the patch

2. If you want to passthru your NVIDIA gpu to your Windows 10,add to x86.c the string "static const char bhyve_id[12] = "bhyve bhyve "; according with this thread :

https://forums.freebsd.org/threads/trying-to-understand-why-only-some-kind-of-vmm-ko-module-allows-to-passthru-my-nvidia-gpu-to-a-windows-11-vm.93690/post-662088

3. Rebuild vmm.ko
4. Rebuild bhyve
Comment 4 Mark Peek freebsd_committer freebsd_triage 2024-11-20 16:28:56 UTC
(In reply to Sean Farley from comment #2)
> Is there a particular reason to choose the *_IMM flag?

Good catch, yes, it should be VIE_OP_F_IMM8. I didn't apply the patch but did copy/paste/edit from the existing code.

Feel free to commit since you have the right patch.
Comment 5 Sean Farley freebsd_committer freebsd_triage 2024-11-21 03:01:05 UTC
(In reply to Mark Peek from comment #4)
> Feel free to commit since you have the right patch.

Thank you for the approval.  However, I am an alumni, so I am unable to commit anything.
Comment 6 Jose Luis Duran freebsd_committer freebsd_triage 2025-03-03 07:16:28 UTC
Went ahead and created: https://reviews.freebsd.org/D49208
I can also confirm that this patch fixes the issue.
Comment 7 commit-hook freebsd_committer freebsd_triage 2025-03-04 13:54:29 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=49a4838a0d94e145a826abf02aa03ff444e614e3

commit 49a4838a0d94e145a826abf02aa03ff444e614e3
Author:     Jose Luis Duran <jlduran@FreeBSD.org>
AuthorDate: 2025-03-04 13:51:47 +0000
Commit:     Jose Luis Duran <jlduran@FreeBSD.org>
CommitDate: 2025-03-04 13:51:47 +0000

    vmm: Emulate testb imm8,r/m8

    Add support for "testb imm8,r/m8" emulation.

    PR:             261940
    PR:             282852
    Reviewed by:    markj, emaste
    Approved by:    emaste (mentor)
    Obtained from:  Illumos (https://www.illumos.org/issues/14483)
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D49208

 sys/amd64/vmm/vmm_instruction_emul.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Comment 8 commit-hook freebsd_committer freebsd_triage 2025-03-11 00:44:20 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3733546de363eda2d7bade2c110b8ba69a8e3c3e

commit 3733546de363eda2d7bade2c110b8ba69a8e3c3e
Author:     Jose Luis Duran <jlduran@FreeBSD.org>
AuthorDate: 2025-03-04 13:51:47 +0000
Commit:     Jose Luis Duran <jlduran@FreeBSD.org>
CommitDate: 2025-03-11 00:41:29 +0000

    vmm: Emulate testb imm8,r/m8

    Add support for "testb imm8,r/m8" emulation.

    PR:             261940
    PR:             282852
    Reviewed by:    markj, emaste
    Approved by:    emaste (mentor)
    Obtained from:  Illumos (https://www.illumos.org/issues/14483)
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D49208

    (cherry picked from commit 49a4838a0d94e145a826abf02aa03ff444e614e3)

 sys/amd64/vmm/vmm_instruction_emul.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Comment 9 commit-hook freebsd_committer freebsd_triage 2025-03-11 00:44:23 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=20083496866c379aef98fc352a4b6fd49801894d

commit 20083496866c379aef98fc352a4b6fd49801894d
Author:     Jose Luis Duran <jlduran@FreeBSD.org>
AuthorDate: 2025-03-04 13:51:47 +0000
Commit:     Jose Luis Duran <jlduran@FreeBSD.org>
CommitDate: 2025-03-11 00:42:30 +0000

    vmm: Emulate testb imm8,r/m8

    Add support for "testb imm8,r/m8" emulation.

    PR:             261940
    PR:             282852
    Reviewed by:    markj, emaste
    Approved by:    emaste (mentor)
    Obtained from:  Illumos (https://www.illumos.org/issues/14483)
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D49208

    (cherry picked from commit 49a4838a0d94e145a826abf02aa03ff444e614e3)

 sys/amd64/vmm/vmm_instruction_emul.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Comment 10 Jose Luis Duran freebsd_committer freebsd_triage 2025-03-11 00:45:57 UTC
Thank you for your bug report!
Comment 11 Chris Petrik 2025-11-14 23:11:12 UTC
After trying to run a WIndows 11 VM i get this after the final pin creation

Failed to emulate instruction sequence [ 8908f0091c244883c2044883c00441 ] at 0xfffff8004e6018b2

which I think differs from this error. This is 14.3-p5
Comment 12 Ed Maste freebsd_committer freebsd_triage 2025-11-17 16:01:03 UTC
(In reply to Chris Petrik from comment #11)
Yes that is a new issue and will need a new bug report.