Bug 277326 - FreeBSD guest on QEMU with Hyper-V enlightenment on is no longer able to detect Hyper-V devices
Summary: FreeBSD guest on QEMU with Hyper-V enlightenment on is no longer able to dete...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 15.0-CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-26 09:56 UTC by Zhenlei Huang
Modified: 2024-02-27 17:16 UTC (History)
4 users (show)

See Also:


Attachments
dmesg of 14.0 (6.67 KB, text/plain)
2024-02-26 09:57 UTC, Zhenlei Huang
no flags Details
dmesg of current 15 (6.35 KB, text/plain)
2024-02-26 09:57 UTC, Zhenlei Huang
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Zhenlei Huang freebsd_committer freebsd_triage 2024-02-26 09:56:12 UTC
With recent change 6744fd8e7503 (x86/cpu: improve hypervisor detection), FreeBSD guest on QEMU with Hyper-V enlightenment on detects the hypervisor(KVM) correctly, but the emulated Hyper-V devices are no longer be detected during boot.

Highlight the diff:

< Hyper-V Version: 10.0.14393 [SP0]
<   Features=0x27e<TMREFCNT,SYNIC,SYNTM,APIC,HYPERCALL,VPINDEX,REFTSC>
<   PM Features=0x0 [C0]
<   Features3=0x8<PCPUDPE>
< Timecounter "Hyper-V" frequency 10000000 Hz quality 2000
< Timecounter "Hyper-V-TSC" frequency 10000000 Hz quality 3000
< vmbus0: <Hyper-V Vmbus> on pcib0
Comment 1 Zhenlei Huang freebsd_committer freebsd_triage 2024-02-26 09:57:20 UTC
Created attachment 248755 [details]
dmesg of 14.0
Comment 2 Zhenlei Huang freebsd_committer freebsd_triage 2024-02-26 09:57:54 UTC
Created attachment 248756 [details]
dmesg of current 15
Comment 3 Roger Pau Monné freebsd_committer freebsd_triage 2024-02-26 10:30:30 UTC
Hm, I guess I'm a bit confused.  If the hypervisor is KVM, shouldn't it be detected as KVM rather than HyperV?  If it's QEMU running on KVM and providing HyperV enlightenments, that's quite likely the same that's done on Xen, and if FreeBSD supports Xen enlightenments is much better to use those rather than the HyperV ones (as those are just a subset of the HyperV features aimed at making Windows faster when running on Xen).

What's the benefit of using the HyperV enlightenments when running on QEMU/KVM?
Comment 4 Zhenlei Huang freebsd_committer freebsd_triage 2024-02-27 15:21:05 UTC
(In reply to Roger Pau Monné from comment #3)
> Hm, I guess I'm a bit confused.
While debugging PR 274810, I see the behavior, that FreeBSD guests report HyperV rather than KVM when Hyper-V enlightenment is enabled.

> If the hypervisor is KVM, shouldn't it be detected as KVM rather than HyperV?
I think KVM should be reported. But it seems everything goes right even HyperV is reported.

> What's the benefit of using the HyperV enlightenments when running on QEMU/KVM?
I think Microsoft people are more familiar with that. When debugging PR 274810 I read briefly the design of Hyper-V virtual devices. I think the design is pretty clear and performant.

I see lots of comparing of vm_guest in hyperv device drivers. I guess they should be removed to correctly probe Hyper-V devices, to correctly support Hyper-V enlightenment feature.
Comment 5 Zhenlei Huang freebsd_committer freebsd_triage 2024-02-27 15:25:08 UTC
(In reply to Zhenlei Huang from comment #4)
> I see lots of comparing of vm_guest in hyperv device drivers. I guess they should be removed to correctly probe Hyper-V devices, to correctly support Hyper-V enlightenment feature.

In case KVM should be reported when Hyper-V enlightenmen is enabled.

# grep -r 'VM_GUEST_HV' sys/dev/hyperv
sys/dev/hyperv/storvsc/hv_storvsc_drv_freebsd.c:	if (vm_guest == VM_GUEST_HV) {
sys/dev/hyperv/vmbus/vmbus.c:	if (device_get_unit(parent) != 0 || vm_guest != VM_GUEST_HV ||
sys/dev/hyperv/vmbus/vmbus.c:	if (device_get_unit(dev) != 0 || vm_guest != VM_GUEST_HV ||
sys/dev/hyperv/vmbus/vmbus.c:	if (vm_guest != VM_GUEST_HV || sc == NULL)
sys/dev/hyperv/vmbus/hyperv.c:		if (vm_guest == VM_GUEST_HV)
sys/dev/hyperv/vmbus/hyperv.c:	if (vm_guest != VM_GUEST_HV)
sys/dev/hyperv/vmbus/vmbus_res.c:	if (device_get_unit(dev) != 0 || vm_guest != VM_GUEST_HV ||
sys/dev/hyperv/vmbus/x86/hyperv_x86.c:	if (vm_guest != VM_GUEST_HV)
sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c:	vm_guest = VM_GUEST_HV;
sys/dev/hyperv/hvsock/hv_sock.c:	if (vm_guest != VM_GUEST_HV)
sys/dev/hyperv/netvsc/if_hn.c:	if (vm_guest != VM_GUEST_HV)
Comment 6 Miroslav Lachman 2024-02-27 16:52:27 UTC
According to this document https://www.qemu.org/docs/master/system/i386/hyperv.html
some thing looks like Hyper-V, but for others you can tell it's a KVM hypervisor.

"When any set of the Hyper-V enlightenments is enabled, QEMU changes hypervisor identification (CPUID 0x40000000..0x4000000A) to Hyper-V. KVM identification and features are kept in leaves 0x40000100..0x40000101."
Comment 7 Roger Pau Monné freebsd_committer freebsd_triage 2024-02-27 17:16:44 UTC
(In reply to Miroslav Lachman from comment #6)
That's the same on Xen, the first CPUID signature is the HyperV one, but the following one is the Xen one, and we want to use the Xen one, as the Viridian set of features emulated by Xen is not very complete and just tested against Windows guests.  This was already discussed in the differential revision of the change for Xen: https://reviews.freebsd.org/D43508

So the question boils down to whether FreeBSD wants to treat KVM with HyperV extensions as HyperV or as KVM.  I think the latter, as KVM is the real hypervisor, disguising as HyperV.