Bug 250865

Summary: Usage of recent windows virtio-blk driver crashes the bhyve VM
Product: Base System Reporter: Michael Reifenberger <mr>
Component: bhyveAssignee: freebsd-virtualization (Nobody) <virtualization>
Status: New ---    
Severity: Affects Only Me CC: igorz, jason, olevole
Priority: ---    
Version: 12.2-STABLE   
Hardware: amd64   
OS: Any   

Description Michael Reifenberger freebsd_committer freebsd_triage 2020-11-04 16:05:08 UTC
Hi,
if using the virtio-blk drivers version 1.189-1 the VM exits with:
...
bhyve exited with status 134
...

If using the virtio-blk drivers version 0.1.141 are used nothing crashes
but the disk is not found...

My config:

loader="uefi"
cpu=2
cpu_sockets=1
cpu_cores=2
cpu_threads=1
wired_memory="yes"
memory=8G
virt_random="yes"
network0_type="virtio-net"
network0_switch="private"
disk0_type="nvme"
disk0_dev="zvol"
disk0_name="disk0"
disk1_type="nvme"
disk1_dev="zvol"
disk1_name="disk1"
disk2_type="ahci-cd"
disk2_name="disk2.img"
disk3_type="virtio-blk"
disk3_dev="zvol"
disk3_name="disk3"
uuid="a9656f03-7f7a-11e7-983e-98e7f4868540"
graphics="yes"
graphics_port="5900"
graphics_listen="0.0.0.0"
graphics_res="1600x900"
graphics_wait="no"
xhci_mouse="yes"

Any ideas or how to debug this further?
Comment 1 Yuri Pankov 2020-11-04 16:34:42 UTC
The "stable" version (0.1.185) from fedora works for me, you can find it linked on the following page:

https://docs.fedoraproject.org/en-US/quick-docs/creating-windows-virtual-machines-using-virtio-drivers/index.html

Look for "Stable virtio-win iso".
Comment 2 Michael Reifenberger freebsd_committer freebsd_triage 2020-11-04 17:14:23 UTC
Hi,
many thanks for the hint.

Yes, this version is working fine (for some time).
Unfortunately I get a bluescreen in Windows and:
...
pid 39852 (bhyve), jid 0, uid 0: exited on signal 6
...
when running a CrystalDiskMark benchmark on this disk :-(

So at least its not finally stable somewhere.

Could you try to stress the virtio-blk disks a bit.

BTW if its relevant: 
I put the virtio-blk disk on a ZFS vol with 4k blocksize.
Comment 3 Jason Tubnor 2020-11-05 01:08:10 UTC
There is a known issues with 1.189.  https://bugzilla.redhat.com/show_bug.cgi?id=1890810

If you are either presenting the zvol as 4k or the underlying setting of it is 4k, then you'll run into corruption pretty quickly.

I have tested and confirmed this. 1.187 does not exhibit the same issue.  Testing was conducted using 12.2-RELEASE and Windows 10.

Our Windows Server 2019 fleet are using 1.185 without issue.
Comment 4 Michael Reifenberger freebsd_committer freebsd_triage 2020-11-05 09:35:21 UTC
(In reply to Jason Tubnor from comment #3)
What blocksize would you recommend for NTFS?
The ZFS pool is on a 4k SSD.
Comment 5 Michael Reifenberger freebsd_committer freebsd_triage 2020-11-05 09:56:34 UTC
I tested now with volblocksizes of 32k and 8k (the default).
This settings last longer but both produce bluescreens during random write.
Driver version was 1.185.
Comment 6 Igor Zabelin 2021-07-29 10:54:20 UTC
I have installed Windows 10 on 13.0-STABLE. 
Zvol sparse volume with default 8k volblocksize.
The latest working driver 0.1.187
With the newer version installation is interrupted.