Hello.

When I try to boot a linux / windows / freebsd guest os installed physically on a sata or usb disk (it makes no difference) with bhyve on FreeBSD 13R. So,I've created some cases to show you in a incontrovertible way that the passthrough of any device (I tried with my nvidia geforce RTX 2080 ti and with my USB renesas controller),interferes with the booting of any OS installed physically on the disks. it happens if I use virtio-blk and also ahci-hd. If I use virtio-blk I have an additional problem,in addition to the fact that the passthrough does not work,without passthrough,at some point the VM can't load the root partition,while using ahci-hd it can). In addition to this,when I enable and disable the framebuffer,I get two additional errors,explained on the title.

case 1)

bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H \
-s 0,hostbridge \
-s 1,virtio-blk,/dev/da1 \
-s 2:0,passthru,2/0/0,rom=TU102.rom \
-s 31,lpc \
-l com1,stdio \                           
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm1

= NO (da1 with freebsd 14 installed does not boot)

case 2)

bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H \
-s 0,hostbridge \
-s 1,virtio-blk,/dev/da1 \
-s 31,lpc \
-l com1,stdio \                           
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm1

= yes.

case 3)

bhyve -s 0,hostbridge \
-s 1,virtio-blk,/dev/da1 \
-s 31,lpc \
-l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm1

= yes

case 4)

bhyve -S -s 0,hostbridge \
-s 1,virtio-blk,/dev/da1 \
-s 31,lpc \
-l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm1

= yes

case 5)

bhyve -S -s 0,hostbridge \
-s 1,virtio-blk,/dev/da1 \
-s 2:0,passthru,2/0/0,rom=TU102.rom \
-s 31,lpc \
-l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm1

= no

[reply] [−] Comment 4 mario felicioni 2022-02-04 18:56:30 UTC

I've passed through the graphic card and the renesas controller (because they are on the same IOMMU group) and I've got a precise bug :

This command :

bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/dev/da1,sectorsize=512 \
-s 3:0,passthru,2/0/0 \
-s 3:1,passthru,2/0/1 \
-s 3:2,passthru,2/0/2 \
-s 3:3,passthru,2/0/3 \
-s 4:0,passthru,1/0/0 \
-s 8,virtio-net,tap1 \
-s 9,virtio-9p,sharename=/ \
-s 30,xhci,tablet \
-s 31,lpc \                                                      
-s 29,fbuf,tcp=0.0.0.0:5901,w=1440,h=900 \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
-l com1,stdio \
vm1

produces this error :

root@marietto:/usr/home/marietto/bhyve # ./freebsd14.sh

VM:vm0 is not created.
VM:vm1 is not created.
VM:vm0 is not created.
VM:vm1 is not created.
fbuf frame buffer base: 0x945e00000 [sz 16777216]
Assertion failed: (baridx == 0), function pci_fbuf_write, file /usr/src/usr.sbin/bhyve/pci_fbuf.c, line 134

Abort : core dumped.


this is what happens without the framebuffer defined in bhyve : 

Assertion failed: (pi->pi_bar[baridx].type == PCIBAR_IO), function passthru_read, file /usr/src/usr.sbin/bhyve/pci_passthru.c, line 942.

I think that here there are some concatenated bugs.