Created attachment 227380 [details]
bhyve.core zipped file

bhyve.core zipped file

so,these are the informations that I've collected doing the debug of the core file :

root@marietto:/usr/home/marietto/Desktop/Files/bhyve # lldb -c bhyve.core

(lldb) target create --core "bhyve.core"
Core file '/usr/home/marietto/Desktop/Files/bhyve/bhyve.core' (x86_64) was loaded.

(lldb) thread backtrace all
* thread #1, name = 'bhyve', stop reason = signal SIGABRT
  * frame #0: 0x00000008015f62ea
    frame #1: 0x000000080156b064
  thread #2, name = 'blk-1:0-0', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #3, name = 'blk-1:0-1', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #4, name = 'blk-1:0-2', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #5, name = 'blk-1:0-3', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #6, name = 'blk-1:0-4', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #7, name = 'blk-1:0-5', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #8, name = 'blk-1:0-6', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #9, name = 'blk-1:0-7', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #10, name = 'vtnet-6:0 tx', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
  thread #11, name = 'hda-audio-output', stop reason = signal SIGABRT
    frame #0: 0x000000080149cb3c
    frame #1: 0x00000008014ac660
(lldb)

I tried to boot Ubuntu with these arguments :

Code:

bhyve -S -c 4 -m 8G -w -H \
        -s 0,hostbridge \
        -s 1,virtio-blk,/mnt/da1p1/vms/os/ubuntu-budgie-gpu/ubuntu-2104-gpu.img \
        -s 6,virtio-net,tap0 \
        -s 2,passthru,1/0/0 \
        -s 2:1,passthru,1/0/1 \
        -s 2:2,passthru,1/0/2 \
        -s 2:3,passthru,1/0/3 \
        -s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900 \
        -s 30,xhci,tablet \
        -s 31,lpc -l com1,stdio \
        -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
        vm0

(I have removed : -s 20,hda,play=/dev/dsp8,rec=/dev/dsp8)

the previous error is gone,but ubuntu is not able to boot. It is freezed on a black screen,before to start loading the img file. I hear the noise that it tells that is trying to do something,but it can't for some reason.

lldb -c bhyve.core -- bhyve

(lldb) thread backtrace all

Program aborted due to an unhandled Error:
Error value was Success. (Note: Success values must still be checked prior to being destroyed).
PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace.
Stack dump:
0.    Program arguments: lldb -c bhyve.core -- bhyve
1.    HandleCommand(command = "thread backtrace all")
#0 0x0000000003ae7aee PrintStackTrace /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:564:13
#1 0x0000000003ae5fa5 RunSignalHandlers /usr/src/contrib/llvm-project/llvm/lib/Support/Signals.cpp:69:18
#2 0x0000000003ae8060 SignalHandler /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
#3 0x0000000804c35e00 handle_signal /usr/src/lib/libthr/thread/thr_sig.c:0:3
Segnale di annullamento(creato file core)

Here we go. I've installed FreeBSD instead of Linux with bhyve and I tried to pass thru the rtx 2080 ti using this command :

Code:

bhyve   -S -c 4 -m 8G -w -H \
        -s 0,hostbridge \
        -s 1,passthru,1/0/0 \
        -s 1:1,passthru,1/0/2 \
        -s 1:2,passthru,1/0/3 \
        -s 2,virtio-blk,/mnt/da1p1/vms/os/freebsd-13R/freebsd-13R.img \
        -s 3,virtio-net,tap0 \
        -s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900 \
        -s 30,xhci,tablet \
        -s 31,lpc -l com1,stdio \
        -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
        vm0


as u can see :

1) I didn't pass thru the device 1/0/1 because it is the audio device integrated with the rtx 2080 ti
2) I have removed the emulated audio device
3) I have installed freebsd 13R instead of linux
4) without the passthru lines it works

but it didn't work. it freezes on the black screen before to itizialize the disk. On /boot/loader.conf I have added this :

Code:

pptdevs="1/0/0 1/0/2 1/0/3"


Code:

pciconf -v -l

ppt0@pci0:1:0:0:    class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1e04 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 [GeForce RTX 2080 Ti]'
    class      = display
    subclass   = VGA

hdac0@pci0:1:0:1:    class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f7 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 High Definition Audio Controller'
    class      = multimedia
    subclass   = HDA

ppt1@pci0:1:0:2:    class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad6 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 USB 3.1 Host Controller'
    class      = serial bus
    subclass   = USB

ppt2@pci0:1:0:3:    class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad7 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 USB Type-C UCSI Controller'
    class      = serial bus

Comment 6 Peter Grehan 2021-08-25 11:12:03 UTC

Is this using the proprietary Nvidia driver in the guest ?

How can I install the nvidia proprietary driver in the guest if the guest can't recognize the presence of the nvidia graphic card ?

this is the error message I get if I try to install the nvidia proprietary driver on the ubuntu guest without that I haven't been able to pass trhu the card before :

https://ibb.co/XtKqZTk

as I've said,the virtual machine freezes on a black screen when I try to pass it. I've found that it happens also to this guy :

https://www.youtube.com/watch?v=Y1FuxpahDrM&t=10s&ab_channel=MarkoTasic

he tried for 1.5 hours to pass his 2070,but without success.

this :

/boot/loader.conf

pptdevs="1/0/0 1/0/1 1/0/2 1/0/3"

bhyve -S -c 4 -m 11G -w -H \
-s 0,hostbridge \
-s 1,virtio-blk,/mnt/da1p1/vms/os/ubuntu-budgie-gpu/ubuntu-2104-gpu.img \
-s 2:0,passthru,1/0/0 \
-s 2:1,passthru,1/0/1 \
-s 2:2,passthru,1/0/2 \
-s 2:3,passthru,1/0/3 \
-s 6,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900 \
-s 30,xhci,tablet \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
vm0

produced this :

root@marietto:/home/marietto/Desktop/Files/bhyve # ./os-uefi-hirsute.sh

VM:vm0 is not created.
fbuf frame buffer base: 0xb04600000 [sz 16777216]
vm exit[0]
	reason		VMX
	rip		0x000000000000fff0
	inst_length	3
	status		0
	exit_reason	48 (EPT violation)
	qualification	0x0000000000000784
	inst_type		0
	inst_error		0
Segnale di annullamento

[drm] Unable to create a private tmpfs mount, hugepage support will be disabled(-19).
Failed to add WC MTRR for [0x40000000-0x5fffffff]: -22; performance may suffer
[drm] Got stolen memory base rx0, size 0x0
[drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[drm] Driver supports precise vblank timestamp query.
[drm] Connector eDP-1: get mode from tunables:
[drm]   - kern.vt.fb.modes.eDP-1
[drm]   - kern.vt.fb.default_mode
drmn1: successfully loaded firmware image 'i915/kbl_dmc_ver1_04.bin'
[drm] Finished loading DMC firmware i915/kbl_dmc_ver1_04.bin (v1.4)
[drm] failed to retrieve link info, disabling eDP
[drm] Connector HDMI-A-1: get mode from tunables:
[drm]   - kern.vt.fb.modes.HDMI-A-1
[drm]   - kern.vt.fb.default_mode
drmn1: Failed to program MOCS registers; expect performance issues.
sysctl_warn_reuse: can't re-use a leaf (hw.dri.debug)!
[drm] Initialized i915 1.6.0 20190822 for drmn1 on minor 0
WARNING: Device "fb" is Giant locked and may be deleted before FreeBSD 14.0.
VT: Replacing driver "efifb" with new "fb".
start FB_INFO:
type=11 height=1080 width=1920 depth=32
cmsize=16 size=8294400
pbase=0x40040000 vbase=0xfffff80040040000
name=drmn1 flags=0x0 stride=7680 bpp=32
cmap[0]=0 cmap[1]=7f0000 cmap[2]=7f00 cmap[3]=c4a000
end FB_INFO
drmn1: fb0: i915drmfb frame buffer device
WARNING: /mnt/da1p1 was not properly dismounted
tap0: link state changed to UP
pid 1841 (bhyve), jid 0, uid 0: exited on signal 6
tap0: link state changed to DOWN

I have installed the nvidia driver and then I did :

Code:

bhyve -S -c 4 -m 11G -w -H \
-s 0,hostbridge \
-s 1,virtio-blk,/mnt/da1p1/vms/os/ubuntu-budgie-gpu/ubuntu-2104-gpu.img \
-s 2:0,passthru,1/0/0 \
-s 2:1,passthru,1/0/1 \
-s 2:2,passthru,1/0/2 \
-s 2:3,passthru,1/0/3 \
-s 6,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900 \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
vm0


mount: /dev/da1p1: Device busy
VM:vm0 is not created.
fbuf frame buffer base: 0xb04600000 [sz 16777216]
Assertion failed: (error == 0), function modify_bar_registration, file /usr/src/usr.sbin/bhyve/pci_emul.c, line 501.
Segnale di annullamento

root@marietto:/home/marietto/Desktop/Files/bhyve # ./os-uefi-hirsute.sh
mount: /dev/da1p1: Device busy
vm_open: Invalid argument

root@marietto:/home/marietto/Desktop/Files/bhyve # ./os-uefi-hirsute.sh
mount: /dev/da1p1: Device busy
VM:vm0 is not created.
fbuf frame buffer base: 0xb04600000 [sz 16777216]

BLACK SCREEN and nothing happens.

root@marietto:/home/marietto # pciconf -vlb

hostb0@pci0:0:0:0:	class=0x060000 rev=0x0d hdr=0x00 vendor=0x8086 device=0x3e30 subvendor=0x1458 subdevice=0x5000
    vendor     = 'Intel Corporation'
    device     = '8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S]'
    class      = bridge
    subclass   = HOST-PCI
pcib1@pci0:0:1:0:	class=0x060400 rev=0x0d hdr=0x01 vendor=0x8086 device=0x1901 subvendor=0x1458 subdevice=0x5000
    vendor     = 'Intel Corporation'
    device     = '6th-10th Gen Core Processor PCIe Controller (x16)'
    class      = bridge
    subclass   = PCI-PCI
pcib2@pci0:0:1:1:	class=0x060400 rev=0x0d hdr=0x01 vendor=0x8086 device=0x1905 subvendor=0x1458 subdevice=0x5000
    vendor     = 'Intel Corporation'
    device     = 'Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8)'
    class      = bridge
    subclass   = PCI-PCI
vgapci1@pci0:0:2:0:	class=0x030000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x3e98 subvendor=0x1458 subdevice=0xd000
    vendor     = 'Intel Corporation'
    device     = 'CoffeeLake-S GT2 [UHD Graphics 630]'
    class      = display
    subclass   = VGA
    bar   [10] = type Memory, range 64, base rx98000000, size 16777216, enabled
    bar   [18] = type Prefetchable Memory, range 64, base rx40000000, size 536870912, enabled
    bar   [20] = type I/O Port, range 32, base rx5000, size 64, enabled
none0@pci0:0:18:0:	class=0x118000 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa379 subvendor=0x1458 subdevice=0x8888
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH Thermal Controller'
    class      = dasp
    bar   [10] = type Memory, range 64, base rx9933d000, size 4096, enabled
xhci0@pci0:0:20:0:	class=0x0c0330 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa36d subvendor=0x1458 subdevice=0x5007
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH USB 3.1 xHCI Host Controller'
    class      = serial bus
    subclass   = USB
    bar   [10] = type Memory, range 64, base rx99320000, size 65536, enabled
none1@pci0:0:20:2:	class=0x050000 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa36f subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH Shared SRAM'
    class      = memory
    subclass   = RAM
    bar   [10] = type Memory, range 64, base rx99336000, size 8192, enabled
    bar   [18] = type Memory, range 64, base rx9933c000, size 4096, enabled
none2@pci0:0:22:0:	class=0x078000 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa360 subvendor=0x1458 subdevice=0x1c3a
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH HECI Controller'
    class      = simple comms
    bar   [10] = type Memory, range 64, base rx9933b000, size 4096, enabled
ahci0@pci0:0:23:0:	class=0x010601 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa352 subvendor=0x1458 subdevice=0xb005
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH SATA AHCI Controller'
    class      = mass storage
    subclass   = SATA
    bar   [10] = type Memory, range 32, base rx99334000, size 8192, enabled
    bar   [14] = type Memory, range 32, base rx9933a000, size 256, enabled
    bar   [18] = type I/O Port, range 32, base rx5090, size 8, enabled
    bar   [1c] = type I/O Port, range 32, base rx5080, size 4, enabled
    bar   [20] = type I/O Port, range 32, base rx5060, size 32, enabled
    bar   [24] = type Memory, range 32, base rx99339000, size 2048, enabled
pcib3@pci0:0:27:0:	class=0x060400 rev=0xf0 hdr=0x01 vendor=0x8086 device=0xa340 subvendor=0x1458 subdevice=0x5001
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:28:0:	class=0x060400 rev=0xf0 hdr=0x01 vendor=0x8086 device=0xa338 subvendor=0x1458 subdevice=0x5001
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib5@pci0:0:28:5:	class=0x060400 rev=0xf0 hdr=0x01 vendor=0x8086 device=0xa33d subvendor=0x1458 subdevice=0x5001
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
pcib6@pci0:0:29:0:	class=0x060400 rev=0xf0 hdr=0x01 vendor=0x8086 device=0xa330 subvendor=0x1458 subdevice=0x5001
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
isab0@pci0:0:31:0:	class=0x060100 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa305 subvendor=0x1458 subdevice=0x5001
    vendor     = 'Intel Corporation'
    device     = 'Z390 Chipset LPC/eSPI Controller'
    class      = bridge
    subclass   = PCI-ISA
hdac1@pci0:0:31:3:	class=0x040300 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa348 subvendor=0x1458 subdevice=0xa0c3
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH cAVS'
    class      = multimedia
    subclass   = HDA
    bar   [10] = type Memory, range 64, base rx99330000, size 16384, enabled
    bar   [20] = type Memory, range 64, base rx99000000, size 1048576, enabled
ichsmb0@pci0:0:31:4:	class=0x0c0500 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa323 subvendor=0x1458 subdevice=0x5001
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH SMBus Controller'
    class      = serial bus
    subclass   = SMBus
    bar   [10] = type Memory, range 64, base rx99338000, size 256, enabled
    bar   [20] = type I/O Port, range 32, base rxefa0, size 32, enabled
none3@pci0:0:31:5:	class=0x0c8000 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa324 subvendor=0x8086 subdevice=0x7270
    vendor     = 'Intel Corporation'
    device     = 'Cannon Lake PCH SPI Controller'
    class      = serial bus
    bar   [10] = type Memory, range 32, base rxfe010000, size 4096, enabled
em0@pci0:0:31:6:	class=0x020000 rev=0x10 hdr=0x00 vendor=0x8086 device=0x15bc subvendor=0x1458 subdevice=0xe000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection (7) I219-V'
    class      = network
    subclass   = ethernet
    bar   [10] = type Memory, range 32, base rx99300000, size 131072, enabled
ppt0@pci0:1:0:0:	class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1e04 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 [GeForce RTX 2080 Ti]'
    class      = display
    subclass   = VGA
    bar   [10] = type Memory, range 32, base rx96000000, size 16777216, enabled
    bar   [14] = type Prefetchable Memory, range 64, base rx60000000, size 268435456, enabled
    bar   [1c] = type Prefetchable Memory, range 64, base rx94000000, size 33554432, enabled
    bar   [24] = type I/O Port, range 32, base rx4000, size 128, enabled
ppt1@pci0:1:0:1:	class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f7 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 High Definition Audio Controller'
    class      = multimedia
    subclass   = HDA
    bar   [10] = type Memory, range 32, base rx97080000, size 16384, disabled
ppt2@pci0:1:0:2:	class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad6 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 USB 3.1 Host Controller'
    class      = serial bus
    subclass   = USB
    bar   [10] = type Prefetchable Memory, range 64, base rx70000000, size 262144, disabled
    bar   [1c] = type Prefetchable Memory, range 64, base rx70040000, size 65536, disabled
ppt3@pci0:1:0:3:	class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad7 subvendor=0x19da subdevice=0x2503
    vendor     = 'NVIDIA Corporation'
    device     = 'TU102 USB Type-C UCSI Controller'
    class      = serial bus
    bar   [10] = type Memory, range 32, base rx97084000, size 4096, disabled
vgapci0@pci0:2:0:0:	class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1c02 subvendor=0x19da subdevice=0x2438
    vendor     = 'NVIDIA Corporation'
    device     = 'GP106 [GeForce GTX 1060 3GB]'
    class      = display
    subclass   = VGA
    bar   [10] = type Memory, range 32, base rx92000000, size 16777216, enabled
    bar   [14] = type Prefetchable Memory, range 64, base rx80000000, size 268435456, enabled
    bar   [1c] = type Prefetchable Memory, range 64, base rx90000000, size 33554432, enabled
    bar   [24] = type I/O Port, range 32, base rx3000, size 128, enabled
hdac0@pci0:2:0:1:	class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f1 subvendor=0x19da subdevice=0x2438
    vendor     = 'NVIDIA Corporation'
    device     = 'GP106 High Definition Audio Controller'
    class      = multimedia
    subclass   = HDA
    bar   [10] = type Memory, range 32, base rx93080000, size 16384, enabled
nvme0@pci0:3:0:0:	class=0x010802 rev=0x03 hdr=0x00 vendor=0xc0a9 device=0x5403 subvendor=0xc0a9 subdevice=0x2100
    vendor     = 'Micron/Crucial Technology'
    class      = mass storage
    subclass   = NVM
    bar   [10] = type Memory, range 64, base rx99200000, size 16384, enabled
xhci1@pci0:5:0:0:	class=0x0c0330 rev=0x03 hdr=0x00 vendor=0x1912 device=0x0014 subvendor=0x1912 subdevice=0x0015
    vendor     = 'Renesas Technology Corp.'
    device     = 'uPD720201 USB 3.0 Host Controller'
    class      = serial bus
    subclass   = USB
    bar   [10] = type Memory, range 64, base rx99100000, size 8192, enabled