https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256722 Having same issues as this thread, I normally boot from refind usb stick on r720 to nvme EFI. I have tried freebsd-BETA5, freebsd 15 stable etc nothing seems to work for loader.conf options or kernel options. I can reboot with 14.3 release or 14 stable kernel and no issues. I can even install freebsd 15 kernel and boot from 14.3 kernel.old fine with all same files in /boot as freebsd 15. There must be some regression with kernel in 15 up with EFI. For me it hits that framebuffer at beginning of boot, then machine just reboots itself.
Unfortunately that does not provide any information to help debug. The EFI framebuffer information is the last thing printed by the loader before the kernel starts booting. The only information we have is that the system restarted before the kernel produced any output.
So with the same issue hitting 14 and 15 ... what exactly do you need to debug this issue?
Yeah it is elusive bug for sure, what I can confirm is it must have been a PR into EFI kernel code itself, as booting FreeBSD 14 kernel with same /boot environment of FreeBSD 15 fixes it. Then again Dell r720 firmware is so old, won't get fixes for any new UEFI code, so problematic for people still running it in home lab. The only fix I could do for this was to create a FreeBSD 15-BETA5 memstick, set its bios to boot in BIOS mode, boot the USB stick and modify /boot/loader.conf with vfs.root.mountfrom="zfs:zroot/ROOT/default", to boot from nvme drives. Essentially just getting rid of refind completely and coping my own /boot environment to USB stick. After more testing on FreeBSD 15-BETA5, I found at least 2 more issues. Modifying /etc/pkg/FreeBSD.conf to pull latest ports, I found programs like dhcpcd to hang, killall dhcpcd does nothing, a firm "kill -9 <of each pid>" was needed. Last one is quite a serious one with bhyve, same environment for vm-bhyve I had on FreeBSD 14, I can confirm passthrough is broken on guests. In /boot/loader.conf: pptdevs="9/0/0 10/0/0 66/0/0" works fine, I can confirm the 2 USB ports and nvidia GPU are available for passthrough, however when passing it through on guest it is not working for any of them. 2 USB(zigbee and zwave) sticks are missing on home assistant, and ubuntu VM with nvidia GPU, mind you I had to use FreeBSD 14 stable for nvidia patches. Checking USB stick passthrough on home assistant: # dmesg | grep -i xhci [ 0.745789] xhci_hcd 0000:00:06.0: xHCI Host Controller [ 0.748133] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 1 [ 0.756495] xhci_hcd 0000:00:06.0: hcc params 0x014051c7 hci version 0x100 quirks 0x0000000100000090 [ 0.763687] xhci_hcd 0000:00:06.0: xHCI Host Controller [ 0.766148] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 2 [ 0.769165] xhci_hcd 0000:00:06.0: Host supports USB 3.0 SuperSpeed [ 1.500133] xhci_hcd 0000:00:06.0: Host took too long to start, waited 32000 microseconds. [ 1.656982] xhci_hcd 0000:00:06.0: startup error -19 [ 1.659106] xhci_hcd 0000:00:06.0: USB bus 2 deregistered [ 1.662286] xhci_hcd 0000:00:06.0: remove, state 1 [ 1.670295] xhci_hcd 0000:00:06.0: USB bus 1 deregistered [ 1.671337] xhci_hcd 0000:00:07.0: xHCI Host Controller [ 1.672184] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 1 [ 1.678468] xhci_hcd 0000:00:07.0: hcc params 0x014051c7 hci version 0x100 quirks 0x0000000100000090 [ 1.680689] xhci_hcd 0000:00:07.0: xHCI Host Controller [ 1.681523] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 2 [ 1.682613] xhci_hcd 0000:00:07.0: Host supports USB 3.0 SuperSpeed [ 1.685692] xhci_hcd 0000:00:07.0: Host took too long to start, waited 32000 microseconds. [ 2.412808] xhci_hcd 0000:00:07.0: startup error -19 [ 2.414863] xhci_hcd 0000:00:07.0: USB bus 2 deregistered [ 2.417269] xhci_hcd 0000:00:07.0: remove, state 1 [ 2.425292] xhci_hcd 0000:00:07.0: USB bus 1 deregistered [ 2.428056] xhci_hcd 0000:00:09.0: xHCI Host Controller [ 2.430338] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 1 [ 2.435170] xhci_hcd 0000:00:09.0: hcc params 0x03401281 hci version 0x100 quirks 0x000000000000b930 [ 2.440101] xhci_hcd 0000:00:09.0: xHCI Host Controller [ 2.442350] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 2 [ 2.445446] xhci_hcd 0000:00:09.0: Host supports USB 3.0 SuperSpeed [ 2.454254] usb usb1: Product: xHCI Host Controller [ 2.456179] usb usb1: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.474092] usb usb2: Product: xHCI Host Controller [ 2.476031] usb usb2: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.704998] usb 2-1: new SuperSpeed USB device number 2 using xhci_hcd # dmesg | grep -i usb [ 0.310422] ACPI: bus type USB registered [ 0.311371] usbcore: registered new interface driver usbfs [ 0.312082] usbcore: registered new interface driver hub [ 0.312358] usbcore: registered new device driver usb [ 0.748133] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 1 [ 0.766148] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 2 [ 0.769165] xhci_hcd 0000:00:06.0: Host supports USB 3.0 SuperSpeed [ 1.659106] xhci_hcd 0000:00:06.0: USB bus 2 deregistered [ 1.670295] xhci_hcd 0000:00:06.0: USB bus 1 deregistered [ 1.672184] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 1 [ 1.681523] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 2 [ 1.682613] xhci_hcd 0000:00:07.0: Host supports USB 3.0 SuperSpeed [ 2.414863] xhci_hcd 0000:00:07.0: USB bus 2 deregistered [ 2.425292] xhci_hcd 0000:00:07.0: USB bus 1 deregistered [ 2.430338] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 1 [ 2.442350] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 2 [ 2.445446] xhci_hcd 0000:00:09.0: Host supports USB 3.0 SuperSpeed [ 2.448159] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.12 [ 2.451450] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.454254] usb usb1: Product: xHCI Host Controller [ 2.456179] usb usb1: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.458520] usb usb1: SerialNumber: 0000:00:09.0 [ 2.460786] hub 1-0:1.0: USB hub found [ 2.464830] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM. [ 2.468061] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.12 [ 2.471267] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.474092] usb usb2: Product: xHCI Host Controller [ 2.476031] usb usb2: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.478394] usb usb2: SerialNumber: 0000:00:09.0 [ 2.480588] hub 2-0:1.0: USB hub found [ 2.484681] usbcore: registered new interface driver usblp [ 2.487332] usbcore: registered new interface driver usb-storage [ 2.489765] usbcore: registered new interface driver usbserial_generic [ 2.492948] usbserial: USB Serial support registered for generic [ 2.557619] usbcore: registered new interface driver usbhid [ 2.559706] usbhid: USB HID core driver [ 2.704998] usb 2-1: new SuperSpeed USB device number 2 using xhci_hcd [ 2.721813] usb 2-1: New USB device found, idVendor=fb5d, idProduct=0001, bcdDevice= 0.00 [ 2.725380] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 2.728498] usb 2-1: Product: HID Tablet [ 2.730828] usb 2-1: Manufacturer: BHYVE [ 2.733740] usb 2-1: SerialNumber: 01 [ 2.739082] input: BHYVE HID Tablet as /devices/pci0000:00/0000:00:09.0/usb2/2-1/2-1:1.0/0003:FB5D:0001.0001/input/input4 [ 2.745058] hid-generic 0003:FB5D:0001.0001: input,hidraw0: USB HID v10.01 Mouse [BHYVE HID Tablet] on usb-0000:00: So its like ppt driver not being initialized correctly for passing through 2 USB sticks that works on FreeBSD 14. I have confirmed same problem with FreeBSD 15 stable. My config on FreeBSD 15 host is: router:/usr/local/etc # grep ppt /boot/loader.conf pptdevs="9/0/0 10/0/0 66/0/0" router:/usr/local/etc # 9 and 10 are USB sticks for home assistant, 66 is nvidia GPU. router:/usr/local/etc # grep passthru /vm/ha/ha.conf passthru0="9/0/0" passthru1="10/0/0" router:/usr/local/etc # router:/usr/local/etc # tail -5 /vm/ubuntu/ubuntu.conf # Passthru nvidia card passthru0="66/0/0" bhyve_options="-A -H -P" router:/usr/local/etc # router:/usr/local/etc # tail -10 /vm/ha/vm-bhyve.log Nov 10 14:14:52: [primary disk dev: file] Nov 10 14:14:52: initialising network device tap1 Nov 10 14:14:52: setting mtu of tap1 to 9000 Nov 10 14:14:52: adding tap1 -> br0 (services addm) Nov 10 14:14:52: bring up tap1 -> br0 (services addm) Nov 10 14:14:52: booting Nov 10 14:14:52: [bhyve options: -c 10,sockets=1,cores=10 -m 16G -AHPw -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -U a5811c38-567e-11ed-bdeb-e2335bf65f00 -u -S] Nov 10 14:14:52: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,nvme,/vm/ha/disk0.img -s 5:0,virtio-net,tap1,mac=58:9c:11:0c:5a:a9 -s 6:0,passthru,9/0/0 -s 7:0,passthru,10/0/0 -s 8:0,fbuf,tcp=192.168.0.1:5915 -s 9:0,xhci,tablet] Nov 10 14:14:52: [bhyve console: -l com1,stdio] Nov 10 14:14:52: starting bhyve (run 1) router:/usr/local/etc # router:/usr/local/etc # pciconf -lv|grep ppt ppt0@pci0:9:0:0: class=0x0c0330 rev=0x02 hdr=0x00 vendor=0x1912 device=0x0015 subvendor=0x1912 subdevice=0x0015 ppt1@pci0:10:0:0: class=0x0c0330 rev=0x02 hdr=0x00 vendor=0x1912 device=0x0015 subvendor=0x1912 subdevice=0x0015 ppt2@pci0:66:0:0: class=0x030200 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1db4 subvendor=0x10de subdevice=0x1214 router:/usr/local/etc #
I am going to snapshot back to FreeBSD 14, I can't live without my passthroughs for my smart home lol.
Back on FreeBSD 14 stable: Home assistant VM: For zigbee and zwave USB sticks: # lsusb Bus 005 Device 001: ID 1d6b:0002 Linux 6.12.51-haos xhci-hcd xHCI Host Controller Bus 003 Device 001: ID 1d6b:0002 Linux 6.12.51-haos xhci-hcd xHCI Host Controller Bus 001 Device 001: ID 1d6b:0002 Linux 6.12.51-haos xhci-hcd xHCI Host Controller Bus 006 Device 001: ID 1d6b:0003 Linux 6.12.51-haos xhci-hcd xHCI Host Controller Bus 001 Device 002: ID 303a:4001 Nabu Casa ZWA-2 Bus 004 Device 001: ID 1d6b:0003 Linux 6.12.51-haos xhci-hcd xHCI Host Controller Bus 002 Device 001: ID 1d6b:0003 Linux 6.12.51-haos xhci-hcd xHCI Host Controller Bus 003 Device 002: ID 10c4:ea60 Itead Sonoff Zigbee 3.0 USB Dongle Plus V2 Bus 006 Device 002: ID fb5d:0001 BHYVE HID Tablet Ubuntu VM: With Nvidia GPU: ubuntu:~ # nvidia-smi Mon Nov 10 19:12:54 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla V100-PCIE-16GB Off | 00000000:00:06.0 Off | 0 | | N/A 82C P0 55W / 250W | 12778MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 1150 C python3 1268MiB | | 0 N/A N/A 1766 C /usr/local/bin/ollama 11506MiB | +-----------------------------------------------------------------------------------------+ ubuntu:~ # No issues. Maybe the vmm.ko driver on FreeBSD 15? No idea....
sys/dev/vmm/vmm.c is suspect here: Symptom: The GPU failed with "no GSI" / "can't derive routing". GSI (Global System Interrupt) is a Linux concept, but it means the physical interrupt signal from the passed-through PCI device (GPU or USB controller) is not being correctly mapped by the hypervisor into a signal the guest OS can understand. Commits touching the core interrupt handling between the host's IOMMU hardware (VT-d/AMD-Vi) and the virtualized environment are the prime suspects. Look for PRs that: Refactor or clean up the MSI/MSI-X handling within the vmm driver. Update core IOMMU register access or error handling code. Change the logic for x2APIC or IO-APIC interrupt injection into the guest.
For PCI passthrough, there are several potential things that it would be good to confirm: 1) Is the host an AMD system or Intel? (I think it's AMD from some of the comments.) 2) For the Nvidia GPU, there was a recent commit in 15: commit effcd0ded0ef1c92bef29c1e9bbf517c97bb45b8 Author: Corvin Köhne <corvink@FreeBSD.org> AuthorDate: Thu Mar 17 13:11:14 2022 +0100 Commit: Corvin Köhne <corvink@FreeBSD.org> CommitDate: Tue Sep 16 08:55:09 2025 +0200 bhyve: assign a valid INTPIN to NVIDIA GPUs When passing an NVIDIA GPU to a Linux VM, the GPU driver refuses to work and complains about a missing IRQ: [ 77.208984] NVRM: Can't find an IRQ for your NVIDIA card! [ 77.212697] NVRM: Please check your BIOS settings. [ 77.212699] NVRM: [Plug & Play OS] should be set to NO [ 77.212700] NVRM: [Assign IRQ to VGA] should be set to YES [ 77.212702] nvidia: probe of 0000:00:07.0 failed with error -1 ... However, this commit isn't present in 14. You could try reverting that commit locally perhaps? 3) For the USB pass through, it would be really handy to have a full Linux dmesg (and lspci output perhaps) inside the guest when booted under 14 vs booted under 15.
I may have a similar issue, not sure yet. Host is a FreeBSD 14.3-STABLE running pfSense as a VM, couple NICs are in passthrough mode from host to VM ppt0@pci0:7:0:0: class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x0003 ppt1@pci0:7:0:1: class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x0003 ppt2@pci0:9:0:0: class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1563 subvendor=0x8086 subdevice=0x0022 ppt3@pci0:9:0:1: class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1563 subvendor=0x8086 subdevice=0x0022 NICs are both ix driver controlled Ethernet Controller X550 82599ES 10-Gigabit SFI/SFP+ Network Connection installing 15.0-STABLE (buildworld/installworld/buildkernel/installkernel) and pfSense can no longer see any traffic. VM sees the NICs as before, can see link status going UP/DOWN but absolutely no traffic, no arp, no nothing. Revert back to 14.3-STABLE (installworld/kernel) - all is good again, no other changes. Intel CPU FreeBSD 14.3-STABLE stable/14-n272956-1fbba4fbf0f8 GENERIC amd64 FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2) VT(efifb): resolution 800x600 CPU microcode: no matching update found CPU: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz (2693.68-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x7fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x1<LAHF> Structured Extended Features=0x281<FSGSBASE,SMEP,ERMS> Structured Extended Features3=0x9c000400<MD_CLEAR,IBPB,STIBP,L1DFL,SSBD> XSAVE Features=0x1<XSAVEOPT> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 206158430208 (196608 MB) avail memory = 200668577792 (191372 MB)
After upgrading to FreeBSD 15, PCI passthrough of a Fresco Logic FL1100 USB 3.0 Host Controller (pci0:3:0:0) to a bhyve VM no longer functions. The xHCI device is detected inside the Linux guest (Home Assistant OS), but USB enumeration fails and the controller becomes unresponsive. No USB devices appear (no /dev/ttyUSB*, /dev/ttyACM*, etc.) even when directly attached to the PCI card. This configuration worked previously on FreeBSD 14.x on the same hardware. I suspect this is related to recent changes involving bhyve PCI/USB/xHCI handling in FreeBSD 15, as the controller fails at the moment the guest tries to issue commands/drain the command ring. Environment: Host OS: FreeBSD 15 (fresh upgrade from 14.x) Hypervisor: bhyve Passed through device: Fresco Logic FL1100 USB 3.0 Host Controller ppt0@pci0:3:0:0 vendor=0x1b73 device=0x1100 Guest OS: Home Assistant OS / Linux kernel 6.12.x IOMMU/VT-d: Enabled pptdevs entry in loader.conf: pptdevs="3/0/0" USB devices are physically connected to this PCI card (not motherboard USB). Symptoms inside guest (Linux dmesg): xhci_hcd 0000:00:06.0: xHCI Host Controller xhci_hcd 0000:00:06.0: new USB bus registered xhci_hcd 0000:00:06.0: Host supports USB 3.0 SuperSpeed ... Workqueue: events xhci_handle_command_timeout xhci_hcd 0000:00:06.0: Abort failed to stop command ring: -110 xhci_hcd 0000:00:06.0: xHCI host controller not responding, assume dead xhci_hcd 0000:00:06.0: HC died; cleaning up usb usb1-port1: couldn't allocate usb_device lsusb only shows the two root hubs, no devices. No /dev/serial/by-id, /dev/ttyUSB*, /dev/ttyACM* appears.
Having the same issue. Details on virtualization@ https://marc.info/?l=freebsd-virtualization&m=176485824804266&w=2
Same issue. Upgrade 14.3 to 15 broke bhyve passthru. (yes, homeassistant in bhyve, as many here do). maybe release of a new bhyve update with the planned xhid per device passthru would be the solution. I known there is a working patch for that. would also save us the need to passthru whole controler.
Same issue for me, upgrading from amd64 14.3-RELEASE-p6 to 15.0-RELEASE using vm-bhyve. My OPNSense bhyve guest starts without error and the two onboard network devices appear correctly in the guest, but packets don't seem to be moving. CPU: Intel(R) Xeon(R) CPU E3-1280 V2 @ 3.60GHz MoBo: Supermicro X9SCI-LN4 Passthru NICs: 2x (of 4) Intel 82574L (em driver) vm-bhyve guest config file: loader="bhyveload" priority="1" cpu="3" memory="8G" disk0_type="nvme" disk0_name="disk0.img" uuid="3b52dad1-c916-11ed-a8a9-002590247e86" passthru0="3/0/0" # LAN passthru1="5/0/0" # Modem /boot/loader.conf: boot_serial="YES" comconsole_port="0x3e8" console="comconsole" autoboot_delay="3" security.bsd.allow_destructive_dtrace="0" kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" cryptodev_load="YES" zfs_load="YES" coretemp_load="YES" mlx4en_load="YES" kern.racct.enable="1" cpu_microcode_load="YES" cpu_microcode_name="/boot/firmware/intel-ucode.bin" # PCI passthrough of em2 and em3 to OPNSense vmm_load="YES" pptdevs="3/0/0 5/0/0"
Same here on 15.0-RELEASE. Intel proc, Realtek RTL8111/8168/8211∕8411 PCI NIC. bhyve passthru ok, but the passthru NIC doesn't work inside the VM. And, of course, all was perfectly working under 14.3-RELEASE.
Created attachment 265939 [details] Logs/configs Hi John, I've created attachment of anything I figure might help be relevant, full commands with no edits of VMs and main host. Reason I am running 14 stable is to get at the Corvin patches, which was only committed to 14/15 stable and 16 current, USB passthrough works in 14 release but would fail to pass through nvidia GPU for reasons you listed unless using stable. Hopefully they were included in 15 release. Appreciate you looking at this, seems a lot of people are having same issue running FreeBSD 15 as main host, seems a lot of people are going to be stuck on 14 unless they running 15 in a VM.
I'm not sure if anything else is relevant but I'll include my VM configs and what I have enabled in loader.conf if you can spot a change in 15. ########Home Assistant /vm/ha/ha.conf: loader="uefi" graphics="yes" graphics_listen="192.168.0.1" graphics_port="5915" xhci_mouse="yes" cpu=10 cpu_sockets=1 cpu_cores=10 memory=16G network0_type="virtio-net" network0_switch="services" disk0_type="nvme" disk0_name="disk0.img" # bhyve pci interrupt passthrough attempt fix on freebsd 15 #x2apic="yes" uuid="a5811c38-567e-11ed-bdeb-e2335bf65f00" network0_mac="58:9c:11:0c:5a:a9" passthru0="9/0/0" passthru1="10/0/0" #######Ubuntu VM with Nvidia card passthrough loader="uefi" graphics="yes" graphics_listen="192.168.0.1" graphics_port="5906" xhci_mouse="yes" cpu=10 cpu_sockets=1 cpu_cores=10 memory=32G network0_type="virtio-net" network0_switch="services" disk0_type="nvme" disk0_name="disk0.img" uuid="61496056-8da5-11ee-906d-e4434bf65f00" network0_mac="58:9c:fc:0e:25:3f" passthru0="66/0/0" bhyve_options="-A -H -P" ######Loader.conf /boot/loader.conf: comconsole_speed="115200" boot_multicons="YES" boot_serial="YES" vfs.root.mountfrom="zfs:zroot/ROOT/default" kernels="kernel kernel.old" kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" cryptodev_load="YES" zfs_load="YES" #bhyve #only need this one for AMD systems and put before vmm_load #hw.vmm.amdvi.enable=1 vmm_load="YES" nmdm_load="YES" if_bridge_load="YES" if_tap_load="YES" #these needed for windows hw.usb.usbhid.enable="1" usbhid_load="YES" pptdevs="9/0/0 10/0/0 66/0/0" cc_htcp_load="YES" cc_cubic_load="YES" cc_dctcp_load="YES" #NetFlix TCP RACK STACK #tcp_rack_load="YES" #net.link.ifqmaxlen=2048 #net.inet.tcp.soreceive_stream=1 vfs.zfs.arc.max=37748736000
(In reply to Emrion from comment #13) I worked on the subject. I have to clarify what is this VM. It's a pfSense one, version 2.7.2-RELEASE (so, not up-to-date but that's not important). I don't use vm-bhyve, just bhyve command line. The realtek NIC is set for DHCP, it's the "wan" side for the pfSense VM. There is something random in this regression as most of the time, the NIC doesn't get an ip, but not always (I saw that with numerous tries to stop and restart the VM). Often, I have no connectivity despite it has a valid ip address, but in some occasions all seems to properly function. What I've seen in the dmesg of the pfSense VM is, when it fails: SRAT: Ignoring memory at addr 0xff000000 SRAT: Ignoring memory at addr 0xffc84000 I don't know if it's relevant.
just FYI, the patch from https://marc.info/?l=freebsd-virtualization&m=176531257403805&w=2 worked for me. HASS(home assistant linux) usb over pci passthrough works again
(In reply to martin.larsson2 from comment #17) Thanks for this detail. I've added the author of the commit, but I'm not sure yet if we want to just restore the ordering or if we need to fix the kernel side to not care about the ordering.