Greetings, I have a problem with forwarding an LSI 2308 via vt-d in KVM to a FreeBSD virtual machine. FreeBSD (9.2 and 10.0 beta 3) will hang during the boot. Hardware Setup: Supermicro X10SL7-F with LSI 2308 flashed to IT mode 8x4 GB ecc ram Haswell Xeon E3-1230V3 Software Setup: Ubuntu 12.04.3 LTS 64 bit + latest KVM version. uname -a Linux Secretum 3.8.0-33-generic #48~precise1-Ubuntu SMP Thu Oct 24 16:28:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux dpkg --list | grep kvm ii kvm 1:84+dfsg-0ubuntu16+1.0+noroms+0ubuntu14.12 dummy transitional package from kvm to qemu-kvm ii kvm-ipxe 1.0.0+git-3.55f6c88-0ubuntu1 PXE ROM's for KVM ii qemu-kvm 1.0+noroms-0ubuntu14.12 Full virtualization on i386 and amd64 hardware Under KVM I have the following 3 virtual machines installed, I have tried forwarding the LSI 2308 to all three virtual machines. It works perfectly under Ubuntu, but both FreeBSD vms will hang during the boot. FreeBSD 9.2 FreeBSD 10.0 beta 3 FreeBSD 10.0 live cd Ubuntu 12.04 LTS If I run FreeBSD 10.0 beta 3 directly on the hardware, it does recognize the raid controller and it'll use the mps0 driver. Everything works fine then. So the problem is that for some reason FreeBSD hangs during boot if you forward the LSI 2308 via vt-d, and I have no idea why. It will hang and give the following error: http://i.imgur.com/hAMxwR7.png http://i.imgur.com/rKALeXZ.png While doing so the FreeBSD virtual machine uses 300% cpu, so it maxes out 3 cores. And it will stay like that. After googling a bit some people suggested turning off msi / msix in the loader.conf hw.pci.enable_msi="0" hw.pci.enable_msix="0" I have tried this on both freebsd virtual machines, it makes no difference. It still hangs. After having tried all this a collegue suggested trying to turn off drivers in the kernel. After compiling a few kernels and seeing if I could boot I managed to trace the root of the problem to the mps SCSI driver. If I compile a kernel without that driver and pass through the raid controller via vt-d, the freebsd virtual machine will no longer hang during boot. I obviously also don't have a driver for my raid controller then.. So something in the mps driver is causing the deadlock which causes FreeBSD to hang / lockup during boot. I can give you more information if necessarry. Kind Regards Robert Sevat Fix: Change the mps SCSI driver. How-To-Repeat: Install FreeBSD as a virtual machine under KVM and pass through a LSI 2308 raid controller via vt-d. This will cause the mps driver to lock up during boot.
Responsible Changed From-To: freebsd-bugs->freebsd-scsi Over to maintainer(s).
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped
This behavior isn't just KVM. I have a similar but different setup that is hanging at mps0 on boot under ESXi 5.5. I'm using VMware ESXi 5.5.0 build 2068190 on a Supermicro X10SLL-F with an E3-1230v3 and 32Gb RAM with PCIe pass through for the LSI card. The card is a LSI2116 with the latest IT firmware (20.00.07.00). I'm using FreeNAS11 built as of Jan6-2018 for the guest image. On bootup, the screen hangs at the mps0: prompt. If I disable PCI pass through, the system boots normally (without the drives connected to the LSI card, of course.) I tested the exact same card and FreeNAS image on a non-virtualized physical machine (AMD A8 APU 8Gb RAM) and it worked perfectly for weeks. The exact same FreeNAS image and card locks up on boot in the Supermicro system - I made the VM image from the USB stick that worked in the physical machine. I know that the VM Host is at least somewhat FreeBSD friendly as I'm posting this through a pfSense firewall hosted on the same box. (Don't virtualize your firewall, I know I know I know) I know that PCIe passthrough works on this host because it has been running pass through on a 8 port LSI3081E card for years with zero trouble and Freenas8. I also had PCIe pass through running on this host with a LSI2108 card with older firmware. However, I experienced some errors due to a firmware mismatch between the card and FreeBSD's driver, which is a known issue for these cards. Prepared firmware, tested LSI2116 card on physical machine with FreeNAS11 - worked perfect. Pulled LSi3081E, replaced with flashed LSI2116 card in VMHost: FreeNAS8 sees mps0 but won't talk to it, doesn't detect any disks but boots fine. Upgrade FreeNAS8 to current version via web updater - hangs at boot at mps0 Copy exact FreeNAS image which worked on physical machine to VMHost - identical behavior, hang at mps0 on boot. dmesg from VM with PCI passthrough DISABLED: Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.1-STABLE #0 r321665+d4625dcee3e(freenas/11.1-stable): Wed Dec 13 16:33:42 UTC 2017 root@gauntlet:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64 FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn) CPU: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz (3300.00-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3 Features=0x1fa3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,HTT> Features2=0x9ed83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,HV> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant Hypervisor: Origin = "VMwareVMware" real memory = 13958643712 (13312 MB) avail memory = 12416217088 (11841 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <PTLTD APIC > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. MADT: Forcing active-low polarity and level trigger for SCI ioapic0 <Version 1.1> irqs 0-23 on motherboard SMP: AP CPU #3 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! random: entropy device external interface kbd1 at kbdmux0 nexus0 cryptosoft0: <software crypto> on motherboard aesni0: <AES-CBC,AES-XTS,AES-GCM,AES-ICM> on motherboard padlock0: No ACE support. acpi0: <INTEL 440BX> on motherboard acpi0: Power Button (fixed) hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 cpu0: <ACPI CPU> numa-domain 0 on acpi0 cpu1: <ACPI CPU> numa-domain 0 on acpi0 cpu2: <ACPI CPU> numa-domain 0 on acpi0 cpu3: <ACPI CPU> numa-domain 0 on acpi0 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0 pci1: <ACPI PCI bus> on pcib1 isab0: <PCI-ISA bridge> at device 7.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1060-0x106f at device 7.1 on pci0 ata0: <ATA channel> at channel 0 on atapci0 ata1: <ATA channel> at channel 1 on atapci0 pci0: <bridge> at device 7.3 (no driver attached) vgapci0: <VGA-compatible display> port 0x1070-0x107f mem 0xec000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0 vgapci0: Boot video device mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci0 mpt0: MPI Version=1.2.0.0 pcib2: <ACPI PCI-PCI bridge> at device 17.0 on pci0 pci2: <ACPI PCI bus> on pcib2 em0: <Intel(R) PRO/1000 Legacy Network Connection 1.1.0> port 0x2000-0x203f mem 0xfd5c0000-0xfd5dffff,0xfdff0000-0xfdffffff irq 18 at device 0.0 on pci2 em0: Ethernet address: 00:0c:29:30:58:66 em0: link state changed to UP em1: <Intel(R) PRO/1000 Legacy Network Connection 1.1.0> port 0x2040-0x207f mem 0xfd5a0000-0xfd5bffff,0xfdfe0000-0xfdfeffff irq 16 at device 2.0 on pci2 em1: Ethernet address: 00:0c:29:30:58:70 em1: link state changed to UP pcib3: <ACPI PCI-PCI bridge> at device 21.0 on pci0 pcib3: [GIANT-LOCKED] pcib4: <ACPI PCI-PCI bridge> at device 21.1 on pci0 pcib4: [GIANT-LOCKED] pcib5: <ACPI PCI-PCI bridge> at device 21.2 on pci0 pcib5: [GIANT-LOCKED] pcib6: <ACPI PCI-PCI bridge> at device 21.3 on pci0 pcib6: [GIANT-LOCKED] pcib7: <ACPI PCI-PCI bridge> at device 21.4 on pci0 pcib7: [GIANT-LOCKED] pcib8: <ACPI PCI-PCI bridge> at device 21.5 on pci0 pcib8: [GIANT-LOCKED] pcib9: <ACPI PCI-PCI bridge> at device 21.6 on pci0 pcib9: [GIANT-LOCKED] pcib10: <ACPI PCI-PCI bridge> at device 21.7 on pci0 pcib10: [GIANT-LOCKED] pcib11: <ACPI PCI-PCI bridge> at device 22.0 on pci0 pcib11: [GIANT-LOCKED] pcib12: <ACPI PCI-PCI bridge> at device 22.1 on pci0 pcib12: [GIANT-LOCKED] pcib13: <ACPI PCI-PCI bridge> at device 22.2 on pci0 pcib13: [GIANT-LOCKED] pcib14: <ACPI PCI-PCI bridge> at device 22.3 on pci0 pcib14: [GIANT-LOCKED] pcib15: <ACPI PCI-PCI bridge> at device 22.4 on pci0 acpi_acad0: <AC Adapter> on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: <PS/2 Mouse> irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse, device ID 3 acpi_syscontainer0: <System Container> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xcb000-0xcbfff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 coretemp0: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu0 coretemp0: Tj(target) value 0 does not seem right. coretemp1: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu1 coretemp1: Tj(target) value 0 does not seem right. coretemp2: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu2 coretemp2: Tj(target) value 0 does not seem right. coretemp3: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu3 coretemp3: Tj(target) value 0 does not seem right. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec freenas_sysctl: adding account. freenas_sysctl: adding directoryservice. freenas_sysctl: adding middlewared. freenas_sysctl: adding network. freenas_sysctl: adding services. ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled usb_needs_explore_all: no devclass nvme cam probe device init da0 at mpt0 bus 0 scbus2 target 0 lun 0 da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 16384MB (33554432 512 byte sectors) da0: quirks=0x140<RETRY_BUSY,STRICT_UNMAP> cd0 at ata1 bus 0 scbus1 target 0 lun 0 cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI device cd0: Serial Number 10000000000000000001 cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present cd0: quirks=0x40<RETRY_BUSY> random: unblocking device. Trying to mount root from zfs:freenas-boot/ROOT/11.1-RELEASE []... warning: KLD '/boot/kernel/smbus.ko' is newer than the linker.hints file warning: KLD '/boot/kernel/vmm.ko' is newer than the linker.hints file vmx_init: processor does not support VMX operation module_register_init: MOD_LOAD (vmm, 0xffffffff82824400, 0) error 6 GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7) warning: KLD '/boot/kernel/sdt.ko' is newer than the linker.hints file warning: KLD '/boot/kernel/systrace.ko' is newer than the linker.hints file warning: KLD '/boot/kernel/systrace_freebsd32.ko' is newer than the linker.hints file warning: KLD '/boot/kernel/profile.ko' is newer than the linker.hints file hwpc_core: unknown PMC architecture: 1 hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> warning: KLD '/boot/kernel/t3_tom.ko' is newer than the linker.hints file warning: KLD '/boot/kernel/toecore.ko' is newer than the linker.hints file warning: KLD '/boot/kernel/t4_tom.ko' is newer than the linker.hints file VMware memory control driver initialized Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `bufdaemon' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 0 done All buffers synced. Uptime: 1m45s Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.1-STABLE #0 r321665+d4625dcee3e(freenas/11.1-stable): Wed Dec 13 16:33:42 UTC 2017 root@gauntlet:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64 FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn) CPU: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz (3300.00-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3 Features=0x1fa3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,HTT> Features2=0x9ed83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,HV> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant Hypervisor: Origin = "VMwareVMware" real memory = 13958643712 (13312 MB) avail memory = 12416217088 (11841 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <PTLTD APIC > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. MADT: Forcing active-low polarity and level trigger for SCI ioapic0 <Version 1.1> irqs 0-23 on motherboard SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! random: entropy device external interface kbd1 at kbdmux0 nexus0 cryptosoft0: <software crypto> on motherboard aesni0: <AES-CBC,AES-XTS,AES-GCM,AES-ICM> on motherboard padlock0: No ACE support. acpi0: <INTEL 440BX> on motherboard acpi0: Power Button (fixed) hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 cpu0: <ACPI CPU> numa-domain 0 on acpi0 cpu1: <ACPI CPU> numa-domain 0 on acpi0 cpu2: <ACPI CPU> numa-domain 0 on acpi0 cpu3: <ACPI CPU> numa-domain 0 on acpi0 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0 pci1: <ACPI PCI bus> on pcib1 isab0: <PCI-ISA bridge> at device 7.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1060-0x106f at device 7.1 on pci0 ata0: <ATA channel> at channel 0 on atapci0 ata1: <ATA channel> at channel 1 on atapci0 pci0: <bridge> at device 7.3 (no driver attached) vgapci0: <VGA-compatible display> port 0x1070-0x107f mem 0xec000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0 vgapci0: Boot video device mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci0 mpt0: MPI Version=1.2.0.0 pcib2: <ACPI PCI-PCI bridge> at device 17.0 on pci0 pci2: <ACPI PCI bus> on pcib2 em0: <Intel(R) PRO/1000 Legacy Network Connection 1.1.0> port 0x2000-0x203f mem 0xfd5c0000-0xfd5dffff,0xfdff0000-0xfdffffff irq 18 at device 0.0 on pci2 em0: Ethernet address: 00:0c:29:30:58:66 em0: link state changed to UP em1: <Intel(R) PRO/1000 Legacy Network Connection 1.1.0> port 0x2040-0x207f mem 0xfd5a0000-0xfd5bffff,0xfdfe0000-0xfdfeffff irq 16 at device 2.0 on pci2 em1: Ethernet address: 00:0c:29:30:58:70 em1: link state changed to UP pcib3: <ACPI PCI-PCI bridge> at device 21.0 on pci0 pcib3: [GIANT-LOCKED] pcib4: <ACPI PCI-PCI bridge> at device 21.1 on pci0 pcib4: [GIANT-LOCKED] pcib5: <ACPI PCI-PCI bridge> at device 21.2 on pci0 pcib5: [GIANT-LOCKED] pcib6: <ACPI PCI-PCI bridge> at device 21.3 on pci0 pcib6: [GIANT-LOCKED] pcib7: <ACPI PCI-PCI bridge> at device 21.4 on pci0 pcib7: [GIANT-LOCKED] pcib8: <ACPI PCI-PCI bridge> at device 21.5 on pci0 pcib8: [GIANT-LOCKED] pcib9: <ACPI PCI-PCI bridge> at device 21.6 on pci0 pcib9: [GIANT-LOCKED] pcib10: <ACPI PCI-PCI bridge> at device 21.7 on pci0 pcib10: [GIANT-LOCKED] pcib11: <ACPI PCI-PCI bridge> at device 22.0 on pci0 pcib11: [GIANT-LOCKED] pcib12: <ACPI PCI-PCI bridge> at device 22.1 on pci0 pcib12: [GIANT-LOCKED] pcib13: <ACPI PCI-PCI bridge> at device 22.2 on pci0 pcib13: [GIANT-LOCKED] pcib14: <ACPI PCI-PCI bridge> at device 22.3 on pci0 pcib14: [GIANT-LOCKED] pcib15: <ACPI PCI-PCI bridge> at device 22.4 on pci0 pcib15: [GIANT-LOCKED] pcib16: <ACPI PCI-PCI bridge> at device 22.5 on pci0 pcib16: [GIANT-LOCKED] pcib17: <ACPI PCI-PCI bridge> at device 22.6 on pci0 pcib17: [GIANT-LOCKED] pcib18: <ACPI PCI-PCI bridge> at device 22.7 on pci0 pcib18: [GIANT-LOCKED] pcib19: <ACPI PCI-PCI bridge> at device 23.0 on pci0 pcib19: [GIANT-LOCKED] pcib20: <ACPI PCI-PCI bridge> at device 23.1 on pci0 pcib20: [GIANT-LOCKED] pcib21: <ACPI PCI-PCI bridge> at device 23.2 on pci0 pcib21: [GIANT-LOCKED] pcib22: <ACPI PCI-PCI bridge> at device 23.3 on pci0 pcib22: [GIANT-LOCKED] pcib23: <ACPI PCI-PCI bridge> at device 23.4 on pci0 pcib23: [GIANT-LOCKED] pcib24: <ACPI PCI-PCI bridge> at device 23.5 on pci0 pcib24: [GIANT-LOCKED] pcib25: <ACPI PCI-PCI bridge> at device 23.6 on pci0 pcib25: [GIANT-LOCKED] pcib26: <ACPI PCI-PCI bridge> at device 23.7 on pci0 pcib26: [GIANT-LOCKED] pcib27: <ACPI PCI-PCI bridge> at device 24.0 on pci0 pcib27: [GIANT-LOCKED] pcib28: <ACPI PCI-PCI bridge> at device 24.1 on pci0 pcib28: [GIANT-LOCKED] pcib29: <ACPI PCI-PCI bridge> at device 24.2 on pci0 pcib29: [GIANT-LOCKED] pcib30: <ACPI PCI-PCI bridge> at device 24.3 on pci0 pcib30: [GIANT-LOCKED] pcib31: <ACPI PCI-PCI bridge> at device 24.4 on pci0 pcib31: [GIANT-LOCKED] pcib32: <ACPI PCI-PCI bridge> at device 24.5 on pci0 pcib32: [GIANT-LOCKED] pcib33: <ACPI PCI-PCI bridge> at device 24.6 on pci0 pcib33: [GIANT-LOCKED] pcib34: <ACPI PCI-PCI bridge> at device 24.7 on pci0 pcib34: [GIANT-LOCKED] acpi_acad0: <AC Adapter> on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: <PS/2 Mouse> irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse, device ID 3 acpi_syscontainer0: <System Container> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xcb000-0xcbfff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 coretemp0: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu0 coretemp0: Tj(target) value 0 does not seem right. coretemp1: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu1 coretemp1: Tj(target) value 0 does not seem right. coretemp2: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu2 coretemp2: Tj(target) value 0 does not seem right. coretemp3: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu3 coretemp3: Tj(target) value 0 does not seem right. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec freenas_sysctl: adding account. freenas_sysctl: adding directoryservice. freenas_sysctl: adding middlewared. freenas_sysctl: adding network. freenas_sysctl: adding services. ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled usb_needs_explore_all: no devclass nvme cam probe device init da0 at mpt0 bus 0 scbus2 target 0 lun 0 da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 16384MB (33554432 512 byte sectors) da0: quirks=0x140<RETRY_BUSY,STRICT_UNMAP> cd0 at ata1 bus 0 scbus1 target 0 lun 0 cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI device cd0: Serial Number 10000000000000000001 cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present acpi_syscontainer0: <System Container> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xcb000-0xcbfff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 coretemp0: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu0 coretemp0: Tj(target) value 0 does not seem right. coretemp1: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu1 coretemp1: Tj(target) value 0 does not seem right. coretemp2: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu2 coretemp2: Tj(target) value 0 does not seem right. coretemp3: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu3 coretemp3: Tj(target) value 0 does not seem right. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec freenas_sysctl: adding account. freenas_sysctl: adding directoryservice. freenas_sysctl: adding middlewared. freenas_sysctl: adding network. freenas_sysctl: adding services. ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled usb_needs_explore_all: no devclass nvme cam probe device init da0 at mpt0 bus 0 scbus2 target 0 lun 0 da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 16384MB (33554432 512 byte sectors) da0: quirks=0x140<RETRY_BUSY,STRICT_UNMAP> cd0 at ata1 bus 0 scbus1 target 0 lun 0 cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI device cd0: Serial Number 10000000000000000001 cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present cd0: quirks=0x40<RETRY_BUSY> random: unblocking device. Trying to mount root from zfs:freenas-boot/ROOT/11.1-RELEASE []... vmx_init: processor does not support VMX operation module_register_init: MOD_LOAD (vmm, 0xffffffff82824400, 0) error 6 GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7) hwpc_core: unknown PMC architecture: 1 hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> VMware memory control driver initialized
With boot set to verbose, last messages before lockup with PCIe passthrough ENABLED: pcib3: slot 0 INTA is routed to irq 18 mps0: <Avago Technologies (LSI) SAS2116> port 0x4000-0x40ff mem 0xfd4fc000-0xfd4fffff,0xfd4800000-0xfd4bffff irq 18 at device 0.0 on pci3 mps0: attempting to allocate 1 MSI-X vectors (15 supported) mps0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 257 to local APIC 1 vector 50 mps0: using IRQ 257 for MSI mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd msp0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
^Triage: I'm sorry that this PR did not get addressed in a timely fashion. By now, the version that it was created against is long out of support. Please re-open if it is still a problem on a supported version.