Bug 178782
| Summary: | [ixgbe] 82599EB SFP does not work with passthrough under KVM. | ||
|---|---|---|---|
| Product: | Base System | Reporter: | legumen |
| Component: | kern | Assignee: | freebsd-net (Nobody) <net> |
| Status: | Closed FIXED | ||
| Severity: | Affects Only Me | CC: | erj, jeffrey.e.pieper, sbruno, wl.jarod |
| Priority: | Normal | Keywords: | IntelNetworking |
| Version: | 10.2-RELEASE | ||
| Hardware: | Any | ||
| OS: | Any | ||
Responsible Changed From-To: freebsd-bugs->freebsd-net Over to maintainer(s). Since the adapter works just fine from FreeBSD, I'm not sure how to proceed with a test of the pass through capabilities for Linux. Does this still occur now? (In reply to Sean Bruno from comment #2) Sorry Sean. I'm not sure if it still occur now or not. I changed my job for quite a long time! And I believe my former colleagues may solve(or have workaround for) this, because I noticed their product was announced last year. ;> (In reply to legumen from comment #3) ok, I'm hunting around for some linux folks that can test this and validate. I can test this. I'll be using 10.1 VM on a RHEL 7.1 host. 10.1 (ixgbe-2.5.15) still fails (no link and no interrupts), however I also tested a VM using an 11.0-CURRENT-amd64-20150804-r286285 snapshot, which has ix-3.1.0 and it gets link and passes traffic. Since the driver in HEAD is working, I think this can be closed. (In reply to Jeff Pieper from comment #6) Thanks for testing this, it really saved me quite a bit of time in setup. Do you have time to test 10.2R in the next week or so? I think its about to be pushed out by release engineering. (In reply to Sean Bruno from comment #7) ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.2/ (In reply to Sean Bruno from comment #8) 10.2-RELEASE works too (ix-2.8.3). (In reply to Jeff Pieper from comment #9) I still can reproduce the issue using 10.2-RELEASE. My basic environment is below: Hypervisor: KVM Host: Centos 7.0 Could you help share your basic environment like hypervisor and host version? PS, libvirt XML is below: <domain type='kvm'> <name>test</name> <uuid>7f68743b-d225-4c7d-dc7e-9aa6712584b3</uuid> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <vcpu placement='static'>4</vcpu> <cpu mode="host-passthrough"/> <os> <type arch='x86_64'>hvm</type> </os> <features> <acpi/> <apic/> <pae/> </features> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/opt/FreeBSD-10.2-RELEASE-amd64.qcow2'/> <target dev='hda' bus='ide'/> </disk> <interface type='bridge'> <mac address='52:54:00:72:26:88'/> <source bridge='br0'/> <model type='virtio'/> </interface> <hostdev mode="subsystem" type="pci" managed="yes"> <source> <address domain="0x0000" bus="0x06" slot="0x00" function="0x01"/> </source> </hostdev> <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> <serial type="pty"> <target port="0"/> </serial> <console type="pty"> <target type="serial" port="0"/> </console> </devices> </domain> |
The 10G Intel 82599EB SFP NIC could not work with passthrough on KVM. I did some tests with different Hypervisors and Guest OS, which make me think it's a FreeBSD problem. 1. OpenXen + FreeBSD9.1 guest os. -- [OK] 2. OpenXen + Linux guest os (e.g. Fedora 12). -- [OK] 3. KVM + Linux guest os (e.g. Fedora 12). -- [OK] 4. KVM + FreeBSD9.1 guest os. -- [Failed] How-To-Repeat: 1.Make the pci device of 82599SFP assignale. From lspci on OpenSUSE: 0d:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: QLogic, Corp. Device 00e4 Flags: fast devsel, IRQ 25 Memory at f8fe0000 (64-bit, prefetchable) [size=128K] I/O ports at cc00 [size=32] Memory at f8fdc000 (64-bit, prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [70] MSI-X: Enable- Count=64 Masked- Capabilities: [a0] Express Endpoint, MSI 00 Capabilities: [e0] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 00-00-00-ff-ff-00-00-00 Capabilities: [150] Alternative Routing-ID Interpretation (ARI) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) Kernel driver in use: pci-stub 2. Passthrough it to the guest FreeBSD os. Here is my vm configuration file <domain type='kvm'> <name>FreeBSD9.1</name> <uuid>4d56613e-400c-f37d-2a9a-e0332d81ca63</uuid> <memory>2097152</memory> <currentMemory>2097152</currentMemory> <vcpu>2</vcpu> <os> <type arch='x86_64' machine='pc'>hvm</type> <boot dev='hd'/> </os> <features> <pae/> <apic/> <acpi/> </features> <clock offset='utc'> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='default'/> <source file='/home/vm/FreeBSD9.1/FreeBSD9.1.raw'/> <target dev='hdc'/> </disk> <interface type='bridge'> <source bridge='br0'/> <mac address='52:54:00:0d:92:6f'/> <model type='rtl8139'/> </interface> <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> </source> </hostdev> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes'/> </devices> </domain> 3.The device could be probed and attached by FreeBSD. From dmesg on FreeBSD: ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.4.8> port 0xc020-0xc03f mem 0xfe000000-0xfe01ffff,0xfe020000-0xfe023fff irq 11 at device 4.0 on pci0 ix0: Using MSIX interrupts with 2 vectors ix0: Ethernet address: 00:0c:bd:05:7d:4a ix0: PCI Express Bus: Speed 5.0Gb/s Width x8 But no interrupt could be found. From "vmstat -i": interrupt total rate irq1: atkbd0 143 0 irq4: uart0 177 1 irq15: ata1 1593 10 cpu0:timer 7140 46 Total 9053 58 And ix0 keeps to be "no carrier" (even if I config an IP address on it and ping). From "ifconfig": ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO> ether 00:0c:bd:05:7d:4a nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect status: no carrier