Bug 239849 - INTEL X710, WINDOWS SERVER 2019 HYPER-V, FreeBSD Current, iavf
Summary: INTEL X710, WINDOWS SERVER 2019 HYPER-V, FreeBSD Current, iavf
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2019-08-14 12:20 UTC by Michael
Modified: 2020-10-29 10:10 UTC (History)
3 users (show)

See Also:


Attachments
Make FreeBSD 11.3 to support SRIOV on Intel NIC 82599 VF (0x152E) on HyperV (15.53 KB, patch)
2020-10-29 10:10 UTC, Wei Hu
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michael 2019-08-14 12:20:47 UTC
I am trying to use the SR-IOV ability in a virtual machine. But that
does not work. The host has the latest Intel drivers for the x710
network adapter. Second Generation Virtual Machine with GERERIC
Recompiled FreeBSD-Current Core for today. Host is Windows Server 2019
with Intel drivers (i40ea68.sys 1.10.128.0).

hn0 without SR-IOV function
hn1 with SR-IOV

Log messages from virtual machine:

Aug 14 10:56:18 r03 kernel: ---<<BOOT>>---
Aug 14 10:56:18 r03 kernel: Copyright (c) 1992-2019 The FreeBSD Project.
Aug 14 10:56:18 r03 kernel: Copyright (c) 1979, 1980, 1983, 1986,
1988, 1989, 1991, 1992, 1993, 1994
Aug 14 10:56:18 r03 kernel:     The Regents of the University of
California. All rights reserved.
Aug 14 10:56:18 r03 kernel: FreeBSD is a registered trademark of The
FreeBSD Foundation.
Aug 14 10:56:18 r03 kernel: FreeBSD 13.0-CURRENT
a2166b0cec5-c261904(master) R03 amd64
Aug 14 10:56:18 r03 kernel: FreeBSD clang version 8.0.1
(tags/RELEASE_801/final 366581) (based on LLVM 8.0.1)
Aug 14 10:56:18 r03 kernel: SRAT: Ignoring memory at addr 0x1fc000000
...
Aug 14 10:56:18 r03 kernel: SRAT: Ignoring memory at addr 0xc0000000000
Aug 14 10:56:18 r03 kernel: VT(efifb): resolution 1024x768
Aug 14 10:56:18 r03 kernel: Hyper-V Version: 10.0.17763 [SP0]
Aug 14 10:56:18 r03 kernel:
Features=0x2e7f<VPRUNTIME,TMREFCNT,SYNIC,SYNTM,APIC,HYPERCALL,VPINDEX,REFTSC,IDLE,TMFREQ>
Aug 14 10:56:18 r03 kernel:   PM Features=0x0 [C2]
Aug 14 10:56:18 r03 kernel:
Features3=0xbed7b2<DEBUG,XMMHC,IDLE,NUMA,TMFREQ,SYNCMC,CRASH,NPIEP>
Aug 14 10:56:18 r03 kernel: Timecounter "Hyper-V" frequency 10000000
Hz quality 2000
Aug 14 10:56:18 r03 kernel: module iavf already present!
Aug 14 10:56:18 r03 kernel: CPU: Intel(R) Xeon(R) CPU E5-2623 v3 @
3.00GHz (2996.53-MHz K8-class CPU)
Aug 14 10:56:18 r03 kernel:   Origin="GenuineIntel"  Id=0x306f2
Family=0x6  Model=0x3f  Stepping=2
...
Aug 14 10:56:18 r03 kernel: vmbus0: <Hyper-V Vmbus> on acpi_syscontainer0
Aug 14 10:56:18 r03 kernel: vmbus_res0: <Hyper-V Vmbus Resource> irq 5 on acpi0
...
Aug 14 10:56:18 r03 kernel: vmbus0: version 4.0
Aug 14 10:56:18 r03 kernel: hvet0: <Hyper-V event timer> on vmbus0
Aug 14 10:56:18 r03 kernel: Event timer "Hyper-V" frequency 10000000
Hz quality 1000
Aug 14 10:56:18 r03 kernel: hvkbd0: <Hyper-V KBD> on vmbus0
Aug 14 10:56:18 r03 kernel: hvheartbeat0: <Hyper-V Heartbeat> on vmbus0
Aug 14 10:56:18 r03 kernel: hvkvp0: <Hyper-V KVP> on vmbus0
Aug 14 10:56:18 r03 kernel: hvshutdown0: <Hyper-V Shutdown> on vmbus0
Aug 14 10:56:18 r03 kernel: hvtimesync0: <Hyper-V Timesync> on vmbus0
Aug 14 10:56:18 r03 kernel: hvtimesync0: RTT
Aug 14 10:56:18 r03 kernel: hvvss0: <Hyper-V VSS> on vmbus0
Aug 14 10:56:18 r03 kernel: storvsc0: <Hyper-V SCSI> on vmbus0
Aug 14 10:56:18 r03 kernel: hn0: <Hyper-V Network Interface> on vmbus0
Aug 14 10:56:18 r03 kernel: hn0: Ethernet address: 00:15:5d:00:88:29
Aug 14 10:56:18 r03 kernel: hn0: link state changed to UP
Aug 14 10:56:18 r03 kernel: hn1: <Hyper-V Network Interface> on vmbus0
Aug 14 10:56:18 r03 kernel: hn1: got notify, nvs type 128
Aug 14 10:56:18 r03 kernel: hn1: Ethernet address: 00:15:5d:00:88:2a
Aug 14 10:56:18 r03 kernel: hn1: link state changed to UP
Aug 14 10:56:18 r03 kernel: pcib0: <Hyper-V PCI Express Pass Through> on vmbus0
Aug 14 10:56:18 r03 kernel: pci0: <PCI bus> on pcib0
Aug 14 10:56:18 r03 kernel: pci0: <network, ethernet> at device 2.0
(no driver attached)
Aug 14 10:56:18 r03 kernel: lo0: link state changed to UP
...

root@r03:/# pciconf -lvb
none0@pci2:0:2:0:       class=0x020000 card=0x00018086 chip=0x15718086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Virtual Function 700 Series'
    class      = network
    subclass   = ethernet
    bar   [10] = type Prefetchable Memory, range 64, base r, size 65536, disabled
    bar   [1c] = type Prefetchable Memory, range 64, base r, size 16384, disabled
Comment 1 Andrey V. Elsukov freebsd_committer 2019-08-14 12:37:49 UTC
It looks like iavf_vendor_info_array[] lacks I40E_DEV_ID_VF_HV id, thus the iavf driver doesn't hook it.
You can try add corresponding line into sys/dev/ixl/if_iavf.c and recompile your kernel and modules.

--- if_iavf.c	(revision 350972)
+++ if_iavf.c	(working copy)
@@ -57,6 +57,7 @@
 static pci_vendor_info_t iavf_vendor_info_array[] =
 {
 	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_VF, "Intel(R) Ethernet Virtual Function 700 Series"),
+	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_VF_HV, "Intel(R) Ethernet Virtual Function 700 Series"),
 	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_X722_VF, "Intel(R) Ethernet Virtual Function 700 Series (X722)"),
 	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_ADAPTIVE_VF, "Intel(R) Ethernet Adaptive Virtual Function"),
 	/* required last entry */
Comment 2 Michael 2019-08-15 17:55:26 UTC
(In reply to Andrey V. Elsukov from comment #1)

Add I40E_DEV_ID_VF_HV in if_iavf.c
Result:

root@r03:~ # dmesg
...
hn1: <Hyper-V Network Interface> on vmbus0
hn1: got notify, nvs type 128
hn1: Ethernet address: 00:15:5d:00:81:09
hn1: link state changed to UP
pcib0: <Hyper-V PCI Express Pass Through> on vmbus0
Trying to mount root from ufs:/dev/da0p2 [rw]...
pci0: <PCI bus> on pcib0
iavf0: <Intel(R) Ethernet Virtual Function 700 Series> at device 2.0 on pci0
iavf0: PF API 1.1 / VF API 1.1
iavf0: VSIs 1, QPs 1, MSI-X 5, RSS sizes: key 40 lut 64
iavf0: Using 1024 TX descriptors and 1024 RX descriptors
iavf0: Using 1 RX queues 1 TX queues
iavf0: Using MSI-X interrupts with 2 vectors
iavf0: Ethernet address: 00:15:5d:00:81:09
iavf0: netmap queues/slots: TX 1/1024, RX 1/1024
...

root@r03:~ # ifconfig
...
hn1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=51a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,LRO>
        ether 00:15:5d:00:81:09
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
iavf0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80053a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4,LRO>
        ether 00:15:5d:00:81:09
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

----------------
Disabling SR-IOV on hn1

root@r03:~ # dmesg
...
iavf0: detached
hn1: got notify, nvs type 128
pci0: detached
pcib0: detached

root@r03:~ # ifconfig
...
hn1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=51a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,LRO>
        ether 00:15:5d:00:81:09
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

----------------
... not work :(
Comment 3 Wei Hu 2020-10-29 10:10:42 UTC
Created attachment 219193 [details]
Make FreeBSD 11.3 to support SRIOV on Intel NIC 82599 VF (0x152E) on HyperV

This is code changes to support SRIOV for Intel NIC 82599 VF (Device ID 0x152E) on HyperV. I have done some test on FreeBSD 11.3. The similar code changes will need to make other Intel VFs work on HyperV.