Bug 239849 - INTEL X710, WINDOWS SERVER 2019 HYPER-V, FreeBSD Current, iavf
Summary: INTEL X710, WINDOWS SERVER 2019 HYPER-V, FreeBSD Current, iavf
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-bugs mailing list
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2019-08-14 12:20 UTC by Michael
Modified: 2019-08-15 18:05 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael 2019-08-14 12:20:47 UTC
I am trying to use the SR-IOV ability in a virtual machine. But that
does not work. The host has the latest Intel drivers for the x710
network adapter. Second Generation Virtual Machine with GERERIC
Recompiled FreeBSD-Current Core for today. Host is Windows Server 2019
with Intel drivers (i40ea68.sys 1.10.128.0).

hn0 without SR-IOV function
hn1 with SR-IOV

Log messages from virtual machine:

Aug 14 10:56:18 r03 kernel: ---<<BOOT>>---
Aug 14 10:56:18 r03 kernel: Copyright (c) 1992-2019 The FreeBSD Project.
Aug 14 10:56:18 r03 kernel: Copyright (c) 1979, 1980, 1983, 1986,
1988, 1989, 1991, 1992, 1993, 1994
Aug 14 10:56:18 r03 kernel:     The Regents of the University of
California. All rights reserved.
Aug 14 10:56:18 r03 kernel: FreeBSD is a registered trademark of The
FreeBSD Foundation.
Aug 14 10:56:18 r03 kernel: FreeBSD 13.0-CURRENT
a2166b0cec5-c261904(master) R03 amd64
Aug 14 10:56:18 r03 kernel: FreeBSD clang version 8.0.1
(tags/RELEASE_801/final 366581) (based on LLVM 8.0.1)
Aug 14 10:56:18 r03 kernel: SRAT: Ignoring memory at addr 0x1fc000000
...
Aug 14 10:56:18 r03 kernel: SRAT: Ignoring memory at addr 0xc0000000000
Aug 14 10:56:18 r03 kernel: VT(efifb): resolution 1024x768
Aug 14 10:56:18 r03 kernel: Hyper-V Version: 10.0.17763 [SP0]
Aug 14 10:56:18 r03 kernel:
Features=0x2e7f<VPRUNTIME,TMREFCNT,SYNIC,SYNTM,APIC,HYPERCALL,VPINDEX,REFTSC,IDLE,TMFREQ>
Aug 14 10:56:18 r03 kernel:   PM Features=0x0 [C2]
Aug 14 10:56:18 r03 kernel:
Features3=0xbed7b2<DEBUG,XMMHC,IDLE,NUMA,TMFREQ,SYNCMC,CRASH,NPIEP>
Aug 14 10:56:18 r03 kernel: Timecounter "Hyper-V" frequency 10000000
Hz quality 2000
Aug 14 10:56:18 r03 kernel: module iavf already present!
Aug 14 10:56:18 r03 kernel: CPU: Intel(R) Xeon(R) CPU E5-2623 v3 @
3.00GHz (2996.53-MHz K8-class CPU)
Aug 14 10:56:18 r03 kernel:   Origin="GenuineIntel"  Id=0x306f2
Family=0x6  Model=0x3f  Stepping=2
...
Aug 14 10:56:18 r03 kernel: vmbus0: <Hyper-V Vmbus> on acpi_syscontainer0
Aug 14 10:56:18 r03 kernel: vmbus_res0: <Hyper-V Vmbus Resource> irq 5 on acpi0
...
Aug 14 10:56:18 r03 kernel: vmbus0: version 4.0
Aug 14 10:56:18 r03 kernel: hvet0: <Hyper-V event timer> on vmbus0
Aug 14 10:56:18 r03 kernel: Event timer "Hyper-V" frequency 10000000
Hz quality 1000
Aug 14 10:56:18 r03 kernel: hvkbd0: <Hyper-V KBD> on vmbus0
Aug 14 10:56:18 r03 kernel: hvheartbeat0: <Hyper-V Heartbeat> on vmbus0
Aug 14 10:56:18 r03 kernel: hvkvp0: <Hyper-V KVP> on vmbus0
Aug 14 10:56:18 r03 kernel: hvshutdown0: <Hyper-V Shutdown> on vmbus0
Aug 14 10:56:18 r03 kernel: hvtimesync0: <Hyper-V Timesync> on vmbus0
Aug 14 10:56:18 r03 kernel: hvtimesync0: RTT
Aug 14 10:56:18 r03 kernel: hvvss0: <Hyper-V VSS> on vmbus0
Aug 14 10:56:18 r03 kernel: storvsc0: <Hyper-V SCSI> on vmbus0
Aug 14 10:56:18 r03 kernel: hn0: <Hyper-V Network Interface> on vmbus0
Aug 14 10:56:18 r03 kernel: hn0: Ethernet address: 00:15:5d:00:88:29
Aug 14 10:56:18 r03 kernel: hn0: link state changed to UP
Aug 14 10:56:18 r03 kernel: hn1: <Hyper-V Network Interface> on vmbus0
Aug 14 10:56:18 r03 kernel: hn1: got notify, nvs type 128
Aug 14 10:56:18 r03 kernel: hn1: Ethernet address: 00:15:5d:00:88:2a
Aug 14 10:56:18 r03 kernel: hn1: link state changed to UP
Aug 14 10:56:18 r03 kernel: pcib0: <Hyper-V PCI Express Pass Through> on vmbus0
Aug 14 10:56:18 r03 kernel: pci0: <PCI bus> on pcib0
Aug 14 10:56:18 r03 kernel: pci0: <network, ethernet> at device 2.0
(no driver attached)
Aug 14 10:56:18 r03 kernel: lo0: link state changed to UP
...

root@r03:/# pciconf -lvb
none0@pci2:0:2:0:       class=0x020000 card=0x00018086 chip=0x15718086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Virtual Function 700 Series'
    class      = network
    subclass   = ethernet
    bar   [10] = type Prefetchable Memory, range 64, base r, size 65536, disabled
    bar   [1c] = type Prefetchable Memory, range 64, base r, size 16384, disabled
Comment 1 Andrey V. Elsukov freebsd_committer 2019-08-14 12:37:49 UTC
It looks like iavf_vendor_info_array[] lacks I40E_DEV_ID_VF_HV id, thus the iavf driver doesn't hook it.
You can try add corresponding line into sys/dev/ixl/if_iavf.c and recompile your kernel and modules.

--- if_iavf.c	(revision 350972)
+++ if_iavf.c	(working copy)
@@ -57,6 +57,7 @@
 static pci_vendor_info_t iavf_vendor_info_array[] =
 {
 	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_VF, "Intel(R) Ethernet Virtual Function 700 Series"),
+	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_VF_HV, "Intel(R) Ethernet Virtual Function 700 Series"),
 	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_X722_VF, "Intel(R) Ethernet Virtual Function 700 Series (X722)"),
 	PVID(I40E_INTEL_VENDOR_ID, I40E_DEV_ID_ADAPTIVE_VF, "Intel(R) Ethernet Adaptive Virtual Function"),
 	/* required last entry */
Comment 2 Michael 2019-08-15 17:55:26 UTC
(In reply to Andrey V. Elsukov from comment #1)

Add I40E_DEV_ID_VF_HV in if_iavf.c
Result:

root@r03:~ # dmesg
...
hn1: <Hyper-V Network Interface> on vmbus0
hn1: got notify, nvs type 128
hn1: Ethernet address: 00:15:5d:00:81:09
hn1: link state changed to UP
pcib0: <Hyper-V PCI Express Pass Through> on vmbus0
Trying to mount root from ufs:/dev/da0p2 [rw]...
pci0: <PCI bus> on pcib0
iavf0: <Intel(R) Ethernet Virtual Function 700 Series> at device 2.0 on pci0
iavf0: PF API 1.1 / VF API 1.1
iavf0: VSIs 1, QPs 1, MSI-X 5, RSS sizes: key 40 lut 64
iavf0: Using 1024 TX descriptors and 1024 RX descriptors
iavf0: Using 1 RX queues 1 TX queues
iavf0: Using MSI-X interrupts with 2 vectors
iavf0: Ethernet address: 00:15:5d:00:81:09
iavf0: netmap queues/slots: TX 1/1024, RX 1/1024
...

root@r03:~ # ifconfig
...
hn1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=51a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,LRO>
        ether 00:15:5d:00:81:09
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
iavf0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80053a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4,LRO>
        ether 00:15:5d:00:81:09
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

----------------
Disabling SR-IOV on hn1

root@r03:~ # dmesg
...
iavf0: detached
hn1: got notify, nvs type 128
pci0: detached
pcib0: detached

root@r03:~ # ifconfig
...
hn1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=51a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,LRO>
        ether 00:15:5d:00:81:09
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

----------------
... not work :(