Bug 263568 - ix(4): SR-IOV connection lost after loading VM with Intel X52-DAA2 passthrough device (0x10ED)
Summary: ix(4): SR-IOV connection lost after loading VM with Intel X52-DAA2 passthroug...
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 13.0-RELEASE
Hardware: Any Any
: Normal Affects Many People
Assignee: freebsd-net (Nobody)
URL:
Keywords: IntelNetworking, needs-qa
: 263569 (view as bug list)
Depends on:
Blocks:
 
Reported: 2022-04-25 18:12 UTC by benoitc
Modified: 2022-05-30 22:47 UTC (History)
6 users (show)

See Also:
koobs: maintainer-feedback? (rstone)
koobs: maintainer-feedback? (freebsd)
koobs: maintainer-feedback? (kbowling)
koobs: maintainer-feedback? (krzysztof.galazka)
koobs: maintainer-feedback? (erj)
koobs: mfc-stable13?
koobs: mfc-stable12?


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description benoitc 2022-04-25 18:12:57 UTC
I am trying to use SR-IOV on my machine with an IX driver, but each time i start the vm on the passthrough devices the connections which is started on the virtuall function dedicated for the host, the connection hangs and it is not possible to setup the connection back until I reboot. Do anyone know what i'm missing in my configuration? I don't see any useful logs in dmesg.

The card is the HP Ethernet 10Gb 2-port 560SFP+ Adapter	665249-B21 which is an Intel  X520-DAA2.

Maybe related but when restarting  netif I am getting this message: "Mailbox API negotiation failed in if_init" .I am using the intel-ix-kmod port. I try to update it to the latest 3.3.30 version which provided the same results.



I've the following configuration to setup sr-iov:

```
more /etc/iov-ix0.conf
PF {
        device: "ix0";
        num_vfs: 4;
}

DEFAULT {
        passthrough: true;
        allow-set-mac: true
        allow-promisc: true;
}

VF-0 {
        passthrough: false;
        mac-addr: "36:6b:d2:f9:a9:08";
}
```

and in loader.conf:

```
ifconfig_ixv0="up"

cloned_interfaces="lo1 bgp services"

vlans_ixv0="bgp services"

create_args_bgp="vlan 1000"
create_args_services="vlan 2000"

ifconfig_bgp_ipv6="inet6 XXX/64"

```

vms starts and the device seeps to be setup:

```
ixv0@pci0:8:0:128:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
ppt0@pci0:8:0:130:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
ppt1@pci0:8:0:132:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
ppt2@pci0:8:0:134:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x10ed subvendor=0x103c subdevice=0x17d3
    vendor     = 'Intel Corporation'
    device     = '82599 Ethernet Controller Virtual Function'
    class      = network
    subclass   = ethernet
    cap 11[70] = MSI-X supports 3 messages
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 0 endpoint max data 128(128)
                 max read 128
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected
    ecap 000e[150] = ARI 1
```


VM is started using vm-bhyve:

```
loader="uefi-csm"
cpu=1
memory=2G
passthru0=8/0/130=5:0
disk0_type="virtio-blk"
disk0_name="disk0.img"
uuid="ac3dafab-bedb-11ec-b24d-1402ec690a80"
```
Comment 1 Li-Wen Hsu freebsd_committer freebsd_triage 2022-04-25 19:14:08 UTC
*** Bug 263569 has been marked as a duplicate of this bug. ***
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2022-04-26 00:41:07 UTC
Thank you for your report Benoit. Are you able to reproduce this with the Intel ix driver available in FreeBSD ports?

  net/intel-ix-kmod
Comment 3 benoitc 2022-04-26 05:26:23 UTC
yes same result even after upting the port to 3.3.30.
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2022-05-30 22:42:03 UTC
^Triage: Request feedback from folks that play in the sys/dev/ixgbe area