Bug 230235 - ixl: hangs when adding it to a bridge or enabling iovctl VFs (SR-IOV)
Summary: ixl: hangs when adding it to a bridge or enabling iovctl VFs (SR-IOV)
Status: Closed Not A Bug
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 11.2-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-net mailing list
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2018-07-31 12:18 UTC by Finn
Modified: 2018-12-17 19:31 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Finn 2018-07-31 12:18:31 UTC
this is with driver version v1.9.5 (for v1.9.13: see the next section):

when adding ixl1 to a bridge the interface hangs:

# vm switch create public
# vm switch add public ixl1 

/var/log/messages:

hh:39:49  kernel: bridge0: Ethernet address: ...
hh:39:49  kernel: bridge0: changing name to 'vm-public'
hh:40:12  kernel: ixl1: RX queue 3 still enabled!
hh:40:12  kernel: vm-public: link state changed to UP
hh:40:12  kernel: ixl1: promiscuous mode enabled
hh:40:12  kernel: ixl1: Malicious Driver Detection event 2 on TX queue 773, pf number 1
hh:40:12  kernel: ixl1: MDD TX event is for this function!
hh:40:12  kernel: ixl1: Malicious Driver Detection event 2 on TX queue 774, pf number 1
hh:40:12  kernel: ixl1: MDD TX event is for this function!
hh:40:12  kernel: ixl1: Malicious Driver Detection event 2 on TX queue 772, pf number 1
hh:40:12  kernel: ixl1: MDD TX event is for this function!
hh:40:23  kernel: ixl1: WARNING: queue 7 appears to be hung!
hh:40:24  kernel: ixl1: WARNING: queue 6 appears to be hung!
hh:40:30  kernel: ixl1: WARNING: queue 4 appears to be hung!
hh:40:34  kernel: ixl1: WARNING: queue 5 appears to be hung!

-> rebooted to recover

---------

Since the bridge interface does not work I tried SR-IOV
(this is with driver version v1.9.13 after the reboot)
log when booting up:

kernel: ixl1: <Intel(R) Ethernet Connection 700 Series PF Driver, Version - 1.9.13> mem 0xf8800000-0xf8ffffff,0xf9800000-0xf9807fff irq 32 at device 0.1 numa-domain 0 on pci3
kernel: ixl1: using 1024 tx descriptors and 1024 rx descriptors
kernel: ixl1: fw 6.0.48442 api 1.7 nvm 6.01 etid 800035cf oem 1.262.0
kernel: ixl1: PF-ID[1]: VFs 64, MSIX 129, VF MSIX 5, QPs 768, I2C
kernel: ixl1: Using MSIX interrupts with 9 vectors
kernel: ixl1: Allocating 8 queues for PF LAN VSI; 8 queues active
kernel: ixl1: Ethernet address: ...
kernel: ixl1: PCI Express Bus: Speed 8.0GT/s Width x8
kernel: ixl1: SR-IOV ready
kernel: ixl1: netmap queues/slots: TX 8/1024, RX 8/1024
kernel: ixl1: The device is not iWARP enabled


/etc/iov/ixl1.conf
---
PF {
	  device : "ixl1";
	  num_vfs : 2;
}

DEFAULT {
	  passthrough :	true;
}
--

starting iovctl caused it to hang again:

# service iovctl start

logs:

/var/log/messages:

hh:mm:19  kernel: NMI ISA 38, EISA 0
hh:mm:19  kernel: NMI/cpu10 ... going to debugger
hh:mm:21  kernel: ixl1: RX queue 0 still enabled!
hh:mm:21  kernel: ixl1: RX queue 1 still enabled!
hh:mm:21  kernel: ixl1: RX queue 2 still enabled!
hh:mm:21  kernel: pci3: <network, ethernet> at device 0.80 (no driver attached)
hh:mm:21  kernel: pci3: <network, ethernet> at device 0.81 (no driver attached)
hh:mm:30  kernel: ixl1: WARNING: queue 0 appears to be hung!
hh:mm:30  kernel: ixl1: WARNING: queue 1 appears to be hung!
hh:mm:30  kernel: ixl1: WARNING: queue 2 appears to be hung!
hh:mm:30  kernel: ixl1: WARNING: queue 3 appears to be hung!


-> to recover I disabled iovctl in rc.conf and rebooted


there is about 1Gbps of network traffic when we invoked these commands.

I found similar errors here:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221919
but I'm running a newer driver version than 1.9.9-k

I did not try to disable TSO.

Is there anything else I can provide you with to help solving this?
Comment 1 Finn 2018-11-19 21:50:09 UTC
The SR-IOV part of this was an error on my side

/etc/iov/ixl1.conf
---
PF {
	  device : "ixl1";
	  num_vfs : 2;
}

DEFAULT {
	  passthrough :	true;
}
--

new version with an VF for the host (passthrough: false)

---
PF {
	  device : "ixl1";
	  num_vfs : 3;
}

DEFAULT {
	  passthrough :	true;
}

#for the host:
VF-0 {
	passthrough : false;
}
--

this creates a ixlv0 which can be used by the host (ixl1 becomes unusable after 'service iovctl start')