Created attachment 185963 [details]
some debuginfo from ipmi console
We have several INTEL-E10G42AFDA cards that worked well with Twinax cable, but driver fails to attach if 10G-SR SFP plugged.
The output from patched driver:
% pciconf -l | grep 0x0200
ix0@pci0:4:0:0: class=0x020000 card=0xa21f8086 chip=0x10f18086 rev=0x01 hdr=0x00
ix1@pci0:4:0:1: class=0x020000 card=0xa21f8086 chip=0x10f18086 rev=0x01 hdr=0x00
ix2@pci0:7:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
ix3@pci0:7:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
% grep ^ix /var/run/dmesg.boot
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x7020-0x703f mem 0xdf720000-0xdf73ffff,0xdf6c0000-0xdf6fffff,0xdf744000-0xdf747fff irq 32 at device 0.0 on pci4
ix0: Using MSIX interrupts with 8 vectors
ix0: Ethernet address: 00:1b:21:6c:e1:8f
ix0: PCI Express Bus: Speed 2.5GT/s Width x8
ix0: netmap queues/slots: TX 7/2048, RX 7/2048
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x7000-0x701f mem 0xdf700000-0xdf71ffff,0xdf640000-0xdf67ffff,0xdf740000-0xdf743fff irq 36 at device 0.1 on pci4
ix1: Using MSIX interrupts with 8 vectors
ix1: Ethernet address: 00:1b:21:6c:e1:8e
ix1: PCI Express Bus: Speed 2.5GT/s Width x8
ix1: netmap queues/slots: TX 7/2048, RX 7/2048
ix2: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x6020-0x603f mem 0xde680000-0xde6fffff,0xde704000-0xde707fff irq 42 at device 0.0 on pci7
ix2: Using MSIX interrupts with 8 vectors
ix2: Ethernet address: 90:e2:ba:2b:52:ac
ix2: PCI Express Bus: Speed 5.0GT/s Width x8
ix2: netmap queues/slots: TX 7/2048, RX 7/2048
ix3: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x6000-0x601f mem 0xde600000-0xde67ffff,0xde700000-0xde703fff irq 45 at device 0.1 on pci7
ix3: Using MSIX interrupts with 8 vectors
ix3: Ethernet address: 90:e2:ba:2b:52:ad
ix3: PCI Express Bus: Speed 5.0GT/s Width x8
ix3: netmap queues/slots: TX 7/2048, RX 7/2048
ix0: link state changed to UP
ix2: link state changed to UP
ix3: link state changed to UP
ix1: link state changed to UP
All cards use the same SFP modules
plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
vendor: Upnet PN: SFP+-10G-SR SN: SB3A680013 DATE: 2011-03-14
but 82598 card fails to attach it with message:
ix0: Unsupported SFP+ Module
device_attach: ix0 attach returned 5
I did some debugging to determine where the driver fails. The fail is in ixgbe_get_sfp_init_sequence_offsets() function. As I understand, there should be some values correspoinding to sfp_type, but in the table only one value, that doesn't match.
I modifed the driver that it thinks that SFP module has ixgbe_sfp_type_da_cu type, and after that it successful attached and works, but of course an information about SFP is unavailable:
% ifconfig -v ix0
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
inet6 fe80::21b:21ff:fe6c:e18f%ix0 prefixlen 64 scopeid 0x3
media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
Also there is another version of patch, that seems solves the same issue, but I didn't test it.
Created attachment 185964 [details]
Forgot to mention, in the loader.conf we have these tunables set, and this doesn't help:
Does this patch look right to you?
(In reply to Sean Bruno from comment #3)
I think the patch linked in the original comment might be a better choice than the attached one if it works, but ultimately, the decision on our side should be Jeb's -- I'll cc him.
For bugs matching the following conditions:
- Status == In Progress
- Assignee == "bugs@FreeBSD.org"
- Last Modified Year <= 2017
- Set Status to "Open"
E10G42AFDA does not support SFP+ optics:
SFP+ direct attached cable (twinaxial) only
SFP+ optics and active cables cannot be used with this adapter
Seems like a general issue with the intel driver.
It would be good to get this implemented:
Even better would be to have it permanently enabled without having to use the allow_unsupported_sfp setting.
I mean in what situation would you want your interfaces disappearing during boot because of this.
(In reply to Kristian from comment #7)
If by "issue" you mean that this adapter doesn't support SFP+ optics, I agree...I linked the specs in a previous post and was ignored.
(In reply to Jeff Pieper from comment #8)
Yes I saw the reference to that specific NIC.
By this issue I meant that this issue is also present on intel NICs that support SPF optics when the optic is not programmed with the intel eeprom flags.
Here is talk of a similar fix: