Created attachment 185963 [details] some debuginfo from ipmi console We have several INTEL-E10G42AFDA cards that worked well with Twinax cable, but driver fails to attach if 10G-SR SFP plugged. The output from patched driver: % pciconf -l | grep 0x0200 ix0@pci0:4:0:0: class=0x020000 card=0xa21f8086 chip=0x10f18086 rev=0x01 hdr=0x00 ix1@pci0:4:0:1: class=0x020000 card=0xa21f8086 chip=0x10f18086 rev=0x01 hdr=0x00 ix2@pci0:7:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 ix3@pci0:7:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 % grep ^ix /var/run/dmesg.boot ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x7020-0x703f mem 0xdf720000-0xdf73ffff,0xdf6c0000-0xdf6fffff,0xdf744000-0xdf747fff irq 32 at device 0.0 on pci4 ix0: Using MSIX interrupts with 8 vectors ix0: Ethernet address: 00:1b:21:6c:e1:8f ix0: PCI Express Bus: Speed 2.5GT/s Width x8 ix0: netmap queues/slots: TX 7/2048, RX 7/2048 ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x7000-0x701f mem 0xdf700000-0xdf71ffff,0xdf640000-0xdf67ffff,0xdf740000-0xdf743fff irq 36 at device 0.1 on pci4 ix1: Using MSIX interrupts with 8 vectors ix1: Ethernet address: 00:1b:21:6c:e1:8e ix1: PCI Express Bus: Speed 2.5GT/s Width x8 ix1: netmap queues/slots: TX 7/2048, RX 7/2048 ix2: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x6020-0x603f mem 0xde680000-0xde6fffff,0xde704000-0xde707fff irq 42 at device 0.0 on pci7 ix2: Using MSIX interrupts with 8 vectors ix2: Ethernet address: 90:e2:ba:2b:52:ac ix2: PCI Express Bus: Speed 5.0GT/s Width x8 ix2: netmap queues/slots: TX 7/2048, RX 7/2048 ix3: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.0> port 0x6000-0x601f mem 0xde600000-0xde67ffff,0xde700000-0xde703fff irq 45 at device 0.1 on pci7 ix3: Using MSIX interrupts with 8 vectors ix3: Ethernet address: 90:e2:ba:2b:52:ad ix3: PCI Express Bus: Speed 5.0GT/s Width x8 ix3: netmap queues/slots: TX 7/2048, RX 7/2048 ix0: link state changed to UP ix2: link state changed to UP ix3: link state changed to UP ix1: link state changed to UP All cards use the same SFP modules plugged: SFP/SFP+/SFP28 10G Base-SR (LC) vendor: Upnet PN: SFP+-10G-SR SN: SB3A680013 DATE: 2011-03-14 but 82598 card fails to attach it with message: ix0: Unsupported SFP+ Module device_attach: ix0 attach returned 5 I did some debugging to determine where the driver fails. The fail is in ixgbe_get_sfp_init_sequence_offsets() function. As I understand, there should be some values correspoinding to sfp_type, but in the table only one value, that doesn't match. I modifed the driver that it thinks that SFP module has ixgbe_sfp_type_da_cu type, and after that it successful attached and works, but of course an information about SFP is unavailable: % ifconfig -v ix0 ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=8100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER> ether 00:1b:21:6c:e1:8f inet6 fe80::21b:21ff:fe6c:e18f%ix0 prefixlen 64 scopeid 0x3 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>) status: active Also there is another version of patch, that seems solves the same issue, but I didn't test it. http://www.grosbein.net/freebsd/patches/patch-if_ix.c
Created attachment 185964 [details] used patch
Forgot to mention, in the loader.conf we have these tunables set, and this doesn't help: hw.ixgbe.unsupported_sfp="1" hw.ix.unsupported_sfp="1"
Eric: Does this patch look right to you?
(In reply to Sean Bruno from comment #3) I think the patch linked in the original comment might be a better choice than the attached one if it works, but ultimately, the decision on our side should be Jeb's -- I'll cc him.
For bugs matching the following conditions: - Status == In Progress - Assignee == "bugs@FreeBSD.org" - Last Modified Year <= 2017 Do - Set Status to "Open"
E10G42AFDA does not support SFP+ optics: https://www.intel.com/content/dam/doc/product-brief/10-gigabit-af-da-dual-port-server-adapter-brief.pdf Under specifications: Cabling (passive) SFP+ direct attached cable (twinaxial) only SFP+ optics and active cables cannot be used with this adapter
Seems like a general issue with the intel driver. It would be good to get this implemented: http://www.grosbein.net/freebsd/patches/patch-if_ix.c Even better would be to have it permanently enabled without having to use the allow_unsupported_sfp setting. I mean in what situation would you want your interfaces disappearing during boot because of this.
(In reply to Kristian from comment #7) If by "issue" you mean that this adapter doesn't support SFP+ optics, I agree...I linked the specs in a previous post and was ignored.
(In reply to Jeff Pieper from comment #8) Yes I saw the reference to that specific NIC. By this issue I meant that this issue is also present on intel NICs that support SPF optics when the optic is not programmed with the intel eeprom flags. Here is talk of a similar fix: https://sourceforge.net/p/e1000/mailman/message/28694855/
^Triage: reassign to pool by assignee request. To submitter: is this PR still valid?
(In reply to Andrey V. Elsukov from comment #2) Can you test the patch in comment #7 or is this two distinct problems?
(In reply to Kevin Bowling from comment #11) Sorry, but I'm unable to test this. We have migrated to Mellanox cards.
Intel doesn't support optics in the card. Without an 82598 AF to test it is unclear if there is a clean way of using the sanctioned 'unsupported SFP' sysctl, there are many places that need to be inspected. I'm not interested enough in this card to buy one on eBay with my own money so closing as OBE.