Bug 238540 - mlx5en(4): no link for MT27710 Family [ConnectX-4 Lx]
Summary: mlx5en(4): no link for MT27710 Family [ConnectX-4 Lx]
Status: Closed Works As Intended
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-RELEASE
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-06-13 09:13 UTC by Piotr Kubaj
Modified: 2019-07-05 12:08 UTC (History)
2 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Piotr Kubaj freebsd_committer freebsd_triage 2019-06-13 09:13:03 UTC 
I have a Huawei 2288H V5 server running FreeBSD 12.0-RELEASE.

This server has MT27710 Family [ConnectX-4 Lx] network cards and in dmesg I can see that FreeBSD can see they're plugged in:
mlx5_core0: INFO: Module 0, status: plugged and enabled
mlx5_core1: INFO: Module 1, status: plugged and enabled

The problem is that there's no link (status is no carrier):
mce0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWRXTSTMP>
        ether d0:ef:c1:71:9f:9d
        media: Ethernet autoselect <full-duplex,rxpause,txpause>
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
mce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWRXTSTMP>
        ether d0:ef:c1:71:9f:9e
        media: Ethernet autoselect <full-duplex,rxpause,txpause>
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

The same NIC's in the same network work when Ubuntu Linux (18.04) is used (no configuration changes required).

Status on Ubuntu:
Settings for eno6:
        Supported ports: [ Backplane ]
        Supported link modes:   1000baseKX/Full
                                10000baseKR/Full
                                25000baseCR/Full
                                25000baseKR/Full
                                25000baseSR/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  1000baseKX/Full
                                10000baseKR/Full
                                25000baseCR/Full
                                25000baseKR/Full
                                25000baseSR/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
Cannot get wake-on-lan settings: Operation not permitted
        Current message level: 0x00000004 (4)
                               link
        Link detected: yes

This fits reality, the NIC is connected to 10G port in switch.

I have also tried setting media manually to each of:
                media 1000baseT mediaopt full-duplex,rxpause,txpause
                media 10GBase-ER mediaopt full-duplex,rxpause,txpause
                media 10Gbase-SR mediaopt full-duplex,rxpause,txpause
                media 10GBase-CR1 mediaopt full-duplex,rxpause,txpause
                media 10GBase-KR mediaopt full-duplex,rxpause,txpause
                media 1000Base-KX mediaopt full-duplex,rxpause,txpause
                media 1000Base-CX-SGMII mediaopt full-duplex,rxpause,txpause

But it also didn't work.
Comment 1 Slava Shwartsman freebsd_committer freebsd_triage 2019-06-13 10:11:00 UTC 
Hi Piotr,

Thanks for your report.
Can you please share some more details? 
What FW version do you have on the HCA?
# sysctl dev.mlx5_core.0.hw.fw_version
Do you have any errors in logs?
Did you try to set the if up?
# ifconfig mce0 up


Slava
Comment 2 Piotr Kubaj freebsd_committer freebsd_triage 2019-07-05 12:08:30 UTC 
(In reply to Slava Shwartsman from comment #1)
<feels ashamed>

Yes, it looks like it was about that. Sorry for the wrong bug.