Bug 206168 - Broadcom 57xx fails on any version of FreeBSD
Summary: Broadcom 57xx fails on any version of FreeBSD
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.2-RELEASE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-01-12 20:06 UTC by dave
Modified: 2022-06-02 02:20 UTC (History)
5 users (show)

See Also:


Attachments
console output for bge0 interface (40.66 KB, image/png)
2022-05-02 13:16 UTC, ferdinand.goldmann
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description dave 2016-01-12 20:06:52 UTC
This device:
bge0: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem 0xd50a0000-0xd50affff,0xd50b0000-0xd50bffff,0xd50c0000-0xd50cffff irq 34 at device 0.0 on pci1
bge0: APE FW version: NCSI v1.3.7.0
bge0: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E

Doesn't work.  For anyone I can find anyway.  It apparently initializes, it apparently is in the system, it apparently tries to work, but a cable connected to it fails to move any network packets.

I have it set at all defaults in the bios, tested with Freebsd 9, 10 and 11, with the exact same result.

To duplicate, find a Dell/HP server with a Broadcom 5720 card, install FreeBSD v.any, insert Ethernet Cable, boot it up and ifconfig returns "status: no carrier" and DHCP client says its not there. 

To check my sanity, I left everything in place, and booted to a Linux Mint thumbdrive, the card worked perfect (so I know the network is networking)
Comment 1 Bhargava Chenna. Marreddy 2019-03-21 10:27:08 UTC
Hi Dev,

>> and booted to a Linux Mint thumbdrive, the card worked
>> perfect (so I know the network is networking)
<Bhargava> Thank you for the experiment with Linux, this clearly indicates underlying firmware and hardware are healthy.

Can you please follow these steps and provide me the output files for further debugging.
 
Step #1: Reboot
Step #2: For all bge interfaces (i.e. bge0, bge1, bge2, bge3, etc..)
          ## ifconfig         > output_all.txt
          ## ifconfig -m bge0 > output_bge0.txt
          ## ifconfig -m bge1 > output_bge1.txt
          ## ifconfig -m bge2 > output_bge2.txt
          ## ifconfig -m bge3 > output_bge3.txt
          ## ifconfig -m bge4 > output_bge4.txt
         Please provide me all output files.
Step #3: Run following dhclent command for all bge interfaces.
          ## dhclient bge0
          ## dhclient bge1
          ## dhclient bge2
          ## dhclient bge3
          ## dhclient bge4
Step $4: Repeat Step #2 and see if any interface is in 'Link Up' state.

Thanks,
Bhargava Chenna, Marreddy
Comment 2 Eugene Grosbein freebsd_committer 2019-03-21 11:15:14 UTC
Please try adding "dev.bge.0.msi=0" to /boot/loader.conf and re-do the test.
Comment 3 Bhargava Chenna. Marreddy 2019-03-21 11:22:28 UTC
Thanks, Eugene Grosbein,

Looks like BZ #229727 talks also about watchdog timeouts.
Just to avoid confusion, can we try these Two suggestions separately one after the other.
Comment 4 Bhargava Chenna. Marreddy 2019-04-16 13:29:22 UTC
Hi Dev,

Did you get chance to try the steps as per comment #1?
Comment 5 Marek Zarychta 2019-04-16 15:17:19 UTC
It could be firmware incompatibility flaw. For a couple of motherboards with NetXtreme BCM5761 integrated NICs, these NICs haven't worked for me either. Only FreeBSD was affected. Symptoms were the same and hardware worked fine under Linux. Update of Broadcom Integrated NIC Firmware in my case has resolved the issue.

The exact hardware model was:

bge0@pci0:63:0:0:	class=0x020000 card=0x3047103c chip=0x168114e4 rev=0x10 hdr=0x00
    vendor     = 'Broadcom Inc. and subsidiaries'
    device     = 'NetXtreme BCM5761 Gigabit Ethernet PCIe'
    class      = network
    subclass   = ethernet
Comment 6 dave 2019-04-16 17:30:27 UTC
Revisiting this after 3 years...
Now using Freebsd 12
All 4 interfaces start and operate as would be expected.  Nothing has changed on the server since the extensive attempts to make it work 3 years ago (its mostly retired now) but all 4 ports are alive and well today.  I did try different hardware flashes, and gobs of different settings to try to make them work with no joy.

Could try with Freebsd 11 again I guess, but really, it works, do we care?
Comment 7 Eugene Grosbein freebsd_committer 2019-04-16 20:28:55 UTC
Submitter reports the problem's solved with update to FreeBSD 12.0
Comment 8 Kubilay Kocak freebsd_committer freebsd_triage 2019-04-16 23:23:51 UTC
Since we don't have a identified or definitive resolver (commit(s)), resolution: Overcome By Events

This is technically still an issue on FreeBSD 11.x, which is not EoL. If someone can either identify a set of commits OR PR's that need to be merged to stable/11 or can reproduce the issue on 11.x with the reported hardware, please re-open the issue with further details

While I'm here, correct 'Version', which was set for CURRENT (ambiguous), but the report was for 9.x-11.x
Comment 9 Eugene Grosbein freebsd_committer 2019-04-17 01:12:06 UTC
(In reply to Kubilay Kocak from comment #8)

Just to clarify: initial report dates back to January of 2016 while 11.0-RELEASE occured in October of 2016. It is unknown if 11.2-STABLE or 11.2-RELEASE have the problem and changes are bge(4) is already fixed there.
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2019-04-17 02:05:54 UTC
(In reply to Eugene Grosbein from comment #9)

Thanks Eugene, I'll re-enable earlier versions and set it to 10.2-RELEASE (Release date: 2015-08-06) so its... less incorrect.
Comment 11 dave 2019-04-17 22:35:15 UTC
This controller works fine with 11.2-RELEASE.  Tested with live CD.
Comment 12 Eugene Grosbein freebsd_committer 2019-04-17 23:23:37 UTC
Fixed in all supported branches.
Comment 13 ferdinand.goldmann 2022-05-02 13:14:42 UTC
This still (again?) does not work.

I got a new Dell PowerEdge R6515 with a Broadcom Gigabit Ethernet BCM5720 and can't get it to work. Cisco Switch says the port is connected. The iDRAC of the Dell server says the port is up with 1000Mbps as well. Just in FreeBSD I can't get the port to come up.

I also tried setting dev.bge.0.msi = 0 to no effect. It seems to be the same 0x5720 ASIC as in the original bug report.

To make sure the NIC is basically working I tried booting CentOS and was able to obtain a DHCP lease without problems.

I both tried booting with FreeBSD 12.3 as well as 13.0, no success.

Any ideas? Anybody using a Dell R6515? :-(
Comment 14 ferdinand.goldmann 2022-05-02 13:16:57 UTC
Created attachment 233662 [details]
console output for bge0 interface
Comment 15 Eugene Grosbein freebsd_committer 2022-05-02 13:36:44 UTC
(In reply to ferdinand.goldmann from comment #13)

Please create new PR, specify your FreeBSD version, kernel type (GENERIC or custom), describe hardware and do not disable msi, use defaults to show problem.