Hello, I am using the following system: 540-BCKU : Broadcom 57414 Dual-Port 25GbE SFP28 LOM Mezz Card 405-AAMS : PERC H740P RAID Controller, 8G B NV Cache, Mini card 400-BDSS : 240GB SSD SATA Mixed Use 6Gbps 512e 2.5in Hot-Plug S4610 Dri ve 400-AVEZ : 2.4TB 10K RPM SAS 12Gbps 512e 2.5in Hot-plug Hard Drive 384-BCJD : PowerEdge R7515 Motherboard, with 2 x 1Gb Onboard LOM (BCM57 20) 379-BDSS : SAS/SATA Backplane 370-AEVR : 3200MT/s RDIMMs 370-AEVQ : 16GB RDIMM, 3200MT/s, Dual Ran k 338-BSWO : AMD 7302P 3GHz,16C/32T,128M,15 5W,3200 321-BERX : 2.5" Chassis with up to 24 Hot Plug Hard Drives 210-ASVQ : PowerEdge R7515 Server The network adapter at the top is non-functional in the 12.1 release. I am able to check for the Broadcom 57414 drivers via: >>kldstat | grep if_bnxt It returns the memory space loaded and then also shows it is functional. If I try to see the device or bring it online it does not show nor work: >>ifconfig does not show bnxt0 or bnxt1 >>ifconfig bnxt0 up does not enable or show the device If I use lspci | grep NetX it does show the PCI device as being loaded and lists details within pciconf if I map it to the device ID of pci02:00:0 But if I force unload the if_bnxt and then reload: >>kldunload if_bnxt >>kldload /boot/kernel/if_bnxt.ko It states it fails due to " bus_dma_tag_create failed return: 22" Am I doing something wrong or?
Sometimes when force unloading and reloading I do have a kernel panic that happens. https://i.imgur.com/COwfFZY.png
Can you post dmesg and the pciconf?
Created attachment 215503 [details] dmesg for ticket
truenas# pciconf -l -BbcevV bge0 bge0@pci0:193:0:0: class=0x020000 card=0x08fd1028 chip=0x165f14e4 rev=0x00 hdr=0x00 vendor = 'Broadcom Inc. and subsidiaries' device = 'NetXtreme BCM5720 2-port Gigabit Ethernet PCIe' class = network subclass = ethernet bar [10] = type Prefetchable Memory, range 64, base rxc7030000, size 65536, enabled bar [18] = type Prefetchable Memory, range 64, base rxc7040000, size 65536, enabled bar [20] = type Prefetchable Memory, range 64, base rxc7050000, size 65536, enabled cap 01[48] = powerspec 3 supports D0 D3 current D0 cap 03[50] = VPD cap 05[58] = MSI supports 8 messages, 64 bit enabled with 1 message cap 11[a0] = MSI-X supports 17 messages Table in map 0x20[0x0], PBA in map 0x20[0x1000] cap 10[ac] = PCI-Express 2 endpoint max data 128(256) FLR link x1(x2) speed 5.0(5.0) ClockPM disabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[13c] = Serial 1 00003448ede7ee1a ecap 0004[150] = Power Budgeting 1 ecap 0002[160] = VC 1 max VC0 Corrected = Advisory Non-Fatal Error VPD ident = 'Broadcom NetXtreme Gigabit Ethernet' VPD ro PN = 'BCM95720' VPD ro MN = '1028' VPD ro V0 = 'FFV21.60.2' VPD ro V1 = 'DSV1028VPDR.VER1.0' VPD ro V2 = 'NPY2' VPD ro V3 = 'PMT1' VPD ro V4 = 'NMVBroadcom Corp' VPD ro V5 = 'DTINIC' VPD ro V6 = 'DCM1001008d452101008d45' truenas# clear truenas# pciconf -l -BbcevV bge1 bge1@pci0:193:0:1: class=0x020000 card=0x08fd1028 chip=0x165f14e4 rev=0x00 hdr=0x00 vendor = 'Broadcom Inc. and subsidiaries' device = 'NetXtreme BCM5720 2-port Gigabit Ethernet PCIe' class = network subclass = ethernet bar [10] = type Prefetchable Memory, range 64, base rxc7000000, size 65536, enabled bar [18] = type Prefetchable Memory, range 64, base rxc7010000, size 65536, enabled bar [20] = type Prefetchable Memory, range 64, base rxc7020000, size 65536, enabled cap 01[48] = powerspec 3 supports D0 D3 current D0 cap 03[50] = VPD cap 05[58] = MSI supports 8 messages, 64 bit enabled with 1 message cap 11[a0] = MSI-X supports 17 messages Table in map 0x20[0x0], PBA in map 0x20[0x1000] cap 10[ac] = PCI-Express 2 endpoint max data 128(256) FLR link x1(x2) speed 5.0(5.0) ClockPM disabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[13c] = Serial 1 00003448ede7ee1b ecap 0004[150] = Power Budgeting 1 ecap 0002[160] = VC 1 max VC0 Corrected = Advisory Non-Fatal Error VPD ident = 'Broadcom NetXtreme Gigabit Ethernet' VPD ro PN = 'BCM95720' VPD ro MN = '1028' VPD ro V0 = 'FFV21.60.2' VPD ro V1 = 'DSV1028VPDR.VER1.0' VPD ro V2 = 'NPY2' VPD ro V3 = 'PMT1' VPD ro V4 = 'NMVBroadcom Corp' VPD ro V5 = 'DTINIC' VPD ro V6 = 'DCM1001008d452101008d45' truenas# pciconf -l -BbcevV bge0 bge0@pci0:193:0:0: class=0x020000 card=0x08fd1028 chip=0x165f14e4 rev=0x00 hdr=0x00 vendor = 'Broadcom Inc. and subsidiaries' device = 'NetXtreme BCM5720 2-port Gigabit Ethernet PCIe' class = network subclass = ethernet bar [10] = type Prefetchable Memory, range 64, base rxc7030000, size 65536, enabled bar [18] = type Prefetchable Memory, range 64, base rxc7040000, size 65536, enabled bar [20] = type Prefetchable Memory, range 64, base rxc7050000, size 65536, enabled cap 01[48] = powerspec 3 supports D0 D3 current D0 cap 03[50] = VPD cap 05[58] = MSI supports 8 messages, 64 bit enabled with 1 message cap 11[a0] = MSI-X supports 17 messages Table in map 0x20[0x0], PBA in map 0x20[0x1000] cap 10[ac] = PCI-Express 2 endpoint max data 128(256) FLR link x1(x2) speed 5.0(5.0) ClockPM disabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[13c] = Serial 1 00003448ede7ee1a ecap 0004[150] = Power Budgeting 1 ecap 0002[160] = VC 1 max VC0 Corrected = Advisory Non-Fatal Error VPD ident = 'Broadcom NetXtreme Gigabit Ethernet' VPD ro PN = 'BCM95720' VPD ro MN = '1028' VPD ro V0 = 'FFV21.60.2' VPD ro V1 = 'DSV1028VPDR.VER1.0' VPD ro V2 = 'NPY2' VPD ro V3 = 'PMT1' VPD ro V4 = 'NMVBroadcom Corp' VPD ro V5 = 'DTINIC' VPD ro V6 = 'DCM1001008d452101008d45'
pciconf should have been for bnxt not bge.
Not sure how to obtain that, does not like pciconf bnxt0 or 1
If you do pciconf -l -BbcevV it should be somewhere in the long list. If you can't get it though that's probably okay, there were some pretty good clues in the dmesg log already.
bnxt0: <Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Ethernet> mem 0xe1510000-0xe151ffff,0xe1400000-0xe14fffff,0xe1522000-0xe1523fff irq 54 at device 0.0 numa-domain 7 on pci1 bnxt0: Using 256 TX descriptors and 256 RX descriptors bnxt0: Using 0 RX queues 0 TX queues bnxt0: Using MSI-X interrupts with 1 vectors bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 bnxt0: Unable to allocate device TX queue bnxt0: Unable to allocate queue memory device_attach: bnxt0 attach returned 22 For some reason either the firmware is not reporting the number of queues correctly, or something else is causing the number of queues to be forced to 0. Unfortunately I don't have access to hardware for debugging this myself.
Testing within Ubuntu 18.04 LTS is mapping the queues correctly, although it is loading the driver modules within the kernel for FreeBSD it does not allow for mapping it? Is there any additional steps we can take to identify or solve this allocation of queues?
I have this problem too. This happened after updating the dell firmware to the latest version. BNXT interface disappeared and I see an error in dmesg: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet>mfi0: mem 0x9da10000-0x9da1ffff,0x9d900000-0x9d9fffff,0x9da22000-0x9da23fff irq 34 at device 0.0 numa-domain 0 on pci6 2523 (boot + 26s/0x0002/info) - Inserted: PD 03(e0x20/s3) Info: enclPd=20, scsiType=0, portMap=01, sasAddr=5000cca02f519775,0000000000000000 mfi0: 2524 (boot + 59s/0x0020/info) - Controller operating temperature within normal range, full operation restored bnxt0: mfi0: 2525 (678537358s/0x0020/info) - Time established as 07/02/21 10:35:58; (60 seconds since power on) mfi0: 2526 (678537426s/0x0020/info) - Host driver is loaded and operational using 256 tx descriptors and 256 rx descriptors bnxt0: msix_init qsets capped at 0 bnxt0: pxm cpus: 12 queue msgs: 73 admincnt: 1 bnxt0: using 0 rx queues 0 tx queues bnxt0: Using MSI-X interrupts with 1 vectors bnxt0: iflib_dma_alloc: bus_dma_tag_create failed: 22 bnxt0: device queue allocation failed bnxt0: Unable to allocate queue memory device_attach: bnxt0 attach returned 22
Likely the same issue. Fresh install of FreeBSD 13.0 on Dell PowerEdge T640. ~50% chance of panics after reboot: Fatal trap 12: page fault while in kernel mode cpuid = 11; apic id = 0b fault virtual address = 0x81 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80d1da85 stack pointer = 0x28:0xfffffe00e276a060 frame pointer = 0x28:0xfffffe00e276a150 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 327CSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 (ifconfig)ups: lo trap number = 12 panic: page fault cpuid = 11 time = 1625402939 KDB: stack backtrace: #0 0xffffffff80c57345 at kdb_backtrace+0x65 #1 0xffffffff80c09d21 at vpanic+0x181 #2 0xffffffff80c09b93 at panic+0x43 #3 0xffffffff8108b187 at trap_fatal+0x387 #4 0xffffffff8108b1df at trap_pfault+0x4f #5 0xffffffff8108a83d at trap+0x27d #6 0xffffffff810617a8 at calltrap+0x8 #7 0xffffffff80d1f30c at ifioctl+0x50c #8 0xffffffff80c76ced at kern_ioctl+0x26d #9 0xffffffff80c769e6 at sys_ioctl+0xf6 #10 0xffffffff8108ba8c at amd64_syscall+0x10c #11 0xffffffff810620ce at fast_syscall_common+0xf8 Fatal trap 12: page fault while in kernel mode cpuid = 4; apic id = 04 fault virtual address = 0x81 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80d1da85 stack pointer = 0x28:0xfffffe014267c060 frame pointer = 0x28:0xfffffe014267c150 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current procCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::ess = 337 (ifconfig) trap number = 12 panic: page fault cpuid = 4 time = 1625402828 KDB: stack backtrace: #0 0xffffffff80c57345 at kdb_backtrace+0x65 #1 0xffffffff80c09d21 at vpanic+0x181 #2 0xffffffff80c09b93 at panic+0x43 #3 0xffffffff8108b187 at trap_fatal+0x387 #4 0xffffffff8108b1df at trap_pfault+0x4f #5 0xffffffff8108a83d at trap+0x27d #6 0xffffffff810617a8 at calltrap+0x8 #7 0xffffffff80d1f30c at ifioctl+0x50c #8 0xffffffff80c76ced at kern_ioctl+0x26d #9 0xffffffff80c769e6 at sys_ioctl+0xf6 #10 0xffffffff8108ba8c at amd64_syscall+0x10c #11 0xffffffff810620ce at fast_syscall_common+0xf8 Uptime: 20s dmesg output (only bnxt related): Jul 4 17:22:55 trunk kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xab210000-0xab21ffff,0xab100000-0xab1fffff,0xab222000-0xab223fff at device 0.0 numa-domain 0 on pci5 Jul 4 17:22:55 trunk kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors Jul 4 17:22:55 trunk kernel: bnxt0: Using 0 RX queues 0 TX queues Jul 4 17:22:55 trunk kernel: bnxt0: Using MSI-X interrupts with 1 vectors Jul 4 17:22:55 trunk kernel: bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 Jul 4 17:22:55 trunk kernel: bnxt0: Unable to allocate device TX queue Jul 4 17:22:55 trunk kernel: bnxt0: Unable to allocate queue memory Jul 4 17:22:55 trunk kernel: device_attach: bnxt0 attach returned 22 Jul 4 17:22:55 trunk kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xab200000-0xab20ffff,0xab000000-0xab0fffff,0xab220000-0xab221fff at device 0.1 numa-domain 0 on pci5 Jul 4 17:22:55 trunk kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors Jul 4 17:22:55 trunk kernel: bnxt0: Using 0 RX queues 0 TX queues Jul 4 17:22:55 trunk kernel: bnxt0: Using MSI-X interrupts with 1 vectors Jul 4 17:22:55 trunk kernel: bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 Jul 4 17:22:55 trunk kernel: bnxt0: Unable to allocate device TX queue Jul 4 17:22:55 trunk kernel: bnxt0: Unable to allocate queue memory Jul 4 17:22:55 trunk kernel: device_attach: bnxt0 attach returned 22 Jul 4 17:22:55 trunk kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xab210000-0xab21ffff,0xab100000-0xab1fffff,0xab222000-0xab223fff at device 0.0 numa-domain 0 on pci5 Jul 4 17:22:55 trunk kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors Jul 4 17:22:55 trunk kernel: bnxt0: Using 8 RX queues 8 TX queues Jul 4 17:22:55 trunk kernel: bnxt0: Using MSI-X interrupts with 9 vectors Jul 4 17:22:55 trunk kernel: bnxt0: Ethernet address: b0:7b:25:c8:6d:b6 Jul 4 17:22:55 trunk kernel: bnxt0: netmap queues/slots: TX 8/256, RX 8/256 Jul 4 17:22:55 trunk kernel: bnxt1: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xab200000-0xab20ffff,0xab000000-0xab0fffff,0xab220000-0xab221fff at device 0.1 numa-domain 0 on pci5 Jul 4 17:22:55 trunk kernel: bnxt1: Using 256 TX descriptors and 256 RX descriptors Jul 4 17:22:55 trunk kernel: bnxt1: Using 8 RX queues 8 TX queues Jul 4 17:22:55 trunk kernel: bnxt1: Using MSI-X interrupts with 9 vectors Jul 4 17:22:55 trunk kernel: bnxt1: Ethernet address: b0:7b:25:c8:6d:b7 Jul 4 17:22:55 trunk kernel: bnxt1: netmap queues/slots: TX 8/256, RX 8/256 Jul 4 17:22:55 trunk kernel: bnxt0: Link is UP full duplex, FC - receive & transmit - 1000 Mbps Jul 4 17:22:55 trunk kernel: bnxt0: link state changed to UP I've added to /boot/loader.conf: dev.bnxt.0.iflib.override_nrxqs="8" dev.bnxt.0.iflib.override_ntxqs="8" dev.bnxt.1.iflib.override_nrxqs="8" dev.bnxt.1.iflib.override_ntxqs="8" dmesg afterwards (errors disapeared): Jul 4 17:31:09 trunk kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xab210000-0xab21ffff,0xab100000-0xab1fffff,0xab222000-0xab223fff at device 0.0 numa-domain 0 on pci5 Jul 4 17:31:09 trunk kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors Jul 4 17:31:09 trunk kernel: bnxt0: Using 8 RX queues 8 TX queues Jul 4 17:31:09 trunk kernel: bnxt0: Using MSI-X interrupts with 9 vectors Jul 4 17:31:09 trunk kernel: bnxt0: Ethernet address: b0:7b:25:c8:6d:b6 Jul 4 17:31:09 trunk kernel: bnxt0: netmap queues/slots: TX 8/256, RX 8/256 Jul 4 17:31:09 trunk kernel: bnxt1: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xab200000-0xab20ffff,0xab000000-0xab0fffff,0xab220000-0xab221fff at device 0.1 numa-domain 0 on pci5 Jul 4 17:31:09 trunk kernel: bnxt1: Using 256 TX descriptors and 256 RX descriptors Jul 4 17:31:09 trunk kernel: bnxt1: Using 8 RX queues 8 TX queues Jul 4 17:31:09 trunk kernel: bnxt1: Using MSI-X interrupts with 9 vectors Jul 4 17:31:09 trunk kernel: bnxt1: Ethernet address: b0:7b:25:c8:6d:b7 Jul 4 17:31:09 trunk kernel: bnxt1: netmap queues/slots: TX 8/256, RX 8/256 Jul 4 17:31:09 trunk kernel: bnxt0: Link is UP full duplex, FC - receive & transmit - 1000 Mbps Jul 4 17:31:09 trunk kernel: bnxt0: link state changed to UP
thanks @kestas, same issue on 12.2 and overriding queue sizes resolves the issue!
The same here, after upgrading HP-supplied FW (from 216.x to 218.x) the interface stopped working with: bnxt0: <Broadcom BCM57412 NetXtreme-E 10Gb Ethernet> mem 0xc4210000-0xc421ffff,0xc4100000-0xc41fffff,0xc4222000-0xc4223fff at device 0.0 numa-domain 0 on pci11 bnxt0: Using 256 TX descriptors and 256 RX descriptors bnxt0: Using 0 RX queues 0 TX queues bnxt0: Using MSI-X interrupts with 1 vectors bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 bnxt0: Unable to allocate device TX queue bnxt0: Unable to allocate queue memory device_attach: bnxt0 attach returned 22 Warning: failed attempt to remove oid fc with child autoneg sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%desc)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%driver)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%location)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%pnpinfo)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%parent)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%domain)! bnxt0: <Broadcom BCM57412 NetXtreme-E 10Gb Ethernet> mem 0xc4200000-0xc420ffff,0xc4000000-0xc40fffff,0xc4220000-0xc4221fff at device 0.1 numa-domain 0 on pci11 sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.driver_version)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxqs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxqs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_qs_enable)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.disable_msix)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.rx_budget)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.tx_abdicate)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.core_offset)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.separate_txrx)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.use_logical_cores)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxds)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxds)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.mfg_id)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.device_id)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.sector_size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.reserved_size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.available_size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_key)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_type)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rx_stall)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.vlan_strip)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.if_name)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.enable)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.gro_mode)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_agg_segs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_aggs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.min_agg_len)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.tx)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.rx)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.autoneg)! bnxt0: Using 256 TX descriptors and 256 RX descriptors bnxt0: Using 0 RX queues 0 TX queues bnxt0: Using MSI-X interrupts with 1 vectors bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 bnxt0: Unable to allocate device TX queue bnxt0: Unable to allocate queue memory device_attach: bnxt0 attach returned 22 After adding the boot options, it works again.
+1, bit us at the dayjob on a recent firmware update. Would be good to get this fixed in base. I'm available to test fixes, though this card is only on one of our machines, not across the fleet.
Can people [1] who have reported reproduction of this issue: - if the details are still known, confirm what the firmware version was prior to the issue, and the firmware version that first (for them), introduced the issue, for them. - detail the exact freebsd version (uname -a) they are reproducible with, and additionally, include the 'earliest' version the same hardware the issue was reproducible with. We currently have the reports at 12.1 release, 12.2 and 13.0. Ideas for isolation / resolution - If someone could review the changelog for Linux driver for the same hardware looking for potentially relevent commits/updates around the time of the firmware updates, or mentioning firmware revisions. - Reproduction against latest stable/13 and CURRENT (provide exact versions please) [1] One report ("from 216.x to 218.x") in comment 13 ^Triage: Request feedback from a number of folks who have touched or referenced bnxt/tx/rx in the recent past
Just checked my iDrac, and it conveniently shows the prior version with a rollback option. Current 21.85.21.91 Prior 21.60.22.11 Note that we're running on 12.3-RELEASE-p1 FreeBSD 12.3-RELEASE-p1 GENERIC amd64 We have the ability to remote-boot an ISO. I could try with 13, if you feel that would be helpful.
On 13.0 firmware was 21.80.16.92 (haven't tried any other firmware versions). I'm not 100% sure, but I think it was 13.0-RELEASE (updated since then to 13.0-RELEASE-p6, but didn't tried to reproduce the bug). With applied iflib.override_* as reported before I don't see bnxt0 errors.
I am also getting this error: bnxt0: Using 256 TX descriptors and 256 RX descriptors bnxt0: Using 0 RX queues 0 TX queues bnxt0: Using MSI-X interrupts with 1 vectors bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 bnxt0: Unable to allocate device TX queue bnxt0: Unable to allocate queue memory device_attach: bnxt0 attach returned 22 This FreeBSD 13.0 server has no issues and is using the BCM57416 card. Firmware for the network card is 21.85.21.91 and was installed when I got the server. bnxt0@pci0:25:0:0: class=0x020000 rev=0x01 hdr=0x00 vendor=0x14e4 device=0x16d8 subvendor=0x14e4 subdevice=0x4160 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet bnxt1@pci0:25:0:1: class=0x020000 rev=0x01 hdr=0x00 vendor=0x14e4 device=0x16d8 subvendor=0x14e4 subdevice=0x4160 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet FreeBSD 12.2 server using the BCM57416 card and firmware version 21.65.33.33 works fine. Updated to 21.85.21.92 and I get the 'returned 22 error'. Rolled the firmware back and things are working again. bnxt0@pci0:25:0:0: class=0x020000 rev=0x01 hdr=0x00 vendor=0x14e4 device=0x16d8 subvendor=0x14e4 subdevice=0x4160 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet bnxt1@pci0:25:0:1: class=0x020000 rev=0x01 hdr=0x00 vendor=0x14e4 device=0x16d8 subvendor=0x14e4 subdevice=0x4160 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet I can upgrade the 12.2 server to 12.3 or 13.0 and keep the firmware on .33 then upgrade to .92 as a test.
Updated my 12.2 server with NIC firmware 21.65.33.33 to FreeBSD 12.3. Upgraded just fine and no issues with the card. After the OS upgrade was done I updated the NIC firmware to 21.85.21.92 and things are still working as expected, no issues.
Upgraded another server from FreeBSD 12.2 to 12.3, then upgraded the firmware from 21.65.33.33 to 21.85.21.92. This time the bnxt interfaces failed to show up when the server booted but module was loaded in the kernel. I unloaded and reloaded if_bnxt.ko and kernel panicked the server. After the reboot while freebsd was spinning up the server halted with sysctl_warn_reuse errors and a hard reset we needed. Rolled the firmware back to 21.65.33.33 to get the server working again.
I have a test box (13.0p10) with a dual BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller. https://freefall.freebsd.org/~pi/bnxt-pciconf.txt and https://freefall.freebsd.org/~pi/bnxt-sysctl.txt The NIC does not work 8-(, which is sad 8-( I tried several of the things in this PR, still fails. How do I find the firmware installed ?
Hi everyone, Just wanted to add that this seems to be fixed in 13.1 (RC1). Not sure if will be mentioned on release notes. I have tested today with: bnxt0: Using 256 TX descriptors and 256 RX descriptors bnxt0: Using 16 RX queues 16 TX queues bnxt0: Using MSI-X interrupts with 17 vectors bnxt0: Ethernet address: ac:1f:6b:e7:9f:0e bnxt0: netmap queues/slots: TX 16/256, RX 16/256 bnxt1: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0x18020f00000-0x18020f0ffff,0x18020d00000-0x18020dfffff,0x18020f20000-0x18020f21fff irq 31 at device 0.1 on pci7 bnxt1: Using 256 TX descriptors and 256 RX descriptors bnxt1: Using 16 RX queues 16 TX queues bnxt1: Using MSI-X interrupts with 17 vectors bnxt1: Ethernet address: ac:1f:6b:e7:9f:0f bnxt1: netmap queues/slots: TX 16/256, RX 16/256 bnxt0@pci0:199:0:0: class=0x020000 rev=0x01 hdr=0x00 vendor=0x14e4 device=0x16d8 subvendor=0x15d9 subdevice=0x16d8 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet bnxt1@pci0:199:0:1: class=0x020000 rev=0x01 hdr=0x00 vendor=0x14e4 device=0x16d8 subvendor=0x15d9 subdevice=0x16d8 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet root@hyper02:/home/smartinez # freebsd-version -uk 13.1-RC1 13.1-RC1 bnxt0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: dd=ANK-MGMT-SW01;di=ge-0/0/8 options=4e524bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether ac:1f:6b:e7:9f:0e media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> root@hyper02:/home/smartinez # ifconfig bnxt0.1011 bnxt0.1011: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: BR-DCN-PF options=4600403<RXCSUM,TXCSUM,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether ac:1f:6b:e7:9f:0e groups: vlan vlan: 1011 vlanproto: 802.1q vlanpcp: 0 parent interface: bnxt0 media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>) status: active nd6 options=9<PERFORMNUD,IFDISABLED> root@hyper02:/home/smartinez # root@hyper02:/home/smartinez # ping 192.168.11.1 PING 192.168.11.1 (192.168.11.1): 56 data bytes 64 bytes from 192.168.11.1: icmp_seq=0 ttl=64 time=0.184 ms
(In reply to Kurt Jaeger from comment #21) dev.bnxt.0.ver.hwrm_if: 1.10.1 dev.bnxt.0.ver.hwrm_fw: 214.4.91 see also https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236983 vlans do not work on 13.1-RC1 without ifconfig promisc
Experiencing issues with the BCM57416 on new hardware, with a fresh install of FreeBSD 13.0. Firmware version shipped with card: 22.00.07.60 % uname -a FreeBSD foo 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 On running `kldload /boot/kernel/if_bnxt.ko`, the following appears in /var/log/messages: May 19 10:37:17 <kern.crit> foo kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xb8a10000-0xb8a1ffff,0xb8900000-0xb89fffff,0xb8a22000-0xb8a23fff at device 0.0 numa-domain 0 on pci9 May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Using 0 RX queues 0 TX queues May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Using MSI-X interrupts with 1 vectors May 19 10:37:17 <kern.crit> foo kernel: bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Unable to allocate device TX queue May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Unable to allocate queue memory May 19 10:37:17 <kern.crit> foo kernel: device_attach: bnxt0 attach returned 22 May 19 10:37:17 <kern.crit> foo kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xb8a00000-0xb8a0ffff,0xb8800000-0xb88fffff,0xb8a20000-0xb8a21fff at device 0.1 numa-domain 0 on pci9 May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Using 0 RX queues 0 TX queues May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Using MSI-X interrupts with 1 vectors May 19 10:37:17 <kern.crit> foo kernel: bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Unable to allocate device TX queue May 19 10:37:17 <kern.crit> foo kernel: bnxt0: Unable to allocate queue memory May 19 10:37:17 <kern.crit> foo kernel: device_attach: bnxt0 attach returned 22 % pciconf -l -BbcevV ... vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet bar [10] = type Prefetchable Memory, range 64, base rxb8a10000, size 65536, enabled bar [18] = type Prefetchable Memory, range 64, base rxb8900000, size 1048576, enabled bar [20] = type Prefetchable Memory, range 64, base rxb8a22000, size 8192, enabled cap 01[48] = powerspec 3 supports D0 D3 current D0 cap 03[50] = VPD cap 05[58] = MSI supports 8 messages, 64 bit cap 11[a0] = MSI-X supports 74 messages Table in map 0x20[0x0], PBA in map 0x20[0x4a0] cap 10[ac] = PCI-Express 2 endpoint max data 256(512) FLR RO NS max read 512 link x8(x8) speed 8.0(8.0) ClockPM disabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[13c] = Serial 1 f4ee08fffe25fabd ecap 0004[150] = Power Budgeting 1 ecap 0002[160] = VC 1 max VC0 ecap 000b[180] = Vendor [1] ID 0000 Rev 0 Length 32 ecap 0018[1b0] = LTR 1 ecap 000e[1b8] = ARI 1 ecap 0017[230] = TPH Requester 1 ecap 0019[300] = PCIe Sec 1 lane errors 0 ecap 001f[200] = Precision Time Measurement 1 PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error VPD ident = 'Broadcom Adv. Dual 10GBASE-T Ethernet' VPD ro PN = 'BCM957416' VPD ro MN = '1028' VPD ro V0 = 'FFV22.00.07.60' VPD ro V1 = 'DSV1028VPDR.VER2.1' VPD ro V2 = 'NPY2' VPD ro V3 = 'PMT1' VPD ro V4 = 'NMVBroadcom Corp' VPD ro V5 = 'DTINIC' VPD ro V6 = 'DCM1001FFFFFF1202FFFFFF1403FFFFFF1604FFFFFF1805FFFFFF1A06FFFFFF1C07FFFFFF1E08FFFFFF2101FFFFFF2302FFFFFF2503FFFFFF2704FFFFFF2905FFFFFF2B06FFFFFF2D07FFFFFF2F08FFFFFF' VPD ro V7 = 'L1D0' none89@pci0:94:0:1: class=0x020000 rev=0x01 hdr=0x00 vendor=0x14e4 device=0x16d8 subvendor=0x1028 subdevice=0x1fea vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller' class = network subclass = ethernet bar [10] = type Prefetchable Memory, range 64, base rxb8a00000, size 65536, enabled bar [18] = type Prefetchable Memory, range 64, base rxb8800000, size 1048576, enabled bar [20] = type Prefetchable Memory, range 64, base rxb8a20000, size 8192, enabled cap 01[48] = powerspec 3 supports D0 D3 current D0 cap 03[50] = VPD cap 05[58] = MSI supports 8 messages, 64 bit cap 11[a0] = MSI-X supports 74 messages Table in map 0x20[0x0], PBA in map 0x20[0x4a0] cap 10[ac] = PCI-Express 2 endpoint max data 256(512) FLR RO NS max read 512 link x8(x8) speed 8.0(8.0) ClockPM disabled ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[13c] = Serial 1 f4ee08fffe25fabd ecap 0004[150] = Power Budgeting 1 ecap 000b[180] = Vendor [1] ID 0000 Rev 0 Length 32 ecap 000e[1b8] = ARI 1 ecap 0017[230] = TPH Requester 1 PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error VPD ident = 'Broadcom Adv. Dual 10GBASE-T Ethernet' VPD ro PN = 'BCM957416' VPD ro MN = '1028' VPD ro V0 = 'FFV22.00.07.60' VPD ro V1 = 'DSV1028VPDR.VER2.1' VPD ro V2 = 'NPY2' VPD ro V3 = 'PMT1' VPD ro V4 = 'NMVBroadcom Corp' VPD ro V5 = 'DTINIC' VPD ro V6 = 'DCM1001FFFFFF1202FFFFFF1403FFFFFF1604FFFFFF1805FFFFFF1A06FFFFFF1C07FFFFFF1E08FFFFFF2101FFFFFF2302FFFFFF2503FFFFFF2704FFFFFF2905FFFFFF2B06FFFFFF2D07FFFFFF2F08FFFFFF' VPD ro V7 = 'L1D0'
(In reply to SJ from comment #24) Can you update to 13.1, as it contains some fixes for bnxt ?
(In reply to Kurt Jaeger from comment #25) Done! % uname -a FreeBSD foo 13.1-RELEASE FreeBSD 13.1-RELEASE % kldstat Id Refs Address Size Name 1 27 0xffffffff80200000 1f30590 kernel 2 1 0xffffffff82131000 5ec1d8 zfs.ko 3 1 0xffffffff8271e000 af68 cryptodev.ko 4 1 0xffffffff82c11000 3378 acpi_wmi.ko 5 1 0xffffffff82c15000 3250 ichsmb.ko 6 1 0xffffffff82c19000 2180 smbus.ko 7 1 0xffffffff82c1c000 2110 pchtherm.ko 8 1 0xffffffff82c1f000 2a08 mac_ntpd.ko % kldload /boot/kernel/if_bnxt.ko May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xb8a10000-0xb8a1ffff,0xb8900000-0xb89fffff,0xb8a22000-0xb8a23fff at device 0.0 numa-domain 0 on pci9 May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Using 0 RX queues 0 TX queues May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Using MSI-X interrupts with 1 vectors May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Unable to allocate device TX queue May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Unable to allocate queue memory May 19 12:05:21 <kern.crit> chlorine kernel: device_attach: bnxt0 attach returned 22 May 19 12:05:21 <kern.crit> chlorine kernel: Warning: failed attempt to remove oid fc with child autoneg May 19 12:05:21 <kern.crit> chlorine kernel: pci0:94:0:0: Device leaked memory resources May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%desc)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%driver)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%location)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%pnpinfo)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%parent)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%domain)! May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xb8a00000-0xb8a0ffff,0xb8800000-0xb88fffff,0xb8a20000-0xb8a21fff at device 0.1 numa-domain 0 on pci9 May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.driver_version)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxqs)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxqs)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_qs_enable)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.disable_msix)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.rx_budget)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.tx_abdicate)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.core_offset)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.separate_txrx)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.use_logical_cores)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxds)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxds)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.mfg_id)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.device_id)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.sector_size)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.size)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.reserved_size)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.available_size)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_key)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_type)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rx_stall)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.vlan_strip)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.if_name)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs_irq)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames_irq)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs_irq)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames_irq)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.enable)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.gro_mode)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_agg_segs)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_aggs)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.min_agg_len)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.tx)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.rx)! May 19 12:05:21 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.autoneg)! May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Using 0 RX queues 0 TX queues May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Using MSI-X interrupts with 1 vectors May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: iflib_dma_alloc_align: bus_dma_tag_create failed: 22 May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Unable to allocate device TX queue May 19 12:05:21 <kern.crit> chlorine kernel: bnxt0: Unable to allocate queue memory May 19 12:05:21 <kern.crit> chlorine kernel: device_attach: bnxt0 attach returned 22 May 19 12:05:21 <kern.crit> chlorine kernel: pci0:94:0:1: Device leaked memory resources If I then unload if_bnxt.ko and load it again, I get: May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%desc)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%driver)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%location)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%pnpinfo)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%parent)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%domain)! May 19 12:06:22 <kern.crit> chlorine kernel: bnxt0: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xb8a10000-0xb8a1ffff,0xb8900000-0xb89fffff,0xb8a22000-0xb8a23fff at device 0.0 numa-domain 0 on pci9 May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.driver_version)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxqs)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxqs)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_qs_enable)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.disable_msix)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.rx_budget)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.tx_abdicate)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.core_offset)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.separate_txrx)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.use_logical_cores)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxds)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxds)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.mfg_id)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.device_id)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.sector_size)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.size)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.reserved_size)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.available_size)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_key)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_type)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rx_stall)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.vlan_strip)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.if_name)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs_irq)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames_irq)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs_irq)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames_irq)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.enable)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.gro_mode)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_agg_segs)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_aggs)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.min_agg_len)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.tx)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.rx)! May 19 12:06:22 <kern.crit> chlorine kernel: sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.autoneg)! May 19 12:06:22 <kern.crit> chlorine kernel: bnxt0: Using 256 TX descriptors and 256 RX descriptors May 19 12:06:22 <kern.crit> chlorine kernel: bnxt0: Using 8 RX queues 8 TX queues May 19 12:06:22 <kern.crit> chlorine kernel: bnxt0: Using MSI-X interrupts with 9 vectors May 19 12:06:22 <kern.info> chlorine kernel: bnxt0: Ethernet address: f4:ee:08:25:fa:bd May 19 12:06:22 <kern.info> chlorine kernel: bnxt0: netmap queues/slots: TX 8/256, RX 8/256 May 19 12:06:22 <kern.crit> chlorine kernel: bnxt1: <Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet> mem 0xb8a00000-0xb8a0ffff,0xb8800000-0xb88fffff,0xb8a20000-0xb8a21fff at device 0.1 numa-domain 0 on pci9 May 19 12:06:22 <kern.crit> chlorine kernel: bnxt0: Link is UP full duplex, FC - none - 10000 Mbps May 19 12:06:22 <kern.info> chlorine kernel: bnxt0: link state changed to UP May 19 12:06:22 <kern.crit> chlorine kernel: debugnet_any_ifnet_update: Bad dn_init result from bnxt0 (ifp 0xfffff808b357f800), ignoring. May 19 12:06:22 <kern.crit> chlorine kernel: bnxt1: Using 256 TX descriptors and 256 RX descriptors May 19 12:06:22 <kern.crit> chlorine kernel: bnxt1: Using 8 RX queues 8 TX queues May 19 12:06:22 <kern.crit> chlorine kernel: bnxt1: Using MSI-X interrupts with 9 vectors May 19 12:06:22 <kern.info> chlorine kernel: bnxt1: Ethernet address: foo May 19 12:06:22 <kern.info> chlorine kernel: bnxt1: netmap queues/slots: TX 8/256, RX 8/256 and the interfaces are now showing up in ifconfig bnxt0: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether foo media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> bnxt1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether foo media: Ethernet autoselect (Unknown <full-duplex,rxpause,txpause>) status: no carrier nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> On rebooting again, we're back to just the on-board NIC.
(In reply to SJ from comment #26) If the interfaces can be made to appear, are they usable ? Is the performance appropriate ? Or do they show, but fail to work ?
(In reply to Kurt Jaeger from comment #27) After some testing, I can confirm that they show, but fail to work. When configuring one of the interfaces to use DHCP, it doesn't obtain anything, showing the IP 0.0.0.0. bnxt0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether foo inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> No luck with a static configuration either. I've tried rebooting and reloading if_bnxt.ko after changes to see if there's any difference. % ping -I bnxt0 ... ping: invalid multicast interface: `bnxt0' Thanks!
I tested 13.1-RC1 with the BCM57414. I have no BCM57416 to test 8-(
(In reply to Kurt Jaeger from comment #29) As per #20, it seems like firmware version 21.65.33.33 may work. Whether or not it'll let me upgrade is a different thing altogether! It'll be tomorrow or Monday before I'll be able to test this out, but will report back.
I couldn't find a version matching 21.65.33.33, but downgraded the NIC firmware to 21.60.22.11 and everything worked fine after loading the kernel module. Tried upgrading to 22.00.07.60 again after that and can confirm that now works as well. Bit of an odd one, but happy to report no further issues. Cheers!
*** Bug 262291 has been marked as a duplicate of this bug. ***
FWIW, I've run into the same problem today with 57416. After settings txqueues and rxqueues in kenv, the adapters work. The only problem is that when they are in lagg interface with laggproto failover, then when one of the link fails, the default gateway is not rechable until I ping it (like if gratituous ARP is not being sent. But this can be a problem of the UniFi switch the box is connected to). The other problem is that the link comes up to only 1Gbps, even if plugged into 10Gbps copper transceiver.
I can reproduce this on a build based on 13.1-RELEASE, but only if I load the kernel module late. If I load if from /boot/loader.conf, or from kld_list in /etc/rc.conf, then it works fine. But if I load it by hand with "kldload if_bnxt" after booting has completed, then I get the same "iflib_dma_alloc_align: bus_dma_tag_create failed: 22" error that others report, and no bnxt devices are created. However, if I then unload the module and load it again, it works on the second try. I cannot reproduce the problem on 14.0-CURRENT simply because on that version, something causes the module to load automatically. And I can't unload the module to try again due to bug #270270. My firmware is: dev.bnxt.0.ver.roce_fw: 214.0.194 dev.bnxt.0.ver.netctrl_fw: 214.4.7 dev.bnxt.0.ver.mgmt_fw: 214.4.42 dev.bnxt.0.ver.hwrm_fw: 214.4.91
I'm facing a similar issue with latest TrueNAS CORE 13.0-U5.1 and the Broadcom BCM57414. The NIC has the latest firmware (225.1.95.0), but I'm encountering problems. Interestingly, an older card with firmware (214.4.42) seems to be working fine. I'm currently testing different firmware versions to find which one works. ----------------------------DMESG--------------------------------------------------- sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%driver)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%location)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%pnpinfo)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%parent)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.%domain)! bnxt0: <Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Ethernet> mem 0x39ffffe10000-0x39ffffe1ffff,0x39ffffd00000-0x39ffffdfffff,0x39ffffe22000-0x39ffffe23fff at device 0.0 numa-domain 1 on pci19 sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.driver_version)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxqs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxqs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_qs_enable)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.disable_msix)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.rx_budget)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.tx_abdicate)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.core_offset)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.separate_txrx)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.use_logical_cores)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_ntxds)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.iflib.override_nrxds)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.mfg_id)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.device_id)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.sector_size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.reserved_size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.nvram.available_size)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_key)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rss_type)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.rx_stall)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.vlan_strip)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.if_name)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_usecs_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_rx_frames_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_usecs_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.intr_coal_tx_frames_irq)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.enable)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.gro_mode)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_agg_segs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.max_aggs)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.hw_lro.min_agg_len)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.tx)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.rx)! sysctl_warn_reuse: can't re-use a leaf (dev.bnxt.0.fc.autoneg)! bnxt0: Using 256 TX descriptors and 256 RX descriptors bnxt0: Using 16 RX queues 16 TX queues bnxt0: Using MSI-X interrupts with 17 vectors bnxt0: Ethernet address: 5c:6f:69:XXXXXXX bnxt0: Unknown phy type bnxt1: <Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Ethernet> mem 0x39ffffe00000-0x39ffffe0ffff,0x39ffffc00000-0x39ffffcfffff,0x39ffffe20000-0x39ffffe21fff at device 0.1 numa-domain 1 on pci19 bnxt1: Using 256 TX descriptors and 256 RX descriptors bnxt1: Using 16 RX queues 16 TX queues bnxt1: Using MSI-X interrupts with 17 vectors bnxt1: Ethernet address: 5c:6f:69:XXXXXXX bnxt1: Unknown phy type bnxt2: <Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Ethernet> mem 0x3a3fffe10000-0x3a3fffe1ffff,0x3a3fffd00000-0x3a3fffdfffff,0x3a3fffe22000-0x3a3fffe23fff at device 0.0 numa-domain 1 on pci21 bnxt2: Using 256 TX descriptors and 256 RX descriptors bnxt2: Using 16 RX queues 16 TX queues bnxt2: Using MSI-X interrupts with 17 vectors bnxt2: Ethernet address: 5c:6f:69:XXXXXXXXX bnxt2: Unknown phy type bnxt3: <Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Ethernet> mem 0x3a3fffe00000-0x3a3fffe0ffff,0x3a3fffc00000-0x3a3fffcfffff,0x3a3fffe20000-0x3a3fffe21fff at device 0.1 numa-domain 1 on pci21 bnxt3: Using 256 TX descriptors and 256 RX descriptors bnxt3: Using 16 RX queues 16 TX queues bnxt3: Using MSI-X interrupts with 17 vectors bnxt3: Ethernet address: 5c:6f:69:XXXXXXXXX bnxt3: Unknown phy type
Did you retry with a recent version of FreeBSD, like 14.0 ?
(In reply to Kurt Jaeger from comment #36) We're not using 14 in production yet, preferring to wait for the rest of the world to resolve issues that may crop up in dot-zero releases. Sadly this equipment is kind of bespoke and out in the field. (Was provided by one of our partners). I know that's not super helpful from our end -- just trying to set expectations.
^Triage: clear unneeded flags. Nothing has yet been committed to be merged.
There's a long list of reviews that had constitute a patch: https://reviews.freebsd.org/D36436 https://reviews.freebsd.org/D36437 https://reviews.freebsd.org/D36438 https://reviews.freebsd.org/D36439 https://reviews.freebsd.org/D36440 https://reviews.freebsd.org/D36441 https://reviews.freebsd.org/D36442 https://reviews.freebsd.org/D36443 https://reviews.freebsd.org/D36444 https://reviews.freebsd.org/D36445 https://reviews.freebsd.org/D36446 https://reviews.freebsd.org/D36447 https://reviews.freebsd.org/D36448 I think they went into 14.x sometime. I have some testhardware somewhere and have a look in the next few weeks.
Hi, I'm in the same case. 14.1-RELEASE-p6 Dell PowerEdge R7425 BCM57412 NetXtreme-E 10Gb RDMA Ethernet Controller I've to downgrade the firmware of the BCM57412 to 21.60.22.11 to have it working.