Bug 239240 - igb: TX(2) desc avail = 1024, pidx = 0 messages appear when the network card loses ethernet link
Summary: igb: TX(2) desc avail = 1024, pidx = 0 messages appear when the network card ...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-net mailing list
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2019-07-16 05:37 UTC by Richard Gallamore
Modified: 2019-07-16 19:25 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Gallamore freebsd_committer 2019-07-16 05:37:23 UTC
Currently, I have a router running FreeBSD 12.0-RELEASE on atom c2758. My switch seems to have become unresponsive which required a hard reset.

Sometime around when the switch became unresponsive, (I think it was at least a few minutes after) I started receiving "igb6: TX(2) desc avail = 1024, pidx = 0". The interface no longer works and will continue to print the message until the interface is down. Bringing it back up will start the message again.

Most of the settings are below. There is a lot that were removed as I don't think they have any relevance to this issue. Let me know if there is anything else that may help.

pciconf -lv
igb6@pci0:0:20:3:       class=0x020000 card=0x1f4115d9 chip=0x1f418086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet

sysctl -a | grep igb\.0
dev.igb.0.interrupts.rx_overrun: 0
dev.igb.0.interrupts.rx_desc_min_thresh: 0
dev.igb.0.interrupts.tx_queue_min_thresh: 65832053
dev.igb.0.interrupts.tx_queue_empty: 71983081
dev.igb.0.interrupts.tx_abs_timer: 0
dev.igb.0.interrupts.tx_pkt_timer: 0
dev.igb.0.interrupts.rx_abs_timer: 0
dev.igb.0.interrupts.rx_pkt_timer: 65832053
dev.igb.0.interrupts.asserts: 74311168
dev.igb.0.mac_stats.tso_ctx_fail: 0
dev.igb.0.mac_stats.tso_txd: 46548
dev.igb.0.mac_stats.tx_frames_1024_1522: 48022265
dev.igb.0.mac_stats.tx_frames_512_1023: 301529
dev.igb.0.mac_stats.tx_frames_256_511: 900639
dev.igb.0.mac_stats.tx_frames_128_255: 3062062
dev.igb.0.mac_stats.tx_frames_65_127: 16876524
dev.igb.0.mac_stats.tx_frames_64: 2820062
dev.igb.0.mac_stats.mcast_pkts_txd: 27545
dev.igb.0.mac_stats.bcast_pkts_txd: 211
dev.igb.0.mac_stats.good_pkts_txd: 71983081
dev.igb.0.mac_stats.total_pkts_txd: 71983081
dev.igb.0.mac_stats.good_octets_txd: 75469715088
dev.igb.0.mac_stats.good_octets_recvd: 59838583722
dev.igb.0.mac_stats.rx_frames_1024_1522: 37344025
dev.igb.0.mac_stats.rx_frames_512_1023: 5784017
dev.igb.0.mac_stats.rx_frames_256_511: 472870
dev.igb.0.mac_stats.rx_frames_128_255: 2642368
dev.igb.0.mac_stats.rx_frames_65_127: 11918975
dev.igb.0.mac_stats.rx_frames_64: 7669798
dev.igb.0.mac_stats.mcast_pkts_recvd: 43168
dev.igb.0.mac_stats.bcast_pkts_recvd: 1268
dev.igb.0.mac_stats.good_pkts_recvd: 65832053
dev.igb.0.mac_stats.total_pkts_recvd: 65832053
dev.igb.0.mac_stats.xoff_txd: 0
dev.igb.0.mac_stats.xoff_recvd: 0
dev.igb.0.mac_stats.xon_txd: 0
dev.igb.0.mac_stats.xon_recvd: 0
dev.igb.0.mac_stats.coll_ext_errs: 0
dev.igb.0.mac_stats.alignment_errs: 0
dev.igb.0.mac_stats.crc_errs: 0
dev.igb.0.mac_stats.recv_errs: 0
dev.igb.0.mac_stats.recv_jabber: 0
dev.igb.0.mac_stats.recv_oversize: 0
dev.igb.0.mac_stats.recv_fragmented: 0
dev.igb.0.mac_stats.recv_undersize: 0
dev.igb.0.mac_stats.recv_no_buff: 0
dev.igb.0.mac_stats.missed_packets: 0
dev.igb.0.mac_stats.defer_count: 0
dev.igb.0.mac_stats.sequence_errors: 0
dev.igb.0.mac_stats.symbol_errors: 0
dev.igb.0.mac_stats.collision_count: 0
dev.igb.0.mac_stats.late_coll: 0
dev.igb.0.mac_stats.multiple_coll: 0
dev.igb.0.mac_stats.single_coll: 0
dev.igb.0.mac_stats.excess_coll: 0
dev.igb.0.queue_rx_3.rx_irq: 0
dev.igb.0.queue_rx_3.rxd_tail: 942
dev.igb.0.queue_rx_3.rxd_head: 944
dev.igb.0.queue_rx_2.rx_irq: 0
dev.igb.0.queue_rx_2.rxd_tail: 369
dev.igb.0.queue_rx_2.rxd_head: 371
dev.igb.0.queue_rx_1.rx_irq: 0
dev.igb.0.queue_rx_1.rxd_tail: 1010
dev.igb.0.queue_rx_1.rxd_head: 1012
dev.igb.0.queue_rx_0.rx_irq: 0
dev.igb.0.queue_rx_0.rxd_tail: 341
dev.igb.0.queue_rx_0.rxd_head: 343
dev.igb.0.queue_tx_3.tx_irq: 0
dev.igb.0.queue_tx_3.txd_tail: 748
dev.igb.0.queue_tx_3.txd_head: 748
dev.igb.0.queue_tx_2.tx_irq: 0
dev.igb.0.queue_tx_2.txd_tail: 667
dev.igb.0.queue_tx_2.txd_head: 667
dev.igb.0.queue_tx_1.tx_irq: 0
dev.igb.0.queue_tx_1.txd_tail: 48
dev.igb.0.queue_tx_1.txd_head: 48
dev.igb.0.queue_tx_0.tx_irq: 0
dev.igb.0.queue_tx_0.txd_tail: 363
dev.igb.0.queue_tx_0.txd_head: 363
dev.igb.0.fc_low_water: 32752
dev.igb.0.fc_high_water: 32768
dev.igb.0.rx_control: 71335966
dev.igb.0.device_control: 404488769
dev.igb.0.watchdog_timeouts: 0
dev.igb.0.rx_overruns: 0
dev.igb.0.tx_dma_fail: 0
dev.igb.0.mbuf_defrag_fail: 0
dev.igb.0.link_irq: 2
dev.igb.0.dropped: 0
dev.igb.0.eee_control: 1
dev.igb.0.itr: 488
dev.igb.0.tx_abs_int_delay: 66
dev.igb.0.rx_abs_int_delay: 66
dev.igb.0.tx_int_delay: 66
dev.igb.0.rx_int_delay: 0
dev.igb.0.rs_dump: 0
dev.igb.0.reg_dump: General Registers
dev.igb.0.fc: 0
dev.igb.0.debug: -1
dev.igb.0.nvm: -1
dev.igb.0.iflib.rxq3.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq3.rxq_fl0.cidx: 944
dev.igb.0.iflib.rxq3.rxq_fl0.pidx: 943
dev.igb.0.iflib.rxq2.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq2.rxq_fl0.cidx: 371
dev.igb.0.iflib.rxq2.rxq_fl0.pidx: 370
dev.igb.0.iflib.rxq1.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq1.rxq_fl0.cidx: 1012
dev.igb.0.iflib.rxq1.rxq_fl0.pidx: 1011
dev.igb.0.iflib.rxq0.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq0.rxq_fl0.cidx: 343
dev.igb.0.iflib.rxq0.rxq_fl0.pidx: 342
dev.igb.0.iflib.txq3.r_abdications: 0
dev.igb.0.iflib.txq3.r_restarts: 0
dev.igb.0.iflib.txq3.r_stalls: 0
dev.igb.0.iflib.txq3.r_starts: 15127348
dev.igb.0.iflib.txq3.r_drops: 0
dev.igb.0.iflib.txq3.r_enqueues: 15165787
dev.igb.0.iflib.txq3.ring_state: pidx_head: 0347 pidx_tail: 0347 cidx: 0347 state: IDLE
dev.igb.0.iflib.txq3.txq_cleaned: 15730369
dev.igb.0.iflib.txq3.txq_processed: 15730409
dev.igb.0.iflib.txq3.txq_in_use: 43
dev.igb.0.iflib.txq3.txq_cidx_processed: 745
dev.igb.0.iflib.txq3.txq_cidx: 705
dev.igb.0.iflib.txq3.txq_pidx: 748
dev.igb.0.iflib.txq3.no_tx_dma_setup: 0
dev.igb.0.iflib.txq3.txd_encap_efbig: 0
dev.igb.0.iflib.txq3.tx_map_failed: 0
dev.igb.0.iflib.txq3.no_desc_avail: 0
dev.igb.0.iflib.txq3.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq3.m_pullups: 59
dev.igb.0.iflib.txq3.mbuf_defrag: 0
dev.igb.0.iflib.txq2.r_abdications: 0
dev.igb.0.iflib.txq2.r_restarts: 0
dev.igb.0.iflib.txq2.r_stalls: 0
dev.igb.0.iflib.txq2.r_starts: 29491761
dev.igb.0.iflib.txq2.r_drops: 0
dev.igb.0.iflib.txq2.r_enqueues: 29560202
dev.igb.0.iflib.txq2.ring_state: pidx_head: 1418 pidx_tail: 1418 cidx: 1418 state: IDLE
dev.igb.0.iflib.txq2.txq_cleaned: 43517553
dev.igb.0.iflib.txq2.txq_processed: 43517593
dev.igb.0.iflib.txq2.txq_in_use: 42
dev.igb.0.iflib.txq2.txq_cidx_processed: 665
dev.igb.0.iflib.txq2.txq_cidx: 625
dev.igb.0.iflib.txq2.txq_pidx: 667
dev.igb.0.iflib.txq2.no_tx_dma_setup: 0
dev.igb.0.iflib.txq2.txd_encap_efbig: 0
dev.igb.0.iflib.txq2.tx_map_failed: 0
dev.igb.0.iflib.txq2.no_desc_avail: 0
dev.igb.0.iflib.txq2.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq2.m_pullups: 33
dev.igb.0.iflib.txq2.mbuf_defrag: 0
dev.igb.0.iflib.txq1.r_abdications: 0
dev.igb.0.iflib.txq1.r_restarts: 0
dev.igb.0.iflib.txq1.r_stalls: 0
dev.igb.0.iflib.txq1.r_starts: 16060858
dev.igb.0.iflib.txq1.r_drops: 0
dev.igb.0.iflib.txq1.r_enqueues: 16161222
dev.igb.0.iflib.txq1.ring_state: pidx_head: 0454 pidx_tail: 0454 cidx: 0454 state: IDLE
dev.igb.0.iflib.txq1.txq_cleaned: 22249477
dev.igb.0.iflib.txq1.txq_processed: 22249517
dev.igb.0.iflib.txq1.txq_in_use: 43
dev.igb.0.iflib.txq1.txq_cidx_processed: 45
dev.igb.0.iflib.txq1.txq_cidx: 5
dev.igb.0.iflib.txq1.txq_pidx: 48
dev.igb.0.iflib.txq1.no_tx_dma_setup: 0
dev.igb.0.iflib.txq1.txd_encap_efbig: 0
dev.igb.0.iflib.txq1.tx_map_failed: 0
dev.igb.0.iflib.txq1.no_desc_avail: 0
dev.igb.0.iflib.txq1.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq1.m_pullups: 90
dev.igb.0.iflib.txq1.mbuf_defrag: 0
dev.igb.0.iflib.txq0.r_abdications: 0
dev.igb.0.iflib.txq0.r_restarts: 0
dev.igb.0.iflib.txq0.r_stalls: 0
dev.igb.0.iflib.txq0.r_starts: 11114832
dev.igb.0.iflib.txq0.r_drops: 0
dev.igb.0.iflib.txq0.r_enqueues: 11144763
dev.igb.0.iflib.txq0.ring_state: pidx_head: 1595 pidx_tail: 1595 cidx: 1595 state: IDLE
dev.igb.0.iflib.txq0.txq_cleaned: 11860289
dev.igb.0.iflib.txq0.txq_processed: 11860329
dev.igb.0.iflib.txq0.txq_in_use: 42
dev.igb.0.iflib.txq0.txq_cidx_processed: 361
dev.igb.0.iflib.txq0.txq_cidx: 321
dev.igb.0.iflib.txq0.txq_pidx: 363
dev.igb.0.iflib.txq0.no_tx_dma_setup: 0
dev.igb.0.iflib.txq0.txd_encap_efbig: 0
dev.igb.0.iflib.txq0.tx_map_failed: 0
dev.igb.0.iflib.txq0.no_desc_avail: 0
dev.igb.0.iflib.txq0.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq0.m_pullups: 116
dev.igb.0.iflib.txq0.mbuf_defrag: 0
dev.igb.0.iflib.override_nrxds: 0
dev.igb.0.iflib.override_ntxds: 0
dev.igb.0.iflib.tx_abdicate: 0
dev.igb.0.iflib.rx_budget: 0
dev.igb.0.iflib.disable_msix: 0
dev.igb.0.iflib.override_qs_enable: 0
dev.igb.0.iflib.override_nrxqs: 0
dev.igb.0.iflib.override_ntxqs: 0
dev.igb.0.iflib.driver_version: 7.6.1-k
dev.igb.0.%parent: pci4
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x1533 subvendor=0x15d9 subdevice=0x1533 class=0x020000
dev.igb.0.%location: slot=0 function=0 dbsf=pci0:4:0:0
dev.igb.0.%driver: igb
dev.igb.0.%desc: Intel(R) PRO/1000 PCI-Express Network Driver

rc.conf:
powerd_enable="YES"
powerd_flags="-a min -b min -m 1200/1000 -M 2400/10000"
ifconfig_igb6="up"
ifconfig_bridge1="inet 10.93.232.1/24 addm lagg1 addm igb6 addm igb5"
ifconfig_bridge1_ipv6="inet6 auto_linklocal"
gateway_enable="YES"
ipv6_gateway_enable="YES"
ipv6_cpe_wanif="igb0"
ipv6_privacy="NO"

sysctl.conf:
net.link.bridge.pfil_member=0
net.link.bridge.pfil_onlyip=0
net.link.stf.permit_rfc1918=1
net.inet.tcp.fastopen.server_enable=1
net.inet.tcp.cc.algorithm=cubic
net.inet.tcp.rfc6675_pipe=1
net.inet.tcp.recvbuf_inc=65536
net.inet.tcp.recvspace=131072
net.inet.tcp.sendbuf_inc=65536
net.inet.tcp.sendspace=131072
net.inet.tcp.mssdflt=1460
net.inet.tcp.minmss=536
net.inet.tcp.syncache.rexmtlimit=3
net.inet.tcp.abc_l_var=44
net.inet.tcp.initcwnd_segments=16
net.inet.tcp.syncookies=0
dev.igb.0.fc=0

loader.conf
net.link.bridge.inherit_mac="1"
net.link.ifqmaxlen="2048"
hw.igb.rx_process_limit="-1"
hw.igb.num_queues="2"
net.pf.request_maxcount=1048575
Comment 1 IPTRACE 2019-07-16 17:00:35 UTC
Hello!

I'm affected as well for FreeBSD 12.0-RELEASE-p7.

kernel: igb0: TX(0) desc avail = 1024, pidx = 0
...
kernel: igb2: TX(5) desc avail = 1024, pidx = 0


igb0@pci0:129:0:0:      class=0x020000 card=0x00001458 chip=0x15218086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet


igb2@pci0:132:0:0:      class=0x020000 card=0xa02c8086 chip=0x10e88086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82576 Gigabit Network Connection'
    class      = network
    subclass   = ethernet


Use case (forced by me but generally it happens randomly):
1. All was working.
2. I detached Ethernet cable from modem side.
3. Connected again and saw "no carrier" on igb0 interface.
4. System logged as follow:
      Jul  5 22:31:35 hpv kernel: igb0: TX(0) desc avail = 1024, pidx = 0
      Jul  5 22:31:38 hpv kernel: igb0: TX(0) desc avail = 1024, pidx = 0
      Jul  5 22:31:40 hpv kernel: igb0: TX(0) desc avail = 1024, pidx = 0
5. I was trying "server:# service netif restart igb0 && service routing restart" without success.
6. Resarted OS to back to normal.

Is there any workaround instead of restart a system?
I was trying below without progress as well.

# ifconfig igb0 down
# ifconfig igb0 up

I think the problem is when the network card loses ethernet link and then errors occurs with non-working interface?!