Bug 220198

Summary: igb stats double counting in 12-CURRENT
Product: Base System Reporter: Ben Woods <woodsb02>
Component: kernAssignee: freebsd-net (Nobody) <net>
Status: Closed FIXED    
Severity: Affects Some People CC: amigan, kaho, sbruno
Priority: --- Keywords: IntelNetworking
Version: CURRENT   
Hardware: Any   
OS: Any   

Description Ben Woods freebsd_committer freebsd_triage 2017-06-21 23:24:27 UTC
My networking statistics appear to be double counting on two of my FreeBSD 12-CURRENT boxes running r319025.

I discovered the double counting by comparing the output of "systat -ifstat" (which reports 988 KB/s) with the output of "iftop -B" (which reports 491 KB/s). I believe systat uses the kernel interface counters to infer bandwidth usage, whereas iftop gathers statistics using pcap. I know that iftop is the correct one, because my internet speed only gives me around ~500 KB/s unfortunately.

# systat -ifstat -match igb0
...
      Interface           Traffic               Peak                Total
           igb0  in    988.008 KB/s        988.008 KB/s           36.295 GB
                 out    16.953 KB/s         16.953 KB/s          295.774 GB


# iftop -i igb0 -B
...
TX:    cum:   31.7KB   peak:   9.69KB    rates:   9.69KB  7.92KB  7.92KB
RX:           1.54MB            491KB              491KB   394KB   394KB


The two interfaces in these boxes are:

$ pciconf -lvv | grep igb1 -A4
igb1@pci0:4:0:0:        class=0x020000 card=0x00008086 chip=0x15398086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I211 Gigabit Network Connection'
    class      = network
    subclass   = ethernet

$ pciconf -lvv | grep igb0 -A4
igb0@pci0:8:0:0:        class=0x020000 card=0x15331849 chip=0x15338086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
Comment 1 commit-hook freebsd_committer freebsd_triage 2017-08-10 03:12:10 UTC
A commit references this bug:

Author: sbruno
Date: Thu Aug 10 03:11:05 UTC 2017
New revision: 322337
URL: https://svnweb.freebsd.org/changeset/base/322337

Log:
  Export IFCAP_HWSTATS so that we don't experience double stats counting
  on iflib enabled devices.

  PR:		220198
  Submitted by:	Matt Macy <matt@mattmacy.io>
  Reported by:	Ben Woods <woodsb02@freebsd.org>
  Sponsored by:	Limelight Networks

Changes:
  head/sys/net/iflib.c
Comment 2 Ben Woods freebsd_committer freebsd_triage 2017-08-22 15:22:38 UTC
Hi Sean,
I can confirm this has fixed the error.
Thanks for your help!
-Ben
Comment 3 Kaho Toshikazu 2017-08-24 09:55:56 UTC
(In reply to commit-hook from comment #1)

I think the bug is fixed by this commit, but it is not a good way.

1. Every driver using iflib should set IFCAP_HWSTATS flag within the driver.
For example, in the em(4) driver, add IFCAP_HWSTATS flag
into #define EM_CAPS and #define IGB_CAPS lines of if_em.c.

2. The IFCAP_FLAGS in iflib.c seems to hold flags which can be changed by
a ioctl function. Who want to set or unset the IFCAP_HWSTATS flag
by a function?
Comment 4 Daniel Ponte 2021-02-24 21:15:00 UTC
I have cherry-picked this into my local stable/13 and can also confirm that this fixes the issue.
Comment 5 Daniel Ponte 2021-02-24 21:25:03 UTC
Oops, disregard.