Bug 266048 - [ixgbe] 82599ES reports IPv4 UDP with zero checksum as input errors
Summary: [ixgbe] 82599ES reports IPv4 UDP with zero checksum as input errors
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Gordon Tetlow
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-08-26 10:59 UTC by Andrey V. Elsukov
Modified: 2024-01-08 05:25 UTC (History)
8 users (show)

See Also:


Attachments
proposed patch (608 bytes, patch)
2022-10-19 07:14 UTC, Andrey V. Elsukov
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey V. Elsukov freebsd_committer freebsd_triage 2022-08-26 10:59:52 UTC
After https://reviews.freebsd.org/D27191 ixgbe(4) uses sum of all errors registers in the IFCOUNTER_IERRORS. Especially IXGBE_XEC is now included in this sum. And now we have noticeable input errors rate:

% netstat -hw1 -I ix0
            input            ix0           output
   packets  errs idrops      bytes    packets  errs      bytes colls
      7.7k    14     0       2.0M       4.3k     0       1.9M     0
       16k    20     0        27M        30k     0        29M     0
       10k    40     0        18M        20k     0        19M     0
       12k    23     0        18M        21k     0        20M     0
^C
% sysctl dev.ix.0 | grep errs
dev.ix.0.mac_stats.checksum_errs: 25190176
dev.ix.0.mac_stats.rec_len_errs: 0
dev.ix.0.mac_stats.byte_errs: 0
dev.ix.0.mac_stats.ill_errs: 0
dev.ix.0.mac_stats.crc_errs: 0
dev.ix.0.mac_stats.rx_errs: 25190176
% pciconf -l | grep ^ix0
ix0@pci0:6:0:0:	class=0x020000 card=0x00068086 chip=0x10fb8086 rev=0x01 hdr=0x00

I found similar report about OpenSense: https://forum.opnsense.org/index.php?topic=27051.0

It looks like 82599ES has errata related to IPv4 UDP packets and zero checksum:

 44. Integrity Error Reported for IPv4/UDP Packets With Zero Checksum
 
 Problem:According to the UDP specification “an all zero transmitted checksum value
 means that the transmitter generated no checksum (for debugging or for higher
 level protocols that don’t care)”, these packets should be received without a
 checksum error notation. The 82599 reports an L4 integrity error if such packets
 are received.

 Implication: UDP packets without a checksum will have an L4 integrity error indication in the
 Rx descriptor.

See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1922430
Comment 1 Eric Joyner freebsd_committer freebsd_triage 2022-08-26 17:16:23 UTC
Thanks for the link to the Red Hat (and inside, dpdk commits) bug tracker -- it looks like we need to add a 82599-specific workaround when it comes to counting errors, as well as marking these bad checksum packets as having no checksum calculated so that SW can properly re-compute them instead of assuming them as being bad packets.
Comment 2 Anthony Capone 2022-10-17 17:53:41 UTC
I believe I am also affected by this driver bug. I have tried this on a vanilla Freebsd install and the errors IN are still there... however, I did first notice this on an OPNsense install. 

See the details here:

https://forum.opnsense.org/index.php?topic=30645
Comment 3 Andrey V. Elsukov freebsd_committer freebsd_triage 2022-10-19 07:14:30 UTC
Created attachment 237452 [details]
proposed patch

I think for now we can use the same solution as here: https://git.dpdk.org/dpdk/commit/?id=2ee14c8905e9
Comment 4 Franco Fichtner 2022-10-19 07:16:40 UTC
We were testing https://github.com/opnsense/src/commit/4a788be44e0 yesterday and it was confirmed working. Your patch is shorter, happy to see it committed!


Cheers,
Franco
Comment 5 commit-hook freebsd_committer freebsd_triage 2022-11-10 09:48:06 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=8526120ad41ca47367b43f8f4459e0fa61285571

commit 8526120ad41ca47367b43f8f4459e0fa61285571
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2022-11-10 09:34:40 +0000
Commit:     Andrey V. Elsukov <ae@FreeBSD.org>
CommitDate: 2022-11-10 09:34:40 +0000

    ixgbe: workaround errata about UDP frames with zero checksum

    Intel 82599 has errata related to IPv4 UDP frames with zero checksum.
    It reports such datagrams with L4 integrity errors in IXGBE_XEC
    register. And after afb1aa4e6df2 commit such errors are reported
    via IFCOUNTER_IERRORS. This confuses users, since actually all frames
    are handled correctly by the system.
    To workaround the problem, let's ignore the XEC register value for
    82599 cards for now.

    PR:             266048
    Discussed with: erj
    MFC after:      1 week
    Sponsored by:   Yandex LLC

 sys/dev/ixgbe/if_ix.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 6 commit-hook freebsd_committer freebsd_triage 2022-11-17 20:15:27 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=daf3d88ac1849c9a6c5f05d2a663c7fe57453b37

commit daf3d88ac1849c9a6c5f05d2a663c7fe57453b37
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2022-11-10 09:34:40 +0000
Commit:     Andrey V. Elsukov <ae@FreeBSD.org>
CommitDate: 2022-11-17 20:13:43 +0000

    ixgbe: workaround errata about UDP frames with zero checksum

    Intel 82599 has errata related to IPv4 UDP frames with zero checksum.
    It reports such datagrams with L4 integrity errors in IXGBE_XEC
    register. And after afb1aa4e6df2 commit such errors are reported
    via IFCOUNTER_IERRORS. This confuses users, since actually all frames
    are handled correctly by the system.
    To workaround the problem, let's ignore the XEC register value for
    82599 cards for now.

    PR:             266048
    Discussed with: erj
    Sponsored by:   Yandex LLC

    (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571)

 sys/dev/ixgbe/if_ix.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 7 commit-hook freebsd_committer freebsd_triage 2022-11-17 20:17:29 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2

commit fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2022-11-10 09:34:40 +0000
Commit:     Andrey V. Elsukov <ae@FreeBSD.org>
CommitDate: 2022-11-17 20:16:15 +0000

    ixgbe: workaround errata about UDP frames with zero checksum

    Intel 82599 has errata related to IPv4 UDP frames with zero checksum.
    It reports such datagrams with L4 integrity errors in IXGBE_XEC
    register. And after afb1aa4e6df2 commit such errors are reported
    via IFCOUNTER_IERRORS. This confuses users, since actually all frames
    are handled correctly by the system.
    To workaround the problem, let's ignore the XEC register value for
    82599 cards for now.

    PR:             266048
    Discussed with: erj
    Sponsored by:   Yandex LLC

    (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571)

 sys/dev/ixgbe/if_ix.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 8 Eric Joyner freebsd_committer freebsd_triage 2022-11-18 20:16:59 UTC
Thanks for fixing this and MFC-ing back to the older branches! Can this be marked as closed?
Comment 9 Andrey V. Elsukov freebsd_committer freebsd_triage 2022-11-19 11:59:17 UTC
(In reply to Eric Joyner from comment #8)

I didn't meet the deadline before 12.4-RELEASE, so Glen said it probably will be included in the post-release Errata. I think then we will close this PR.
Comment 10 commit-hook freebsd_committer freebsd_triage 2023-02-08 18:16:21 UTC
A commit in branch releng/13.1 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f3e20eb8d8f0b87ae828741c48e9ebd7913086ac

commit f3e20eb8d8f0b87ae828741c48e9ebd7913086ac
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2022-11-10 09:34:40 +0000
Commit:     Gordon Tetlow <gordon@FreeBSD.org>
CommitDate: 2023-02-08 16:30:38 +0000

    ixgbe: workaround errata about UDP frames with zero checksum

    Intel 82599 has errata related to IPv4 UDP frames with zero checksum.
    It reports such datagrams with L4 integrity errors in IXGBE_XEC
    register. And after afb1aa4e6df2 commit such errors are reported
    via IFCOUNTER_IERRORS. This confuses users, since actually all frames
    are handled correctly by the system.
    To workaround the problem, let's ignore the XEC register value for
    82599 cards for now.

    PR:             266048
    Discussed with: erj
    Sponsored by:   Yandex LLC
    Approved by:    so
    Security:       FreeBSD-EN-23:04.ixgbe

    (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571)
    (cherry picked from commit daf3d88ac1849c9a6c5f05d2a663c7fe57453b37)

 sys/dev/ixgbe/if_ix.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 11 commit-hook freebsd_committer freebsd_triage 2023-02-08 18:28:25 UTC
A commit in branch releng/12.3 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f31403bfdd79be98e11a00cccf4b6707e006ba7a

commit f31403bfdd79be98e11a00cccf4b6707e006ba7a
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2022-11-10 09:34:40 +0000
Commit:     Gordon Tetlow <gordon@FreeBSD.org>
CommitDate: 2023-02-08 16:32:26 +0000

    ixgbe: workaround errata about UDP frames with zero checksum

    Intel 82599 has errata related to IPv4 UDP frames with zero checksum.
    It reports such datagrams with L4 integrity errors in IXGBE_XEC
    register. And after afb1aa4e6df2 commit such errors are reported
    via IFCOUNTER_IERRORS. This confuses users, since actually all frames
    are handled correctly by the system.
    To workaround the problem, let's ignore the XEC register value for
    82599 cards for now.

    PR:             266048
    Discussed with: erj
    Sponsored by:   Yandex LLC
    Approved by:    so
    Security:       FreeBSD-EN-23:04.ixgbe

    (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571)
    (cherry picked from commit fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2)

 sys/dev/ixgbe/if_ix.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 12 commit-hook freebsd_committer freebsd_triage 2023-02-08 18:30:27 UTC
A commit in branch releng/12.4 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=36a39f0cc68fab01d2c33f4738e865664db31192

commit 36a39f0cc68fab01d2c33f4738e865664db31192
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2022-11-10 09:34:40 +0000
Commit:     Gordon Tetlow <gordon@FreeBSD.org>
CommitDate: 2023-02-08 16:32:02 +0000

    ixgbe: workaround errata about UDP frames with zero checksum

    Intel 82599 has errata related to IPv4 UDP frames with zero checksum.
    It reports such datagrams with L4 integrity errors in IXGBE_XEC
    register. And after afb1aa4e6df2 commit such errors are reported
    via IFCOUNTER_IERRORS. This confuses users, since actually all frames
    are handled correctly by the system.
    To workaround the problem, let's ignore the XEC register value for
    82599 cards for now.

    PR:             266048
    Discussed with: erj
    Sponsored by:   Yandex LLC
    Approved by:    so
    Security:       FreeBSD-EN-23:04.ixgbe

    (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571)
    (cherry picked from commit fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2)

 sys/dev/ixgbe/if_ix.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 13 Mark Linimon freebsd_committer freebsd_triage 2024-01-08 05:25:05 UTC
^Triage: assign to committer that resolved.  This fix is now in all supported branches.