After https://reviews.freebsd.org/D27191 ixgbe(4) uses sum of all errors registers in the IFCOUNTER_IERRORS. Especially IXGBE_XEC is now included in this sum. And now we have noticeable input errors rate: % netstat -hw1 -I ix0 input ix0 output packets errs idrops bytes packets errs bytes colls 7.7k 14 0 2.0M 4.3k 0 1.9M 0 16k 20 0 27M 30k 0 29M 0 10k 40 0 18M 20k 0 19M 0 12k 23 0 18M 21k 0 20M 0 ^C % sysctl dev.ix.0 | grep errs dev.ix.0.mac_stats.checksum_errs: 25190176 dev.ix.0.mac_stats.rec_len_errs: 0 dev.ix.0.mac_stats.byte_errs: 0 dev.ix.0.mac_stats.ill_errs: 0 dev.ix.0.mac_stats.crc_errs: 0 dev.ix.0.mac_stats.rx_errs: 25190176 % pciconf -l | grep ^ix0 ix0@pci0:6:0:0: class=0x020000 card=0x00068086 chip=0x10fb8086 rev=0x01 hdr=0x00 I found similar report about OpenSense: https://forum.opnsense.org/index.php?topic=27051.0 It looks like 82599ES has errata related to IPv4 UDP packets and zero checksum: 44. Integrity Error Reported for IPv4/UDP Packets With Zero Checksum Problem:According to the UDP specification “an all zero transmitted checksum value means that the transmitter generated no checksum (for debugging or for higher level protocols that don’t care)”, these packets should be received without a checksum error notation. The 82599 reports an L4 integrity error if such packets are received. Implication: UDP packets without a checksum will have an L4 integrity error indication in the Rx descriptor. See also: https://bugzilla.redhat.com/show_bug.cgi?id=1922430
Thanks for the link to the Red Hat (and inside, dpdk commits) bug tracker -- it looks like we need to add a 82599-specific workaround when it comes to counting errors, as well as marking these bad checksum packets as having no checksum calculated so that SW can properly re-compute them instead of assuming them as being bad packets.
I believe I am also affected by this driver bug. I have tried this on a vanilla Freebsd install and the errors IN are still there... however, I did first notice this on an OPNsense install. See the details here: https://forum.opnsense.org/index.php?topic=30645
Created attachment 237452 [details] proposed patch I think for now we can use the same solution as here: https://git.dpdk.org/dpdk/commit/?id=2ee14c8905e9
We were testing https://github.com/opnsense/src/commit/4a788be44e0 yesterday and it was confirmed working. Your patch is shorter, happy to see it committed! Cheers, Franco
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=8526120ad41ca47367b43f8f4459e0fa61285571 commit 8526120ad41ca47367b43f8f4459e0fa61285571 Author: Andrey V. Elsukov <ae@FreeBSD.org> AuthorDate: 2022-11-10 09:34:40 +0000 Commit: Andrey V. Elsukov <ae@FreeBSD.org> CommitDate: 2022-11-10 09:34:40 +0000 ixgbe: workaround errata about UDP frames with zero checksum Intel 82599 has errata related to IPv4 UDP frames with zero checksum. It reports such datagrams with L4 integrity errors in IXGBE_XEC register. And after afb1aa4e6df2 commit such errors are reported via IFCOUNTER_IERRORS. This confuses users, since actually all frames are handled correctly by the system. To workaround the problem, let's ignore the XEC register value for 82599 cards for now. PR: 266048 Discussed with: erj MFC after: 1 week Sponsored by: Yandex LLC sys/dev/ixgbe/if_ix.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=daf3d88ac1849c9a6c5f05d2a663c7fe57453b37 commit daf3d88ac1849c9a6c5f05d2a663c7fe57453b37 Author: Andrey V. Elsukov <ae@FreeBSD.org> AuthorDate: 2022-11-10 09:34:40 +0000 Commit: Andrey V. Elsukov <ae@FreeBSD.org> CommitDate: 2022-11-17 20:13:43 +0000 ixgbe: workaround errata about UDP frames with zero checksum Intel 82599 has errata related to IPv4 UDP frames with zero checksum. It reports such datagrams with L4 integrity errors in IXGBE_XEC register. And after afb1aa4e6df2 commit such errors are reported via IFCOUNTER_IERRORS. This confuses users, since actually all frames are handled correctly by the system. To workaround the problem, let's ignore the XEC register value for 82599 cards for now. PR: 266048 Discussed with: erj Sponsored by: Yandex LLC (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571) sys/dev/ixgbe/if_ix.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2 commit fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2 Author: Andrey V. Elsukov <ae@FreeBSD.org> AuthorDate: 2022-11-10 09:34:40 +0000 Commit: Andrey V. Elsukov <ae@FreeBSD.org> CommitDate: 2022-11-17 20:16:15 +0000 ixgbe: workaround errata about UDP frames with zero checksum Intel 82599 has errata related to IPv4 UDP frames with zero checksum. It reports such datagrams with L4 integrity errors in IXGBE_XEC register. And after afb1aa4e6df2 commit such errors are reported via IFCOUNTER_IERRORS. This confuses users, since actually all frames are handled correctly by the system. To workaround the problem, let's ignore the XEC register value for 82599 cards for now. PR: 266048 Discussed with: erj Sponsored by: Yandex LLC (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571) sys/dev/ixgbe/if_ix.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Thanks for fixing this and MFC-ing back to the older branches! Can this be marked as closed?
(In reply to Eric Joyner from comment #8) I didn't meet the deadline before 12.4-RELEASE, so Glen said it probably will be included in the post-release Errata. I think then we will close this PR.
A commit in branch releng/13.1 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f3e20eb8d8f0b87ae828741c48e9ebd7913086ac commit f3e20eb8d8f0b87ae828741c48e9ebd7913086ac Author: Andrey V. Elsukov <ae@FreeBSD.org> AuthorDate: 2022-11-10 09:34:40 +0000 Commit: Gordon Tetlow <gordon@FreeBSD.org> CommitDate: 2023-02-08 16:30:38 +0000 ixgbe: workaround errata about UDP frames with zero checksum Intel 82599 has errata related to IPv4 UDP frames with zero checksum. It reports such datagrams with L4 integrity errors in IXGBE_XEC register. And after afb1aa4e6df2 commit such errors are reported via IFCOUNTER_IERRORS. This confuses users, since actually all frames are handled correctly by the system. To workaround the problem, let's ignore the XEC register value for 82599 cards for now. PR: 266048 Discussed with: erj Sponsored by: Yandex LLC Approved by: so Security: FreeBSD-EN-23:04.ixgbe (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571) (cherry picked from commit daf3d88ac1849c9a6c5f05d2a663c7fe57453b37) sys/dev/ixgbe/if_ix.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
A commit in branch releng/12.3 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f31403bfdd79be98e11a00cccf4b6707e006ba7a commit f31403bfdd79be98e11a00cccf4b6707e006ba7a Author: Andrey V. Elsukov <ae@FreeBSD.org> AuthorDate: 2022-11-10 09:34:40 +0000 Commit: Gordon Tetlow <gordon@FreeBSD.org> CommitDate: 2023-02-08 16:32:26 +0000 ixgbe: workaround errata about UDP frames with zero checksum Intel 82599 has errata related to IPv4 UDP frames with zero checksum. It reports such datagrams with L4 integrity errors in IXGBE_XEC register. And after afb1aa4e6df2 commit such errors are reported via IFCOUNTER_IERRORS. This confuses users, since actually all frames are handled correctly by the system. To workaround the problem, let's ignore the XEC register value for 82599 cards for now. PR: 266048 Discussed with: erj Sponsored by: Yandex LLC Approved by: so Security: FreeBSD-EN-23:04.ixgbe (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571) (cherry picked from commit fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2) sys/dev/ixgbe/if_ix.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
A commit in branch releng/12.4 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=36a39f0cc68fab01d2c33f4738e865664db31192 commit 36a39f0cc68fab01d2c33f4738e865664db31192 Author: Andrey V. Elsukov <ae@FreeBSD.org> AuthorDate: 2022-11-10 09:34:40 +0000 Commit: Gordon Tetlow <gordon@FreeBSD.org> CommitDate: 2023-02-08 16:32:02 +0000 ixgbe: workaround errata about UDP frames with zero checksum Intel 82599 has errata related to IPv4 UDP frames with zero checksum. It reports such datagrams with L4 integrity errors in IXGBE_XEC register. And after afb1aa4e6df2 commit such errors are reported via IFCOUNTER_IERRORS. This confuses users, since actually all frames are handled correctly by the system. To workaround the problem, let's ignore the XEC register value for 82599 cards for now. PR: 266048 Discussed with: erj Sponsored by: Yandex LLC Approved by: so Security: FreeBSD-EN-23:04.ixgbe (cherry picked from commit 8526120ad41ca47367b43f8f4459e0fa61285571) (cherry picked from commit fe9c4deda9d4aa2c5bed75071f8006bd2a0734a2) sys/dev/ixgbe/if_ix.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
^Triage: assign to committer that resolved. This fix is now in all supported branches.