Created attachment 233823 [details] python script for sending a good/bad checksum packet The dwc driver's RXCSUM hardware offloading is not able to detect the packet with a bad checksum. In my development environment, I will manually send a packet with a good checksum and a packet with a bad checksum respectively to the dwc0 interface. The python script and Wireshark log are attached. The setup of the environment is as follows On host $ ifconfig enp0s31f6 enp0s31f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.3.2 netmask 255.255.255.0 broadcast 192.168.3.255 ether 8c:8c:aa:c1:2b:c3 txqueuelen 1000 (Ethernet) RX packets 2378 bytes 991956 (991.9 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 14312 bytes 1872875 (1.8 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 16 memory 0xae380000-ae3a0000 On the Freebsd current image root@generic:~ # uname -a FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n254961-b91a48693a5: Thu Apr 21 09:35:51 UTC 2022 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 root@generic:~ # ifconfig dwc0 192.168.3.129/24 root@generic:~ # ifconfig dwc0 -rxcsum -txcsum root@generic:~ # ifconfig dwc0 dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80008<VLAN_MTU,LINKSTATE> ether fa:97:92:f6:f1:09 inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> Test Procedure 1. Check dwc0 config showing that the RXCSUM offloading is off in Freebsd root@generic:~ # ifconfig dwc0 dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80008<VLAN_MTU,LINKSTATE> ether fa:97:92:f6:f1:09 inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> 2. Check the current netstat in the Freebsd root@generic:~ # netstat -s -p ip ip: 42 total packets received 5 bad header checksums 3. Send a good packet from the host to dwc0 sudo python3 send_raw_ping.py -s good 4. Check the current netstat in Freebsd root@generic:~ # netstat -s -p ip ip: 43 total packets received 5 bad header checksums 5. Send a bad packet from the host to dwc0 sudo python3 send_raw_ping.py -s bad 6. Check the current netstat in Freebsd. The bad checksum packet is detected. root@generic:~ # netstat -s -p ip ip: 44 total packets received 6 bad header checksums 7. Check the dwc0 interface config in Freebsd showing the RXCSUM is off. root@generic:~ # ifconfig dwc0 dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80008<VLAN_MTU,LINKSTATE> ether fa:97:92:f6:f1:09 inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> 8. Turn on the dwc0's RXCSUM capability in Freebsd root@generic:~ # ifconfig dwc0 rxcsum 9. Check the dwc0 interface config in Freebsd showing the RXCSUM is on root@generic:~ # ifconfig dwc0 dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80009<RXCSUM,VLAN_MTU,LINKSTATE> ether fa:97:92:f6:f1:09 inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> 11. Check the current netstat in Freebsd root@generic:~ # netstat -s -p ip ip: 44 total packets received 6 bad header checksums 11. Send a good packet from the host to dwc0 sudo python3 send_raw_ping.py -s good 12. Check the current netstat in Freebsd. root@generic:~ # netstat -s -p ip ip: 45 total packets received 6 bad header checksums 13. Send a bad packet from the host to dwc0 sudo python3 send_raw_ping.py -s bad 14. Check the current netstat in Freebsd. The bad checksum packet is not detected. root@generic:~ # netstat -s -p ip ip: 46 total packets received 6 bad header checksums Expected Results: At step 14, the results should be root@generic:~ # netstat -s -p ip ip: 46 total packets received 7 bad header checksums The packet with bad checksum should be detected and recorded at the netstat. Do I miss any configuration to enable the RXCSUM hardware offloading? Or the RXCSUM is not supported at the dwc driver now.
Created attachment 233824 [details] network traffic captured by Wireshark. Turn on the IP checksum validation in Wireshark to show the good/bad checksum in the packet
Comment on attachment 233824 [details] network traffic captured by Wireshark. Turn on the IP checksum validation in Wireshark to see the good/bad checksum packet.
Created attachment 233961 [details] enable IP csum offload engine Hi, Thanks for the report. I noticed that we do not enable the IP checksum engine in the configuration register, so this may be all that is missing. The attached patch should enable this feature. I have not tested it myself, can you please give it a try? Mitchell
(In reply to Mitchell Horne from comment #3) Hi, Thanks for the patch. This patch can fix the issue of not detecting the packets with a bad RX checksum. It would be nice if the enable/disable of RX/TX checksum hardware offloading can be integrated into dwc_ioctl(). Is there a plan in Freebsd to support this kind of configuration?
Created attachment 234238 [details] enable_rx_csum_by_init_ioctl Hi, I tried to make the enable/disable of RX CSUM integrated into dwc interface's init and ioctl for my development purposes. The first "return" at the function dwc_rxfinish_one() is deleted. The reason is that the counters, IFCOUNTER_IPACKETS and IFCOUNTER_IMCASTS, are not updated globally anymore based on your patch in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263817.
(In reply to Jiahao LI from comment #5) Thank you for testing and completing the patch! I have posted it for review with a couple of small tweaks. The return from dwc_rxfinish_one() I have incorporated to the patch for PR263817. While testing the change I found that packets with a bad checksum were simply dropped on my device, unless I also set bit 26 of the OPMODE register ("Disable Dropping of TCP/IP Checksum Error Frames"). I don't think this is an issue, but it took me a moment to figure out why the counters weren't increasing.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=35c9edab4169f99de7e8fcada6d9b499c8405f87 commit 35c9edab4169f99de7e8fcada6d9b499c8405f87 Author: Jiahao Li <jiahali@blackberry.com> AuthorDate: 2022-06-21 13:23:43 +0000 Commit: Mitchell Horne <mhorne@FreeBSD.org> CommitDate: 2022-06-23 18:15:10 +0000 if_dwc: enable RX checksum offload feature We claim support in ifcaps, but don't actually enable it. PR: 263886 Reviewed by: manu MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D35498 sys/dev/dwc/if_dwc.c | 21 +++++++++++++++++++++ sys/dev/dwc/if_dwc.h | 1 + 2 files changed, 22 insertions(+)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=164bdd8f9a49762742f9d2a466d427dd9c95a674 commit 164bdd8f9a49762742f9d2a466d427dd9c95a674 Author: Jiahao Li <jiahali@blackberry.com> AuthorDate: 2022-06-21 13:23:43 +0000 Commit: Mitchell Horne <mhorne@FreeBSD.org> CommitDate: 2022-07-04 16:34:56 +0000 if_dwc: enable RX checksum offload feature We claim support in ifcaps, but don't actually enable it. PR: 263886 Reviewed by: manu MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D35498 (cherry picked from commit 35c9edab4169f99de7e8fcada6d9b499c8405f87) sys/dev/dwc/if_dwc.c | 21 +++++++++++++++++++++ sys/dev/dwc/if_dwc.h | 1 + 2 files changed, 22 insertions(+)