Problem with large UDP packets. Vendor : 0x13b1 (Linksys) Product : 0x0041 (Linksys USB3GIGV1) After increasing net.inet.ip.maxfragsperpacket from 16 to 64. ping -c1 x.x.x.x succeeds ping -c1 -s30000 x.x.x.x succeeds ping -c1 -s 60000 x.x.x.x fails. The packet is successfully sent. The reply in not successfully received. This does work in Linux. I believe it is related to their support for Realtek RTL8152/RTL8153 Based USB Ethernet Adapters.
Hi, I believe this is the expected behaviour. I think FreeBSD has also a size limitation for fragments. @bz added. --HPS
A large ping will work with other devices. e.g. Vendor : 0x0b95 (ASIX Elec. Corp.) Product : 0x7720 (AX88x72A) # ping -c1 -s60000 192.168.1.2 PING 192.168.1.2 (192.168.1.2): 60000 data bytes 60008 bytes from 192.168.1.2: icmp_seq=0 ttl=64 time=54.000 ms --- 192.168.1.2 ping statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 54.000/54.000/54.000/0.000 ms
Can you do a tcpdump of the receive and sender side, to see exactly which packets are lost? --HPS
Usually defragging is done in software and not hardware. Might have something to do with the MTU ...
It might also be the software queue length which overflows and drops packets. I believe the stats are updated when this happens. I.E. the burst of 64 packets in a row may simply be too much for the TX queue to handle.
Thanks. I will take a look at the stats and I will get the traces. It will take me a bit of time.
I had captured some traces, but realized I made a mistake. And in trying to recapture I discovered something that I found confusing. My original testing was against a Linux PC with an on board NIC. It failed consistently. For capturing the traces, I used a QNX target and most pings were successful. I don't believe there is a fundamental issue with the Linux PC as it will work with other devices. From FreeBSD->QNX, the large pings worked a lot of the time and a trace of a failure showed a missing fragment at the remote side. (local = FreeBSD). I have two other device that I have been using and one of them also seems to show problems with larger UDP packets. (axge): Vendor : 0x0b95 (ASIX Elec. Corp.) Product : 0x1790 (AX88179) However, the following device seems to work fine (axe): Vendor : 0x0b95 (ASIX Elec. Corp.) Product : 0x7720 (AX88x72A) (FYI...the original context of this issue is nfs failing) I need to systematically capture more information and will present it more clearly with traces. Btw...unless I am mistaken, the send queue appears to be large enough, net.link.ifqmaxlen: 50 Regards.
Created attachment 223079 [details] Packet traces.
I configured a fresh system with FreeBSD 12.2 RELEASE r366954 amd64. (It's an old DELL PC.) The remote side is a Linux PC (192.168.10.2) running Ubuntu 16.04. I am using the onboard interface. (RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller). To allow large UDP packets on FreeBSD, I changed net.inet.ip.maxfragsperpacket from 16 to 64. (The default for net.inet6.ip6.maxfragsperpacket is 64). Everything is from the perspective of the FreeBSD PC. All pings originate from FreeBSD. A large ping from the on board interface is successful. I also ran tests with 4 different USB devices. test10. Cisco Linksys USB300M (successful) Vendor : 0x0b95 (ASIX Elec. Corp.) Product : 0x7720 (AX88x72A) test11. D-Link DUB-E100 (successful) Vendor : 0x2001 (D-Link Corporation) Product : 0x3c05 (DUB-E100) test12. Startech USB3200SPT (fails fo 32k.) Vendor : 0x0b95 (ASIX Elec. Corp.) Product : 0x1790 (AX88179) test13. Linksys USB3GIG (fails for 64k) Vendor : 0x13b1 (Linksys) Product : 0x0041 (Linksys USB3GIGV1) The following commands were issued: ping -c1 192.168.1.2 ping -c1 -s 8000 192.168.1.2 ping -c1 -s 32000 192.168.1.2 ping -c1 -s 64000 192.168.1.2 I tested with additional devices to highlight that some devices work and some do not. The first two were successful. The second two were not. I originally suspected that this issue was specific to the USB3GIG as there was a Linux patch for this device. The pcap traces on the local side were captured a second time, as I terminated tcpdump with a SIGKILL out of habit. This resulted in truncated files so I recaptured the local traces a second time. So they may not match exactly to with the respective capture for the Linux side, but the Linux traces do not show any missing packets. test11b - successful pings with D-Link device test12b - shows missing fragments for ping with size 32000. test13b - shows missing fragments for ping with size 64000. 32000 was successful. test12remote - captured in linux side. all packets present. test13remote - captured on Linux side. all packets present. Le me know if I can provide more information.
I'll try to digest the information you provided tomorrow. Thank you for the packet traces!
Hi, sysctl -a | grep ifqmax net.link.ifqmaxlen: 50 A wild guess until tomorrow: Can you set net.link.ifqmaxlen to 128, then re-plug the adapter and redo the failing test? --HPS
I tried it with net.link.ifqmaxlen set to 128, and it does not appear to change the behaviour. That one is a tunable, so a reboot was required after adding it to boot/loader.conf. I also included setting maxfragsperpacket in loader.conf so that I don't forget to change it again. :)
Can you confirm which USB class driver is in use for the different devices? --HPS
axe test10. Cisco Linksys USB300M (successful) Vendor : 0x0b95 (ASIX Elec. Corp.) Product : 0x7720 (AX88x72A) axe test11. D-Link DUB-E100 (successful) Vendor : 0x2001 (D-Link Corporation) Product : 0x3c05 (DUB-E100) axge test12. Startech USB3200SPT (fails fo 32k.) Vendor : 0x0b95 (ASIX Elec. Corp.) Product : 0x1790 (AX88179) cdce test13. Linksys USB3GIG (fails for 64k) Vendor : 0x13b1 (Linksys) Product : 0x0041 (Linksys USB3GIGV1) There is another device that probably fails but I do not have it right now. Vendor : 0x0b95 (SMC) Product : 0x1720 (SMC2209USB/ETH) It would use the axe driver. I can get it if we need it. The person who ran into the nfs problem has this device. I have managed to get my hands on a USB analyzer. If there is something specific you would like me to look for, I can look at a USB trace. So far it looks like the frames are probably getting dropped in the device, as there are no USB transfers for the missing frames.