At CacheIQ we discovered a problem where sosend() would occassionally return EINVAL. This is pretty bad, as it leads our software to believe something was wrong with the calling arguments or perhaps the socket is in a problematic state. Digging into it we found that every now and then requests larger than dmat->maxsize are rejected by bus_dmamap_load_mbuf_sg(). It's important to note that in our environment we are using the Intel 82599 10Gb/s adapter with TSO enabled and VLAN tagging (QINQ). The problem only occurs for TSO packets generated by tcp_output(). When the failure occurs, we see that tcp_ouput() submits an mbuf chain to ip_output() that is 65520 bytes long. ip_output() adds on the link header bringing it to 65534 bytes. Finally, vlan_tag() adds 4 more bytes bringing it to 65538 bytes, and thus failing the lower level checks. As a quick fix we factor in max_linkhdr into the TSO length adjustment in tcp_output(). Fix: Factor max_linkhdr into the TSO length adjustment in tcp_output(). Patch attached with submission follows: How-To-Repeat: Enable TSO and VLAN tagging for 82599 NIC, and drive a lot of data through the system. We use a custom tool, but any networking tool that can keep the link saturated should illustrate the problem. Note, our software is based on FreeBSD-8.2, but includes a number of fixes and select updates.
Responsible Changed From-To: freebsd-bugs->freebsd-net Over to maintainer(s).
Looks like https://svnweb.freebsd.org/base?view=revision&revision=287775 fixed this. Greg, thanks for your submission.