Bug 214171 - [tcp] Out-of-order transmission severely degrades performance
Summary: [tcp] Out-of-order transmission severely degrades performance
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.3-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-net (Nobody)
Depends on:
Reported: 2016-11-03 18:04 UTC by noah.bergbauer
Modified: 2016-11-16 00:28 UTC (History)
3 users (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description noah.bergbauer 2016-11-03 18:04:52 UTC
Downloading data from a FreeBSD server over a fast (~700Mbps with 0.1% loss) internet connection that suffers from a lot of packet reordering, I'm seeing very bad transmission rates (~4Mbps whereas Linux gets ~40Mbps on the same connection).

Traffic analysis shows that when the client (Windows in my tests) receives out-of-order packets, it first sends out SACKs until the missing packet arrives (usually just ~1ms late). Then it ACKs everything normally.

Linux reacts to this by just carrying on - after all, no packets were dropped. FreeBSD however appears to react to each SACK immediately and indepentently, assuming that the late packets were actually lost. Because of that, it retransmits them and lowers the transmission rate (I tested with several different congestion control modules and it affects all of them).
Comment 1 Hiren Panchasara freebsd_committer 2016-11-03 20:30:09 UTC
Yes, you are right. We don't do well with loss in general and ooo packets in particular. We are trying to fix a bunch of issues around that right now.

Would it be possible to you to test -CURRENT? That should work a little better.
Comment 2 noah.bergbauer 2016-11-03 21:51:55 UTC
(In reply to Hiren Panchasara from comment #1)

>Would it be possible to you to test -CURRENT? That should work a little better.

I grabbed the latest snapshot (FreeBSD 12.0-CURRENT #0 r308137: Mon Oct 31 19:21:13 UTC 2016     root@releng3.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64) and did not see any significant improvements.
Comment 3 Hiren Panchasara freebsd_committer 2016-11-03 22:01:43 UTC
(In reply to noah.bergbauer from comment #2)
You'd also want to get r308180 so probably try next snapshot.

But let me clarify a few things. We don't currently have a way to detect reordering and thus react to it in sensible way. So that's something we'd have to deal with at least for next few months. The fixes I mentioned should help the case where previously we were going down to 1MSS on any loss, thus slowing down all the way to reacting a little better.