Bug 204409 - Uploading to a FreeBSD sshd server drops bandwidth
Summary: Uploading to a FreeBSD sshd server drops bandwidth
Status: Closed Unable to Reproduce
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.2-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs mailing list
Depends on:
Reported: 2015-11-09 23:12 UTC by VK
Modified: 2015-12-02 14:31 UTC (History)
0 users

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description VK freebsd_triage 2015-11-09 23:12:56 UTC
It appears that UPLOADING to sshd (eg. scp localfile remote.tld:remotefile) will happen at very limited bandwidth.

I haven't been able to isolate which circumstances those are precisely, but, between two servers when the command is reversed (eg. scp remote.tld:remotefile ./localfile) the transfer happens at full available speed.

To make this clear, for a particular set of servers, for example:

on server A:
scp localfile B:remotefile   ->   40Mbps  (upload to sshd)
scp B:remotefile localfile   ->  120Mbps  (download from sshd)

scp localfile B:remotefile   ->  200Mbps  (upload to sshd, Debian on B)
scp B:remotefile localfile   ->  120Mbps  (download from sshd, Debian on B)

on server B:
scp localfile A:remotefile   ->   30Mbps  (upload to sshd)
scp A:remotefile localfile   ->  120Mbps  (download from sshd)

scp localfile A:remotefile   ->   40Mbps  (upload to sshd, Debian on B)
scp B:remotefile localfile   ->  200Mbps  (download from sshd, Debian on B)

NOTE: When upload is _TO_ FreeBSD sshd, the bandwidth is 30-40Mbps. 

Which means the differences between upload and download are not due to network congestion or particular datacenter setup, whether a server is dedicated or VPS, UFS2 filesystem or ZFS, 1GB of RAM or 16GB. Though all were 64bit. Tested between Leaseweb Amsterdam, DigitalOcean Amsterdam, DigitalOcean Frankfurt, Hetzner Germany (Falkenstein) and Rackspace in London.

This also affects rsync over ssh, so it's not just scp. The only thing that changes are absolute bandwidth numbers for a given set of servers. 

Testing other protocols (http, https, netcat, ...) shows full network speed available between testing servers.

Different IPQoS or Cipher settings do not improve anything.
Comment 1 VK freebsd_triage 2015-11-09 23:15:14 UTC
(In reply to Vlad K. from comment #0)

Errata, that last scp should be:

scp A:remotefile localfile ...
Comment 2 VK freebsd_triage 2015-11-10 21:29:20 UTC
It appears that using OpenSSH-portable on the same servers mostly fixes the issue.

Why is it "mostly" and not for all servers, I'm still investigating, could be that one particular pair of servers I'm still seeing 3Mpbs (on a 100MBps link, 8 hops and 11msec away from each other) are affected by something in the provider's network configuration.
Comment 3 VK freebsd_triage 2015-12-02 14:31:16 UTC
I'm closing my own report, I can't replicate the issue, in some networks it works as expected in some it doesn't and I can't find the common denominator. I can easily file another report when I'm certain of the cause.