Bug 240827 - ftp(1) is slower than fetch(1)
Summary: ftp(1) is slower than fetch(1)
Status: In Progress
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Hiroki Sato
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-26 01:43 UTC by Yuichiro NAITO
Modified: 2019-10-10 00:54 UTC (History)
1 user (show)

See Also:
koobs: mfc-stable11?
koobs: mfc-stable12?


Attachments
ftp.patch (1.77 KB, patch)
2019-09-26 01:43 UTC, Yuichiro NAITO
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yuichiro NAITO 2019-09-26 01:43:41 UTC
Created attachment 207828 [details]
ftp.patch

Of course this problem depends on network environment,
I can see this problem in my home network only.

In my home network,
the bandwidth of the internet connection is 100Mbps.
Fetch(1) downloads in mostly full speed of the bandwidth.
But ftp(1) seems about half speed of fetch(1).

I have tried to download FreeBSD iso image file
from the fastest mirror from my home.
The results are shown below.

```
$ fetch ftp://ftp.jp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso
FreeBSD-12.0-RELEASE-amd64-disc1.iso                   851 MB   10 MBps 01m20s
```

```
$ ftp ftp://ftp.jp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso
Trying 2405:f000:202:2598:210:231:212:2:21 ...
Connected to ftp.lab.bbtower.net.
220- Welcome to ftp.lab.BBTower.net,
220-            Located in Tokyo, Japan.
220-
220- This server maintenanced by BroadBand Tower, Inc.
220-
220-             BBTower Web site is at
220-             http://www.bbtower.co.jp/
220-
220- If you have any problem, please contact ftp-contact@bbtower.co.jp.
220 FTP server ready.
331 Guest login ok, send your email address as password.
230 Guest login ok, access restrictions apply.
Remote system type is UNKNOWN.
200 Type set to I.
250 CWD command successful.
250 CWD command successful.
250 CWD command successful.
250 CWD command successful.
250 CWD command successful.
local: FreeBSD-12.0-RELEASE-amd64-disc1.iso remote: FreeBSD-12.0-RELEASE-amd64-disc1.iso
229 Entering Extended Passive Mode (|||63019|)
150 Opening BINARY mode data connection for 'FreeBSD-12.0-RELEASE-amd64-disc1.iso' (892467200 bytes).
100% |***********************************|   851 MiB    5.14 MiB/s    00:00 ETA
226 Transfer complete.
892467200 bytes received in 02:45 (5.14 MiB/s)
221 Goodbye.
```

This is because socket buffer size is not large enough to ftp(1).
Ftp(1) always set the buffer size to the default value.
It prevents FreeBSD kernel from increasing socket buffer size automatically.

So setting `sysctl net.inet.tcp.recvspace=131072` speeds up the ftp download
for workaround.

```
$ sudo sysctl net.inet.tcp.recvspace=131072
Password:
net.inet.tcp.recvspace: 65536 -> 131072
$ ftp ftp://ftp.jp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso
Trying 2405:f000:202:2598:210:231:212:2:21 ...
Connected to ftp.lab.bbtower.net.
220- Welcome to ftp.lab.BBTower.net,
220-            Located in Tokyo, Japan.
220-
220- This server maintenanced by BroadBand Tower, Inc.
220-
220-             BBTower Web site is at
220-             http://www.bbtower.co.jp/
220-
220- If you have any problem, please contact ftp-contact@bbtower.co.jp.
220 FTP server ready.
331 Guest login ok, send your email address as password.
230 Guest login ok, access restrictions apply.
Remote system type is UNKNOWN.
200 Type set to I.
250 CWD command successful.
250 CWD command successful.
250 CWD command successful.
250 CWD command successful.
250 CWD command successful.
local: FreeBSD-12.0-RELEASE-amd64-disc1.iso remote: FreeBSD-12.0-RELEASE-amd64-disc1.iso
229 Entering Extended Passive Mode (|||59743|)
150 Opening BINARY mode data connection for 'FreeBSD-12.0-RELEASE-amd64-disc1.iso' (892467200 bytes).
100% |***********************************|   851 MiB    9.67 MiB/s    00:00 ETA
226 Transfer complete.
892467200 bytes received in 01:27 (9.67 MiB/s)
221 Goodbye.
```

Basically ftp(1) should allow FreeBSD kernel to adjust socket buffer size.
I suggest the attached patch that prevents ftp(1) from setting socket buffer size
unless user input 'rcvbuf' or 'sndbuf' command.
Comment 1 Hiroki Sato freebsd_committer 2019-09-26 04:09:58 UTC
Take.
Comment 2 Yuichiro NAITO 2019-10-10 00:54:03 UTC
Two weeks have been passed.
Are there any problem in my patch?
Or are you busy now?