getsockopt(SO_SNDBUF) does not return the value set by getsockopt(SO_SNDBUF). HPN-SSH would like to determine the value set by setsockopt(SO_SNDBUF) so that it can fill it up. This helps to use more bandwidth on a high BDP link. HPN-SSH tries to determine this value by periodically calling getsockopt(SO_SNDBUF) at [1]. However, on FreeBSD, getsockopt(SO_SNDBUF) does not return the value set by setsockopt(SO_SNDBUF). Rather it returns the "high watermark". The net result is that on high BDP links, HPN-SSH does nothing to increase bytes-in-flight. [1] /usr/src/crypto/openssh/roaming_common.c line 59 in get_snd_buf_size: getsockopt(fd, SOL_SOCKET, SO_SNDBUF, &optval, &optvallen)
Typo on the first line of comment #0: get->set getsockopt(SO_SNDBUF) does not return the value set by setsockopt(SO_SNDBUF).
des@, is this of interest?
No, this is a kernel issue.
Maybe I miss something, but looks like there is no bug in FreeBSD. Yes, on FreeBSD getsockopt(..., SO_SNDBUF, ...) returns "high watermark", but setsockopt(..., SO_SNDBUF, ...) attempts to resize the buffer and, on success, sets "high watermark" to the new buffer size. So getsockopt() returns the value set by getsockopt(). I have written a simple test and run it on the 10.1-RELEASE and 10.2-RELEASE: -----8<----- #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <err.h> #include <stdio.h> int main () { int s, size; socklen_t sizelen; s = socket(AF_INET6, SOCK_RAW, IPPROTO_IP); sizelen = sizeof size; getsockopt(s, SOL_SOCKET, SO_SNDBUF, &size, &sizelen); printf("Current socket send buffer size: %i\n", size); size /= 2; setsockopt(s, SOL_SOCKET, SO_SNDBUF, &size, sizelen); getsockopt(s, SOL_SOCKET, SO_SNDBUF, &size, &sizelen); printf("New socket send buffer size: %i\n", size); } -----8<----- $ cc test.so_sndbuf.c $ sudo ./a.out Current socket send buffer size: 9216 New socket send buffer size: 4608
(In reply to timon from comment #4) Thank you. I modified the example to increase the buffer size (vs. decrease) and I see the same thing. It must be that the conclusions reached in this discussion are wrong: == begin quote of [1] == So, I looked at what getsockopt SO_RCVBUF returns, and it returns: case SO_RCVBUF: optval = so->so_rcv.sb_hiwat; Which is NOT S-BMAX, so it looks like OpenSSH only ever gets 66052 bytes for the buffer... If it's decided to base it's waiting for ack on SO_RCVBUF, then this is probably where the issue is... == end quote of [1] === So it sounds like this bug should be closed/worksforme. Chris [1] http://freebsd.1045724.n5.nabble.com/ssh-over-WAN-TCP-window-too-small-tp6035112p6035301.html