Bug 203922 - The kern.ipc.acceptqueue limit is too low
Summary: The kern.ipc.acceptqueue limit is too low
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Alfred Perlstein
URL:
Keywords: needs-qa, patch, performance
Depends on:
Blocks:
 
Reported: 2015-10-21 11:48 UTC by White Knight
Modified: 2016-03-20 22:29 UTC (History)
9 users (show)

See Also:
koobs: mfc-stable10?
koobs: mfc-stable9?


Attachments
A patch that changes kern.ipc.acceptqueue to unsigned integer. (7.46 KB, patch)
2015-10-21 11:48 UTC, White Knight
no flags Details | Diff
A patch which keeps the ABI of xsctp_inpc intact. (8.65 KB, patch)
2016-01-27 12:04 UTC, White Knight
no flags Details | Diff
A revisited patch, without copyright statements. (5.97 KB, patch)
2016-01-30 11:55 UTC, White Knight
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description White Knight 2015-10-21 11:48:53 UTC
Created attachment 162281 [details]
A patch that changes kern.ipc.acceptqueue to unsigned integer.

The kernel parameter kern.ipc.acceptqueue, formerly known as kern.ipc.somaxconn, is limited to unsigned short.

This limit is way too low for very active sites.

Attached is a patch that widens the data type to unsigned integer.
Comment 1 Sean Bruno freebsd_committer freebsd_triage 2015-10-28 15:48:15 UTC
Hrm ... this is interesting.  Adding network people to this issue for evaluation.
Comment 2 Chris Hutchinson 2016-01-16 06:52:24 UTC
(In reply to Sean Bruno from comment #1)
> Hrm ... this is interesting.  Adding network people to this issue for
> evaluation.

Interesting indeed. I'd go one further, and say it makes good sense.
Any reason this can't be implemented? I'm not sure how, but would an
exp-run be of any value?

--Chris
Comment 3 Konstantin Belousov freebsd_committer freebsd_triage 2016-01-16 10:00:09 UTC
(In reply to Chris Hutchinson from comment #2)
The patch breaks the ABI, e.g. for struct xsctp_inpcb.
Comment 4 White Knight 2016-01-16 13:29:44 UTC
(In reply to Konstantin Belousov from comment #3)

Is there another consumer of that structure than netstat?

Can you suggest how to change the structure and maintain the ABI?
Comment 5 Konstantin Belousov freebsd_committer freebsd_triage 2016-01-16 19:32:50 UTC
(In reply to White Knight from comment #4)
I have no idea about any users of the structure, I am only sure that it was written with intent of keeping its ABI.

The extra_padding field was added to allow extending the structure content.
Comment 6 Alfred Perlstein freebsd_committer freebsd_triage 2016-01-16 21:15:06 UTC
The correct way would be to retain the field as-is but rename it to "_old".

Then add a field with the correct width.

Then when exporting the field make sure to continue to export the _old struct but carefully assign it using code as follows:
xpcb->qlen = newval
xpcb->qlen_old = qlen->qlen > (uint32_t)MAX_ISHORT ? MAX_USHORT : qlen->qlen

Then possibly mark the old field for deprecation, but still really shouldn't deprecate if we intend to be forever backwards compatible.

To _really_ fix it, there really should be an accessor library written so that no one has to even know the size of xpcb other than the library.
Comment 7 White Knight 2016-01-27 12:04:45 UTC
Created attachment 166184 [details]
A patch which keeps the ABI of xsctp_inpc intact.

This patch addresses comment #6 and keeps the xsctp_inpc ABI intact.

It has also been updated to base r294778.
Comment 8 Konstantin Belousov freebsd_committer freebsd_triage 2016-01-29 11:34:05 UTC
(In reply to White Knight from comment #7)
The patch is mostly fine.  I do not see a need in the casts to uint32_t in sctp_sysctl_handle_assoclist(), the normal promotion rules would do the right thing.

Also, I believe that somebody else at some other place already noted that such change does not deserve addition of the copyright lines to the affected files.  I do not see the patch with these claims acceptable, and we cannot strip your copyright notice.
Comment 9 Alfred Perlstein freebsd_committer freebsd_triage 2016-01-30 00:16:54 UTC
At a glance the patch looks good.  Thank you for doing this work.  I will commit shortly.
Comment 10 Alfred Perlstein freebsd_committer freebsd_triage 2016-01-30 00:19:40 UTC
(In reply to White Knight from comment #7)

Please authorize removal of the copyright notice, there is not sufficient new work here to allow for copyright changes.
Comment 11 Steven Hartland freebsd_committer freebsd_triage 2016-01-30 01:31:07 UTC
Does the extrapadding in the struct need reducing by the size of the new fields?
Comment 12 Alfred Perlstein freebsd_committer freebsd_triage 2016-01-30 04:03:21 UTC
(In reply to Steven Hartland from comment #11)

Yes it does.
Comment 13 White Knight 2016-01-30 11:55:22 UTC
Created attachment 166298 [details]
A revisited patch, without copyright statements.

A patch without copyright statements, as per comment #8 and comment #9.

Also without the extranenous typecast.

Tested with base r295065.
Comment 14 White Knight 2016-01-30 11:57:23 UTC
(In reply to Alfred Perlstein from comment #9)

No problem.  You can choose the new patch, or edit the old one.

Thank you.
Comment 15 Alfred Perlstein freebsd_committer freebsd_triage 2016-01-31 10:04:05 UTC
OK, I am trying out this patch, it should work, however cursory testing isn't working out as I'd hope.  When I am running the following python program:

import socket
import struct

serversocket = socket.socket(
    socket.AF_INET, socket.SOCK_STREAM)
#bind the socket to a public host,
# and a well-known port
serversocket.bind(('0.0.0.0', 8080))
#become a server socket
serversocket.listen(5)
qlimit = serversocket.getsockopt(socket.SOL_SOCKET, 0x1011)
print qlimit
#print struct.unpack("i", qlimit)
s = serversocket.accept()

And then I run in another terminal "netstat -L" I get the following output:
spigot# netstat -L
Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen         Local Address         
spigot# 

It seems like qlimit is being set to zero as I've added this snippet of code to instrument:
@@ -460,8 +460,10 @@
 			xo_emit("\n");
 			first = 0;
 		}
-		if (Lflag && so->so_qlimit == 0)
+		if (Lflag && so->so_qlimit == 0) {
+			fprintf(stderr, "qlimit==0\n");
 			continue;
+		}
 		xo_open_instance("socket");
 		if (Aflag) {
 			if (istcp)

I will test some more tomorrow, I am probably just using netstat incorrectly or something else is broken with my setup.

However if you want to maybe tell me if this works with testing and why my test program fails to show listening queue max, that would be splendid.

Thank you.
Comment 16 Alfred Perlstein freebsd_committer freebsd_triage 2016-02-01 05:35:40 UTC
OK, I have sorted this out, I needed to add "-a" to netstat while testing, surprised that "-L" did not implicitly set "-a".

Will be committed shortly.
Comment 17 Alfred Perlstein freebsd_committer freebsd_triage 2016-02-01 05:37:29 UTC
Oh, last question before commit, what do you want your attribution to be under?  

White Knight <white_knight@2ch.net> or some other name/email?
Comment 18 White Knight 2016-02-01 08:33:28 UTC
(In reply to Alfred Perlstein from comment #17)

White Knight <white_knight@2ch.net> is the correct name/address.  If possible some kudos to 2ch.net for sponsoring the work would be nice too.

Thank you.
Comment 19 commit-hook freebsd_committer freebsd_triage 2016-02-02 05:58:43 UTC
A commit references this bug:

Author: alfred
Date: Tue Feb  2 05:58:00 UTC 2016
New revision: 295136
URL: https://svnweb.freebsd.org/changeset/base/295136

Log:
  Increase max allowed backlog for listen sockets
  from short to int.

  PR: 203922
  Submitted by: White Knight <white_knight@2ch.net>
  MFC After: 4 weeks

Changes:
  head/sys/kern/uipc_debug.c
  head/sys/kern/uipc_socket.c
  head/sys/netinet/sctp_sysctl.c
  head/sys/netinet/sctp_uio.h
  head/sys/sys/socketvar.h
  head/usr.bin/netstat/inet.c
  head/usr.bin/netstat/sctp.c
  head/usr.bin/netstat/unix.c
Comment 20 Alfred Perlstein freebsd_committer freebsd_triage 2016-02-02 07:30:17 UTC
Committed, thank you!
Comment 21 Alfred Perlstein freebsd_committer freebsd_triage 2016-03-20 22:29:41 UTC
Code committed in r295136, unfortunately can't be pushed into -stable/10.x as-is