It appears that in sysctl_tcp_hc_list, the function used for 'sysctl net.inet.tcp.hostcache.list', there is a somewhat rare chance of cache_count being lower than the actual number of entries. Not sure if this is happening by a race condition after execution time, or if the value just isn't updated constantly enough. On heavily loaded boxes (1000+ more or less unique req/s) it's not too tough to cause memory corruption and a panic running 'sysctl net.inet.tcp.hostcache.list' with the current code. A colleague of mine had spotted the issue, and believes this patch would do the trick. I've been testing it by simply running the hostcache.list in a loop, and have had success where prior it would have caused a panic in fairly short order. Patch by: Nils McCarthy <nils@shkoo.com> Fix: Possible patch supplied Patch attached with submission follows: How-To-Repeat: Run sysctl net.inet.tcp.hostcache.list continuously.
Responsible Changed From-To: freebsd-bugs->freebsd-net Over to maintainer(s).
It appears this was committed to HEAD verbatim in r241735. Not sure who 'az' is though :)
My mistake, this is still an open issue, only the small RTT value patch of this diff is live. - (RTM_RTTUNIT / (hz * TCP_RTT_SCALE))), + (RTM_RTTUNIT / (hz * TCP_RTTVAR_SCALE))),
This typo was commited in r238083
Race in cache_count change is still not fixed after more than two years :( atomic_add_int / atomic_subtract_int is expensive operations. Better to use counter(9) API.
Created attachment 151584 [details] sbuf_hc_list.patch It is true that cache_count updates are very racy, and it might be worth fixing them to be atomic. However, those would not prevent the corruption. If the cache grew during the sysctl even with the atomic ops you could still overflow the buffer. I started out by adding explicit checks of 'p - buf >= bufsize' to break out of the loop. However, the sbuf(9) API already provides a nice way to handle variable output into a fixed size buffer while handling overflow correctly, etc. I've attached a patch which takes this route and converts the sysctl to use an sbuf instead. Please test.
A commit references this bug: Author: jhb Date: Sun Jan 25 19:45:45 UTC 2015 New revision: 277709 URL: https://svnweb.freebsd.org/changeset/base/277709 Log: Use an sbuf to generate the output of the net.inet.tcp.hostcache.list sysctl to avoid a possible buffer overflow if the cache grows while the text is being generated. PR: 172675 MFC after: 2 weeks Changes: head/sys/netinet/tcp_hostcache.c
A commit references this bug: Author: jhb Date: Tue Feb 10 21:41:57 UTC 2015 New revision: 278534 URL: https://svnweb.freebsd.org/changeset/base/278534 Log: MFC 277709: Use an sbuf to generate the output of the net.inet.tcp.hostcache.list sysctl to avoid a possible buffer overflow if the cache grows while the text is being generated. PR: 172675 Changes: _U stable/10/ stable/10/sys/netinet/tcp_hostcache.c _U stable/9/sys/ stable/9/sys/netinet/tcp_hostcache.c
John's fix was brought to 9 / 10, where I can confirm it works as intended. Thanks!