Bug 94997 - [patch] port databases/memcached performance (TCP_NODELAY)
Summary: [patch] port databases/memcached performance (TCP_NODELAY)
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Sean Chittenden
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-03-27 10:20 UTC by Anton Yuzhaniov
Modified: 2006-07-03 21:12 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Anton Yuzhaniov 2006-03-27 10:20:13 UTC
memcached has very big response time when reading stored objects size approximately
from 1400 to 2800 - when reply from server sent in two ip packets.

This caused by badly interaction between TCP delayed acknowledgements
(enabled in FreeBSD by default) and Nagle's algorithm.

Nagle's explanation of why interaction between delayed ACK and Nagle's
algorithm is bad:
http://developers.slashdot.org/comments.pl?sid=174457&threshold=1&commentsort=0&mode=thread&cid=14515105

So solution to improve latency is disable Nagle's algorithm for
memcached via socket option TCP_NODELAY.

This don't done by developers of memcached because on Linux TCP_CORK
(analog of TCP_NOPUSH) not compatible with TCP_NODELAY. And on
TCP_CORK show on Linux better performance than TCP_NODLEAY

From memcached Changelog:

2003-08-12 (Brad Fitzpatrick)
        * use TCP_CORK on Linux or TCP_PUSH on BSD
        * only use TCP_NODELAY when we don't have alternatives
        
2003-08-10
        * disable Nagel's Algorithm (TCP_NODELAY) for better performance (avva)

I run tests and it show, that TCP_NODELAY significantly decrease reading response time:
http://citrin.ru/stuff/memcached/read.png
and practically don't affect writing response time:
http://citrin.ru/stuff/memcached/write.png
Script used for this test:
http://citrin.ru/stuff/memcached/bench.pl
        
In result of using TCP_NODLEAY will be some network overhead, but in
environment where usually used memcached (100Mb or 1Gb LAN) it
negligible.

Fix: Apply this patch:



/*
      * the memset call clears nonstandard fields in some impementations--7rfBa7h8kM0KCcJMTPRyYJJE3buTpXegUTFV7SsIZpoSNQfI
Content-Type: text/plain; name="file.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="file.diff"

--- memcached.c.orig    Thu Mar 23 18:51:47 2006
+++ memcached.c Thu Mar 23 18:57:56 2006
@@ -1146,9 +1146,7 @@
     setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR, &flags, sizeof(flags));
     setsockopt(sfd, SOL_SOCKET, SO_KEEPALIVE, &flags, sizeof(flags));
     setsockopt(sfd, SOL_SOCKET, SO_LINGER, &ling, sizeof(ling));
-#if !defined(TCP_NOPUSH)
     setsockopt(sfd, IPPROTO_TCP, TCP_NODELAY, &flags, sizeof(flags));
-#endif
How-To-Repeat: Measure response time when reading from memcached objets size 1000, 2000 and 3000 bytes
(or in more wide range of sizes), e. g. using this script:
http://citrin.ru/stuff/memcached/test.pl
Comment 1 Edwin Groothuis freebsd_committer 2006-03-27 10:24:10 UTC
Responsible Changed
From-To: freebsd-ports-bugs->seanc

Over to maintainer
Comment 2 Sean Chittenden freebsd_committer 2006-07-03 21:11:38 UTC
State Changed
From-To: open->closed

Patch applied, thanks!