There are several potential improvements to the NFS read-ahead heuristics. First, the nfsheur table size is sized too small for large servers. Second, the sequentiality hueristic can be modified to recognize and handle read patterns that have a large sequential component (such as stride reads) that are missed by the current sequentiality metric. Third, the sequentiality metric is fragile in the face of jitter in the request stream (which happens when client nfsiods are stressed). Fix: The first problem (nfsheur too small) is easy to fix, and I recommend this be done for all systems: 1. In sys/nfs/nfs_serv.c: change NUM_HEURISTIC from 64 to 1021. The exact value isn't important, but it should be at least a few hundred, and prime. 2. Change all the calculations of the nfsheur hash bucket location from (something) & (NUM_HEURISTIC - 1); to (something) % NUM_HEURISTIC; This is necessary because if NUM_HEURISTIC isn't a power of two, then we can't use bit-fiddling to do the %. 2. In function nfsrv_read, file sys/nfs/nfs_serv.c: change the number of probes from 4 to something larger: /* * Calculate seqcount for heuristic */ { int try = 4; In my experiments, 8 works well, but YMMV. The downside of this fix is that nfsheur uses more memory (approx 20K) and if you have a slow CPU then doing the % instead of & will hurt. But if you have a busy NFS server, the reduction in hash table collisions will be worth the extra computation. Fixing the second and third problems involves much more extensive changes to the code. The ideas are described in the latter half of a paper I am working on for FreeNIX (URL below). I have implemented these changes in a new file, using sysctl to control the parameters and whether or not the new code is used at all. A snapshot of my code and the paper is available at: http://www.eecs.harvard.edu/~ellard/NFS/ ellard-freenix03.pdf - paper snapshot, with description of the new algorithms. nfs_serv.c - modified nfs_serv.c nfs_serv.c.diff - listing of the diffs from the original nfs_serv.c nfs_tweak.c - new source code nfs_tweak.h - new header file Please contact me if you need more information, or have suggestions for how I can improve my code. Thanks, -Dan How-To-Repeat: Beat on your NFS server
The NFS code has been completely rototilled since this patch was submitted.