Bug 48894 - [nfs] Suggested improvements to the NFS read-ahead heuristic code
Summary: [nfs] Suggested improvements to the NFS read-ahead heuristic code
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 4.6.2-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-03-03 20:10 UTC by Daniel Ellard
Modified: 2017-06-27 00:56 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Ellard 2003-03-03 20:10:10 UTC
	There are several potential improvements to the NFS read-ahead
	heuristics.  First, the nfsheur table size is sized too small for
	large servers.  Second, the sequentiality hueristic can be
	modified to recognize and handle read patterns that have a large
	sequential component (such as stride reads) that are missed by
	the current sequentiality metric.  Third, the sequentiality metric
	is fragile in the face of jitter in the request stream (which happens
	when client nfsiods are stressed).

Fix: 

The first problem (nfsheur too small) is easy to fix, and I recommend
this be done for all systems:

	1.  In sys/nfs/nfs_serv.c:  change NUM_HEURISTIC from 64 to
		1021.  The exact value isn't important, but it should
		be at least a few hundred, and prime.

	2.  Change all the calculations of the nfsheur hash bucket location
		from 

			(something) & (NUM_HEURISTIC - 1);

		to

			(something) % NUM_HEURISTIC;

		This is necessary because if NUM_HEURISTIC isn't a power
		of two, then we can't use bit-fiddling to do the %.

	2.  In function nfsrv_read, file sys/nfs/nfs_serv.c: change the
		number of probes from 4 to something larger:

			/*
			 * Calculate seqcount for heuristic
			 */

			{
				int try = 4;

		In my experiments, 8 works well, but YMMV.  

The downside of this fix is that nfsheur uses more memory (approx 20K)
and if you have a slow CPU then doing the % instead of & will hurt. 
But if you have a busy NFS server, the reduction in hash table
collisions will be worth the extra computation.

Fixing the second and third problems involves much more extensive
changes to the code.  The ideas are described in the latter half of a
paper I am working on for FreeNIX (URL below).  I have implemented
these changes in a new file, using sysctl to control the parameters
and whether or not the new code is used at all.  A snapshot of my code
and the paper is available at:

http://www.eecs.harvard.edu/~ellard/NFS/

	ellard-freenix03.pdf - paper snapshot, with description of
			the new algorithms.
	nfs_serv.c - modified nfs_serv.c
	nfs_serv.c.diff - listing of the diffs from the original nfs_serv.c
	nfs_tweak.c - new source code
	nfs_tweak.h - new header file

Please contact me if you need more information, or have suggestions
for how I can improve my code.

Thanks,
	-Dan
How-To-Repeat: 	Beat on your NFS server
Comment 1 Josh Paetzel freebsd_committer freebsd_triage 2017-06-27 00:56:51 UTC
The NFS code has been completely rototilled since this patch was submitted.