The new dirhash function ufsdirhash_lowmem() is called in low-memory situations. I have a KDE repository mirror on a machine with 1.25 GiB of memory, with 2 directories of each more than 10e6 entries. Due to memory pressure, the hashes are now freed multiple times even during a single subversion command, such that subversion exhibits extremely long waiting times and is nearly unusable. This is also due to bug http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/117954 , which is still unsolved and leads to several tens of seconds of complete freezes of the machine whenever the dirhash is (re-) computed. Fix: Probably go back to the old dirhash behavior (7.2), or free dirhashes only as last resort in low-mem situations. Note: I never had memory problems in 7.2 with the old dirhash behavior, so clearly there was still enough memory (to be gotten from somewhere else, probably). Can I revert the dirhash changes by just reverting ufs_dirhash.c? How-To-Repeat: Mirror the SVN repo (rsync://rsync.kde.org/svnmirror/) on a FreeBSD 7.3 machine with 1.25 GiB of memory. Note: This does not succeed any more because the rsync server (on rsync.kde.org) times out before the local dirhash can be created for the two directories in question.
I reverted to SVN revision 195783, and it's working o.k. again (except for the old issue http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/117954, of course).
Responsible Changed From-To: freebsd-bugs->freebsd-fs Over to maintainer(s).
I suspect the fix here is to provide a tunable/sysctl that can be used to disable the dirhash lowmem handler for workloads where it does more harm than good. -- John Baldwin
I believe that there should be some notion of "value" given to resources which may potentially be destroyed and recreated - computing the dirhash takes a lot of time (see http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/117954 ), and this work should not be thrown away lightly. (This is why dirhash exists in the first place.) Maybe the lowmem handler could be modified to only use the dirhash lowmem handler as a last resort. In my case, the kernel did not really seem to be that short on memory, as it has now been running with moderate to high load for 24 hours with the reversion to SVN rev 195783 for ufs_dirhash.c and dirhash.h. I have configured vfs.ufs.dirhash_maxmem=33554432 (32M) by the way. Compared to 1.25G, this is only 2.5%, so I believe not really something that's eating lots of memory. Regards, Martin
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped