Bug 258974

I recently migrated my work desktop e-mail client from KMail1 on an old install of Ubuntu Linux to KMail2 on FreeBSD 12.2-RELEASE. Since I've been working at the same place for over 20 years, my old KMail instance had accumulated over 400,000 e-mails in various maildir folders. This may sound like a lot, but the old setup had no real problems handling this.

Unfortunately KMail2 on FreeBSD did have some problems, which I traced down to an interaction between the KDE KDirWatch class and the libinotify library that FreeBSD uses to emulate the inotify(7) facility from Linux.

I use fetchmail and procmail to take incoming messages from a Microsoft IMAP server and deposit them directly in the inbox folder in KMail's maildir directory (under ~/.local/share/local-mail). KMail needs to be able to detect when new files appear in the folder. Behind the scenes, this is handled by the akonadi_maildir_resource program (plugin?) which in turn uses the KDirWatch class from libKF5CoreAddons.so. The KDirWatch class supports a couple of different options for directory monitoring, including using FAM, using inotify(7) and (as a last resort) periodically calling stat(2) on the directory to detect updates. The default is to use inotify(7), because that's the preferred strategy for Linux.

Since inotify(7) is a Linux custom API, the FreeBSD kernel doesn't have a direct analog for it. Instead, there is a devel/libinotify port which provides a compatible API built on top of kqueue(2). Note however that in Linux, you can call inotify_add_watch() on a directory and monitor it for a number of different kinds of events, including changes to files within the directory.

There is a limitation with libinotify though: it has to maintain open file descriptors for the filesystem objects that it monitors. If you choose to monitor a directory and monitor changes to files within it, then libinotify will need to maintain an open file descriptor not just for the directory itself but also for _every_ regular file it contains. This happens for exaple if you call inotify_add_watch() with the IN_ATTRIB and/or the IN_MODIFY flags, as described in the README here:

https://github.com/libinotify-kqueue/libinotify-kqueue

Note the admonition about possibly running out of file descriptors, and the possibility of needing to increase the file descriptor limit. (I wouldn't consider that a reasonable way to deal with the matter though: it's like saying "if you shoot yourself in the foot, just add more feet.")

It turns out that the KDirWatch class does exactly this:

https://invent.kde.org/frameworks/kcoreaddons/-/blob/master/src/lib/io/kdirwatch.cpp#L710

The end result:

- If you use KMail with maildir folders
- And you have hundreds of thousands of messages in those folders
- Then akonadi_maildir_resource will end up with hundreds of thousands of open file descriptors (one for _every_ message)

This makes KMail/Akonadi very sluggish and has a fairly negative impact on the system in general. akonadi_maildir_resource may get stuck consuming gobs of CPU cycles without getting any real work done, and if you run fstat(1) it will just hang. (lsof won't, but it'll spit out 400000 lines of output.)

Also, the only reason KMail works at all in this scenario is that the default process limit for open file descriptors in FreeBSD these days can be over 900000. And if you reduce the descriptor limit to something less than the number of messages in your maildir folder, Akonadi and KMail will crash.

I'm not exactly sure of the right way to address this, but something should be done, otherwise KMail is just not usable on FreeBSD. It may *seem* like it is if you have only a moderately small number of messages, but it will get progressively worse the more you use it.

I think the best thing would be if someone found a way to rework libinotify to eliminate this behavior, but I'm not sure of the best way to do that, since it may be beyond the limits of what's possible with kqueue(2) or the FreeBSD kernel in general. (Also I wasn't sure if I should file this bug against the kcoreaddons port or the libinotify port either, so flipped a coin.)

I did want to file a PR though just so there would be some publicly visible info on the matter, and so that people could potentially take advantage of the workarounds I'm about to describe.

There are basically two workarounds:

1) Rebuild the libinotify.so.0.0.0 shared library from the devel/libinotify port using the patch shown here:

https://people.freebsd.org/~wpaul/libinotify/

This forces off the IN_ATTRIB and IN_MODIFY flags for directories. This at least prevents KDirWatch from making it spiral out of control.

2) The KDirWatch class can be configured to change the preferred monitoring mechanism, as described here:

https://api.kde.org/frameworks/kcoreaddons/html/classKDirWatch.html

I don't think that KDirWatch is compiled with FAM support on FreeBSD, so the only reasonable alternative is to select the "Stat" method, which while not as ideal as modifying libinotify, at least doesn't require recompiling any code.

I *think* what you can do is modify the .kde/share/config/kdeglobals file to include the following:

[DirWatch]
PreferredMethod=Stat

Alternatively, you can use the KDIRWATCH_METHOD environment variable, e.g.:

% setenv KDIRWATCH_METHOD Stat

You have to make sure you do this before you launch KMail though.

I elected to use option 1, because I also discovered that the Qt library also has a directory/file watcher class, and it also uses the inotify interface and sets the IN_ATTRIB flag when monitoring directories. I decided fixing both problems with one change would be better.

Now that I've patched libinotify, KMail/Akonadi is much more responsive, even with my 20 years worth of mail. So I'm not in urgent need of a fix myself. But responsible parties might want to at least consider changing the default watch method for FreeBSD from "inotify" to "Stat", otherwise KMail isn't really very usable on FreeBSD.

-Bill