Bug 258010 - kqueue shortcoming for desktop usage
Summary: kqueue shortcoming for desktop usage
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: Unspecified
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords: needs-patch, needs-qa
Depends on:
Blocks: 258974
  Show dependency treegraph
 
Reported: 2021-08-23 17:28 UTC by Tobias C. Berner
Modified: 2023-10-24 17:38 UTC (History)
13 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tobias C. Berner freebsd_committer freebsd_triage 2021-08-23 17:28:58 UTC
Moin moin


FreeBSDs file monitoring via kqueue/kevent requires files to be opened to be monitored.


This has a few draw back which mostly crop up in desktop systems (gnome, kde, ...) that 
have support for monitoring and indexing the users home directory.



Issue 1: A big ~ leads to file handle exhaustion

As all files have to be opened the kern.maxfiles sysctl needs to be tuned by users. 

=> desktops won't work out of the box for those users
=> unnecessary crashes happening on startup of desktop sessions


Issue 2: Socket files cannot be monitored

As open() cannot be called on socket files, you cannot monitor a socket file for its deletion.




Linux's file monitoring seems to be able to support these use cases -- maybe a 
re-implementation of their file monitoring should be considered inside FreeBSD. 
An other option would probably be to extend kqueue for these use cases.



mfg Tobias
Comment 1 Mark Johnston freebsd_committer freebsd_triage 2021-10-21 14:29:08 UTC
Regarding issue 2, it's now possible to open unix sockets with open(O_PATH): https://cgit.freebsd.org/src/commit/?id=2bd9826995ca6b23f8b088cfa035c0ad1c578ac3

For now kevent() on such fds is not permitted, but I think we could enable EVFILT_VNODE events on them as was done for named pipes recently:
https://cgit.freebsd.org/src/commit/?id=7259ca31048e5ced8e7f90657a3d7084aeafdf51

Would that be sufficient here?
Comment 2 Jacob Taylor 2022-08-30 14:52:32 UTC
I cannot answer the question if this change would suffice, but this is another example of the kqueue limitations causing issues for users: https://github.com/syncthing/syncthing/issues/7855

Syncthing has shipped an in-app notification so bsd users know they may slow their system down due to this. As somebody who uses syncthing server-side on freebsd... I would love to see something helping this :D
Comment 3 Graham Perrin freebsd_committer freebsd_triage 2022-10-23 02:17:42 UTC
There's a documentation aspect to bug 256269 comment 1: 

* the absence of a package message for sysutils/kf5-baloo
* the FreeBSD Handbook …
Comment 4 Mark C 2023-06-05 07:21:21 UTC
This is an issue that I've been struggling with lately too, also with syncthing.  The kqueue workaround works for smaller directory trees, but is unworkable for large collections of files.  The number of file descriptors can be increased, but with millions of files to watch, the system quickly becomes unusable.

There was at least one previous attempt to add inotify-type functionality to FreeBSD in a GSoC project, but I don't know what happened with that:

https://www.freebsd.org/status/report-2010-04-2010-06.html#File-System-Changes-Notification

A long-running discussion on the forums of the differences between inotify (Linux), fsevents (Mac), and kqueue is here:

https://forums.freebsd.org/threads/inotify-for-freebsd.38162/page-3

I also wonder if something could be implemented based on the existing audit framework, which seems to allow file alterations to be watched, although it doesn't have an easy API to use from code.