Bug 230726 - sysutils/kf5-baloo: freezes the system
Summary: sysutils/kf5-baloo: freezes the system
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Po-Chuan Hsieh
URL:
Keywords:
Depends on: 256269
Blocks:
  Show dependency treegraph
 
Reported: 2018-08-18 04:42 UTC by Ting-Wei Lan
Modified: 2021-07-17 09:45 UTC (History)
7 users (show)

See Also:
tcberner: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ting-Wei Lan 2018-08-18 04:42:08 UTC
After upgrading from x11/kde4 to x11/kde5, I found there is a process called 'baloo_file' which seems to use a lot of system resources. If I forget to kill it within minutes after starting a GNOME session, the entire system will freeze. Yes, installing KDE can affect GNOME, as /usr/local/etc/xdg/autostart/baloo_file.desktop has X-GNOME-Autostart-enabled=true. I can change it to false to avoid the problem in GNOME, but it sounds like a workaround.

When baloo_file freezes the system, I can still move the mouse but no desktop applications are responding. SSH server becomes almost non-responsive and it takes several minutes just to get a login shell. Typing any simple commands such as 'killall' takes minutes to complete, and even logging in from a local VT is hard. When you type your user name, it takes several minutes to show the password prompt. If you press Ctrl-T at that time, you will see the following message.

load: 0.47  cmd: login 1141 [vlruwk] 204838.88r 0.00u 0.00s 0% 240

Most of the time it is stuck at [vlruwk] or [ufs] while the system load is low. I am not sure whether it is related to the number of files under my home directory. My home directory has about 2500000 files because I store hundreds of git repositories, mostly GNOME source code, and builds of GNOME and FreeBSD in it.
Comment 1 Tobias C. Berner freebsd_committer 2018-12-26 11:07:12 UTC
You can disable baloo via "balooctl disable". Or you can configure it to not index certain directories.

It would be interesting to know how many files baloo_files has open, and how many are free / available by the system.
Comment 2 Edward Tomasz Napierala freebsd_committer 2019-01-17 11:39:59 UTC
I suspect this might be related to the fact that SDDM doesn't apply resource limits.  There's a patch for this: https://reviews.freebsd.org/D17297
Comment 3 Martin Birgmeier 2019-01-20 10:27:38 UTC
Could this be related to bug #234830?
Comment 4 Tobias C. Berner freebsd_committer 2019-01-20 10:29:22 UTC
(In reply to Martin Birgmeier from comment #3)
I don't think so. The file in this one just seems to hang libtiff.
Comment 5 Edward Tomasz Napierala freebsd_committer 2019-02-20 17:49:34 UTC
I've just realized that even with SDDM patched to apply resource limits, there is no default limit for "openfiles".

Would it be possible to modify libinotify to limit itself to eg half of kern.maxvnodes?
Comment 6 Tobias C. Berner freebsd_committer 2020-01-19 07:53:24 UTC
(In reply to Edward Tomasz Napierala from comment #5)
Probably yes -- over to the maintainer of libinotify.
Comment 7 Graham Perrin 2021-07-17 09:28:41 UTC
From bug 257227 comment 1 (emphasis more on baloo_file_extractor than on baloo_file): 

> … I should, ideally, positively identify the cause of 
> hogging in my case, but this will be a bug for another day.

(In reply to Tobias C. Berner from comment #1)

> … It would be interesting to know how many files baloo_files 
> has open, and how many are free / available by the system.

Reading the above _loosely_ alongside bug 234830 (closed, fixed) …

If it's true that some types of file are more likely to trigger high CPU usage, when handled by baloo_file or baloo_file_extractor, will it help to identify those types?
Comment 8 Graham Perrin 2021-07-17 09:45:17 UTC
<https://invent.kde.org/frameworks/baloo/-/tree/master#user-content-file-indexing-plugins>: 

> Baloo relies on KFileMetaData to extract content from the files. 
> KFileMetadata ships with a number of plugins which can be enabled 
> or disabled. We recommend shipping all KFileMetaData plugins. Specially 
> ffmpeg by default. Without the indexers, Baloo cannot function to its 
> full potential.

% pkg info --list devel/kf5-kfilemetadata | grep plugins
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_epubextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_exiv2extractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_ffmpegextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_odfextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_office2007extractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_officeextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_plaintextextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_poextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_popplerextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_postscriptdscextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_taglibextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/kfilemetadata_xmlextractor.so
        /usr/local/lib/qt5/plugins/kf5/kfilemetadata/writers/kfilemetadata_taglibwriter.so
% uname -KrU
14.0-CURRENT 1400025 1400025
% pkg info -x metadata
kf5-kfilemetadata-5.83.0
py38-importlib-metadata-3.3.0_1
%