Bug 273669 - x11/kde5: kde5 opens too many files and freezes the system (with disabled Baloo) - KDE is unusable when the home directory has a lot of files
Summary: x11/kde5: kde5 opens too many files and freezes the system (with disabled Bal...
Status: Closed DUPLICATE of bug 256269
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-kde (group)
URL:
Keywords: needs-qa, performance
Depends on:
Blocks:
 
Reported: 2023-09-10 03:47 UTC by Yuri Victorovich
Modified: 2024-03-30 16:27 UTC (History)
7 users (show)

See Also:
bugzilla: maintainer-feedback? (kde)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2023-09-10 03:47:37 UTC
KDE5 works fine when the home directory doesn't have too many files.

Baloo causes this type of problem, but 'balooctl status' says that baloo is disabled, so something else opens too many files.

But on my system the home directory has a lot of files and KDE runs out of file descriptors.

kern.maxfiles can be increased, of course, but it is obviously a bad idea to open every available file, and this definitely would cause performance issues when kern.maxfiles would be sufficiently large.

The dmesg log looks like this:
kern.maxfiles limit exceeded by uid 1001, (dolphin) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (plasmashell) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (date) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (DiscoverNotifier) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (date) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (kgpg) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (DiscoverNotifier) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (chrome) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (chrome) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (chrome) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (plasmashell) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (DiscoverNotifier) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (kwin_killer_helper) please see tuning(7).
pid 94202 (kwin_killer_helper), jid 0, uid 1001: exited on signal 5
kern.maxfiles limit exceeded by uid 1001, (kwin_killer_helper) please see tuning(7).
pid 94204 (kwin_killer_helper), jid 0, uid 1001: exited on signal 5
kern.maxfiles limit exceeded by uid 1001, (DiscoverNotifier) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (date) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (qtox) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (date) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (kgpg) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (date) please see tuning(7).
kern.maxfiles limit exceeded by uid 1001, (plasmashell) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (sh) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (sleep) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (sleep) please see tuning(7).
kern.maxfiles limit exceeded by uid 1015, (sleep) please see tuning(7).
pid 93733 (firefox), jid 0, uid 1001: exited on signal 11
kern.maxfiles limit exceeded by uid 1001, (firefox) please see tuning(7).
pid 8573 (firefox), jid 0, uid 1001: exited on signal 11
kern.maxfiles limit exceeded by uid 1001, (firefox) please see tuning(7).
pid 8611 (firefox), jid 0, uid 1001: exited on signal 11
pid 8604 (firefox), jid 0, uid 1001: exited on signal 11
Comment 1 Graham Perrin 2023-09-10 10:10:32 UTC
What's your max, and which version of FreeBSD? Packages from latest or quarterly?

Here: 

% balooctl status
Baloo is currently disabled. To enable, please run balooctl enable
% sysctl kern.maxfiles
kern.maxfiles: 519809
% grep kern.maxfiles /etc/sysctl.conf
% grep kern.maxfiles /boot/loader.conf
% pkg iinfo recoll
recoll-1.33.1_5
% date ; uname -aKU
Sun 10 Sep 2023 11:09:07 BST
FreeBSD mowa219-gjp4-8570p-freebsd 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #10 main-n265135-07bc20e4740d-dirty: Sat Sep  2 17:36:59 BST 2023     grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 1500000 1500000
%
Comment 2 Yuri Victorovich freebsd_committer freebsd_triage 2023-09-10 16:30:38 UTC
kern.maxfiles=25000

FreeBSD 13.2, latest packages.
Comment 3 Yuri Victorovich freebsd_committer freebsd_triage 2023-09-10 19:00:44 UTC
(In reply to Graham Perrin ◐ from comment #1)

I think "kern.maxfiles: 519809" is excessive. If the system needs so many files to be open simultaneously - there's something wrong with it.

As a comparison, XFCE4 works happily with kern.maxfiles=25000.

IMO, KDE probably opens all available files, which is probably more than 25k on y system, I didn't count.

This must be taking toll on overall system performance.

This problem needs to be fixed.
Comment 5 Yuri Victorovich freebsd_committer freebsd_triage 2023-09-11 01:09:27 UTC
(In reply to Graham Perrin ◐ from comment #4)

Yes, I changed kern.maxfiles to 25000 years ago in 2012 and forgot about this, because the default was 12328 and kde4 complained at the time.

But this still doesn't invalidate this problem, because (1) opening all files is a performance problem and (2) number of files can exceed any given threshold.
Comment 6 Graham Perrin 2023-09-11 03:17:39 UTC
> … doesn't invalidate this problem, 

Agreed; note that I set the status to 'Open'.

> … opening all files is a performance problem and …

Agreed; the performance keyword.

I wondered whether (see also) 258010 should become a blocker. Maybe not whilst needs-qa is here. 

To begin telling whether there's any factor other than meta port kde5 in your case: 

pkg query -e '%a = 0' %o
Comment 7 Matthias Apitz 2023-10-05 05:37:13 UTC
I was hit this morning by this bug: firefox crashed on startup with "Too many open files" errors on open(2) sys calls. I found this with truss and then search for the reason and found this PR. On my 1400094 FreeBSD maxfiles is 62558 (don't found from where this value comes from).

The problem was caused by having copied the full /usr/ports tree from my poudriere build system to my HOME to investigate something. The day after FF didn't started anymore.

This bug in KDE must be found which process opens so many files. At the moment kern.openfiles shows ~3200.
Comment 8 Matthias Apitz 2023-10-05 06:45:27 UTC
[guru@c720-1400094 ~]$ LANG=C balooctl status
Baloo File Indexer is not running
Total files indexed: 2
Files waiting for content indexing: 1
Files failed to index: 0
Current size of index is 96.00 KiB
[guru@c720-1400094 ~]$ grep baloo_file /var/log/messages
Oct  5 06:03:04 c720-1400094 kernel: kern.maxfiles limit exceeded by uid 1001, (baloo_file) please see tuning(7).
Oct  5 07:06:25 c720-1400094 kernel: pid 1395 (baloo_file), jid 0, uid 1001: exited on signal 5
Oct  5 07:29:12 c720-1400094 kernel: kern.maxfiles limit exceeded by uid 1001, (baloo_file) please see tuning(7).
Oct  5 07:29:12 c720-1400094 kernel: pid 1988 (baloo_file), jid 0, uid 1001: exited on signal 5


How this baloo-stuff can be switched off?
Comment 9 Yuri Victorovich freebsd_committer freebsd_triage 2023-10-05 08:11:35 UTC
(In reply to Matthias Apitz from comment #8)

> How this baloo-stuff can be switched off?

'balooctl disable' disables baloo.
Comment 10 Gleb Popov freebsd_committer freebsd_triage 2024-03-30 09:50:19 UTC
Closing as a dup of 256269 (which is also created by you, Yuri).

*** This bug has been marked as a duplicate of bug 256269 ***
Comment 11 Yuri Victorovich freebsd_committer freebsd_triage 2024-03-30 16:23:48 UTC
(In reply to Gleb Popov from comment #10)

Hi Gleb,

FYI: These are 2 different issues.

Both are in KDE5, but one is in Baloo, and another one is somewhere outside of Baloo.

This is the reason why there are 2 bug reports.

They are not exactly duplicates, because they are about different issues, which have similar symptoms.

To my knowledge, none of these 2 issues are fixed yet.


Regards,
Yuri
Comment 12 Gleb Popov freebsd_committer freebsd_triage 2024-03-30 16:27:05 UTC
Yes, I already was pointed out that this isn't really a duplicate. However, I doubt anything can be acted on this matter without clear reproduction steps or some reasearch from your side.

I'm using Plasma on many machines with different contexts and never bumped into issue you're describing. It might be that Firefox or Plasma freezes are just red herrings and the problem is somewhere deeper.