|Summary:||linprocfs: /proc/meminfo values do not add up|
|Product:||Base System||Reporter:||Alexey Dokuchaev <danfe>|
|Component:||kern||Assignee:||Mark Johnston <markj>|
|Severity:||Affects Some People||CC:||markj|
Description Alexey Dokuchaev 2020-08-04 08:31:18 UTC
While porting some software which originates from GNU/Linux and collects memory usage information from /proc/meminfo, I've noticed one common pattern of failure when the same code runs on FreeBSD: the amount of estimated "free" memory is larger than total memory. (Obligatory disclaimer: It's been known for years that reasoning about free/used memory is to a certain extent short of black magic, and it's probably better to avoid broad generalisations like "used" and "free" at all, but people still do this in otherwise nice and useful software, so we'd better make sure it runs properly on FreeBSD.) Back to the popular calculation formula: the amount of memory actually in use is often assessed, in the /proc/meminfo field terms, as "MemTotal - (MemFree + Buffers + Cached)". Now, the problem is that on FreeBSD, apparently, "MemFree + Cached" is always larger than MemTotal (Buffers are always zero, because "bufspace is internal to vfs_bio.c and we don't feel like unstaticizing it just for linprocfs's sake" per the comment in the linprocfs.c). Reading the code in /sys/compat/linprocfs/linprocfs.c around line 163 (search for "cheat") shows that while trying to avoid misleading Linux binaries into thinking there is very little memory left by reporting that all memory that isn't wired down is free, we instead mislead them in a different way by screwing their calculations as the amount of used memory becomes negative. SVN-blaming shows that this "cheating" logic had been added more than 20 years ago. Perhaps it's time to revisit it?
Comment 1 Mark Johnston 2020-08-05 14:08:10 UTC
Created attachment 217028 [details] proposed patch Here is a patch which fixes some problems with the meminfo handler.
Comment 2 Alexey Dokuchaev 2020-08-06 06:13:58 UTC
(In reply to Mark Johnston from comment #1) Thanks, new code makes a lot more sense. I've tried it yesterday, but observed some strange inconsistencies in the inactive memory numbers reported between top(1) and /compat/linux/proc/meminfo, but that could be due to my pretty old kernel version (Mayish 1300096). I've just svn up'ed to the latest -CURRENT and will reconduct my tests shortly. I also want to compare with the values reported by Fedora Linux on the same laptop.
Comment 3 Mark Johnston 2020-08-06 14:04:06 UTC
(In reply to Alexey Dokuchaev from comment #2) meminfo does not report inactive memory with my patch, it reports the amount of memory in all page queues.
Comment 4 Alexey Dokuchaev 2020-08-06 14:12:08 UTC
(In reply to Mark Johnston from comment #3) > meminfo does not report inactive memory with my patch Sorry for this confusion; I've modified your patch to report individual values for active, inactive, and laundry pages while trying to understand the differences reported by top(1) and /compat/linux/proc/meminfo.
Comment 5 Mark Johnston 2020-08-11 17:56:56 UTC
Ping? Any problems with the patch?
Comment 6 Alexey Dokuchaev 2020-08-12 08:22:47 UTC
(In reply to Mark Johnston from comment #5) Ah, sorry, I got side-tracked. Everything looks good now, please commit and thank you Mark!
Comment 7 commit-hook 2020-08-12 16:09:41 UTC
A commit references this bug: Author: markj Date: Wed Aug 12 16:08:45 UTC 2020 New revision: 364168 URL: https://svnweb.freebsd.org/changeset/base/364168 Log: linprocfs: Fix some inaccuracies in meminfo. - Fill out MemFree correctly. Delete an ancient comment suggesting that we don't want to advertise the true quantity of free memory. - Populate the Buffers field by reading vfs.bufspace. - The page cache consists of all pages in page queues, not just the inactive queue. PR: 248463 Reported and tested by: danfe MFC after: 1 week Sponsored by: The FreeBSD Foundation Changes: head/sys/compat/linprocfs/linprocfs.c
Comment 8 commit-hook 2020-08-19 13:44:33 UTC
A commit references this bug: Author: markj Date: Wed Aug 19 13:44:08 UTC 2020 New revision: 364392 URL: https://svnweb.freebsd.org/changeset/base/364392 Log: MFC r364168: linprocfs: Fix some inaccuracies in meminfo. PR: 248463 Changes: _U stable/12/ stable/12/sys/compat/linprocfs/linprocfs.c
Comment 9 Mark Johnston 2020-08-19 13:48:19 UTC
Thanks for the report.