Bug 265937 - [Linuxulator]: linprocfs: process size incorrect
Summary: [Linuxulator]: linprocfs: process size incorrect
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.3-RELEASE
Hardware: amd64 Any
: --- Affects Some People
Assignee: Dmitry Chagin
URL:
Keywords:
Depends on:
Blocks: 247219
  Show dependency treegraph
 
Reported: 2022-08-19 09:18 UTC by Ray Bellis
Modified: 2023-03-02 19:01 UTC (History)
3 users (show)

See Also:


Attachments
Remove P2K when printing ki_size in 'stat' file (660 bytes, patch)
2023-02-26 09:44 UTC, Fernando Apesteguía
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ray Bellis 2022-08-19 09:18:55 UTC
On our FreeBSD 12.3 systems, I've detected that the process size
column in a `linprocfs` process status file (`<pid>/stat`) derived
from the `ki_size` structure is reading four times higher than it
should be, compared to values returned by e.g. `ps` and `top`.

The reason appears to be the use of the P2K macro in this line:

sys/compat/linprocfs/linprocfs.c:1019
```C
PS_ADD("vsize",		"%ju",	P2K((uintmax_t)kp.ki_size));
```

https://github.com/freebsd/freebsd-src/blob/c8e7070ce3d98c3aa0b08249afe1d6d887f37f31/sys/compat/linprocfs/linprocfs.c#L1019

ki_size is calculated elsewhere from vm_map.size, but I could find
nothing to say explicitly whether .size is counted in pages or kB.

The issue is still present in the 14.x source.
Comment 1 Ray Bellis 2022-08-19 09:29:55 UTC
A later line strongly hints that .ki_size is measured in bytes:

compat/linprocfs/linprocfs.c:1064:
  sbuf_printf(sb, "%ju ", B2P((uintmax_t)kp.ki_size));
Comment 3 Fernando Apesteguía freebsd_committer freebsd_triage 2023-02-26 09:44:01 UTC
Created attachment 240417 [details]
Remove P2K when printing ki_size in 'stat' file

I can confirm. This is what I see in my Ubuntu system:

fernape~ cat /proc/1993/stat
1993 (gnome-session-b) S 1990 1990 1990 1026 1990 4194304 1726 5624 2 6 4 2 6 2 20 0 3 0 9929 --->230969344<--- 3912 18446744073709551615 94683959459840 94683959602729 140731149451200 0 0 0 0 4096 16386 0 0 0 17 0 0 0 0 0 0 94683959675184 94683959689872 94683990056960 140731149454254 140731149454305 140731149454305 140731149455318 0

 fernape~ cat /proc/1993/status | grep VmSize
VmSize:  225556 kB <---

 fernape~ cat /proc/1993/statm
56389 <--- 3912 3485 35 0 4481 0

According to https://www.kernel.org/doc/html/latest/filesystems/proc.html, the three marked values should be the same, but in different units:

status is in kB
statm is in pages
stat is in bytes

Let's see if they check out:

In status, we have 225556 kb divided by 4 (since my pages in Ubuntu are 4 kB each) gives me 56389 pages and if I multiply by 4046 bytes in each page, we get the 230969344 bytes in the 'stat' file.

This is what I see in current for my fluxbox process in the linprocfs:

fernape@hammer:~$ grep VmSize /compat/linux/proc/1549/status 
VmSize:    24708 kB

fernape@hammer:~$ cat /compat/linux/proc/1549/statm
6177 2884 0 208 36 5932 0

fernape@hammer:~$ cut -f23 -d' ' /compat/linux/proc/1549/stat
101203968

This does not check out: 24708 kb bytes = 6177 pages (I have 4 kb pages) = 25300992 bytes, not the 101203968 reported by 'stat'. It is, in effect, exactly 4 times higher.

With the attached patch, the result is the expected one:

fernape@hammer:~$ grep VmSize /compat/linux/proc/1470/status 
VmSize:    24708 kB

fernape@hammer:~$ cut -f23 -d' ' /compat/linux/proc/1470/stat
25300992 <--- This is now 24708 * 1024

fernape@hammer:~$ cat /compat/linux/proc/1470/statm
6177 2810 0 208 36 5932 0 <--- This is still 24708/4
Comment 4 Dmitry Chagin freebsd_committer freebsd_triage 2023-02-26 11:22:37 UTC
(In reply to Fernando Apesteguía from comment #3)
Thank you for such exelent explanation, I’ll take it.
Comment 5 commit-hook freebsd_committer freebsd_triage 2023-02-26 13:43:04 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=a55b5a9aed3981fcec205e246436991d138c78a2

commit a55b5a9aed3981fcec205e246436991d138c78a2
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2023-02-26 13:42:22 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2023-02-26 13:42:22 +0000

    linprocfs(4): Fixup process size in the /proc/pid/stat file

    According to the Linux sources the kernel exposes a proces virtual
    memory size via proc filesystem into the three files - stat, status
    and statm. This is the struct mm->total_vm value adjusted to the
    corresponding units - bytes, kilobytes and pages.

    The fix is based on a fernape@ analysis.

    PR:             265937
    Reported by:    Ray Bellis
    MFC after:      3 days

 sys/compat/linprocfs/linprocfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 6 commit-hook freebsd_committer freebsd_triage 2023-03-01 09:47:05 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=867e8404efed7a33553646b89588831db1a2102b

commit 867e8404efed7a33553646b89588831db1a2102b
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2023-02-26 13:42:22 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2023-03-01 09:45:56 +0000

    linprocfs(4): Fixup process size in the /proc/pid/stat file

    According to the Linux sources the kernel exposes a proces virtual
    memory size via proc filesystem into the three files - stat, status
    and statm. This is the struct mm->total_vm value adjusted to the
    corresponding units - bytes, kilobytes and pages.

    The fix is based on a fernape@ analysis.

    PR:             265937
    Reported by:    Ray Bellis
    MFC after:      3 days

    (cherry picked from commit a55b5a9aed3981fcec205e246436991d138c78a2)

 sys/compat/linprocfs/linprocfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 7 commit-hook freebsd_committer freebsd_triage 2023-03-02 17:48:55 UTC
A commit in branch releng/13.2 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=413a08a9b66871fe599310e358555edf54dd61a8

commit 413a08a9b66871fe599310e358555edf54dd61a8
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2023-02-26 13:42:22 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2023-03-02 17:47:36 +0000

    linprocfs(4): Fixup process size in the /proc/pid/stat file

    According to the Linux sources the kernel exposes a proces virtual
    memory size via proc filesystem into the three files - stat, status
    and statm. This is the struct mm->total_vm value adjusted to the
    corresponding units - bytes, kilobytes and pages.

    The fix is based on a fernape@ analysis.

    Approved by:    re (delphij)
    PR:             265937
    Reported by:    Ray Bellis
    MFC after:      3 days

    (cherry picked from commit a55b5a9aed3981fcec205e246436991d138c78a2)
    (cherry picked from commit 867e8404efed7a33553646b89588831db1a2102b)

 sys/compat/linprocfs/linprocfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 8 commit-hook freebsd_committer freebsd_triage 2023-03-02 19:00:08 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3c4af6fe955fa8427ebc1206046bdc0d35a692bb

commit 3c4af6fe955fa8427ebc1206046bdc0d35a692bb
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2023-02-26 13:42:22 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2023-03-02 18:59:19 +0000

    linprocfs(4): Fixup process size in the /proc/pid/stat file

    According to the Linux sources the kernel exposes a proces virtual
    memory size via proc filesystem into the three files - stat, status
    and statm. This is the struct mm->total_vm value adjusted to the
    corresponding units - bytes, kilobytes and pages.

    The fix is based on a fernape@ analysis.

    PR:             265937
    Reported by:    Ray Bellis
    MFC after:      3 days

    (cherry picked from commit a55b5a9aed3981fcec205e246436991d138c78a2)

 sys/compat/linprocfs/linprocfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 9 Dmitry Chagin freebsd_committer freebsd_triage 2023-03-02 19:01:29 UTC
Thanks