Bug 278476

Summary: fd and memory leak in ls
Product: Base System Reporter: Paul Floyd <pjfloyd>
Component: miscAssignee: Mark Johnston <markj>
Status: In Progress ---    
Severity: Affects Only Me CC: markj, tamelingdaniel
Priority: ---    
Version: 14.0-RELEASE   
Hardware: Any   
OS: Any   

Description Paul Floyd 2024-04-20 06:11:39 UTC
This is on 14.0-RELEASE amd64 (and arm64). I'm using Valgrind built from git head. If you use the package / ports version you will get one or two more 'still reachable' reports.

If I run

valgrind --leak-check=full --show-leak-kinds=all --track-fds=all  ls

then I get

==65577== FILE DESCRIPTORS: 4 open (3 std) at exit.
==65577== Open file descriptor 3: .
==65577==    at 0x49F0BCA: _open (in /lib/libc.so.7)
==65577==    by 0x4950AF8: fts_open (in /lib/libc.so.7)
==65577==    by 0x10CA22: ??? (in /bin/ls)
==65577==    by 0x10C9D6: ??? (in /bin/ls)
==65577==    by 0x493CAF9: __libc_start1 (in /lib/libc.so.7)
==65577==    by 0x10BA6C: ??? (in /bin/ls)
==65577==    by 0x4823007: ???
==65577== 
==65577== Open file descriptor 2: /dev/pts/3
==65577==    <inherited from parent>
==65577== 
==65577== Open file descriptor 1: /dev/pts/3
==65577==    <inherited from parent>
==65577== 
==65577== Open file descriptor 0: /dev/pts/3
==65577==    <inherited from parent>
==65577== 
==65577== 
==65577== HEAP SUMMARY:
==65577==     in use at exit: 10,176 bytes in 5 blocks
==65577==   total heap usage: 141 allocs, 136 frees, 36,463 bytes allocated
==65577== 
==65577== 984 bytes in 1 blocks are still reachable in loss record 1 of 5
==65577==    at 0x4852321: realloc (vg_replace_malloc.c:1806)
==65577==    by 0x10EB7D: ??? (in /bin/ls)
==65577==    by 0x10D78D: ??? (in /bin/ls)
==65577==    by 0x10CBF4: ??? (in /bin/ls)
==65577==    by 0x10C9D6: ??? (in /bin/ls)
==65577==    by 0x493CAF9: __libc_start1 (in /lib/libc.so.7)
==65577==    by 0x10BA6C: ??? (in /bin/ls)
==65577==    by 0x4823007: ???
==65577== 
==65577== 1,280 bytes in 1 blocks are indirectly lost in loss record 2 of 5
==65577==    at 0x4852613: reallocf (vg_replace_malloc.c:1808)
==65577==    by 0x49506CD: fts_open (in /lib/libc.so.7)
==65577==    by 0x10CA22: ??? (in /bin/ls)
==65577==    by 0x10C9D6: ??? (in /bin/ls)
==65577==    by 0x493CAF9: __libc_start1 (in /lib/libc.so.7)
==65577==    by 0x10BA6C: ??? (in /bin/ls)
==65577==    by 0x4823007: ???
==65577== 
==65577== 1,368 bytes in 1 blocks are indirectly lost in loss record 3 of 5
==65577==    at 0x4852613: reallocf (vg_replace_malloc.c:1808)
==65577==    by 0x4951BF3: ??? (in /lib/libc.so.7)
==65577==    by 0x4951FC5: fts_children (in /lib/libc.so.7)
==65577==    by 0x10CBE3: ??? (in /bin/ls)
==65577==    by 0x10C9D6: ??? (in /bin/ls)
==65577==    by 0x493CAF9: __libc_start1 (in /lib/libc.so.7)
==65577==    by 0x10BA6C: ??? (in /bin/ls)
==65577==    by 0x4823007: ???
==65577== 
==65577== 5,096 (2,448 direct, 2,648 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
==65577==    at 0x4852105: calloc (vg_replace_malloc.c:1679)
==65577==    by 0x4950625: fts_open (in /lib/libc.so.7)
==65577==    by 0x10CA22: ??? (in /bin/ls)
==65577==    by 0x10C9D6: ??? (in /bin/ls)
==65577==    by 0x493CAF9: __libc_start1 (in /lib/libc.so.7)
==65577==    by 0x10BA6C: ??? (in /bin/ls)
==65577==    by 0x4823007: ???
==65577== 
==65577== LEAK SUMMARY:
==65577==    definitely lost: 2,448 bytes in 1 blocks
==65577==    indirectly lost: 2,648 bytes in 2 blocks
==65577==      possibly lost: 0 bytes in 0 blocks
==65577==    still reachable: 984 bytes in 1 blocks
==65577==         suppressed: 4,096 bytes in 1 blocks

Piece by piece that is

The following is the leaking file descriptor

==65577== Open file descriptor 3: .
==65577==    at 0x49F0BCA: _open (in /lib/libc.so.7)
==65577==    by 0x4950AF8: fts_open (in /lib/libc.so.7)
==65577==    by 0x10CA22: ??? (in /bin/ls)
==65577==    by 0x10C9D6: ??? (in /bin/ls)
==65577==    by 0x493CAF9: __libc_start1 (in /lib/libc.so.7)
==65577==    by 0x10BA6C: ??? (in /bin/ls)
==65577==    by 0x4823007: ???


I guess the following are allocations related to opening the file descriptor.


==65577== 1,280 bytes in 1 blocks are indirectly lost in loss record 2 of 5
==65577==    at 0x4852613: reallocf (vg_replace_malloc.c:1808)
==65577==    by 0x49506CD: fts_open (in /lib/libc.so.7)
...
==65577== 1,368 bytes in 1 blocks are indirectly lost in loss record 3 of 5
==65577==    at 0x4852613: reallocf (vg_replace_malloc.c:1808)
==65577==    by 0x4951BF3: ??? (in /lib/libc.so.7)
==65577==    by 0x4951FC5: fts_children (in /lib/libc.so.7)


I don't know what this is, possibly not related (if so another one for my wishlist https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259294)

==65577== 984 bytes in 1 blocks are still reachable in loss record 1 of 5
==65577==    at 0x4852321: realloc (vg_replace_malloc.c:1806)
==65577==    by 0x10EB7D: ??? (in /bin/ls)
==65577==    by 0x10D78D: ??? (in /bin/ls)
==65577==    by 0x10CBF4: ??? (in /bin/ls)
==65577==    by 0x10C9D6: ??? (in /bin/ls)
==65577==    by 0x493CAF9: __libc_start1 (in /lib/libc.so.7)
==65577==    by 0x10BA6C: ??? (in /bin/ls)
==65577==    by 0x4823007: ???
Comment 1 Mark Johnston freebsd_committer freebsd_triage 2024-04-20 15:27:30 UTC
Is this report effectively just saying that ls doesn't close all of its fds before exiting?  I don't think that's a bug: there's no point in calling close() when the kernel's going to clean up anyway.

Does valgrind have any way of annotating code to suppress reports like that?
Comment 2 Paul Floyd 2024-04-20 16:40:01 UTC
There is a suppression mechanism but I can't put it the default suppression file as it would affect all other applications.

It's not a serious problem. I use 'ls' and similar common userland applications (like sleep, false, id, pwd) for quick smoketests. About half of them don't free everything.
Comment 3 Paul Floyd 2024-04-20 16:42:19 UTC
If you do want something with annotation you can do something like

#include <valgrind.h>

...

if (RUNNING_ON_VALGRIND)
{
   close(fd);
}

That adds a Valgrind dependency. It'd be a lot easier to just close the file descriptor.
Comment 4 Daniel Tameling 2024-04-21 05:18:06 UTC
The cause of the leak is that fts_open is called in the traverse function without a final fts_close:
https://github.com/freebsd/freebsd-src/blob/main/bin/ls/ls.c#L636
After finishing the traverse function ls exits.
Comment 5 commit-hook freebsd_committer freebsd_triage 2025-01-14 15:26:35 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e6c9c463384d11a44af3e5f2cc947fb69f3a1968

commit e6c9c463384d11a44af3e5f2cc947fb69f3a1968
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2025-01-14 14:20:26 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2025-01-14 14:20:26 +0000

    ls: Release resources before returning from traverse()

    PR:             278476
    MFC after:      2 weeks
    Reported by:    valgrind

 bin/ls/ls.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)