Created attachment 271006 [details] reproducer script `casper_ttymsg()` in `usr.sbin/syslogd/syslogd_cap_log.c` leaks the string array returned by `nvlist_take_string_array()`. The function takes ownership of both the array pointer and the individual strings, but only frees the `iov` wrapper. The strings and array itself are never freed. This leaks memory on every F_CONSOLE and F_TTY log message (e.g. anything matching `*.err` routed to `/dev/console` in the default syslog.conf). On long-running systems, the `syslogd.casper` child process grows to hundreds of MB. The sibling function `casper_wallmsg()` in the same file handles cleanup correctly and can serve as a reference for the fix. --- Here are the results of the reproducer script: root@devbsd:~/lab/default.jj/oss/freebsd-src/default.jj/usr.sbin/syslogd # ../../../prs/freebsd-mem-leak/leak-test.sh syslogd not running? (check /var/run/syslog.pid). --- 150000 user.info messages to /tmp/leak-test.log --- casper RSS (KB): 2676 syslogd: syslogd.casper (syslogd) --- 150000 user.err messages to /dev/console --- casper RSS (KB): 31336 syslogd: syslogd.casper (syslogd) Starting syslogd.
Created attachment 271007 [details] reproducer script
fix in https://github.com/freebsd/freebsd-src/pull/2222 --- root@devbsd:~/lab/default.jj/oss/freebsd-src/default.jj/usr.sbin/syslogd # PATH=$(make -V .OBJDIR):$PATH ../../../prs/freebsd-mem-leak/leak-test.sh syslogd not running? (check /var/run/syslog.pid). --- 150000 user.info messages to /tmp/leak-test.log --- casper RSS (KB): 2680 syslogd: syslogd.casper (syslogd) --- 150000 user.err messages to /dev/console --- casper RSS (KB): 2684 syslogd: syslogd.casper (syslogd) Starting syslogd.
Thanks Pat for the excellent repro. Confirmed on 15.1-BETA3: ``` --- 500000 user.info messages to /tmp/leak-test.log --- casper RSS (KB): 3020 syslogd: syslogd.casper (syslogd) --- 500000 user.err messages to /dev/console --- casper RSS (KB): 107284 syslogd: syslogd.casper (syslogd) ``` The impact of this is particularly severe for users in Google Cloud envs, and possibly elsewhere, that log extensively to /dev/console, it's trivial to leak GB of memory. I believe this is introduced in https://reviews.freebsd.org/D41465
Checked on 15.1-BETA3, use `sleep 0.1` in the repro script to save some time. https://patch-diff.githubusercontent.com/raw/freebsd/freebsd-src/pull/2222.patch - before ``` root@wintermute /tmp# ./wat.sh Stopping syslog_ng. Waiting for PIDS: 47041. --- 500000 user.info messages to /tmp/leak-test.log --- casper RSS (KB): 3020 syslogd: syslogd.casper (syslogd) --- 500000 user.err messages to /dev/console --- casper RSS (KB): 107284 syslogd: syslogd.casper (syslogd) ``` - after ``` Starting syslog_ng. root@wintermute /tmp# ./wat.sh Stopping syslog_ng. Waiting for PIDS: 73590. --- 500000 user.info messages to /tmp/leak-test.log --- casper RSS (KB): 3020 syslogd: syslogd.casper (syslogd) --- 500000 user.err messages to /dev/console --- casper RSS (KB): 3012 syslogd: syslogd.casper (syslogd) Starting syslog_ng. root@wintermute /tmp# ```
I would like to get this into CURRENT, MFC and do an erratum. I need a src approver tho. For re@ Wont Fix for 15.1-RELEASE as that ship has sailed, and this bug is already present in 15.0-RELEASE.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=c783d7181d6a71cb2453f06e40c08c892510c2f2 commit c783d7181d6a71cb2453f06e40c08c892510c2f2 Author: Pat Maddox <pat@patmaddox.com> AuthorDate: 2026-05-22 21:45:30 +0000 Commit: Dave Cottlehuber <dch@FreeBSD.org> CommitDate: 2026-05-22 21:45:30 +0000 syslogd: fix memory leak in casper_ttymsg() nvlist_take_string_array(9) takes ownership of the array and its strings. casper_ttymsg() freed neither, leaking memory on every F_CONSOLE and F_TTY message. On long-running systems with high error-rate syslog traffic routed to /dev/console, syslogd.casper grew to hundreds of MB. Use nvlist_get_string_array(9) to borrow the array instead. Update casper_wallmsg() similarly. Approved by: src (des) Closes: https://github.com/freebsd/freebsd-src/pull/2222 Fixes: 61a29eca550b ("syslogd: Log messages using libcasper") MFC after: 3 days MFC to: stable/15 PR: 295488 Reported by: Pat Maddox <pat@patmaddox.com> Reviewed by: markj Tested by: dch usr.sbin/syslogd/syslogd_cap_log.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-)
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=be03b0fb2241260ec94db431cf4f2954161f227e commit be03b0fb2241260ec94db431cf4f2954161f227e Author: Pat Maddox <pat@patmaddox.com> AuthorDate: 2026-05-22 21:45:30 +0000 Commit: Dave Cottlehuber <dch@FreeBSD.org> CommitDate: 2026-05-26 20:37:18 +0000 syslogd: fix memory leak in casper_ttymsg() nvlist_take_string_array(9) takes ownership of the array and its strings. casper_ttymsg() freed neither, leaking memory on every F_CONSOLE and F_TTY message. On long-running systems with high error-rate syslog traffic routed to /dev/console, syslogd.casper grew to hundreds of MB. Use nvlist_get_string_array(9) to borrow the array instead. Update casper_wallmsg() similarly. Approved by: src (des) Closes: https://github.com/freebsd/freebsd-src/pull/2222 Fixes: 61a29eca550b ("syslogd: Log messages using libcasper") MFC after: 3 days MFC to: stable/15 PR: 295488 Reported by: Pat Maddox <pat@patmaddox.com> Reviewed by: markj Tested by: dch (cherry picked from commit c783d7181d6a71cb2453f06e40c08c892510c2f2) usr.sbin/syslogd/syslogd_cap_log.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-)
erratum submitted, lets close this. Thanks Pat for the find & the fix!
A commit in branch releng/15.1 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=d51d91b07f5bd1066652eb3d09fc5db07790824b commit d51d91b07f5bd1066652eb3d09fc5db07790824b Author: Pat Maddox <pat@patmaddox.com> AuthorDate: 2026-05-22 21:45:30 +0000 Commit: Colin Percival <cperciva@FreeBSD.org> CommitDate: 2026-05-28 22:15:34 +0000 syslogd: fix memory leak in casper_ttymsg() nvlist_take_string_array(9) takes ownership of the array and its strings. casper_ttymsg() freed neither, leaking memory on every F_CONSOLE and F_TTY message. On long-running systems with high error-rate syslog traffic routed to /dev/console, syslogd.casper grew to hundreds of MB. Use nvlist_get_string_array(9) to borrow the array instead. Update casper_wallmsg() similarly. Approved by: re (cperciva) Approved by: src (des) Closes: https://github.com/freebsd/freebsd-src/pull/2222 Fixes: 61a29eca550b ("syslogd: Log messages using libcasper") MFC after: 3 days MFC to: stable/15 PR: 295488 Reported by: Pat Maddox <pat@patmaddox.com> Reviewed by: markj Tested by: dch (cherry picked from commit c783d7181d6a71cb2453f06e40c08c892510c2f2) (cherry picked from commit be03b0fb2241260ec94db431cf4f2954161f227e) usr.sbin/syslogd/syslogd_cap_log.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-)
A commit in branch releng/15.0 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=998de2d14e25c1246b8fe75f85c053e0b9781a8f commit 998de2d14e25c1246b8fe75f85c053e0b9781a8f Author: Pat Maddox <pat@patmaddox.com> AuthorDate: 2026-05-22 21:45:30 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2026-05-29 19:50:43 +0000 syslogd: fix memory leak in casper_ttymsg() nvlist_take_string_array(9) takes ownership of the array and its strings. casper_ttymsg() freed neither, leaking memory on every F_CONSOLE and F_TTY message. On long-running systems with high error-rate syslog traffic routed to /dev/console, syslogd.casper grew to hundreds of MB. Use nvlist_get_string_array(9) to borrow the array instead. Update casper_wallmsg() similarly. Approved by: so Security: FreeBSD-EN-26:14.syslogd Approved by: src (des) Closes: https://github.com/freebsd/freebsd-src/pull/2222 Fixes: 61a29eca550b ("syslogd: Log messages using libcasper") PR: 295488 Reported by: Pat Maddox <pat@patmaddox.com> Reviewed by: markj Tested by: dch (cherry picked from commit c783d7181d6a71cb2453f06e40c08c892510c2f2) (cherry picked from commit be03b0fb2241260ec94db431cf4f2954161f227e) usr.sbin/syslogd/syslogd_cap_log.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-)