When a jail exits, POSIX shared memory segments allocated by it will stay around, unless removed from within the jail in an orderly fashion. As a side effect, this keeps the jail in a dying state forever/until non-anonymous shared memory segments are removed. Basic example: See below for the most basic example: [root@jailhost ~]# jail -c path=/ command=/bin/sh # posixshmcontrol create /removeme # exit [root@jailhost ~]# jls -dv -j shmtest dying true So at this point, the jail is stuck in a dying state. Checking POSIX shared memory segments shows the shared memory segment which is stopping the jail from crossing the Styx: [root@jailhost ~]# posixshmcontrol list MODE OWNER GROUP SIZE PATH rw------- root wheel 0 /removeme After removing the shared memory segment manually... [root@jailhost ~]# posixshmcontrol rm /removeme the jail passes away peacefully: [root@jailhost ~]# jls -dv -j shmtest dying jls: jail "shmtest" not found In our discussion on the jails mailing list[0] it was suggested that it might be the easiest and most straightforward way to solve this would be to free shared memory segments based on their path, as access is controlled by the path too. The only questions is how to handle multiple jails using the same jailroot in this case (and if this is legitimate/important enough use case to bother). [0]https://lists.freebsd.org/archives/freebsd-jail/2021-June/000029.html
Hi Jamie, you already fixed bug #257554 and bug #257556, did you already have a chance to look into this one as well? Thanks Michael
Created attachment 234850 [details] Remove a prison's shm segments when the prison is removed It turns out that the path wasn't the easiest way to identify the segments to remove. The cred, which is what causes the very problem, can just be checked directly. I wrapped this inside a general function to clean up removed prisons, which I have some other plans for at some points. Your simple test worked as expected, but I wouldn't mind of you had any more extensive tests.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=7060da62ff18e8e52c5e41f0794cc4f10dadfc6e commit 7060da62ff18e8e52c5e41f0794cc4f10dadfc6e Author: Jamie Gritton <jamie@FreeBSD.org> AuthorDate: 2022-06-29 17:47:39 +0000 Commit: Jamie Gritton <jamie@FreeBSD.org> CommitDate: 2022-06-29 17:47:39 +0000 jail: Remove a prison's shared memory when it dies Add shm_remove_prison(), that removes all POSIX shared memory segments belonging to a prison. Call it from prison_cleanup() so a prison won't be stuck in a dying state due to the resources still held. PR: 257555 Reported by: grembo sys/kern/kern_jail.c | 2 ++ sys/kern/uipc_shm.c | 37 ++++++++++++++++++++++++++++++++----- sys/sys/mman.h | 3 +++ 3 files changed, 37 insertions(+), 5 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=cf18a61708d30866f3e97701daa3729f75878094 commit cf18a61708d30866f3e97701daa3729f75878094 Author: Jamie Gritton <jamie@FreeBSD.org> AuthorDate: 2022-06-29 17:47:39 +0000 Commit: Jamie Gritton <jamie@FreeBSD.org> CommitDate: 2022-07-03 19:25:43 +0000 MFC jail: Remove a prison's shared memory when it dies Add shm_remove_prison(), that removes all POSIX shared memory segments belonging to a prison. Call it from prison_cleanup() so a prison won't be stuck in a dying state due to the resources still held. PR: 257555 Reported by: grembo (cherry picked from commit 7060da62ff18e8e52c5e41f0794cc4f10dadfc6e) sys/kern/kern_jail.c | 2 ++ sys/kern/uipc_shm.c | 37 ++++++++++++++++++++++++++++++++----- sys/sys/mman.h | 3 +++ 3 files changed, 37 insertions(+), 5 deletions(-)
@Jamie Thank you!