Bug 238517

Summary: jail.conf Mount option does not unmount Filesystem on Jail stop anymore
Product: Base System Reporter: matthias+freebsd+bugzilla
Component: binAssignee: Jamie Gritton <jamie>
Status: Closed FIXED    
Severity: Affects Many People CC: andreas.sommer87, chris, crest, jamie
Priority: --- Keywords: regression
Version: 12.0-STABLE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
mymail jail.conf none

Description matthias+freebsd+bugzilla 2019-06-12 09:11:57 UTC
I use "Mount" in Jail.conf to Mount the jail root dynamicaly.
Up to STABLE 11.2 then mounted filesystem was unmounted on Jail stop.
After update to STABLE-12.0 the unmount does not happen anymore.
Comment 1 Jamie Gritton freebsd_committer 2019-06-12 15:19:04 UTC
Nothing has changed in the unmounting of filesystems on jail stop, and this is a feature known to still work.  It would be useful to see your jail.conf, to get to the bottom of what's not working in your case.

There is the issue that is the jail stops "on its own", i.e. when its last process dies (and the jail was created without "persist"), then no on-stop action will be taken because it's jail(8) that takes these actions.  This includes exec.stop and related commands, unmounting filesystems (e.g. from the "mount" option), and removing IP aliases.  But this has always been the case, and is nothing new with STABLE-12.
Comment 2 matthias+freebsd+bugzilla 2019-06-13 09:25:09 UTC
Created attachment 205029 [details]
mymail jail.conf
Comment 3 matthias+freebsd+bugzilla 2019-06-13 09:29:25 UTC
Perhaps the culprit is that I mount the jail root.
Comment 4 Jamie Gritton freebsd_committer 2019-06-13 16:29:37 UTC
OK, now I'm able to replicate the issue (I haven't yet checked it on STABLE-11).  It is indeed only a problem when the mount point in question is the jail root.  Interestingly, a ZFS or nullfs mount even at the jail root is no problem, but a UFS mount is.
Comment 5 Jamie Gritton freebsd_committer 2019-06-18 17:02:31 UTC
There's a fix that works most of the time: add a "-f" flag to the /sbin/umount that use.sbin/jail/command.c runs.

But I'm not going to do that just yet, as I'm currently down a rabbit hole of what appears to be either cred or buffer leaks.  If in the end I'm unable to figure that one out, I'll just go with the workaround.
Comment 6 commit-hook freebsd_committer 2019-06-18 23:49:37 UTC
A commit references this bug:

Author: jamie
Date: Tue Jun 18 23:49:14 UTC 2019
New revision: 349180
URL: https://svnweb.freebsd.org/changeset/base/349180

Log:
  Unmount filesystems on jail removal with "-f", to get around a situation
  where the jail root vnode reference is stopping the filesystem from
  unmounting, when the jail is removed by still exists in a dying state.

  PR:		238517
  Reported by:	matthias at harz.de

Changes:
  stable/12/usr.sbin/jail/command.c
Comment 7 Jamie Gritton freebsd_committer 2019-06-18 23:51:49 UTC
I've gone with the workaround in this situation.  I don't know what has changed since STABLE-11, but I can't say that the current behavior of UFS is incorrect (even though ZFS and nullfs don't seem to have this problem).

The fix is only to STABLE-12, and not to CURRENT, because I have bigger plans for CURRENT that should obviate this problem.
Comment 8 matthias+freebsd+bugzilla 2019-06-19 11:30:08 UTC
"manually" unmounting via "exec.poststop" work reliable.

perhaps unmounting Needs to be done later (exec.poststop time)
Comment 9 Jamie Gritton freebsd_committer 2019-06-19 14:55:56 UTC
The manual unmount in poststop would owrk, but not because it's later.  In this particular issue, the difference between STABL-11 and STABLE-12 seems to be that the unmount that used to work the first time now *almost* works - it allows the jail to release its hold on the mount point and then go away, which means that the next unmount attempt will then succeed.

So it's not that the poststop unmount works because it ran later, but it works because it's the second attempt.

In an unpatched system, you could see this by taking the mounting entirely out of jail.conf: Hand-mount the filesystem, start and then stop the jail, and then hand-unmount the filesystem.  You should see it fail (no matter when you try it), and then try it again and it will succeed.