Bug 241474 - autofs fails to unmount exFAT filesystems automatically
Summary: autofs fails to unmount exFAT filesystems automatically
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 11.3-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: Edward Tomasz Napierala
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-10-24 15:38 UTC by Victor Sudakov
Modified: 2021-12-21 23:20 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Victor Sudakov 2019-10-24 15:38:23 UTC
The /etc/autofs/special_media map has support for exfat (requires sysutils/fusefs-exfat) but it fails to unmount it automatically after a 600 sec timeout.

I suppose the reason is that autounmountd is supposed to unmount filesystems that have the automounted mount option set. However, the exfat (that is, fuse) filesystem mounted automatically by autofs does not display this option:

$ mount | tail -n2
map -media on /media1 (autofs)
/dev/fuse on /media1/da0 (fusefs, local, synchronous)
$ 

So unmounting never happens and the filesystem stays mounted forever.
Comment 1 Victor Sudakov 2019-10-24 15:43:27 UTC
Compare with what happens when an msdosfs pen drive is mounted by autofs:

$ mount | tail -n2
map -media on /media1 (autofs)
/dev/da0 on /media1/da0 (msdosfs, asynchronous, local, noatime, nosuid, automounted)
$ 


The "automounted" option is there all right.
Comment 2 Victor Sudakov 2019-10-24 16:00:22 UTC
I have not tried yet but I expect this to be a problem with fuse-ntfs too.
Comment 3 Victor Sudakov 2019-10-26 13:56:55 UTC
Modifying the -media map to include "automounted" did NOT help.

-- /etc/autofs/special_media.bak       2019-10-26 14:02:02.810753000 +0700
+++ /etc/autofs/special_media   2019-10-26 14:41:13.597922000 +0700
@@ -41,7 +41,7 @@
        case "${_fstype}" in
        "exfat")
                if [ -f "/usr/local/sbin/mount.exfat" ]; then
-                       echo "-mountprog=/usr/local/sbin/mount.exfat,fstype=${_fstype},nosuid   :/dev/${_p}"
+                       echo "-mountprog=/usr/local/sbin/mount.exfat,fstype=${_fstype},nosuid,automounted       :/dev/${_p}"
                else
                        /usr/bin/logger -p info -t "special_media[$$]" \
                            "Cannot mount ${_fstype} formatted device /dev/${_p}: Install sysutils/fusefs-exfat first"
Comment 4 Edward Tomasz Napierala freebsd_committer freebsd_triage 2019-11-25 12:29:27 UTC
I've fixed something quite similar to this in r279489.  Which version of FreeBSD are you running?
Comment 5 Victor Sudakov 2019-11-25 13:20:08 UTC
(In reply to Edward Tomasz Napierala from comment #4)
> Which version of FreeBSD are you running?

11.3 as stated in the PR header.
Comment 6 Jason W. Bacon freebsd_committer freebsd_triage 2019-12-15 20:10:08 UTC
FYI, I see the same issue on 12.0 and 12.1.

Maybe a fuse update since r279489 broke your previous fix?
Comment 7 Jason W. Bacon freebsd_committer freebsd_triage 2019-12-18 23:53:31 UTC
I tried NTFS and it would not even automount for me.  I was able to mount manually with

   ntfs-3g /dev/da1 /media/subdir

Note that this USB stick was formatted without a partition using

   mkntfs --fast --label label /dev/da1

This oddly works while adding an MBR and using /dev/da1s1 hangs due to a known bug.

https://unix.stackexchange.com/questions/513732/why-is-mkntfs-taking-such-a-long-time

Using the unpartitioned device works fine under autofs with msdosfs.
Comment 8 Victor Sudakov 2021-01-29 11:45:00 UTC
This autofs thing is unusable for any practical purpose. Use sysutils/automount from ports for removable media, it works out of the box. This is probably the reason why noone is willing to fix autofs.
Comment 9 Jason W. Bacon freebsd_committer freebsd_triage 2021-01-29 15:34:59 UTC
(In reply to Victor Sudakov from comment #8)

Agreed.  External media mounting and unmount should probably always involve user interaction so the user develops the habit of unmounting before unplugging.

Mounting/unmounting automatically via autofs will leave the user wondering when it's safe to remove the media.

I've been using sysutils/automount for some time.  It works flawlessly for mounting.  A more convenient unmount option can be provided by having automount trigger the right external tool.  I've been using pcmanfm, which supports unmounting via a right-click on the device in the left panel.  The only bug is it does not cd out of an unmounted directory, but no harm comes from this.
Comment 10 Victor Sudakov 2021-01-29 17:00:00 UTC
(In reply to Jason W. Bacon from comment #9)
I'm certain there is some white magic involved but Thunar somehow learns about external media mounted at /media and provides a convenient soft button for unmounting.
Comment 11 Robert Wing freebsd_committer freebsd_triage 2021-01-31 20:02:52 UTC
(In reply to Victor Sudakov from comment #10)

agreed on it being unusable - it needs to be deprecated or fixed.

It looks like the original work was sponsored by the FreeBSD Foundation about 6 years ago. Unless they sponsor another effort, I wouldn't expect to see any significant improvements/fixes in automount, automountd, autounmountd, or autofs.
Comment 12 Edward Tomasz Napierala freebsd_committer freebsd_triage 2021-01-31 20:25:13 UTC
Autofs can't really help with bugs in completely unrelated code, in this case the fusefs-exfat.  I've fixed several of those, and somewhere in the comments below I've included a reference showing what the problem is and how to fix it.

The key to fixing the unmount problem is to simply set the autounmountd timeout low enough, eg to two seconds.

But the main thing is: autofs (both the general idea, in Sun 30 years ago, and this particular implementation) was created for network filesystems, not for removable media.  Adapting it for removable media is something that... well, works good enough for me, but it might very well be not suited for people expecting more typical workflow.  Which means: you might be better off with just using sysutils/automount; that's what it was created for.
Comment 13 Robert Wing freebsd_committer freebsd_triage 2021-02-01 01:23:31 UTC
(In reply to Edward Tomasz Napierala from comment #12)

So poll all the automounted filesystems with unmount every 2 seconds?

If all mounts/unmounts went through automountd, then filtering out all the mounted filesystems by checking the 'automounted' flag wouldn't even be necessary - automountd would already know which mounts that it's responsible for.

In short, I think all mount/unmount requests for an automounted filesystem should be done through automountd(8).
Comment 14 Victor Sudakov 2021-02-01 04:52:49 UTC
(In reply to Edward Tomasz Napierala from comment #12)
> Autofs can't really help with bugs in completely unrelated code, in this case the fusefs-exfat

In fact, I think the "automounted" option problem exists for any fuse filesystem, not just exfat. Which makes it pretty much unusable.

> autofs [...] was created for network filesystems

We have the old good amd for network filesystems, it works, why did they reinvent the wheel?
Comment 15 Edward Tomasz Napierala freebsd_committer freebsd_triage 2021-02-01 09:33:57 UTC
You don't need to poll filesystems to check which ones are automounted; the kernel notifies you.  It is possible to design automount in a way you suggest, it's just that it results in a horrible mess due to having to keep state in multiple places, and doesn't really improve anything.

Of course the "automounted" option works for FUSE filesystems; I've just tested it with ntfs-3g.  You don't need much: just make sure your filesystem doesn't clear it.

As for amd(8): even when it wasn't obsolete, it was 1. weird and 2. used a horrible hack with pretending it was an NFS server to create "virtual filesystems" in userspace.
Comment 16 Victor Sudakov 2021-02-01 10:02:42 UTC
(In reply to Edward Tomasz Napierala from comment #15)
> Of course the "automounted" option works for FUSE filesystems; I've just tested it with ntfs-3g. 

Well, I stand corrected.

Can you please share the output of "mount -t fuse" ?
Comment 17 Robert Wing freebsd_committer freebsd_triage 2021-02-01 10:47:52 UTC
(In reply to Edward Tomasz Napierala from comment #15)


> You don't need to poll filesystems to check which ones are automounted; the 
> kernel notifies you.  

I get that, but when you set a low timeout for autounmountd, you're basically polling all your automounted filesystems with unmount calls. It'd be better if there was no timeout at all. Ideally, the kernel would notify you when there are no more references to that mount, and then unmount it. Is there a kernel event that might grab that?

> It is possible to design automount in a way you suggest, it's just that it
> results in a horrible mess due to having to keep state in multiple places, and
> doesn't really improve anything.

I'm not suggesting keep the state in multiple places, exactly the opposite. All state would live in automountd. automount and autounmountd would defer to automountd to perform the requested operations.
Comment 18 Edward Tomasz Napierala freebsd_committer freebsd_triage 2021-02-01 11:37:22 UTC
(In reply to Victor Sudakov from comment #16)

Sure:

% mount -t fusefs
/dev/fuse on /media/md0 (fusefs, noatime, nosuid, automounted)
Comment 19 Edward Tomasz Napierala freebsd_committer freebsd_triage 2021-02-01 11:41:44 UTC
(In reply to Robert Wing from comment #17)

You're right about umount(2) polling, but in my experience it's quite cheap.  (It wasn't always that way, see https://reviews.freebsd.org/D7047.)  The mechanism for the kernel to notify that a filesystem is not active anymore, using eg kqueue(2), sounds like a good idea, but I don't think we have anything like that.

As for the state - it's not just automountd; there would also be some state keeping in the kernel.  Again, results of doing things that way are... messy.  Restarting automountd - quite useful for debugging, both when developing it, and when trying to debug automount configuration - becomes quite non-trivial.  See Linux, where (at least couple of years ago) it would often involve reboot.
Comment 20 Jason W. Bacon freebsd_committer freebsd_triage 2021-02-01 14:15:56 UTC
Just to clarify for everyone, this conversation is mainly about external media, not autofs in general.  I use autofs regularly for NFS and it works flawlessly there.
Comment 21 Victor Sudakov 2021-02-02 10:32:40 UTC
(In reply to Edward Tomasz Napierala from comment #18)
> /dev/fuse on /media/md0 (fusefs, noatime, nosuid, automounted)

Interesting. Why does it not show "automounted" for exfat then (you can see the output in the description of this PR)?
Comment 22 Edward Tomasz Napierala freebsd_committer freebsd_triage 2021-02-02 14:18:30 UTC
Like I said - it looks like a bug in fusefs-exfat.  The only requirement from autofs point of view is for the filesystems to not lose the "automounted" flag.
Comment 23 Robert Wing freebsd_committer freebsd_triage 2021-02-02 16:54:50 UTC
(In reply to Edward Tomasz Napierala from comment #22)

> The only requirement from autofs point of view is for the filesystems to not
> lose the "automounted" flag.
This is the real bug - automount should be able gracefully handle filesystems that lose the "automounted" flag.

The need to hack external programs for automount to work properly seems like the wrong approach. As some might say, this tends to get a bit...messy.
Comment 24 Edward Tomasz Napierala freebsd_committer freebsd_triage 2021-02-02 18:39:24 UTC
I don't agree.  Buggy filesystems should be fixed instead of having to complicate everything around them to work around their bugs.  It's not particularly hard, I've done that couple of times; it's a few of hours worth of work.
Comment 25 Robert Wing freebsd_committer freebsd_triage 2021-02-02 20:14:36 UTC
(In reply to Edward Tomasz Napierala from comment #24)

> I don't agree.  
Fair enough.

If you have a change of mind, let me know - I'd be up for fixing the real bug.

> Buggy filesystems should be fixed instead of having
> to complicate everything around them to work around their bugs.
The filesystems aren't broke.

> It's not particularly hard, I've done that couple of times; it's a few of 
> hours worth of work.
And you'll do it a couple more.

here's some reference info:

This bug was reported on 2019-10-24 15:38:23 UTC
There's about 15 bugs related to 'autofs' or 'automount' in bugzilla.
The last bug fix to to userland autofs: 2019-08-09

development and bugfixes have all but stopped for the autofs suite.
Comment 26 Graham Perrin freebsd_committer freebsd_triage 2021-06-29 20:27:17 UTC
Maybe of interest, bug 255213: 

> sysutils/fusefs-exfat: Pass "automounted" option into kernel
Comment 27 Victor Sudakov 2021-06-30 02:44:25 UTC
(In reply to Graham Perrin from comment #26)
> Maybe of interest, bug 255213: 

Thank you, it's quite to the point.