Bug 196726 - [FUSEFS] fuse truncates files on read, sometimes returns wrong data.
Summary: [FUSEFS] fuse truncates files on read, sometimes returns wrong data.
Status: Closed Unable to Reproduce
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.1-RELEASE
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-fs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-01-14 14:59 UTC by Carl Drougge
Modified: 2019-04-03 18:31 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Carl Drougge 2015-01-14 14:59:48 UTC
I made a python program to tickle two fusefs problems that may well be the same problem in different forms, both loosing data. I assume the problem is in the kernel and not the library, but have not verified this. (I have only looked at the symptoms.)

#!/usr/bin/env python
#
# FreeBSD (10.1-RELEASE) fuse seems pretty buggy.
# File attributes are cached forever (not tested here), and reported size
# is enforced. (So appending to a file outside the fuse layer has no effect
# if the file is already seen.) And even worse, the first time a file is
# closed (even ro) the reported size is truncated to. This updates the
# mtime, and of course risks discarding data written from somewhere else.
# (This is in test1.)
#
# File contents are also sometimes wrong in surprising ways when a file is
# overwritten. (This is in test2.)
#
# This program takes two directories as arguments, the first one a normal
# mount and the second a fuse mount (e.g. fusexmp or sshfs) of the same dir.

def test2(normal_dir, fuse_dir, test_fn):
    from os.path import exists, isdir, join
    assert isdir(normal_dir)
    assert isdir(fuse_dir)
    assert not exists(join(normal_dir, test_fn))
    assert not exists(join(fuse_dir, test_fn))
    
    with open(join(normal_dir, test_fn), "wb") as fh: 
        fh.write("foo")
    
    with open(join(fuse_dir, test_fn), "rb") as fh: 
        assert fh.read() == "foo", "this should work"
    
    with open(join(normal_dir, test_fn), "wb") as fh: 
        fh.write("longer")
    
    with open(join(fuse_dir, test_fn), "rb") as fh: 
        data = fh.read()
        assert data == "longer", "fuse misread %s ('longer' != %r)" % (test_fn, data,)

def test1(normal_dir, fuse_dir, test_fn):
    from os.path import exists, isdir, join
    assert isdir(normal_dir)
    assert isdir(fuse_dir)
    assert not exists(join(normal_dir, test_fn))
    
    with open(join(normal_dir, test_fn), "wb") as normal_fh:
        normal_fh.write("foo")
        normal_fh.flush()
        with open(join(fuse_dir, test_fn), "rb") as fuse_fh:
            assert fuse_fh.read() == "foo", "this should work"
            normal_fh.write("bar")
            normal_fh.flush()
    with open(join(normal_dir, test_fn), "rb") as normal_fh:
        data = normal_fh.read()
        assert data == "foobar", "fuse truncated %s ('foobar' != %r)" % (test_fn, data,)

if __name__ == "__main__":
    from sys import argv
    from traceback import print_exc
    
    normal_dir, fuse_dir = argv[1:]
    try:
        test1(normal_dir, fuse_dir, "TEST1")
    except Exception:
        print "Test 1 failed:"
        print_exc()
    
    try:
        test2(normal_dir, fuse_dir, "TEST2")
    except Exception:
        print "Test 2 failed:"
        print_exc()
Comment 1 Alan Somers freebsd_committer freebsd_triage 2019-04-03 15:19:32 UTC
I can't reproduce the problem on 13.0-CURRENT using libfuse's passthrough example.  It looks like you experienced three distinct problems:

1) File attributes are cached forever.  Fixed by r344183.

2) Truncate on close, even for RO files.  I can't reproduce using your test script.  Possibly fixed by r344185 or r344187.

3) Update mtime on close of RO files.  I can't reproduce either.

If you still experience the problem, then by all means reopen this bug.
Comment 2 Conrad Meyer freebsd_committer freebsd_triage 2019-04-03 18:31:55 UTC
Fwiw, I wouldn't claim r344183 fixes attributes being cached forever — it depends on what the filesystem tells us.  High-level defaults are non-zero and therefore still cached forever, because we don't do timeouts.

I might guess this is just being hidden by r344186 (no dirty data cache by default) or perhaps r299753 (converting buffer writes into direct writes).  But the conclusion is basically the same — can't reproduce this specific set of issues anymore, and Alan has a good handle on what our remaining issues are and a good dialogue with upstream libfuse on what the formal requirements are.