When we want to use direct IO flag it's mostly working ok.
But in very specific situation still uses cache omitting DIRECT FLAG.
This situation appears only when we are creating file.
We suspect that this problem is connected with function:
fuse_vnode_open(vp, 0, td)
in /usr/src/sys/fs/fuse/fuse_fnops.c file.
The second parameter of this function is always set to 0 value and in some cases removing DIRECT flag set by other functions.
Are you referring to cases where the MooseFS fuse file system
sets FOPEN_DIRECT_IO in the reply to a fuse open?
Are you referring to processes on FreeBSD that open with O_DIRECT?
Btw, if you look at PR#194293, I have a patch there that forces
DIRECT_IO for the case of WRONLY opens, so that a small buffered
write doesn't get into trouble when the buffer cache code tries
to read the entire block in.
Created attachment 165690 [details]
patch fuse so it forces DIRECT_IO for WRONLY opens
Created attachment 165691 [details]
patch fuse so that a create with DIRECT_IO opens with DIRECT_IO
Please try applying both of these patches in order.
The second one fixes your problem, I think?
(Assuming it was a create with FOPEN_DIRECT_IO in
The first is from PR#194293 and forces DIRECT_IO when
a file is WRONLY opened.
Please let me know if this fixes your problem.
Thank you for your patches.
We applied them and all seems to work fine now.
We have a question.
When it will be added to official kernel?
MooseFS Support Team
A guess w.r.t. getting these patches in a release would
be 6mon->1year. Now, the patches will probably make it into
what FreeBSD calls head/current in April and in stable/10
in May. (This assumes that head isn't frozen for the FreeBSD11
release in April.)
(It's too late for the 10.3 release and I'm not sure if/when
a 10.4 release is planned.)
The reporter comfirmed that the patches fixed the problem
for them. I am planning on committing these patches in April 2016.
just to remember you about this :)
Peter from MooseFS Team
A commit references this bug:
Date: Sat May 14 20:03:22 UTC 2016
New revision: 299753
Fix fuse to use DIRECT_IO when required.
When a file is opened write-only and a partial block was written,
buffered I/O would try and read the whole block in. This would
result in a hung thread, since there was no open (fuse filehandle)
that allowed reading. This patch avoids the problem by forcing
DIRECT_IO for this case.
It also sets DIRECT_IO when the file system specifies the FN_DIRECTIO
flag in its reply to the open.
Tested by: email@example.com, firstname.lastname@example.org
PR: 194293, 206238
MFC after: 2 weeks
The patches that fix this have been committed and MFC'd to stable/10.
Peter from MooseFS Team