Bug 206238

Summary:

FUSE can't enter to DIRECT_IO mode during file create.

Product:

Base System

Reporter:

MooseFS FreeBSD Team <freebsd>

Component:

kern

Assignee:

Rick Macklem <rmacklem>

Status:

Closed FIXED

Severity:

Affects Some People

CC:

amd64, emaste, rmacklem, rmacklem

Priority:

---

Version:

10.2-STABLE

Hardware:

amd64

OS:

Any

Attachments:

Description	Flags
patch fuse so it forces DIRECT_IO for WRONLY opens	none
patch fuse so that a create with DIRECT_IO opens with DIRECT_IO	none

Description MooseFS FreeBSD Team 2016-01-14 14:59:02 UTC

When we want to use direct IO flag it's mostly working ok.
But in very specific situation still uses cache omitting DIRECT FLAG.

This situation appears only when we are creating file.
We suspect that this problem is connected with function:

fuse_vnode_open(vp, 0, td)

in /usr/src/sys/fs/fuse/fuse_fnops.c file.
The second parameter of this function is always set to 0 value and in some cases removing DIRECT flag set by other functions.

Comment 1 rmacklem 2016-01-15 04:05:01 UTC

Are you referring to cases where the MooseFS fuse file system
sets FOPEN_DIRECT_IO in the reply to a fuse open?
or
Are you referring to processes on FreeBSD that open with O_DIRECT?

Btw, if you look at PR#194293, I have a patch there that forces
DIRECT_IO for the case of WRONLY opens, so that a small buffered
write doesn't get into trouble when the buffer cache code tries
to read the entire block in.

Comment 2 rmacklem 2016-01-16 23:22:02 UTC

Created attachment 165690 [details]
patch fuse so it forces DIRECT_IO for WRONLY opens

Comment 3 rmacklem 2016-01-16 23:24:50 UTC

Created attachment 165691 [details]
patch fuse so that a create with DIRECT_IO opens with DIRECT_IO

Please try applying both of these patches in order.
The second one fixes your problem, I think?
(Assuming it was a create with FOPEN_DIRECT_IO in
 its reply.)
The first is from PR#194293 and forces DIRECT_IO when
a file is WRONLY opened.

Please let me know if this fixes your problem.

Comment 4 MooseFS FreeBSD Team 2016-01-19 07:49:53 UTC

Thank you for your patches.
We applied them and all seems to work fine now.

We have a question.
When it will be added to official kernel?

Best regards
MooseFS Support Team

Comment 5 rmacklem 2016-01-20 04:50:17 UTC

A guess w.r.t. getting these patches in a release would
be 6mon->1year. Now, the patches will probably make it into
what FreeBSD calls head/current in April and in stable/10
in May. (This assumes that head isn't frozen for the FreeBSD11
release in April.)
(It's too late for the 10.3 release and I'm not sure if/when
 a 10.4 release is planned.)

Comment 6 Rick Macklem freebsd_committer

2016-02-05 01:09:20 UTC

The reporter comfirmed that the patches fixed the problem
for them. I am planning on committing these patches in April 2016.

Comment 7 MooseFS FreeBSD Team 2016-04-28 16:14:59 UTC

Hi,

just to remember you about this :)

Peter from MooseFS Team

Comment 8 commit-hook freebsd_committer

2016-05-14 20:04:11 UTC

A commit references this bug:

Author: rmacklem
Date: Sat May 14 20:03:22 UTC 2016
New revision: 299753
URL: https://svnweb.freebsd.org/changeset/base/299753

Log:
  Fix fuse to use DIRECT_IO when required.

  When a file is opened write-only and a partial block was written,
  buffered I/O would try and read the whole block in. This would
  result in a hung thread, since there was no open (fuse filehandle)
  that allowed reading. This patch avoids the problem by forcing
  DIRECT_IO for this case.
  It also sets DIRECT_IO when the file system specifies the FN_DIRECTIO
  flag in its reply to the open.

  Tested by:	nishida@asusa.net, freebsd@moosefs.com
  PR:		194293, 206238
  MFC after:	2 weeks

Changes:
  head/sys/fs/fuse/fuse_file.c
  head/sys/fs/fuse/fuse_vnops.c

Comment 9 Rick Macklem freebsd_committer

2016-05-28 22:48:35 UTC

The patches that fix this have been committed and MFC'd to stable/10.

Comment 10 MooseFS FreeBSD Team 2016-05-29 12:20:16 UTC

Thanks! :)

Peter from MooseFS Team