Bug 206652 - [ext2fs][patch] EXT4: sparse file fixes and mmap optimizations
Summary: [ext2fs][patch] EXT4: sparse file fixes and mmap optimizations
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Pedro F. Giffuni
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2016-01-26 19:22 UTC by Damjan Jovanovic
Modified: 2016-02-11 00:49 UTC (History)
1 user (show)

See Also:


Attachments
fix handling files with sparse blocks before extent's index, and implement runb in mmap (4.87 KB, patch)
2016-01-26 19:22 UTC, Damjan Jovanovic
no flags Details | Diff
clean up (3.91 KB, patch)
2016-02-10 19:59 UTC, Pedro F. Giffuni
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Damjan Jovanovic 2016-01-26 19:22:03 UTC
Created attachment 166156 [details]
fix handling files with sparse blocks before extent's index, and implement runb in mmap

It turns out that the ei_blk on the first struct ext4_extent_index in an EXT4 file's root extent isn't 0 if the file starts with sparse blocks. This will cause ext4_ext_binsearch_index() to keep passing the "if (lbn < m->ei_blk)" test, moving r to the left, until r < l, then set "path->ep_index = l - 1;" resulting in garbage data being read from wrong disk blocks since path->ep_index is pointing to before the first element of the array.

I've attached a patch that keeps track of the first and last block in each extent as it descends down the extent tree, thus being able to work out that some blocks are sparse earlier, preventing the previous out of range problem. Tracking the first and last block in each extent also lets us calculate where sparse extents end better, passing larger values to read ahead in mmap.

In ext4_bmapext() I've also started supporting the runb parameter (which is apparently the number of adjacent blocks prior to the block being converted in the same way that runp is the number of blocks after), which has sped up random access to mmaped files.
Comment 1 Pedro F. Giffuni freebsd_committer 2016-02-10 18:29:20 UTC
Grab.
Comment 2 Pedro F. Giffuni freebsd_committer 2016-02-10 19:59:21 UTC
Created attachment 166850 [details]
clean up

Some style(9) cleanups:

When returning a value enclose it in parenthesis.
Lines too long.
Comment 3 commit-hook freebsd_committer 2016-02-11 00:35:14 UTC
A commit references this bug:

Author: pfg
Date: Thu Feb 11 00:34:12 UTC 2016
New revision: 295494
URL: https://svnweb.freebsd.org/changeset/base/295494

Log:
  Ext4: fix handling of files with sparse blocks before extent's index.

  This is ongoing work from Damjan Jovanovic to improve ext4 read support
  with sparse files:

  Keep track of the first and last block in each extent as it descends down
  the extent tree, thus being able to work out that some blocks are sparse
  earlier. This solves an issue on r293680.

  In ext4_bmapext() start supporting the runb parameter, which appears to be
  the number of adjacent blocks prior to the block being converted in the
  same way that runp is the number of blocks after, speding up random access
  to mmaped files.

  PR:	206652

Changes:
  head/sys/fs/ext2fs/ext2_bmap.c
  head/sys/fs/ext2fs/ext2_extents.c
Comment 4 Pedro F. Giffuni freebsd_committer 2016-02-11 00:49:51 UTC
Thank you!

I think, ep_is_sparse should be a boolean, but it is better to change that in a subsequent commit.

Note that I won't be merging this code to the stable branches.