Bug 25974

Summary: Holes in files do not zero-fill
Product: Base System Reporter: Jim Zelenka <jimz>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Unspecified   
Hardware: Any   
OS: Any   

Description Jim Zelenka 2001-03-21 22:10:00 UTC
If a process opens a file, writes some bytes, seeks past the end of
the file, writes some bytes, then reads the intermediate (hole) area,
it does not see zeroes in the hole. The attached program does just this.
It initializes the buffer that the read goes into to contain all 0x7f,
and then reads into it. Sometimes the buffer (correctly) contains
zeroes in the hole, sometimes some bytes contain 0x7f, suggesting that
it did not write into the buffer, and sometimes the buffer contains
"other" bytes- it is not clear to me if these are uninitialized bytes
from an in-core cache block or uninitialized bytes on the disk.

How-To-Repeat: Here is a simple program one of our engineers wrote to reproduce this case:

/*
 * freebsd-zerofill.c 
 * 
 * FreeBSD 4.1.1 seems to have a bug where seeking past the end of a file and then writing
 * the data when read back in, may or may not be zero-filled.
 *
 * @author Edward Hogan
 * @version 1.0
 *
 */
/*
 * @PANASAS_COPYRIGHT@
 */


#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/param.h>
#include <fcntl.h>

#define TOTAL_BUF_LEN (72*1024)
#define BUF1_LEN      (63*1024)
#define HOLE_OFFSET   BUF1_LEN
#define HOLE_LEN      (1025)
#define BUF2_OFFSET   (BUF1_LEN+HOLE_LEN)
#define BUF2_LEN      (1023)
#define READ1_LEN     (BUF2_OFFSET+BUF2_LEN)
#define BUF3_OFFSET   (BUF2_OFFSET+BUF2_LEN)
#define BUF3_LEN      (TOTAL_BUF_LEN-BUF3_OFFSET)

#define WRITE1_LEN    (32*1024)

static char *fname = "freebsd-zerofill.xyz";

int main(){
  char *orig_buf, *target_buf;
  int test_num = 0;
  int fd = -1, ret;
  int i, err_count;
  int bytes_written, bytes_read, new_off;
  
  orig_buf = malloc(TOTAL_BUF_LEN);
  if (orig_buf == NULL) {
    printf("alloc buffer failed\n");
    goto test_failed;
  }
  target_buf = malloc(2*TOTAL_BUF_LEN);
  if (target_buf == NULL) {
    printf("alloc buffer failed\n");
    goto test_failed;
  }
  /* fill orig_buf with the alphabet */
  for(i = 0; i < TOTAL_BUF_LEN; i++) {
    orig_buf[i]   = (char) ('a' + (char)(i % 26));
    target_buf[i] = (char) (0x7f);
  }
  /* fill target_buf completely with 0x7f characters */
  for(i = TOTAL_BUF_LEN; i < (2 * TOTAL_BUF_LEN); i++) {
    target_buf[i] = (char) (0x7f);
  }
  /* */
  fd = open(fname,( O_RDWR | O_CREAT | O_TRUNC ), 0666);
  if (fd < 0) {
    printf("failed opening and creating file\n");
    goto test_failed;
  }
  else {
    printf("file opened               [SUCCESS]\n");
  }
  /* write to the test file */
  bytes_written = write(fd, orig_buf, WRITE1_LEN);
  if (bytes_written != WRITE1_LEN) {
    printf("error writing 1\n");
    goto test_failed;
  }
  else {
    printf("data written 1            [SUCCESS]\n");
  }
  /* write again to the test file */
  bytes_written = write(fd, orig_buf+WRITE1_LEN, BUF1_LEN-WRITE1_LEN);
  if (bytes_written != BUF1_LEN-WRITE1_LEN) {
    printf("write 2 failed\n");
    goto test_failed;
  }
  else {
    printf("data written 2            [SUCCESS]\n");
  }
  /* seek forward in file */
  new_off = lseek(fd, HOLE_LEN, SEEK_CUR);
  if (new_off != BUF2_OFFSET) { 
    printf("seek failed\n");
    goto test_failed;
  }
  else {
    printf("seek past end of file     [SUCCESS]\n");
  }
  /* write at seek point */
  bytes_written = write(fd, orig_buf + BUF2_OFFSET, BUF2_LEN);
  if (bytes_written != BUF2_LEN) {
    printf("write 3 failed\n");
    goto test_failed;
  }
  else {
    printf("data written 3            [SUCCESS]\n");
  }
  /* seek to beginning of file */
  new_off = lseek(fd, 0, SEEK_SET);
  if (new_off != 0) {
    printf("seek to start failed\n");
    goto test_failed;
  }
  else {
    printf("seek to start             [SUCCESS]\n");
  }
  /* read from the beginning */
  bytes_read = read(fd, target_buf, READ1_LEN);
  if (bytes_read != READ1_LEN) {
    printf("read failed\n");
  }
  else {
    printf("read entire file          [SUCCESS]\n");
  }
  /* **************************************************************** */
  /* examine result of read */
  if (memcmp(orig_buf, target_buf, BUF1_LEN) != 0) {
    printf("data before seek is bad   [FAILED]\n");
  }
  else {
    printf("data before seek is good  [SUCCESS]\n");
  }
  err_count = 0;
  for(i = BUF1_LEN; i < BUF2_OFFSET; i++) {
    if (target_buf[i]) { 
      printf("target_buf is dirty (char[%d] = (0x%x))\n", i, target_buf[i]);
      err_count++;
      if (err_count > 10){
        printf("target_buf is dirty       [FAILED]\n");
        break;
      }
    }
  }
  if (err_count == 0){
    printf("target_buf is clean       [SUCCESS]\n");
  }
  if (memcmp(orig_buf + BUF2_OFFSET, target_buf + BUF2_OFFSET, BUF2_LEN)) {
    printf("data after seek is bad   [FAILED]\n");
  }
  else {
    printf("data after seek is good   [SUCCESS]\n");
  }
  close(fd);
  return 0;
 test_failed:
  printf("!!! THE TEST DID NOT RUN !!!\n");
  if (fd != -1) {
    close(fd);
  }
  return 1;
}
Comment 1 Peter Pentchev 2001-03-22 13:06:54 UTC
On Wed, Mar 21, 2001 at 02:05:11PM -0800, jimz@panasas.com wrote:
> 
> >Number:         25974
> >Category:       kern
> >Synopsis:       Holes in files do not zero-fill
> >Originator:     Jim Zelenka
> >Release:        4.1.1
> >Organization:
> Panasas, Inc.
> >Environment:
> FreeBSD natasha.panasas.com 4.1.1-RELEASE FreeBSD 4.1.1-RELEASE #0: Thu Mar  8 14:41:32 EST 2001     jimz@natasha.panasas.com:/usr/src/sys/compile/NATASHA-HZ-DBG  i386
> 
> >Description:
> If a process opens a file, writes some bytes, seeks past the end of
> the file, writes some bytes, then reads the intermediate (hole) area,
> it does not see zeroes in the hole. The attached program does just this.
> It initializes the buffer that the read goes into to contain all 0x7f,
> and then reads into it. Sometimes the buffer (correctly) contains
> zeroes in the hole, sometimes some bytes contain 0x7f, suggesting that
> it did not write into the buffer, and sometimes the buffer contains
> "other" bytes- it is not clear to me if these are uninitialized bytes
> from an in-core cache block or uninitialized bytes on the disk.

Can you try updating your system to -STABLE (which is currently 4.3-RC),
and see if the problem persists?  On my 4.3-RC machine, your 'zerofill'
program outputs nothing but SUCCESS lines.

Information on updating your system can be found at:
http://www.FreeBSD.org/handbook/synching.html

G'luck,
Peter

-- 
because I didn't think of a good beginning of it.
Comment 2 Peter Pentchev freebsd_committer freebsd_triage 2001-03-22 13:14:47 UTC
State Changed
From-To: open->feedback

Awaiting response from submitter on whether upgrading fixes the problem.
Comment 3 Jim Zelenka 2001-03-23 21:45:24 UTC
Oops! I've been informed that the machine on which I did not see the zero-fill
problem is not a SCSI machine; it too has IDE disks. In fact, the machines
are identical.

-jim

Jim Zelenka
Senior Software Engineer, Panasas, Inc.
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
jimz@panasas.com
412-323-3500
Comment 4 Kris Kennaway freebsd_committer freebsd_triage 2001-06-02 10:45:36 UTC
State Changed
From-To: feedback->closed

No response from submitter, and this problem appears to be 
no longer present.
Comment 5 Kris Kennaway 2001-06-04 22:36:27 UTC
On Sat, Jun 02, 2001 at 04:45:08PM -0400, Jim Zelenka wrote:
> >For some reason this response did not appear in GNATS.  I asked Dima
> >to re-forward it.
> 
> That's probably my fault; I sent two responses. In the first, I said
> that I'd checked briefly, and I didn't see the problem. In the
> second one, I said that more extensive testing made the problem
> appear. The problem is particularly likely to occur on busy systems
> or systems where lots of files were created and deleted between
> runs of the test program. I suspect that the problem is that something
> is not zeroing out a block that is getting reallocated, and so
> old bits are coming back, but that's only based on empirical observation,
> and not a reading of the code.

We did fix a bug with this kind of symptoms a while back (and released
an advisory; perhaps that is related?)

Kris