| Summary: | Holes in files do not zero-fill | ||
|---|---|---|---|
| Product: | Base System | Reporter: | Jim Zelenka <jimz> |
| Component: | kern | Assignee: | freebsd-bugs (Nobody) <bugs> |
| Status: | Closed FIXED | ||
| Severity: | Affects Only Me | ||
| Priority: | Normal | ||
| Version: | Unspecified | ||
| Hardware: | Any | ||
| OS: | Any | ||
On Wed, Mar 21, 2001 at 02:05:11PM -0800, jimz@panasas.com wrote: > > >Number: 25974 > >Category: kern > >Synopsis: Holes in files do not zero-fill > >Originator: Jim Zelenka > >Release: 4.1.1 > >Organization: > Panasas, Inc. > >Environment: > FreeBSD natasha.panasas.com 4.1.1-RELEASE FreeBSD 4.1.1-RELEASE #0: Thu Mar 8 14:41:32 EST 2001 jimz@natasha.panasas.com:/usr/src/sys/compile/NATASHA-HZ-DBG i386 > > >Description: > If a process opens a file, writes some bytes, seeks past the end of > the file, writes some bytes, then reads the intermediate (hole) area, > it does not see zeroes in the hole. The attached program does just this. > It initializes the buffer that the read goes into to contain all 0x7f, > and then reads into it. Sometimes the buffer (correctly) contains > zeroes in the hole, sometimes some bytes contain 0x7f, suggesting that > it did not write into the buffer, and sometimes the buffer contains > "other" bytes- it is not clear to me if these are uninitialized bytes > from an in-core cache block or uninitialized bytes on the disk. Can you try updating your system to -STABLE (which is currently 4.3-RC), and see if the problem persists? On my 4.3-RC machine, your 'zerofill' program outputs nothing but SUCCESS lines. Information on updating your system can be found at: http://www.FreeBSD.org/handbook/synching.html G'luck, Peter -- because I didn't think of a good beginning of it. State Changed From-To: open->feedback Awaiting response from submitter on whether upgrading fixes the problem. Oops! I've been informed that the machine on which I did not see the zero-fill problem is not a SCSI machine; it too has IDE disks. In fact, the machines are identical. -jim Jim Zelenka Senior Software Engineer, Panasas, Inc. Pioneering Object-Based Network Storage (ONS) www.panasas.com jimz@panasas.com 412-323-3500 State Changed From-To: feedback->closed No response from submitter, and this problem appears to be no longer present. On Sat, Jun 02, 2001 at 04:45:08PM -0400, Jim Zelenka wrote:
> >For some reason this response did not appear in GNATS. I asked Dima
> >to re-forward it.
>
> That's probably my fault; I sent two responses. In the first, I said
> that I'd checked briefly, and I didn't see the problem. In the
> second one, I said that more extensive testing made the problem
> appear. The problem is particularly likely to occur on busy systems
> or systems where lots of files were created and deleted between
> runs of the test program. I suspect that the problem is that something
> is not zeroing out a block that is getting reallocated, and so
> old bits are coming back, but that's only based on empirical observation,
> and not a reading of the code.
We did fix a bug with this kind of symptoms a while back (and released
an advisory; perhaps that is related?)
Kris
|
If a process opens a file, writes some bytes, seeks past the end of the file, writes some bytes, then reads the intermediate (hole) area, it does not see zeroes in the hole. The attached program does just this. It initializes the buffer that the read goes into to contain all 0x7f, and then reads into it. Sometimes the buffer (correctly) contains zeroes in the hole, sometimes some bytes contain 0x7f, suggesting that it did not write into the buffer, and sometimes the buffer contains "other" bytes- it is not clear to me if these are uninitialized bytes from an in-core cache block or uninitialized bytes on the disk. How-To-Repeat: Here is a simple program one of our engineers wrote to reproduce this case: /* * freebsd-zerofill.c * * FreeBSD 4.1.1 seems to have a bug where seeking past the end of a file and then writing * the data when read back in, may or may not be zero-filled. * * @author Edward Hogan * @version 1.0 * */ /* * @PANASAS_COPYRIGHT@ */ #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/param.h> #include <fcntl.h> #define TOTAL_BUF_LEN (72*1024) #define BUF1_LEN (63*1024) #define HOLE_OFFSET BUF1_LEN #define HOLE_LEN (1025) #define BUF2_OFFSET (BUF1_LEN+HOLE_LEN) #define BUF2_LEN (1023) #define READ1_LEN (BUF2_OFFSET+BUF2_LEN) #define BUF3_OFFSET (BUF2_OFFSET+BUF2_LEN) #define BUF3_LEN (TOTAL_BUF_LEN-BUF3_OFFSET) #define WRITE1_LEN (32*1024) static char *fname = "freebsd-zerofill.xyz"; int main(){ char *orig_buf, *target_buf; int test_num = 0; int fd = -1, ret; int i, err_count; int bytes_written, bytes_read, new_off; orig_buf = malloc(TOTAL_BUF_LEN); if (orig_buf == NULL) { printf("alloc buffer failed\n"); goto test_failed; } target_buf = malloc(2*TOTAL_BUF_LEN); if (target_buf == NULL) { printf("alloc buffer failed\n"); goto test_failed; } /* fill orig_buf with the alphabet */ for(i = 0; i < TOTAL_BUF_LEN; i++) { orig_buf[i] = (char) ('a' + (char)(i % 26)); target_buf[i] = (char) (0x7f); } /* fill target_buf completely with 0x7f characters */ for(i = TOTAL_BUF_LEN; i < (2 * TOTAL_BUF_LEN); i++) { target_buf[i] = (char) (0x7f); } /* */ fd = open(fname,( O_RDWR | O_CREAT | O_TRUNC ), 0666); if (fd < 0) { printf("failed opening and creating file\n"); goto test_failed; } else { printf("file opened [SUCCESS]\n"); } /* write to the test file */ bytes_written = write(fd, orig_buf, WRITE1_LEN); if (bytes_written != WRITE1_LEN) { printf("error writing 1\n"); goto test_failed; } else { printf("data written 1 [SUCCESS]\n"); } /* write again to the test file */ bytes_written = write(fd, orig_buf+WRITE1_LEN, BUF1_LEN-WRITE1_LEN); if (bytes_written != BUF1_LEN-WRITE1_LEN) { printf("write 2 failed\n"); goto test_failed; } else { printf("data written 2 [SUCCESS]\n"); } /* seek forward in file */ new_off = lseek(fd, HOLE_LEN, SEEK_CUR); if (new_off != BUF2_OFFSET) { printf("seek failed\n"); goto test_failed; } else { printf("seek past end of file [SUCCESS]\n"); } /* write at seek point */ bytes_written = write(fd, orig_buf + BUF2_OFFSET, BUF2_LEN); if (bytes_written != BUF2_LEN) { printf("write 3 failed\n"); goto test_failed; } else { printf("data written 3 [SUCCESS]\n"); } /* seek to beginning of file */ new_off = lseek(fd, 0, SEEK_SET); if (new_off != 0) { printf("seek to start failed\n"); goto test_failed; } else { printf("seek to start [SUCCESS]\n"); } /* read from the beginning */ bytes_read = read(fd, target_buf, READ1_LEN); if (bytes_read != READ1_LEN) { printf("read failed\n"); } else { printf("read entire file [SUCCESS]\n"); } /* **************************************************************** */ /* examine result of read */ if (memcmp(orig_buf, target_buf, BUF1_LEN) != 0) { printf("data before seek is bad [FAILED]\n"); } else { printf("data before seek is good [SUCCESS]\n"); } err_count = 0; for(i = BUF1_LEN; i < BUF2_OFFSET; i++) { if (target_buf[i]) { printf("target_buf is dirty (char[%d] = (0x%x))\n", i, target_buf[i]); err_count++; if (err_count > 10){ printf("target_buf is dirty [FAILED]\n"); break; } } } if (err_count == 0){ printf("target_buf is clean [SUCCESS]\n"); } if (memcmp(orig_buf + BUF2_OFFSET, target_buf + BUF2_OFFSET, BUF2_LEN)) { printf("data after seek is bad [FAILED]\n"); } else { printf("data after seek is good [SUCCESS]\n"); } close(fd); return 0; test_failed: printf("!!! THE TEST DID NOT RUN !!!\n"); if (fd != -1) { close(fd); } return 1; }