Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192" show incorrect results from "df" for free and used space when mounted immediately after creation. fsck on the new filesystem (before ever mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5. This error hasn't occurred in any runs of fsck immediately after "newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option). How-To-Repeat: "newfs -U -O 1 -b 65536 -f 8192 /dev/da0s1e" (assuming da0s1e is a large enough partition to trigger the problem) and then "fsck /dev/da0s1e".
This error doesn't occur when passing a block size of 32768 to newfs. Debugging fsck_ffs, the following mismatch is seen with the inconsistent filesystem: Starting program: /sbin/fsck_ffs /dev/md0 ** /dev/md0 ** Last Mounted on ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups Breakpoint 1, pass5 () at /usr/src/sbin/fsck_ffs/pass5.c:352 352 if (cursnapshot == 0 && (gdb) print cstotal $1 = {cs_ndir = 2, cs_nbfree = 1631816, cs_nifree = 3333116, cs_nffree = 21, cs_numclusters = 0, cs_spare = {0, 0, 0}} (gdb) print fs->fs_cstotal $2 = {cs_ndir = 0, cs_nbfree = 1631818, cs_nifree = 3333118, cs_nffree = 7, cs_numclusters = 0, cs_spare = {0, 0, 0}}
Responsible Changed From-To: freebsd-bugs->freebsd-fs Over to maintainer(s).
Author: mckusick Date: Fri Sep 24 19:08:56 2010 New Revision: 213119 URL: http://svn.freebsd.org/changeset/base/213119 Log: Reported problem: Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192" show incorrect results from "df" for free and used space when mounted immediately after creation. fsck on the new filesystem (before ever mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5. This error hasn't occurred in any runs of fsck immediately after "newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option). Solution: The default UFS1 superblock is located at offset 8K in the filesystem partition; the default UFS2 superblock is located at offset 64K in the filesystem partition. For UFS1 filesystems with a blocksize of 64K, the first alternate superblock resides at 64K which is the the location used for the default UFS2 superblock. By default, the system first checks for a valid superblock at the default location for a UFS2 filoesystem. For a UFS1 filesystem with a blocksize of 64K, there is a valid UFS1 superblock at this location. Thus, even though it is expected to be a backup superblock, the system will use it as its default superblock. So, we have to ensure that all the statistcs on usage are correct in this first alternate superblock as it is the superblock that will actually be used. While tracking down this problem, another limitation of UFS1 became evident. For UFS1, the number of inodes per cylinder group is stored in an int16_t. Thus the maximum number of inodes per cylinder group is limited to 2^15 - 1. This limit can easily be exceeded for block sizes of 32K and above. Thus when building UFS1 filesystems, newfs must limit the number of inodes per cylinder group to 2^15 - 1. Reported by: Guy Helmer<ghelmer@palisadesys.com> Followup by: Bruce Cran <brucec@freebsd.org> PR: 107692 MFC after: 4 weeks Modified: head/sbin/newfs/mkfs.c Modified: head/sbin/newfs/mkfs.c ============================================================================== --- head/sbin/newfs/mkfs.c Fri Sep 24 19:07:14 2010 (r213118) +++ head/sbin/newfs/mkfs.c Fri Sep 24 19:08:56 2010 (r213119) @@ -376,16 +376,20 @@ restart: * Start packing more blocks into the cylinder group until * it cannot grow any larger, the number of cylinder groups * drops below MINCYLGRPS, or we reach the size requested. + * For UFS1 inodes per cylinder group are stored in an int16_t + * so fs_ipg is limited to 2^15 - 1. */ for ( ; sblock.fs_fpg < maxblkspercg; sblock.fs_fpg += sblock.fs_frag) { sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode), INOPB(&sblock)); - if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS) - break; - if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize) - continue; - if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize) - break; + if (Oflag > 1 || (Oflag == 1 && sblock.fs_ipg <= 0x7fff)) { + if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS) + break; + if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize) + continue; + if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize) + break; + } sblock.fs_fpg -= sblock.fs_frag; sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode), INOPB(&sblock)); @@ -584,8 +588,20 @@ restart: printf("** Exiting on Xflag 3\n"); exit(0); } - if (!Nflag) + if (!Nflag) { do_sbwrite(&disk); + /* + * For UFS1 filesystems with a blocksize of 64K, the first + * alternate superblock resides at the location used for + * the default UFS2 superblock. As there is a valid + * superblock at this location, the boot code will use + * it as its first choice. Thus we have to ensure that + * all of its statistcs on usage are correct. + */ + if (Oflag == 1 && sblock.fs_bsize == 65536) + wtfs(fsbtodb(&sblock, cgsblock(&sblock, 0)), + sblock.fs_bsize, (char *)&sblock); + } for (i = 0; i < sblock.fs_cssize; i += sblock.fs_bsize) wtfs(fsbtodb(&sblock, sblock.fs_csaddr + numfrags(&sblock, i)), sblock.fs_cssize - i < sblock.fs_bsize ? _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
State Changed From-To: open->patched I am taking responsibility for this bug. A fix has been applied to head. If no problems arise it will be MFC'ed to 7-stable and 8-stable in 4 weeks time.
Responsible Changed From-To: freebsd-fs->mckusick I am taking responsibility for this bug. A fix has been applied to head. If no problems arise it will be MFC'ed to 7-stable and 8-stable in 4 weeks time.
Author: mckusick Date: Tue Nov 23 01:24:27 2010 New Revision: 215728 URL: http://svn.freebsd.org/changeset/base/215728 Log: MFC of 213119 Reported problem: Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192" show incorrect results from "df" for free and used space when mounted immediately after creation. fsck on the new filesystem (before ever mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5. This error hasn't occurred in any runs of fsck immediately after "newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option). Solution: The default UFS1 superblock is located at offset 8K in the filesystem partition; the default UFS2 superblock is located at offset 64K in the filesystem partition. For UFS1 filesystems with a blocksize of 64K, the first alternate superblock resides at 64K which is the the location used for the default UFS2 superblock. By default, the system first checks for a valid superblock at the default location for a UFS2 filoesystem. For a UFS1 filesystem with a blocksize of 64K, there is a valid UFS1 superblock at this location. Thus, even though it is expected to be a backup superblock, the system will use it as its default superblock. So, we have to ensure that all the statistcs on usage are correct in this first alternate superblock as it is the superblock that will actually be used. While tracking down this problem, another limitation of UFS1 became evident. For UFS1, the number of inodes per cylinder group is stored in an int16_t. Thus the maximum number of inodes per cylinder group is limited to 2^15 - 1. This limit can easily be exceeded for block sizes of 32K and above. Thus when building UFS1 filesystems, newfs must limit the number of inodes per cylinder group to 2^15 - 1. Reported by: Guy Helmer<ghelmer@palisadesys.com> Followup by: Bruce Cran <brucec@freebsd.org> PR: 107692 Modified: stable/8/sbin/newfs/mkfs.c Directory Properties: stable/8/sbin/newfs/ (props changed) Modified: stable/8/sbin/newfs/mkfs.c ============================================================================== --- stable/8/sbin/newfs/mkfs.c Mon Nov 22 23:49:06 2010 (r215727) +++ stable/8/sbin/newfs/mkfs.c Tue Nov 23 01:24:27 2010 (r215728) @@ -376,16 +376,20 @@ restart: * Start packing more blocks into the cylinder group until * it cannot grow any larger, the number of cylinder groups * drops below MINCYLGRPS, or we reach the size requested. + * For UFS1 inodes per cylinder group are stored in an int16_t + * so fs_ipg is limited to 2^15 - 1. */ for ( ; sblock.fs_fpg < maxblkspercg; sblock.fs_fpg += sblock.fs_frag) { sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode), INOPB(&sblock)); - if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS) - break; - if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize) - continue; - if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize) - break; + if (Oflag > 1 || (Oflag == 1 && sblock.fs_ipg <= 0x7fff)) { + if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS) + break; + if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize) + continue; + if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize) + break; + } sblock.fs_fpg -= sblock.fs_frag; sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode), INOPB(&sblock)); @@ -584,8 +588,20 @@ restart: printf("** Exiting on Xflag 3\n"); exit(0); } - if (!Nflag) + if (!Nflag) { do_sbwrite(&disk); + /* + * For UFS1 filesystems with a blocksize of 64K, the first + * alternate superblock resides at the location used for + * the default UFS2 superblock. As there is a valid + * superblock at this location, the boot code will use + * it as its first choice. Thus we have to ensure that + * all of its statistcs on usage are correct. + */ + if (Oflag == 1 && sblock.fs_bsize == 65536) + wtfs(fsbtodb(&sblock, cgsblock(&sblock, 0)), + sblock.fs_bsize, (char *)&sblock); + } for (i = 0; i < sblock.fs_cssize; i += sblock.fs_bsize) wtfs(fsbtodb(&sblock, sblock.fs_csaddr + numfrags(&sblock, i)), sblock.fs_cssize - i < sblock.fs_bsize ? _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Author: mckusick Date: Tue Nov 23 01:32:44 2010 New Revision: 215729 URL: http://svn.freebsd.org/changeset/base/215729 Log: MFC of 213119 Reported problem: Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192" show incorrect results from "df" for free and used space when mounted immediately after creation. fsck on the new filesystem (before ever mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5. This error hasn't occurred in any runs of fsck immediately after "newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option). Solution: The default UFS1 superblock is located at offset 8K in the filesystem partition; the default UFS2 superblock is located at offset 64K in the filesystem partition. For UFS1 filesystems with a blocksize of 64K, the first alternate superblock resides at 64K which is the the location used for the default UFS2 superblock. By default, the system first checks for a valid superblock at the default location for a UFS2 filoesystem. For a UFS1 filesystem with a blocksize of 64K, there is a valid UFS1 superblock at this location. Thus, even though it is expected to be a backup superblock, the system will use it as its default superblock. So, we have to ensure that all the statistcs on usage are correct in this first alternate superblock as it is the superblock that will actually be used. While tracking down this problem, another limitation of UFS1 became evident. For UFS1, the number of inodes per cylinder group is stored in an int16_t. Thus the maximum number of inodes per cylinder group is limited to 2^15 - 1. This limit can easily be exceeded for block sizes of 32K and above. Thus when building UFS1 filesystems, newfs must limit the number of inodes per cylinder group to 2^15 - 1. Reported by: Guy Helmer<ghelmer@palisadesys.com> Followup by: Bruce Cran <brucec@freebsd.org> PR: 107692 Modified: stable/7/sbin/newfs/mkfs.c Directory Properties: stable/7/sbin/newfs/ (props changed) Modified: stable/7/sbin/newfs/mkfs.c ============================================================================== --- stable/7/sbin/newfs/mkfs.c Tue Nov 23 01:24:27 2010 (r215728) +++ stable/7/sbin/newfs/mkfs.c Tue Nov 23 01:32:44 2010 (r215729) @@ -367,16 +367,20 @@ restart: * Start packing more blocks into the cylinder group until * it cannot grow any larger, the number of cylinder groups * drops below MINCYLGRPS, or we reach the size requested. + * For UFS1 inodes per cylinder group are stored in an int16_t + * so fs_ipg is limited to 2^15 - 1. */ for ( ; sblock.fs_fpg < maxblkspercg; sblock.fs_fpg += sblock.fs_frag) { sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode), INOPB(&sblock)); - if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS) - break; - if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize) - continue; - if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize) - break; + if (Oflag > 1 || (Oflag == 1 && sblock.fs_ipg <= 0x7fff)) { + if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS) + break; + if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize) + continue; + if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize) + break; + } sblock.fs_fpg -= sblock.fs_frag; sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode), INOPB(&sblock)); @@ -568,8 +572,20 @@ restart: printf("** Exiting on Eflag 3\n"); exit(0); } - if (!Nflag) + if (!Nflag) { sbwrite(&disk, 0); + /* + * For UFS1 filesystems with a blocksize of 64K, the first + * alternate superblock resides at the location used for + * the default UFS2 superblock. As there is a valid + * superblock at this location, the boot code will use + * it as its first choice. Thus we have to ensure that + * all of its statistcs on usage are correct. + */ + if (Oflag == 1 && sblock.fs_bsize == 65536) + wtfs(fsbtodb(&sblock, cgsblock(&sblock, 0)), + sblock.fs_bsize, (char *)&sblock); + } for (i = 0; i < sblock.fs_cssize; i += sblock.fs_bsize) wtfs(fsbtodb(&sblock, sblock.fs_csaddr + numfrags(&sblock, i)), sblock.fs_cssize - i < sblock.fs_bsize ? _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
State Changed From-To: patched->closed The fix has been MFC'ed to 7-stable and 8-stable.