The following sequence of commands leads to a panic due to a page fault in the kernel: # mdconfig -s 64m md0 # newfs_msdos -F 12 -n 2 -e 128 -S 4096 -c 2 md0 newfs_msdos: cannot get number of sectors per track: Operation not supported newfs_msdos: cannot get number of heads: Operation not supported newfs_msdos: debug: cls=4084 x1=6 SecPerClust=2 newfs_msdos: warning: FAT type limits file system to 8175 sectors /dev/md0: 8168 sectors in 4084 FAT12 clusters (8192 bytes/cluster) BytesPerSec=4096 SecPerClust=2 ResSectors=1 FATs=2 RootDirEnts=128 Sectors=8175 Media=0xf0 FATsecs=2 SecPerTrack=63 Heads=1 HiddenSecs=0 # mount -t msdosfs /dev/md0 /mnt The cause is the fact that a FAT entry crosses the boundary between the 1st and 2nd FAT sector (byte offset 4095 is the last byte in the 1st sector and 4096 the first byte in the 2nd sector). This reads 1 byte beyond the page allocated for the 1st sector. There are other issues, with less drastic consequences, e.g.: # newfs_msdos -F 12 -s 64m -n 2 -e 1024 -S 512 newfs_msdos: cannot get number of sectors per track: Operation not supported newfs_msdos: cannot get number of heads: Operation not supported newfs_msdos: warning: FAT type limits file system to 32768 sectors /dev/md1: 32672 sectors in 4084 FAT12 clusters (4096 bytes/cluster) BytesPerSec=512 SecPerClust=8 ResSectors=1 FATs=2 RootDirEnts=1024 Sectors=32768 Media=0xf0 FATsecs=12 SecPerTrack=63 Heads=16 HiddenSecs=0 # df -i /mnt Filesystem 1K-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/md1 12288 8 12280 0% 0 1024 0% /mnt # mount -t msdosfs /dev/md0 /mnt # ls -lsd /mnt 32 drwxr-xr-x 1 root wheel 32768 Jan 1 1980 /mnt/ # dd if=/dev/zero of=/mnt/TEST.DAT bs=1k count=12288 dd: /mnt/TEST.DAT: No space left on device 12281+0 records in 12280+0 records out 12574720 bytes transferred in 0.013075 secs (961691614 bytes/sec) This file system has 4084 clusters of 4 KB (or roughly 16 MB), but df shows the total size of the data area as only 12288 KB (12 MB), but trying to write a file of size 12 MB results in an error since apparently 8 KB have already been allocated (for the FAT sectors that actually exist outside the data area), and "ls -lask /mnt" shows a "." entry with an apparent size of 32 KB (which also are allocated outside the data area).
Patches for all issues in this PR are presented in review D39386.
Created attachment 241321 [details] Test script used to validate the patches in review D39386 A rough test script that would have triggered a panic for a FAT12 file system with many data clusters (i.e. more than 1 cluster required for the FAT). It does also verify the calculation and reporting of other limits (correct number of blocks reported by df, inode accounting, disk full report on attempt to store 1 more file, ...). Not verified is that "." in the root directory reports 0 blocks occupied (since the root directory is stored outside the data area). This has been independently verified to have been fixed by the patches in review D39386. Some FAT32 tests at the end report a change of the number of allocated blocks when such a file system is re-mounted. This issue has not been investigated, yet.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=0728695c63efda298feccefb3615c23cb6682929 commit 0728695c63efda298feccefb3615c23cb6682929 Author: Stefan Eßer <se@FreeBSD.org> AuthorDate: 2023-04-25 06:35:16 +0000 Commit: Stefan Eßer <se@FreeBSD.org> CommitDate: 2023-04-25 07:58:29 +0000 fs/msdosfs: Fix potential panic and size calculations Some combinations of FAT12 file system parameters could cause a kernel panic due to an unmapped access if the size of the FAT was larger than the CPU page size. The reason is that FAT12 uses 3 bytes to store 2 FAT pointers, leading to partial FAT pointers at the end of buffers of a size that is not a multiple of 3. With a typical page size of 4 KB, this caused the FAT entry at byte offsets 4095 and 4096 to cross the page boundary, with only the first page mapped. This was fixed by adjusting the mapping to always cover both bytes of each FAT entry. Testing revealed 2 other inconsistencies that are fixed by this commit: 1) The calculation of the size of the data area did not take into account the fact that the first two data block numbers are reserved and that the data area starts with block 2. This could cause a FAT12 file system created with the maximum supported number of blocks to be incorrectly identified as FAT16. 2) The root directory does not take up space in the data area of a FAT12 or FAT16 file system, since it is placed into a reserved area outside of that data area. This commits makes stat() report the logical size of the root directory, but with 0 blocks allocated from the data area. PR: 270587 Reviewed by: mckusick Differential Revision: https://reviews.freebsd.org/D39386 sys/fs/msdosfs/msdosfs_fat.c | 12 +++++++++--- sys/fs/msdosfs/msdosfs_vfsops.c | 15 +++++++++------ sys/fs/msdosfs/msdosfs_vnops.c | 7 +++++-- 3 files changed, 23 insertions(+), 11 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=d767bf361b3ebdb3955473cd378f8a8dcf9c85f0 commit d767bf361b3ebdb3955473cd378f8a8dcf9c85f0 Author: Stefan Eßer <se@FreeBSD.org> AuthorDate: 2023-03-08 16:58:00 +0000 Commit: Stefan Eßer <se@FreeBSD.org> CommitDate: 2023-05-01 08:09:33 +0000 msdosfs: fix debug print format and parameter Building with -DMSDOSFS_DEBUG failed due to a format mismatch and a variable that has been renamed but not updated in the printf() parameter list. (cherry picked from commit 2d8cf575d5778781928699f9b7cfb448bd2f1f8e) fs/msdosfs: add tracking of free root directory entries This update implements tallying of free directory entries during create, delete, or rename operations on FAT12 and FAT16 file systems. Prior to this change, the total number of root directory entries was reported as number of inodes, but 0 as the number of free inodes, causing system health monitoring software to warn about a suspected disk full issue. The FAT12 and FAT16 file systems provide a limited number of root directory entries, e.g. 512 on typical hard disk formats. The valid range of values is 1 to 65535, but the msdosfs code will effectively round up "odd" values to the next multiple of 16 (e.g. 513 would allow for 528 root directory entries). This update implements tracking of directory entries during create, delete, or rename operations, with initial values determined by scanning the directory when the file system is mounted. Total and free directory entries are reported in the f_files and f_ffree elements of struct statfs, despite differences in semantics of these values: - There is no limit on the number of files and directories that can be created on a FAT file system. Only the root directory of FAT12 and FAT16 file systems is limited, any number of files can still be created in sub-directories, even when 0 free "inodes" are reported. - A single file can require 1 to 21 directory entries, depending on the character set, structure, and length of the name. The DOS 8.3 style file name takes up 1 entry, and if the name does not comply with the syntax of a DOS 8.3 file name, 1 additional entry is used for each 13 characters of the file name. Since all these entries have to be contiguous, it is possible that a file or directory with a long name can not be created, despite a sufficient total number of free directory entries. - Renaming a file can require more directory entries than currently allocated to store its long name, which may prevent an in-place update of the name if more entries are needed. This may cause a rename operation to fail if no contiguous range of free entries for the new name can be found. - The volume label is stored in a directory entry. An empty FAT file system with a volume label will therefore show 1 used "inode" in df. - The perceentage of free inodes shown in df or monitoring tools does only represent the state of the root directory of a FAT12 or FAT16 file system. Neither does a reported value of 0% free inodes does prevent files from being created in sub-directories, nor does a value of 50% free inodes guarantee that even a single file with a "long" name can be created in the root directory (if every other directory entry is occupied and there are no 2 contiguous entries). The statfs(2) and df(1) man pages have been updated with a notice regarding the possibly different semantics of values reported as total and free inodes for non-Unix file systems. PR: 270053 Reported by: Ben Woods <woodsb02@freebsd.org> Approved by: mckusick Differential Revision: https://reviews.freebsd.org/D38987 (cherry picked from commit c33db74b5323480fba7adef58e8aa88f6091d134) fs/msdosfs: Fix potential panic and size calculations Some combinations of FAT12 file system parameters could cause a kernel panic due to an unmapped access if the size of the FAT was larger than the CPU page size. The reason is that FAT12 uses 3 bytes to store 2 FAT pointers, leading to partial FAT pointers at the end of buffers of a size that is not a multiple of 3. With a typical page size of 4 KB, this caused the FAT entry at byte offsets 4095 and 4096 to cross the page boundary, with only the first page mapped. This was fixed by adjusting the mapping to always cover both bytes of each FAT entry. Testing revealed 2 other inconsistencies that are fixed by this commit: 1) The calculation of the size of the data area did not take into account the fact that the first two data block numbers are reserved and that the data area starts with block 2. This could cause a FAT12 file system created with the maximum supported number of blocks to be incorrectly identified as FAT16. 2) The root directory does not take up space in the data area of a FAT12 or FAT16 file system, since it is placed into a reserved area outside of that data area. This commits makes stat() report the logical size of the root directory, but with 0 blocks allocated from the data area. PR: 270587 Reviewed by: mckusick Differential Revision: https://reviews.freebsd.org/D39386 (cherry picked from commit 0728695c63efda298feccefb3615c23cb6682929) bin/df/df.1 | 13 ++++- lib/libc/sys/statfs.2 | 18 +++++- sys/fs/msdosfs/msdosfs_denode.c | 2 +- sys/fs/msdosfs/msdosfs_fat.c | 12 +++- sys/fs/msdosfs/msdosfs_lookup.c | 5 +- sys/fs/msdosfs/msdosfs_vfsops.c | 120 +++++++++++++++++++++++++++++++++++++--- sys/fs/msdosfs/msdosfs_vnops.c | 7 ++- sys/fs/msdosfs/msdosfsmount.h | 17 ++++++ 8 files changed, 177 insertions(+), 17 deletions(-)