I can reliably reproduce a deadlock on msdosfs in sendfile(2). I can't reproduce it on tmpfs, nfs, ZFS, UFS, or bfffs (a fusefs file system). However, another user reported a similar hang in a different fuse file system (PR #246886). Stack trace of the hung process: getblkx+0x67 breadn_flags+0x44 vfs_bio_getpages+0x354 msdosfs_getpages+0x3b vop_stdgetpages_async+0x49 vnode_pager_getpages_async+0x43 sendfile_swapin+0x570 vn_sendfile+0x9ba sendfile+0x117 amd64_syscall+0x12e fast_syscall_common+0xf8 Steps to Reproduce: 1) git clone git@github.com:asomers/fsx-rs.git 2) cd fsx-rs 3) git checkout 6882003d89fefb1f815c57c2f027f8ddefe83b58 4) truncate -s 1g /tmp/msdosfs.img 5) newfs_msdos /tmp/msdosfs.img 6) sudo mdconfig -a -t vnode -f /tmp/msdosfs.img 7) mkdir /tmp/msdos 8) sudo mount -t msdos /dev/md0 /tmp/msdos 9) env RUST_LOG=debug cargo run -- -f fsx.toml -N 1000 -P /tmp -S 6953641379939875300 /tmp/msdos/fsx.bin
Could you put a ready binary for your tool somewhere, please?
The executable is too large to attach to bugzilla, so I put it on https://people.freebsd.org/~asomers/fsx.gz , compiled for 14.0 amd64. Also, I forgot to give the fsx.toml content used to reproduce this bug. Here it is: flen = 1048576 nomsyncafterwrite = true [opsize] max = 393216 [weights] read = 0 mapread = 0 mapwrite = 0 truncate = 2 fsync = 1 fdatasync = 1 sendfile = 10
I got this spinning thread at my first shot with fsx: https://people.freebsd.org/~pho/stress/log/log0405.txt
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=0152d453a08fa2bad694dc04a8184fce2b7faa10 commit 0152d453a08fa2bad694dc04a8184fce2b7faa10 Author: Konstantin Belousov <kib@FreeBSD.org> AuthorDate: 2023-02-11 18:09:30 +0000 Commit: Konstantin Belousov <kib@FreeBSD.org> CommitDate: 2023-02-13 22:29:42 +0000 msdosfs deextend: validate pages of the partial buffer Suppose that the cluster size is larger than page size. If the buffer at the old EOF (before extending) was partial and dirty, it cannot be automatically neither written out nor validated by the buffer cache, since extending buffer adds invalid pages at the end. Correct the buffer state by calling vfs_bio_clrbuf() on it, to mark newly added and zeroed pages as valid. Note that UFS is immune to the problem because ffs_truncate() always allocate the block and buffer for the last byte of the file. PR: 269341 Reported by: asomers In collaboration with: pho Reviewed by: markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D38549 sys/fs/msdosfs/msdosfs_denode.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=c2ee668306bbe3edf4a05246ed3a88f52dfc94ae commit c2ee668306bbe3edf4a05246ed3a88f52dfc94ae Author: Konstantin Belousov <kib@FreeBSD.org> AuthorDate: 2023-02-11 18:09:30 +0000 Commit: Konstantin Belousov <kib@FreeBSD.org> CommitDate: 2023-02-19 05:16:25 +0000 msdosfs deextend: validate pages of the partial buffer PR: 269341 (cherry picked from commit 0152d453a08fa2bad694dc04a8184fce2b7faa10) sys/fs/msdosfs/msdosfs_denode.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-)
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b2e3ab898cd2a8517858f32cc7397cf026013548 commit b2e3ab898cd2a8517858f32cc7397cf026013548 Author: Konstantin Belousov <kib@FreeBSD.org> AuthorDate: 2023-02-11 18:09:30 +0000 Commit: Ed Maste <emaste@FreeBSD.org> CommitDate: 2023-09-11 18:04:59 +0000 msdosfs deextend: validate pages of the partial buffer PR: 269341 (cherry picked from commit 0152d453a08fa2bad694dc04a8184fce2b7faa10) (cherry picked from commit c2ee668306bbe3edf4a05246ed3a88f52dfc94ae) sys/fs/msdosfs/msdosfs_denode.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-)
Assigning to the committer and closing.