Created attachment 263385 [details] kyua test report The 'sys/fs/fusefs/last_local_modify:main' testcase fails intermittently on arm64 with the following message: Standard output: [==========] Running 8 tests from 1 test suite. [----------] Global test environment set-up. [----------] 8 tests from LLM/LastLocalModify [ RUN ] LLM/LastLocalModify.lookup/0 [ OK ] LLM/LastLocalModify.lookup/0 (8 ms) [ RUN ] LLM/LastLocalModify.lookup/1 [ OK ] LLM/LastLocalModify.lookup/1 (5 ms) [ RUN ] LLM/LastLocalModify.lookup/2 [ OK ] LLM/LastLocalModify.lookup/2 (5 ms) [ RUN ] LLM/LastLocalModify.lookup/3 /usr/src/tests/sys/fs/fusefs/last_local_modify.cc:337: Failure Expected equality of these values: (off_t)newsize Which is: 15 sb.st_size Which is: 10 [ FAILED ] LLM/LastLocalModify.lookup/3, where GetParam() = "VOP_WRITE" (4 ms) [ RUN ] LLM/LastLocalModify.vfs_vget/0 [ OK ] LLM/LastLocalModify.vfs_vget/0 (5 ms) [ RUN ] LLM/LastLocalModify.vfs_vget/1 [ OK ] LLM/LastLocalModify.vfs_vget/1 (5 ms) [ RUN ] LLM/LastLocalModify.vfs_vget/2 [ OK ] LLM/LastLocalModify.vfs_vget/2 (4 ms) [ RUN ] LLM/LastLocalModify.vfs_vget/3 [ OK ] LLM/LastLocalModify.vfs_vget/3 (5 ms) [----------] 8 tests from LLM/LastLocalModify (45 ms total) [----------] Global test environment tear-down [==========] 8 tests from 1 test suite ran. (45 ms total) [ PASSED ] 7 tests. [ FAILED ] 1 test, listed below: [ FAILED ] LLM/LastLocalModify.lookup/3, where GetParam() = "VOP_WRITE" 1 FAILED TEST ===> Failed tests sys/fs/fusefs/last_local_modify:main -> failed: Returned non-success exit status 1 [0.060s] This failure is also visible in CI here: https://ci.freebsd.org/view/Test/job/FreeBSD-main-aarch64-test/1756/testReport/sys.fs.fusefs/last_local_modify/main/ I have reproduced this error using Bricoler with 100 runs for good measure: $ bricoler run freebsd-src-regression-suite --param freebsd-src:url=/usr/src --param freebsd-src:branch= --param freebsd-src-regression-suite:hypervisor=bhyve --param freebsd-src-regression-suite:memory=4096 --param freebsd-src-regression-suite:ncpus=2 --param freebsd-src-regression-suite:parallelism=1 --param freebsd-src-regression-suite:count=100 --param freebsd-src-regression-suite:tests='sys/fs/fusefs/last_local_modify:main' @asomers I see that you have written the tests in https://cgit.freebsd.org/src/commit/?id=13d593a5b060cf7be40acfa2ca9dc9e0e2339a31 (Fix a race in fusefs that can corrupt a file's size). Please triage as necessary.
Looks like there are more tests failing under the 'fusefs' testsuite. Please also test 'sys/fs/fusefs/io:main' with a count=100, as that is failing intermittently in CI too.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=d1eaa52d10f9b85e5f6358e1a280899b9d55dd07 commit d1eaa52d10f9b85e5f6358e1a280899b9d55dd07 Author: Alan Somers <asomers@FreeBSD.org> AuthorDate: 2025-09-15 00:22:36 +0000 Commit: Alan Somers <asomers@FreeBSD.org> CommitDate: 2025-09-15 00:30:30 +0000 fusefs: fix the last_local_modify LLM/LastLocalModify.lookup/3 test The LastLocalModify tests were originally written to simulate a race condition between VOP_SETATTR and VOP_LOOKUP. They were later extended to cover some other VOPs that can affect file size, including VOP_WRITE. However, the test never correctly simulated the race with VOP_WRITE. So that test only ever passed by accident. Fix it by always opening the file with O_DIRECT. PR: 289237 Reported by: Siva Mahadevan <me@svmhdvn.name> MFC after: 1 week tests/sys/fs/fusefs/last_local_modify.cc | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
With the committed patch, I can run the last_local_modify test for 300,000 iterations without failure. Without the patch, I can usually reproduce the failure in fewer than 500 iterations. Thanks for reporting it.
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3289d383171825622c907df79a0824c8c2175d1c commit 3289d383171825622c907df79a0824c8c2175d1c Author: Alan Somers <asomers@FreeBSD.org> AuthorDate: 2025-09-15 00:22:36 +0000 Commit: Alan Somers <asomers@FreeBSD.org> CommitDate: 2025-09-25 22:35:46 +0000 fusefs: fix the last_local_modify LLM/LastLocalModify.lookup/3 test The LastLocalModify tests were originally written to simulate a race condition between VOP_SETATTR and VOP_LOOKUP. They were later extended to cover some other VOPs that can affect file size, including VOP_WRITE. However, the test never correctly simulated the race with VOP_WRITE. So that test only ever passed by accident. Fix it by always opening the file with O_DIRECT. PR: 289237 Reported by: Siva Mahadevan <me@svmhdvn.name> (cherry picked from commit d1eaa52d10f9b85e5f6358e1a280899b9d55dd07) tests/sys/fs/fusefs/last_local_modify.cc | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=7dc691b703537113c686add53e2789566c8ab9e6 commit 7dc691b703537113c686add53e2789566c8ab9e6 Author: Alan Somers <asomers@FreeBSD.org> AuthorDate: 2025-09-15 00:22:36 +0000 Commit: Alan Somers <asomers@FreeBSD.org> CommitDate: 2025-09-27 14:29:52 +0000 fusefs: fix the last_local_modify LLM/LastLocalModify.lookup/3 test The LastLocalModify tests were originally written to simulate a race condition between VOP_SETATTR and VOP_LOOKUP. They were later extended to cover some other VOPs that can affect file size, including VOP_WRITE. However, the test never correctly simulated the race with VOP_WRITE. So that test only ever passed by accident. Fix it by always opening the file with O_DIRECT. PR: 289237 Reported by: Siva Mahadevan <me@svmhdvn.name> (cherry picked from commit d1eaa52d10f9b85e5f6358e1a280899b9d55dd07) tests/sys/fs/fusefs/last_local_modify.cc | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)