If fusefs.ko is not loaded, or mac_bsdextended.ko is loaded, and the fusefs tests are run via kyua, they will appear to pass but in fact are silently skipped. Knowing this would have saved me a couple of hours while I tried to reproduce the hang from PR 281073. Really, kyua should report the tests as skipped if they were, in fact, skipped. Can we integrate something like this, so that kyua is able to consume test output more easily? https://github.com/kinow/gtest-tap-listener/tree/master I tried it and it mostly works, though I had to tweak tap.h a bit and it doesn't report skipped tests for some reason (though it has code to do so). It also emits more than one test plan, which seems wrong. The point though is that it's pretty easy to change the framework's output format.
> It also emits more than one test plan, which seems wrong. More specifically: markj@xinde> sudo ./access 1..8 ok 1 chmod ok 2 create ok 3 eaccess ok 4 enosys ok 5 Getxattr ok 6 ok ok 7 unlink ok 8 unlink_sticky_directory 1..1 ok 1 erofs and indeed, kyua is unhappy: markj@xinde> kyua test access_tap:main access_tap:main -> broken: TAP test program yielded invalid data: Load of '/tmp/kyua.CFhAcW/2/stdout.txt' failed: Found duplicate plan 1..1 (saw 1..8 earlier) [0.002s] Results file id is usr_tests_sys_fs_fusefs.20240827-200656-893504 Results saved to /home/markj/.kyua/store/results.usr_tests_sys_fs_fusefs.20240827-200656-893504.db 0/1 passed (1 failed)
The problem is in the googletest-ATF integration. If you do "kyua debug access:main", you'll see that each individual test case is indeed SKIPPED. But that information isn't communicated to ATF. And since multiple googletest test cases correspond to each ATF test case, it's not clear what we should do. What if some googletest test cases are skipped but others aren't? I'll dig into the code and see if we can skip the entire thing if fusefs isn't loaded. That should be easy, and it's the most common cause for these tests to get skipped. In the meantime, I'll look at the gtest-tap-listener. BTW, what I do when I want to run a single test case is: $ cd /usr/tests/sys/fs/fusefs $ sudo mkdir mountpoint $ sudo chmod 1777 mountpoint $ sudo chflags uchg mountpoint $ ./access -v --gtest_filter=Access.unlink_sticky_directory Note: Google Test filter = Access.unlink_sticky_directory [==========] Running 1 test from 1 test suite. [----------] Global test environment set-up. [----------] 1 test from Access [ RUN ] Access.unlink_sticky_directory INIT ino= 0 ACCESS ino= 1 mask=0x1 LOOKUP ino= 1 some_file.txt UNLINK ino= 1 [ OK ] Access.unlink_sticky_directory (1 ms) [----------] 1 test from Access (1 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test suite ran. (1 ms total) [ PASSED ] 1 test.
Alan notes that there is a kyua PR which adds googletest support: https://github.com/freebsd/kyua/pull/203