Bug 215690 - kyua framework attempts to open/write test files after running tests; doesn't work when tests enter capability mode
Summary: kyua framework attempts to open/write test files after running tests; doesn't...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Julio Merino,+1 347 694 0576,New York City
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-01-01 02:47 UTC by Enji Cooper
Modified: 2018-01-14 04:52 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer 2017-01-01 02:47:02 UTC
The testcases are failing to create results files and are leaving `testdir` behind.

$ sudo kyua test -k /usr/tests/sys/vfs/Kyuafile lookup_cap_dotdot
lookup_cap_dotdot:capmode__negative  ->  broken: Premature exit; test case received signal 6  [0.005s]
lookup_cap_dotdot:lookup_cap_dotdot__advanced  ->  broken: Premature exit; test case received signal 6  [0.006s]
lookup_cap_dotdot:lookup_cap_dotdot__basic  ->  broken: Premature exit; test case received signal 6  [0.003s]
lookup_cap_dotdot:lookup_cap_dotdot__negative  ->  broken: Premature exit; test case received signal 6  [0.003s]
lookup_cap_dotdot:openat__basic_negative  ->  passed  [0.003s]
lookup_cap_dotdot:openat__basic_positive  ->  passed  [0.003s]

Results file id is usr_tests_sys_vfs.20170101-024440-307762
Results saved to /root/.kyua/store/results.usr_tests_sys_vfs.20170101-024440-307762.db

2/6 passed (4 failed)
$ sudo kyua debug -k /usr/tests/sys/vfs/Kyuafile lookup_cap_dotdot:capmode__negative
FATAL ERROR: Cannot create results file '/tmp/kyua.fAccdD/2/result.atf': Not permitted in capability mode
Files left in work directory after failure: testdir
lookup_cap_dotdot:capmode__negative  ->  broken: Premature exit; test case received signal 6
Comment 1 Conrad Meyer freebsd_committer 2017-01-01 02:52:50 UTC
This is a framework bug and nothing the individual test can fix.  These tests pass when run directly.
Comment 2 Enji Cooper freebsd_committer 2017-01-01 02:56:44 UTC
(In reply to Conrad E. Meyer from comment #1)

*sigh*

It's a test bug. The tests need to fork, enter cap mode, then make sure the invariant does the right thing.
Comment 3 commit-hook freebsd_committer 2017-01-01 04:01:59 UTC
A commit references this bug:

Author: ngie
Date: Sun Jan  1 04:01:27 UTC 2017
New revision: 310994
URL: https://svnweb.freebsd.org/changeset/base/310994

Log:
  Make sys/vfs/lookup_cap_dotdot actually work with "kyua test"

  The tests don't work when reading/writing to file descriptors in the
  sandbox after entering capability mode (and wouldn't have, regardless
  of the framework), so adjust the tests so they function within the
  framework.

  For tests that enter capability mode over the course of the test, the
  following is now done:

    1. Fork child process for capability mode test.
    2. In child...
       i.   Enter capability mode.
       ii.  Test invariants.
       iii. Exit after calling test function.
    3. Collect status for child and determine whether or not it completed
       successfully.

  In order to test the invariants in the child process, they now use assert(3)
  instead of ATF_REQUIRE*, as the atf-c-api functions right to results files
  in the directories in order to determine where and how tests fail.

  While in the area, fix several -Wshadow and -Wunused warnings found when I
  bumped WARNS up to 6, and fix some minor style(9) issues with indentation
  and type alignment.

  PR:	215690

Changes:
  head/tests/sys/vfs/lookup_cap_dotdot.c
Comment 4 Conrad Meyer freebsd_committer 2017-01-14 01:01:13 UTC
As suspected, the framework fix is totally trivial.  Something like this should do fine:

https://github.com/cemeyer/atf/commit/317c80628513337af41b29fceeaa62fbaf498a02
Comment 5 Enji Cooper freebsd_committer 2017-01-14 01:05:16 UTC
(In reply to Conrad Meyer from comment #4)

I agree with the change, but why did you revert the change now (before ATF/kyua has been fixed/updated)?
Comment 6 commit-hook freebsd_committer 2017-01-15 09:14:10 UTC
A commit references this bug:

Author: ngie
Date: Sun Jan 15 09:13:42 UTC 2017
New revision: 312215
URL: https://svnweb.freebsd.org/changeset/base/312215

Log:
  Mark testcases which use cap_enter as expected failures until the
  PR is resolved so those of us that run the tests don't have the
  bogus failures counted against our overall results

  PR:	215690

Changes:
  head/tests/sys/vfs/lookup_cap_dotdot.c
Comment 7 Walter Schwarzenfeld freebsd_triage 2018-01-14 04:52:27 UTC
Forgotten to close?