Bug 191019 - [tests] pgrep/pkill -j testcase failures
Summary: [tests] pgrep/pkill -j testcase failures
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: unspecified
Hardware: Any Any
: --- Affects Many People
Assignee: Enji Cooper
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-13 23:43 UTC by Enji Cooper
Modified: 2015-07-08 19:12 UTC (History)
2 users (show)

See Also:


Attachments
pkill -j test failure log (06/2013) (3.10 KB, text/html)
2014-06-13 23:44 UTC, Enji Cooper
no flags Details
pgrep -j test failure log (06/2013) (2.92 KB, text/html)
2014-06-13 23:45 UTC, Enji Cooper
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer 2014-06-13 23:43:03 UTC
The testcases fail to run successfully via kyua and via prove.

This testcases probably fail because they're using outdated jail commands or because of changes in the security realm (Capsicum, etc), but this requires further investigation.
Comment 1 Enji Cooper freebsd_committer 2014-06-13 23:44:39 UTC
Created attachment 143762 [details]
pkill -j test failure log (06/2013)

From http://kyua3.nyi.freebsd.org/head/data/0-LATEST/results/bin_pkill_pkill-j_test_main.html
Comment 2 Enji Cooper freebsd_committer 2014-06-13 23:45:09 UTC
Created attachment 143763 [details]
pgrep -j test failure log (06/2013)
Comment 3 Enji Cooper freebsd_committer 2014-07-07 22:12:43 UTC
The pgrep issues are test bugs. The assumption that the test script makes is that the forked background process is the same as the value returned by pgrep. pgrep is returning the PID of the process from within the jail:

Script started on Mon Jul  7 15:02:12 2014
command: sh -x ./pgrep-j_test
+ basename ./pgrep-j_test
+ base=pgrep-j_test
+ id -u
+ [ 0 -ne 0 ]
+ echo 1..3
1..3
+ mktemp -d /usr/tests/bin/pkill/pgrep-j_test.XXXXXXX
+ tmpdir=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M
+ cd /usr/tests/bin/pkill/pgrep-j_test.bihmF9M
+ trap 'cd /; jls jid name | grep pgrep-j_test | xargs -n 1 jail -R ; rm -Rf /usr/tests/bin/pkill/pgrep-j_test.bihmF9M' EXIT
+ pwd
+ sleep=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ ln -f /bin/sleep /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ name='pgrep -j <jid>'
+ chpid=63235
+ _jail pgrep-j_test_1-1 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ local name=pgrep-j_test_1-1
+ shift
+ jail -c path=/ host.hostname=pgrep-j_test_1-1 name=pgrep-j_test_1-1 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ chpid2=63236
+ chpid3=63237
+ sleep 0.5
+ _jail pgrep-j_test_1-2 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ local name=pgrep-j_test_1-2
+ shift
+ jail -c path=/ host.hostname=pgrep-j_test_1-2 name=pgrep-j_test_1-2 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ ps auxww
+ grep /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
root  63235   0.0  0.0  18476 1676  0  S+    3:02PM     0:00.00 jail -c path=/ host.hostname=pgrep-j_test_1-1 name=pgrep-j_test_1-1 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63236   0.0  0.0  18476 1676  0  S+    3:02PM     0:00.00 jail -c path=/ host.hostname=pgrep-j_test_1-2 name=pgrep-j_test_1-2 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63237   0.0  0.0   8036 1360  0  S+    3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63239   0.0  0.0   8036 1360  0  S+J   3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63240   0.0  0.0   8036 1360  0  S+J   3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63242   0.0  0.0  18608 1992  0  R+    3:02PM     0:00.00 grep /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ jls jid name
+ awk '$2 == "pgrep-j_test_1-1" {print $1}'
+ jid=136
+ pgrep -f -j 136 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ pid=63239
+ [ 63239 = 63235 ]
+ echo 'not ok 1 - pgrep -j <jid>'
not ok 1 - pgrep -j <jid>
+ sleep 7
+ name='pgrep -j any'
+ chpid=63248
+ _jail pgrep-j_test_2-1 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ local name=pgrep-j_test_2-1
+ shift
+ jail -c path=/ host.hostname=pgrep-j_test_2-1 name=pgrep-j_test_2-1 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ chpid2=63249
+ _jail pgrep-j_test_2-2 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ local name=pgrep-j_test_2-2
+ shift
+ chpid3=63250
+ jail -c path=/ host.hostname=pgrep-j_test_2-2 name=pgrep-j_test_2-2 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ sleep 0.5
+ /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ ps auxww
+ grep /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
root  63248   0.0  0.0  18476 1676  0  S+    3:02PM     0:00.01 jail -c path=/ host.hostname=pgrep-j_test_2-1 name=pgrep-j_test_2-1 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63249   0.0  0.0  18476 1676  0  S+    3:02PM     0:00.00 jail -c path=/ host.hostname=pgrep-j_test_2-2 name=pgrep-j_test_2-2 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63250   0.0  0.0   8036 1360  0  S+    3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63252   0.0  0.0   8036 1360  0  S+J   3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63253   0.0  0.0   8036 1360  0  S+J   3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63255   0.0  0.0  18608 1992  0  R+    3:02PM     0:00.00 grep /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ pgrep -f -j any /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ sort
+ pids='63252
63253'
+ echo 63248
+ echo 63249
+ sort
+ refpids='63248
63249'
+ [ '63252
63253' = '63248
63249' ]
+ echo 'not ok 2 - pgrep -j any'
not ok 2 - pgrep -j any
+ sleep 7
+ name='pgrep -j none'
+ chpid=63263
+ /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ chpid2=63264
+ _jail pgrep-j_test_3 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ local name=pgrep-j_test_3
+ shift
+ jail -c path=/ host.hostname=pgrep-j_test_3 name=pgrep-j_test_3 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
+ sleep 0.5
+ ps auxww
+ grep /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
root  63263   0.0  0.0   8036 1360  0  S+    3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63264   0.0  0.0  18476 1676  0  S+    3:02PM     0:00.00 jail -c path=/ host.hostname=pgrep-j_test_3 name=pgrep-j_test_3 ip4.addr=127.0.0.1 command=/usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63266   0.0  0.0   8036 1360  0  S+J   3:02PM     0:00.00 /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5
root  63268   0.0  0.0  18608 1992  0  R+    3:02PM     0:00.00 grep /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ pgrep -f -j none /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt
+ pid='63264
63263'
+ [ '63264
63263' = 63263 ]
+ echo 'not ok 3 - pgrep -j none'
not ok 3 - pgrep -j none
+ cd /
+ jls jid name
+ grep pgrep-j_test
+ xargs -n 1 jail -R
jail: /usr/tests/bin/pkill/pgrep-j_test.bihmF9M/sleep.txt 5: exited on signal 15
jail: "pgrep-j_test_3" not found
+ rm -Rf /usr/tests/bin/pkill/pgrep-j_test.bihmF9M

Script done on Mon Jul  7 15:02:28 2014

The issue with pkill is valid per the manpage though. In particular the process in the jail is getting killed after it's started up with pkill. It's easy to reproduce (throwing in -f exhibits the problem):

# jls
   JID  IP Address      Hostname                      Path
# jail / foobar 127.0.0.1 sleep 60 &
# jls
   JID  IP Address      Hostname                      Path
    35  127.0.0.1       foobar                        /
# pkill -j none sleep
# jls
   JID  IP Address      Hostname                      Path
    35  127.0.0.1       foobar                        /
# pkill -f -j none sleep
[1]   Terminated              jail / foobar 127.0.0.1 sleep 60
Comment 4 Enji Cooper freebsd_committer 2014-07-07 23:20:19 UTC
Adding more data to the pgrep failures, it's jail(8) that does the forking:

575         pid = fork();
576         if (pid < 0)
577                 err(1, "fork");
578         if (pid > 0) {
579                 if (bg || !add_proc(j, pid)) {
580                         free(j->comline);
581                         j->comline = NULL;
582                         return 0;
583                 } else {
584                         paralimit--;
585                         return 1;
586                 }
587         }
588         if (bg)
589                 setsid();

I fixed the issues with the pgrep -j testcases here:

https://github.com/yaneurabeya/freebsd/commit/95191312475104e2e7afd7cdec1452a031500d33

I'm starting to wonder if this forking in jail(8) is at the cruxt of the issue with "none" as the pkill testcases work by killing the jail process, not the sleep process running in the jail.
Comment 5 Enji Cooper freebsd_committer 2014-07-08 00:15:17 UTC
Aha! The commands are missing the -x option:

     -x          Require an exact match of the process name, or argument list
                 if -f is given.  The default is to match any substring.

With some more work, now the testcases pass 100%: https://github.com/yaneurabeya/freebsd/commit/7091f8f4e13171a7f122ead5a63a54c9d0a0de30

Script started on Mon Jul  7 17:06:51 2014
command: sh -c cd /usr/tests/bin/pkill/ ; kyua test; prove -rv p*
pgrep-F_test:main  ->  passed  [0.345s]^M
pgrep-LF_test:main  ->  passed  [0.693s]^M
pgrep-P_test:main  ->  passed  [0.354s]^M
pgrep-U_test:main  ->  passed  [0.692s]^M
pgrep-_g_test:main  ->  passed  [0.679s]^M
pgrep-_s_test:main  ->  passed  [0.037s]^M
pgrep-g_test:main  ->  passed  [0.691s]^M
pgrep-i_test:main  ->  passed  [0.349s]^M
pgrep-j_test:main  ->  passed  [24.439s]^M
pgrep-l_test:main  ->  passed  [0.322s]^M
pgrep-n_test:main  ->  passed  [0.341s]^M
pgrep-o_test:main  ->  passed  [0.346s]^M
pgrep-q_test:main  ->  passed  [0.353s]^M
pgrep-s_test:main  ->  passed  [0.667s]^M
pgrep-t_test:main  ->  passed  [0.352s]^M
pgrep-v_test:main  ->  passed  [0.357s]^M
pgrep-x_test:main  ->  passed  [0.360s]^M
pkill-F_test:main  ->  passed  [0.336s]^M
pkill-LF_test:main  ->  passed  [0.684s]^M
pkill-P_test:main  ->  passed  [0.346s]^M
pkill-U_test:main  ->  passed  [0.695s]^M
pkill-_g_test:main  ->  passed  [0.687s]^M
pkill-g_test:main  ->  passed  [0.681s]^M
pkill-i_test:main  ->  passed  [0.349s]^M
pkill-j_test:main  ->  passed  [19.325s]^M
pkill-s_test:main  ->  passed  [0.679s]^M
pkill-t_test:main  ->  passed  [0.680s]^M
pkill-x_test:main  ->  passed  [0.690s]^M
^M
28/28 passed (0 failed)^M
Committed action 270^M

...

pgrep-v_test ... ^M
1..2^[[0m^M
ok 1 - pgrep -v^[[0m^M
ok 2 - pgrep -v^[[0m^M
ok^M
pgrep-x_test ... ^M
1..4^[[0m^M
ok 1 - pgrep -x^[[0m^M
ok 2 - pgrep -x^[[0m^M
ok 3 - pgrep -x -f^[[0m^M
ok 4 - pgrep -x -f^[[0m^M
ok^M
pkill-F_test ... ^M
1..1^[[0m^M
ok - pkill -F <pidfile>^[[0m^M
ok^M
pkill-LF_test .. ^M
1..2^[[0m^M
ok 1 - pkill -LF <pidfile>^[[0m^M
ok 2 - pkill -LF <pidfile>^[[0m^M
ok^M
pkill-P_test ... ^M
1..1^[[0m^M
ok - pkill -P <ppid>^[[0m^M
ok^M
pkill-U_test ... ^M
1..2^[[0m^M
ok 1 - pkill -U <uid>^[[0m^M
ok 2 - pkill -U <user>^[[0m^M
ok^M
pkill-_g_test .. ^M
1..2^[[0m^M
ok 1 - pkill -G <gid>^[[0m^M
ok 2 - pkill -G <group>^[[0m^M
ok^M
pkill-g_test ... ^M
1..2^[[0m^M
ok 1 - pkill -g <pgrp>^[[0m^M
ok 2 - pkill -g 0^[[0m^M
ok^M
pkill-i_test ... ^M
1..1^[[0m^M
ok - pkill -i^[[0m^M
ok^M
pkill-j_test ... ^M
1..3^[[0m^M
jail: /usr/tests/bin/pkill/pkill-j_test.ujLT6PV/sleep.txt 5: exited on signal 15^M
kill: 10901: No such process^M
ok 1 - pkill -j <jid>^[[0m^M
jail: /usr/tests/bin/pkill/pkill-j_test.ujLT6PV/sleep.txt 5: exited on signal 15^M
jail: /usr/tests/bin/pkill/pkill-j_test.ujLT6PV/sleep.txt 5: exited on signal 15^M
kill: 10913: No such process^M
kill: 10914: No such process^M
ok 2 - pkill -j any^[[0m^M
ok 3 - pkill -j none^[[0m^M
kill: 10922: No such process^M
jail: "pkill-j_test_1-2" not found^M
jail: "pkill-j_test_3" not found^M
ok^M
pkill-s_test ... ^M
1..2^[[0m^M
ok 1 - pkill -s <sid>^[[0m^M
ok 2 - pkill -s 0^[[0m^M
ok^M
pkill-t_test ... ^M
1..2^[[0m^M
ok 1 - pkill -t <tty>^[[0m^M
ok 2 - pkill -t <tty>^[[0m^M
ok^M
pkill-x_test ... ^M
1..4^[[0m^M
ok 1 - pkill -x^[[0m^M
ok 2 - pkill -x^[[0m^M
ok 3 - pkill -x -f^[[0m^M
ok 4 - pkill -x -f^[[0m^M
ok^M
^[[32mAll tests successful.^M
^[[0mFiles=28, Tests=55, 56 wallclock secs ( 0.18 usr  0.14 sys +  0.18 cusr  0.91 csys =  1.41 CPU)^M
Result: PASS^M

Script done on Mon Jul  7 17:08:44 2014
Comment 6 Enji Cooper freebsd_committer 2014-07-15 00:50:25 UTC
I have fixes for the testcases available here for review: https://github.com/yaneurabeya/freebsd/pull/9 . The patch can be pulled via https://github.com/yaneurabeya/freebsd/pull/9.patch .
Comment 7 Enji Cooper freebsd_committer 2014-08-02 19:11:21 UTC
I can't reproduce this issue anymore, so my guess is that there was a bug in the base system that was causing the testcase to fail. This should be tested on stable/10..
Comment 9 Enji Cooper freebsd_committer 2015-02-21 23:29:54 UTC
I fixed them for a while (r278618, r278633, r278653, r278742, r278776), then they started failing again after jamie's hacking around with jail(8) :(...:

- https://svnweb.freebsd.org/changeset/base/279083
- https://svnweb.freebsd.org/changeset/base/279081
Comment 10 commit-hook freebsd_committer 2015-02-21 23:40:44 UTC
A commit references this bug:

Author: ngie
Date: Sat Feb 21 23:39:44 UTC 2015
New revision: 279121
URL: https://svnweb.freebsd.org/changeset/base/279121

Log:
  Disguising stderr from jls generally considered harmful

  Undisguising it dumps out the following logspew:

  jls: unknown parameter: allow

  PR: 191019

Changes:
  head/bin/pkill/tests/pgrep-j_test.sh
  head/bin/pkill/tests/pkill-j_test.sh
Comment 11 Enji Cooper freebsd_committer 2015-02-21 23:45:27 UTC
Ok, this time it's not the tests fault. There was a regression introduced in r279083 where dotted parameters are not getting parsed properly; it's spewing out loads of warnings about "jls: unknown parameter: allow". Closing this bug and will open another.
Comment 12 Glen Barber freebsd_committer 2015-07-08 18:32:05 UTC
To originators/assignees of this PR:

A commit to the tree references this PR, however the PR is still in a non-closed state.

Please review this PR and close as appropriate, or if closing the PR requires a merge to stable/10, please let re@ know as soon as possible.

Thank you.

Glen
Comment 13 Enji Cooper freebsd_committer 2015-07-08 19:12:27 UTC
This is largely fixed now. Will open a new bug if new issues occur with these testcases.