Bug 211875 - DTrace test failures on ^/head@r304091
Summary: DTrace test failures on ^/head@r304091
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: tests (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-testing (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-15 17:50 UTC by Enji Cooper
Modified: 2019-05-23 22:18 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer 2016-08-15 17:50:47 UTC
Running into the following failures with my amd64 VM host running the DTrace tests.

I'll hopefully get more data for the failures sometime in the next couple days.

$ uname -a
FreeBSD fbsd11 12.0-CURRENT FreeBSD 12.0-CURRENT #5 r304091:304099: Sun Aug 14 13:08:45 PDT 2016     ngie@fbsd11:/usr/obj/usr/src/svn/sys/GENERIC  amd64
$ sysctl hw.physmem
hw.physmem: 3187220480
$ sysctl kern.smp.cpus
kern.smp.cpus: 3

===> Broken tests
cddl/usr.sbin/dtrace/common/probes/t_dtrace_contrib:tst_probestar_d  ->  broken: Test case body timed out  [300.013s]
...
===> Failed tests
cddl/usr.sbin/dtrace/common/dtraceUtil/t_dtrace_contrib:tst_DataModel32_d_ksh  ->  failed: Test case was expecting a failure but none were raised  [0.108s]
cddl/usr.sbin/dtrace/common/misc/t_dtrace_contrib:tst_macroglob_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.528s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:err_D_PDESC_ZERO_badlib_d  ->  failed: atf-check failed; see the output of the test for details  [0.148s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:err_D_PROC_FUNC_badfunc_d  ->  failed: atf-check failed; see the output of the test for details  [0.138s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:err_D_PROC_LIB_libdash_d  ->  failed: atf-check failed; see the output of the test for details  [0.142s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:err_D_PROC_NAME_alldash_d  ->  failed: atf-check failed; see the output of the test for details  [0.168s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:err_D_PROC_NAME_badname_d  ->  failed: atf-check failed; see the output of the test for details  [0.143s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:err_D_PROC_NAME_globdash_d  ->  failed: atf-check failed; see the output of the test for details  [0.144s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:err_D_PROC_OFF_toobig_d  ->  failed: atf-check failed; see the output of the test for details  [0.146s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:tst_addprobes_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.137s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:tst_args1_d  ->  failed: atf-check failed; see the output of the test for details  [0.192s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:tst_coverage_d  ->  failed: atf-check failed; see the output of the test for details  [0.144s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:tst_emptystack_d  ->  failed: atf-check failed; see the output of the test for details  [0.142s]
cddl/usr.sbin/dtrace/common/pid/t_dtrace_contrib:tst_provregex3_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.187s]
cddl/usr.sbin/dtrace/common/pragma/t_dtrace_contrib:err_invalidlibdep_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.168s]
cddl/usr.sbin/dtrace/common/privs/t_dtrace_contrib:tst_kpriv_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.190s]
cddl/usr.sbin/dtrace/common/types/t_dtrace_contrib:tst_complex_d  ->  failed: atf-check failed; see the output of the test for details  [0.158s]
cddl/usr.sbin/dtrace/common/uctf/t_dtrace_contrib:err_user64mode_ksh  ->  failed: atf-check failed; see the output of the test for details  [1.194s]
cddl/usr.sbin/dtrace/common/uctf/t_dtrace_contrib:tst_pidprint_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.147s]
cddl/usr.sbin/dtrace/common/uctf/t_dtrace_contrib:tst_pidprinttarg_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.140s]
cddl/usr.sbin/dtrace/common/uctf/t_dtrace_contrib:tst_userstrings_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.146s]
cddl/usr.sbin/dtrace/common/usdt/t_dtrace_contrib:tst_eliminate_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.291s]
cddl/usr.sbin/dtrace/common/usdt/t_dtrace_contrib:tst_enabled2_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.148s]
cddl/usr.sbin/dtrace/common/usdt/t_dtrace_contrib:tst_fork_ksh  ->  failed: atf-check failed; see the output of the test for details  [1.376s]
cddl/usr.sbin/dtrace/common/usdt/t_dtrace_contrib:tst_include_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.053s]
cddl/usr.sbin/dtrace/common/ustack/t_dtrace_contrib:tst_spin_ksh  ->  failed: atf-check failed; see the output of the test for details  [0.114s]

FWIW, it looks like tst_probestar_d fails intermittently (its completion time is close to the 300 second time limit):

$ sudo kyua debug -k cddl/usr.sbin/dtrace/common/probes/Kyuafile t_dtrace_contrib:tst_probestar_d
Executing command [ /usr/tests/cddl/usr.sbin/dtrace/common/probes/../../dtest /usr/tests/cddl/usr.sbin/dtrace/common/probes/tst.probestar.d ]
t_dtrace_contrib:tst_probestar_d  ->  passed
$ sudo kyua test -k cddl/usr.sbin/dtrace/common/probes/Kyuafile t_dtrace_contrib:tst_probestar_d                                                                                  
t_dtrace_contrib:tst_probestar_d  ->  
load: 0.11  cmd: kyua 17221 [wait] 221.39r 0.00u 0.00s 0% 6656k
passed  [267.179s]

Results file id is usr_tests_cddl_usr.sbin_dtrace_common_probes.20160815-174507-149272
Results saved to /root/.kyua/store/results.usr_tests_cddl_usr.sbin_dtrace_common_probes.20160815-174507-149272.db

1/1 passed (0 failed)
Comment 1 Li-Wen Hsu freebsd_committer 2019-01-21 22:34:40 UTC
We have a continuously running CI job for DTrace tests at https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/

And some of the failures in this tickets have been fixed.  Do you think it's reasonable to close this ticket and file a ticket for each failing test case?
Comment 2 Mark Johnston freebsd_committer 2019-01-22 02:04:06 UTC
Indeed, many tests have been fixed over the past two years.  I think it'd be useful to have individual PRs for failing DTrace tests.
Comment 3 Li-Wen Hsu freebsd_committer 2019-05-23 22:18:57 UTC
https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/ is usually green now and there bug237641 to tracking one flaky test case.