Bug 217973 - devel/kyua: local.kyua.utils.datetime_test.timestamp__subtraction failing non-deterministically after recent upgrade
Summary: devel/kyua: local.kyua.utils.datetime_test.timestamp__subtraction failing non...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Enji Cooper
URL: https://reviews.freebsd.org/D10316
Keywords:
Depends on:
Blocks:
 
Reported: 2017-03-21 09:10 UTC by Enji Cooper
Modified: 2017-04-21 08:37 UTC (History)
3 users (show)

See Also:
bugzilla: maintainer-feedback? (jmmv)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer 2017-03-21 09:10:29 UTC
This commit seems to have broken the local.kyua.utils.datetime_test.timestamp__subtraction in a non-deterministic manner, suggesting that the new test code is potentially racy:

------------------------------------------------------------------------
r436312 | brooks | 2017-03-16 16:12:44 -0700 (Thu, 16 Mar 2017) | 10 lines

Hack around wall clock time going backwards.

As reported in https://github.com/jmmv/kyua/issues/155, the wall clock
time can go backwards resulting in an apparent negative delta. As
a workaround, convert such deltas to 1us. This allows tests to run
successfully in MIPS64 qemu.

Approved by:    jmmv (maintainer)
Sponsored by:   DARPA, AFRL

This test has been largely failing on ci.freebsd.org, as evidenced here:

- Failing example: https://ci.freebsd.org/job/FreeBSD-head-amd64-test/2289/testReport/local.kyua.utils/datetime_test/timestamp__subtraction/
- Trend/summary: https://ci.freebsd.org/job/FreeBSD-head-amd64-test/2289/testReport/
Comment 1 Enji Cooper freebsd_committer 2017-04-08 06:21:54 UTC
I've developed a fix for the CR. Sending out a CR soon.

$ sudo kyua report --results-filter passed | grep subtraction
utils/datetime_test:timestamp__subtraction  ->  passed  [0.007s]
$ sudo kyua report --results-filter passed | tail -n 4       
===> Summary
Results read from /root/.kyua/store/results.usr_local_tests_kyua.20160406-215207-780891.db
Test cases: 1413 total, 3 skipped, 0 expected failures, 0 broken, 0 failed
Total time: 39.398s
Comment 2 Li-Wen Hsu freebsd_committer 2017-04-10 17:22:43 UTC
Julio, please check the patch at https://reviews.freebsd.org/D10316 and provide maintainer feedback , thanks!
Comment 3 commit-hook freebsd_committer 2017-04-21 07:38:39 UTC
A commit references this bug:

Author: ngie
Date: Fri Apr 21 07:38:31 UTC 2017
New revision: 439018
URL: https://svnweb.freebsd.org/changeset/ports/439018

Log:
  Fix utils/datetime_test:timestamp__subtraction

  Don't expect a negative time delta to throw an exception--instead, expect
  1 microsecond to be returned per the behavior change made in r436312.

  Approved by:	brooks (as part of a larger diff), (maintainer timeout)
  Differential Revision:	D10316
  PR:		217973
  Reported by:	Jenkins
  Reviewed by:	asomers
  Sponsored by:	Dell EMC Isilon

Changes:
  head/devel/kyua/Makefile
  head/devel/kyua/files/patch-utils_datetime__test.cpp