Bug 217973

Summary:	devel/kyua: local.kyua.utils.datetime_test.timestamp__subtraction failing non-deterministically after recent upgrade
Product:	Ports & Packages	Reporter:	Enji Cooper <ngie>
Component:	Individual Port(s)	Assignee:	Enji Cooper <ngie>
Status:	Closed FIXED
Severity:	Affects Some People	CC:	brooks, lwhsu, ngie
Priority:	---	Flags:	bugzilla: maintainer-feedback? (jmmv)
Version:	Latest
Hardware:	Any
OS:	Any
URL:	https://reviews.freebsd.org/D10316

Description Enji Cooper freebsd_committer

2017-03-21 09:10:29 UTC

This commit seems to have broken the local.kyua.utils.datetime_test.timestamp__subtraction in a non-deterministic manner, suggesting that the new test code is potentially racy:

------------------------------------------------------------------------
r436312 | brooks | 2017-03-16 16:12:44 -0700 (Thu, 16 Mar 2017) | 10 lines

Hack around wall clock time going backwards.

As reported in https://github.com/jmmv/kyua/issues/155, the wall clock
time can go backwards resulting in an apparent negative delta. As
a workaround, convert such deltas to 1us. This allows tests to run
successfully in MIPS64 qemu.

Approved by:    jmmv (maintainer)
Sponsored by:   DARPA, AFRL

This test has been largely failing on ci.freebsd.org, as evidenced here:

- Failing example: https://ci.freebsd.org/job/FreeBSD-head-amd64-test/2289/testReport/local.kyua.utils/datetime_test/timestamp__subtraction/
- Trend/summary: https://ci.freebsd.org/job/FreeBSD-head-amd64-test/2289/testReport/

Comment 1 Enji Cooper freebsd_committer

2017-04-08 06:21:54 UTC

I've developed a fix for the CR. Sending out a CR soon.

$ sudo kyua report --results-filter passed | grep subtraction
utils/datetime_test:timestamp__subtraction  ->  passed  [0.007s]
$ sudo kyua report --results-filter passed | tail -n 4       
===> Summary
Results read from /root/.kyua/store/results.usr_local_tests_kyua.20160406-215207-780891.db
Test cases: 1413 total, 3 skipped, 0 expected failures, 0 broken, 0 failed
Total time: 39.398s

Comment 2 Li-Wen Hsu freebsd_committer

2017-04-10 17:22:43 UTC

Julio, please check the patch at https://reviews.freebsd.org/D10316 and provide maintainer feedback , thanks!

Comment 3 commit-hook freebsd_committer

2017-04-21 07:38:39 UTC

A commit references this bug:

Author: ngie
Date: Fri Apr 21 07:38:31 UTC 2017
New revision: 439018
URL: https://svnweb.freebsd.org/changeset/ports/439018

Log:
  Fix utils/datetime_test:timestamp__subtraction

  Don't expect a negative time delta to throw an exception--instead, expect
  1 microsecond to be returned per the behavior change made in r436312.

  Approved by:	brooks (as part of a larger diff), (maintainer timeout)
  Differential Revision:	D10316
  PR:		217973
  Reported by:	Jenkins
  Reviewed by:	asomers
  Sponsored by:	Dell EMC Isilon

Changes:
  head/devel/kyua/Makefile
  head/devel/kyua/files/patch-utils_datetime__test.cpp