Bug 240683

Summary:	failing test case: lib.libarchive.functional_test.test_write_filter_zstd
Product:	Base System	Reporter:	Li-Wen Hsu <lwhsu>
Component:	tests	Assignee:	Baptiste Daroussin <bapt>
Status:	Open ---
Severity:	Affects Only Me	CC:	arichardson, bapt, emaste, mm
Priority:	---
Version:	CURRENT
Hardware:	Any
OS:	Any

Description Li-Wen Hsu freebsd_committer

2019-09-19 12:11:11 UTC

lib.libarchive.functional_test.test_write_filter_zstd starts failing
after r352482 (or r352483):

https://ci.freebsd.org/job/FreeBSD-head-amd64-test/12599/testReport/junit/lib.libarchive/functional_test/test_write_filter_zstd/
https://ci.freebsd.org/job/FreeBSD-head-i386-test/6737/testReport/junit/lib.libarchive/functional_test/test_write_filter_zstd/

485: test_write_filter_zstd
/usr/src/contrib/libarchive/libarchive/test/test_write_filter_zstd.c:145: Assertion failed: used2 < used1
   Description: compression-level=6 wrote 746 bytes, default wrote 744 bytes

Comment 1 commit-hook freebsd_committer

2019-09-19 13:24:10 UTC

A commit references this bug:

Author: lwhsu
Date: Thu Sep 19 13:23:25 UTC 2019
New revision: 352515
URL: https://svnweb.freebsd.org/changeset/base/352515

Log:
  Temporarily add test_write_filter_zstd BROKEN_TESTS as it always fails in CI

  There is no trivial way to mark single libarchive test skip currently so just
  add it to BROKEN_TESTS for now.

  PR:		240683
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/lib/libarchive/tests/Makefile

Comment 2 Baptiste Daroussin freebsd_committer

2019-09-20 07:23:46 UTC

What is weird is yes I can reproduce this bug but only under bhyve, not on regular amd64 machines.

I am adding mm@ to see if he has more insight

Comment 3 Alex Richardson freebsd_committer

2020-09-17 17:07:33 UTC

I can't reproduce this either natively or on CHERI RISC-V. Maybe we should try removing it from BROKEN_TESTS?

Comment 4 Li-Wen Hsu freebsd_committer

2020-09-17 17:15:02 UTC

(In reply to Alex Richardson from comment #3)
I haven't checked this for a while, maybe it was fixed silently. Please check it again in bhyve and feel free to enable it.

I think we need a "try broken" CI job to monitor if the skipped/disabled tests get fixed.

Comment 5 Ed Maste freebsd_committer

2020-09-17 17:38:25 UTC

(In reply to Li-Wen Hsu from comment #4)
> I think we need a "try broken" CI job to monitor if the skipped/disabled tests
> get fixed.

Ideally the failing tests would just be XFAIL and not fail the whole test / generate email, but absent that having a regular job that tries the broken/excluded tests indeed sounds like a good idea.

Comment 6 Alex Richardson freebsd_committer

2020-09-17 17:39:19 UTC

We could check the test case name in functional_test.sh?