Bug 240683 - failing test case: lib.libarchive.functional_test.test_write_filter_zstd
Summary: failing test case: lib.libarchive.functional_test.test_write_filter_zstd
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: tests (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Baptiste Daroussin
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-19 12:11 UTC by Li-Wen Hsu
Modified: 2020-09-17 17:39 UTC (History)
4 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Li-Wen Hsu freebsd_committer freebsd_triage 2019-09-19 12:11:11 UTC 
lib.libarchive.functional_test.test_write_filter_zstd starts failing
after r352482 (or r352483):

https://ci.freebsd.org/job/FreeBSD-head-amd64-test/12599/testReport/junit/lib.libarchive/functional_test/test_write_filter_zstd/
https://ci.freebsd.org/job/FreeBSD-head-i386-test/6737/testReport/junit/lib.libarchive/functional_test/test_write_filter_zstd/

485: test_write_filter_zstd
/usr/src/contrib/libarchive/libarchive/test/test_write_filter_zstd.c:145: Assertion failed: used2 < used1
   Description: compression-level=6 wrote 746 bytes, default wrote 744 bytes
Comment 1 commit-hook freebsd_committer freebsd_triage 2019-09-19 13:24:10 UTC 
A commit references this bug:

Author: lwhsu
Date: Thu Sep 19 13:23:25 UTC 2019
New revision: 352515
URL: https://svnweb.freebsd.org/changeset/base/352515

Log:
  Temporarily add test_write_filter_zstd BROKEN_TESTS as it always fails in CI

  There is no trivial way to mark single libarchive test skip currently so just
  add it to BROKEN_TESTS for now.

  PR:		240683
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/lib/libarchive/tests/Makefile
Comment 2 Baptiste Daroussin freebsd_committer freebsd_triage 2019-09-20 07:23:46 UTC 
What is weird is yes I can reproduce this bug but only under bhyve, not on regular amd64 machines.

I am adding mm@ to see if he has more insight
Comment 3 Alex Richardson freebsd_committer freebsd_triage 2020-09-17 17:07:33 UTC 
I can't reproduce this either natively or on CHERI RISC-V. Maybe we should try removing it from BROKEN_TESTS?
Comment 4 Li-Wen Hsu freebsd_committer freebsd_triage 2020-09-17 17:15:02 UTC 
(In reply to Alex Richardson from comment #3)
I haven't checked this for a while, maybe it was fixed silently. Please check it again in bhyve and feel free to enable it.

I think we need a "try broken" CI job to monitor if the skipped/disabled tests get fixed.
Comment 5 Ed Maste freebsd_committer freebsd_triage 2020-09-17 17:38:25 UTC 
(In reply to Li-Wen Hsu from comment #4)
> I think we need a "try broken" CI job to monitor if the skipped/disabled tests
> get fixed.

Ideally the failing tests would just be XFAIL and not fail the whole test / generate email, but absent that having a regular job that tries the broken/excluded tests indeed sounds like a good idea.
Comment 6 Alex Richardson freebsd_committer freebsd_triage 2020-09-17 17:39:19 UTC 
We could check the test case name in functional_test.sh?