Bug 220307 - libxo tests are failing after recent update
Summary: libxo tests are failing after recent update
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: tests (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Many People
Assignee: Phil Shafer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-06-27 17:13 UTC by Enji Cooper
Modified: 2017-07-06 16:30 UTC (History)
2 users (show)

See Also:
ngie: mfc-stable11?
ngie: mfc-stable10-
ngie: mfc-stable9-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer 2017-06-27 17:13:22 UTC
From https://ci.freebsd.org/job/FreeBSD-head-amd64-test/3297/testReport/lib.libxo/functional_test/test_01__E/ :

Fail: stderr does not match golden output
--- /usr/tests/lib/libxo/test_01.E.err	2017-06-27 15:50:56.000000000 +0000
+++ /tmp/kyua.Pf9y2q/5124/work/check.F3sCo2/stderr	2017-06-27 16:28:02.621645000 +0000
@@ -0,0 +1 @@
+test_01: unknown simple option: encoder
Comment 1 Enji Cooper freebsd_committer 2017-06-27 17:26:20 UTC
There's a fair amount that needs to be fixed here from the looks of it. For example:

$ kyua debug -k /usr/tests/lib/libxo/Kyuafile functional_test:test_11__J
Executing command [ env LC_ALL=en_US.UTF-8 LIBXO_OPTIONS=:WJ TZ=EST /usr/tests/lib/libxo/test_11 ]
Fail: stdout does not match golden output
--- /usr/tests/lib/libxo/test_11.J.out  2017-06-27 17:25:06.611615000 +0000
+++ /tmp/kyua.23RGn1/2/work/check.IXSf4A/stdout 2017-06-27 17:25:11.156370000 +0000
@@ -14,5 +14,3 @@
 {{test-program: }}
 {{An application 1011 log entry}}
 
-{"__version": "3.1.4", "top": {}
-}
functional_test:test_11__J  ->  failed: atf-check failed; see the output of the test for details
Comment 2 Phil Shafer freebsd_committer 2017-06-27 20:34:18 UTC
I changed the allowable set of options available via LIBXO_OPTIONS, since one doesn't want this value interfering with sub-processes, etc.  The native libxo "make test" jig was updated but I didn't think to fix the ATF test jig.  Will do.

Thanks,
 Phil
Comment 3 Enji Cooper freebsd_committer 2017-07-06 06:49:35 UTC
(In reply to Phil Shafer from comment #2)

This bug still has issues that need to be resolved. I applied this patch for starters, which fixed a number of items:

====================================================================================

$ svn diff usr.bin/xo
Index: usr.bin/xo/tests/functional_test.sh
===================================================================
--- usr.bin/xo/tests/functional_test.sh (revision 320663)
+++ usr.bin/xo/tests/functional_test.sh (working copy)
@@ -42,8 +42,8 @@
 
        atf_check -s exit:0 -e file:${err_file} -o file:${out_file} \
            env LC_ALL=en_US.UTF-8 \
-               LIBXO_OPTIONS=":W${xo_fmt}" TZ="EST" "${SRCDIR}/${tc}" \
-               ${XO}
+               TZ="EST" "${SRCDIR}/${tc}" \
+               "${XO} -W${xo_fmt}"
 }
 
 add_testcase()

After that, I ran into these issues:

$ (cd usr.bin/xo/tests/; make; sudo make install; make check)
install  -o root  -g wheel -m 555  xo_01  /usr/tests/usr.bin/xo/xo_01
install  -o root  -g wheel -m 555  functional_test  /usr/tests/usr.bin/xo/functional_test
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/usr.bin/xo/Kyuafile
install  -o root -g wheel  -m 444 /usr/src/contrib/libxo/tests/xo/saved/xo_01.H.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.H.out /usr/src/contrib/libxo/tests/xo/saved/xo_01.HIPx.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.HIPx.out /usr/src/contrib/libxo/tests/xo/saved/xo_01.HP.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.HP.out /usr/src/contrib/libxo/tests/xo/saved/xo_01.J.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.J.out /usr/src/contrib/libxo/tests/xo/saved/xo_01.JP.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.JP.out /usr/src/contrib/libxo/tests/xo/saved/xo_01.T.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.T.out /usr/src/contrib/libxo/tests/xo/saved/xo_01.X.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.X.out /usr/src/contrib/libxo/tests/xo/saved/xo_01.XP.err /usr/src/contrib/libxo/tests/xo/saved/xo_01.XP.out /usr/tests/usr.bin/xo/
functional_test:xo_01__H  ->  passed  [0.029s]
functional_test:xo_01__HIPx  ->  failed: atf-check failed; see the output of the test for details  [0.039s]
functional_test:xo_01__HP  ->  failed: atf-check failed; see the output of the test for details  [0.040s]
functional_test:xo_01__J  ->  passed  [0.030s]
functional_test:xo_01__JP  ->  failed: atf-check failed; see the output of the test for details  [0.037s]
functional_test:xo_01__T  ->  passed  [0.029s]
functional_test:xo_01__X  ->  passed  [0.030s]
functional_test:xo_01__XP  ->  failed: atf-check failed; see the output of the test for details  [0.038s]

Results file id is usr_tests_usr.bin_xo.20170705-193650-601725
Results saved to /home/ngie/.kyua/store/results.usr_tests_usr.bin_xo.20170705-193650-601725.db

4/8 passed (4 failed)
*** Error code 1

Stop.
make: stopped in /usr/src/usr.bin/xo/tests

====================================================================================

It seems that -I and -P are no longer supported, which matches what's noted in xo(1).
Comment 4 Enji Cooper freebsd_committer 2017-07-06 08:07:13 UTC
------------------------------------------------------------------------
r320728 | ngie | 2017-07-06 01:05:57 -0700 (Thu, 06 Jul 2017) | 6 lines

Chase upstream change deprecating LIBXO_OPTIONS similar to r320521

LIBXO_OPTIONS was deprecated in favor of the --libxo options.

Inspired by:    https://github.com/Juniper/libxo (02272d5c6d7d)
------------------------------------------------------------------------
r320521 | phil | 2017-06-30 16:53:40 -0700 (Fri, 30 Jun 2017) | 5 lines

Fix functional_test.sh to use --libxo options instead of the deprecated
LIBXO_OPTIONS environment variable.

Submitted by:   phil

------------------------------------------------------------------------
Comment 5 Enji Cooper freebsd_committer 2017-07-06 08:14:38 UTC
It doesn't look like contrib/libxo is going to be MFCed to ^/stable/11 per lack of MFC markers in r319694, r319731. Closing bug.
Comment 6 Phil Shafer freebsd_committer 2017-07-06 16:30:14 UTC
Apologies.  I've been stuck hitting a COMPILER_TYPE issue with
"installworld" and hadn't been able to verify my fix.

Should I MFC the latest libxo back into stable/11?

Thanks,
 Phil