Created attachment 143764 [details] Grabbed from http://kyua3.nyi.freebsd.org/head/data/0-LATEST/results/usr.bin_yacc_legacy_test_main.html on 06/13/2014 The byacc testcases fail today because the copies checked in under usr.bin/yacc/tests are not in synch with the yacc code. The testcases from upstream need to be added/integrated into contrib/yacc .
"into contrib/yacc" -> "from contrib/yacc". Please note that this isn't a small task because the testcases are very ad hoc, use a special "test harness", and doesn't integrate into any existing test harnesses supported by kyua. Just for context: On Mon, May 12, 2014 at 08:01:02PM -0700, Garrett Cooper wrote: > > > On May 12, 2014, at 16:32, Thomas Dickey <dickey { SPAMFREE AT SPAMFREE } his.com> wrote: > > > >> On Mon, May 12, 2014 at 02:50:33PM -0700, Garrett Cooper wrote: > >> Hi Thomas! > >> I'm looking at integrating your byacc tests into the FreeBSD test > >> suite and I'd like to separate out the test driver code from the > >> testcases content and I was wondering if I could get access to the > >> sources so I could adjust the test code and contribute them back to > >> you. > > > > sorry - my archives are all on local machines - I can work with diff's... > > Ok. I'll do some local dev on the last byacc snapshot and send you the diff when I'm done. > > Thanks :)!! no problem. The point of the expanded tests was to help ensure that I didn't break byacc while integrating the changes to allow building a btyacc lookalike. (The two are different - Debian maintainer and I were discussing that). -- Thomas E. Dickey <dickey { SPAMFREE AT SPAMFREE } invisible-island.net> http://invisible-island.net ftp://invisible-island.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAlNx1lgACgkQcCNT4PfkjttmJwCgx2HPX+DE2KzxQGKfHYsy0/M9 6QoAn0oUK1tPx2PDs828adb3yfQkjWzk =TmkH -----END PGP SIGNATURE-----
All of the yacc testcases are failing on CURRENT :( (with both prove and kyua). I'll work on integrating the testcases in with plain.test.mk.
Thomas Dickey fixed the testcases in upstream in byacc (I verified the testcases passed with the byacc-20140715 snapshot). I've integrated the newer testcases in via https://github.com/yaneurabeya/freebsd/commit/42aeba65c4cea60fb77cbab5b96a92c69af89d17 and https://github.com/yaneurabeya/freebsd/commit/6f5366f7d11f08cba987dac7c8c7b838aa1283bc, and will post a more polished github pull request for review and to provide a patch for the work I've done to integrate in the new code. Once a newer version of the code is available with the fixed testcases, then the failing testcases should be pulled in and the XXX commented bits in yacc_tests.sh should be re-enabled . yacc_tests.sh should setup an expected environment to get byacc to work as expected, which includes generating a config.h file and hardlinking yacc to the directory below the test directory (aka PROG_DIR). This will make some of the build hacks I added to usr.bin/yacc/tests/Makefile and usr.bin/yacc/tests/yacc_tests.sh unnecessary.
The proposed changes can be viewed here: https://github.com/yaneurabeya/freebsd/pull/10 / https://github.com/yaneurabeya/freebsd/pull/10.diff . The byacc snapshot code should be updated to the latest snapshot/disabled wherever necessary to obtain a green run as some of the testcases cause yacc to core [on the version we're using], and a number of the testcases fail in unexpected ways according to the test output.
Thank you very much, I'll work on integrating this patch asap
(In reply to Baptiste Daroussin from comment #5) > Thank you very much, I'll work on integrating this patch asap No problem! The testcases will need massaging with the snapshot that's in CURRENT (mostly disabling/backporting the fixed output from later versions of byacc), but it should work once the latest snapshot is brought in.
Committed thank you
A commit references this bug: Author: bapt Date: Thu Jul 17 22:22:56 UTC 2014 New revision: 268811 URL: http://svnweb.freebsd.org/changeset/base/268811 Log: Update to byacc 20140715 (only concerns regression tests being fixed) Directly use regression test from upstream PR: 191020 Submitted by: gcooper (yaneurabeya@gmail.com) Changes: _U head/contrib/byacc/ head/contrib/byacc/CHANGES head/contrib/byacc/MANIFEST head/contrib/byacc/NOTES-btyacc-Changes head/contrib/byacc/NOTES-btyacc-Disposition head/contrib/byacc/VERSION head/contrib/byacc/aclocal.m4 head/contrib/byacc/main.c head/contrib/byacc/package/byacc.spec head/contrib/byacc/package/debian/changelog head/contrib/byacc/package/mingw-byacc.spec head/contrib/byacc/package/pkgsrc/Makefile head/contrib/byacc/test/btyacc/big_b.output head/contrib/byacc/test/btyacc/big_l.output head/contrib/byacc/test/btyacc/err_inherit1.error head/contrib/byacc/test/btyacc/err_inherit2.error head/contrib/byacc/test/btyacc/err_inherit3.error head/contrib/byacc/test/btyacc/err_inherit4.error head/contrib/byacc/test/btyacc/err_inherit5.error head/contrib/byacc/test/btyacc/err_syntax1.error head/contrib/byacc/test/btyacc/err_syntax10.error head/contrib/byacc/test/btyacc/err_syntax11.error head/contrib/byacc/test/btyacc/err_syntax12.error head/contrib/byacc/test/btyacc/err_syntax13.error head/contrib/byacc/test/btyacc/err_syntax14.error head/contrib/byacc/test/btyacc/err_syntax15.error head/contrib/byacc/test/btyacc/err_syntax16.error head/contrib/byacc/test/btyacc/err_syntax17.error head/contrib/byacc/test/btyacc/err_syntax18.error head/contrib/byacc/test/btyacc/err_syntax19.error head/contrib/byacc/test/btyacc/err_syntax2.error head/contrib/byacc/test/btyacc/err_syntax21.error head/contrib/byacc/test/btyacc/err_syntax22.error head/contrib/byacc/test/btyacc/err_syntax23.error head/contrib/byacc/test/btyacc/err_syntax24.error head/contrib/byacc/test/btyacc/err_syntax25.error head/contrib/byacc/test/btyacc/err_syntax26.error head/contrib/byacc/test/btyacc/err_syntax27.error head/contrib/byacc/test/btyacc/err_syntax3.error head/contrib/byacc/test/btyacc/err_syntax4.error head/contrib/byacc/test/btyacc/err_syntax5.error head/contrib/byacc/test/btyacc/err_syntax6.error head/contrib/byacc/test/btyacc/err_syntax7.error head/contrib/byacc/test/btyacc/err_syntax7a.error head/contrib/byacc/test/btyacc/err_syntax7b.error head/contrib/byacc/test/btyacc/err_syntax8.error head/contrib/byacc/test/btyacc/err_syntax8a.error head/contrib/byacc/test/btyacc/err_syntax9.error head/contrib/byacc/test/btyacc/help.output head/contrib/byacc/test/btyacc/no_b_opt.output head/contrib/byacc/test/btyacc/no_output2.output head/contrib/byacc/test/btyacc/no_p_opt.output head/contrib/byacc/test/btyacc/nostdin.output head/contrib/byacc/test/run_test.sh head/contrib/byacc/test/yacc/big_b.output head/contrib/byacc/test/yacc/big_l.output head/contrib/byacc/test/yacc/err_syntax1.error head/contrib/byacc/test/yacc/err_syntax10.error head/contrib/byacc/test/yacc/err_syntax11.error head/contrib/byacc/test/yacc/err_syntax12.error head/contrib/byacc/test/yacc/err_syntax13.error head/contrib/byacc/test/yacc/err_syntax14.error head/contrib/byacc/test/yacc/err_syntax15.error head/contrib/byacc/test/yacc/err_syntax16.error head/contrib/byacc/test/yacc/err_syntax17.error head/contrib/byacc/test/yacc/err_syntax18.error head/contrib/byacc/test/yacc/err_syntax19.error head/contrib/byacc/test/yacc/err_syntax2.error head/contrib/byacc/test/yacc/err_syntax21.error head/contrib/byacc/test/yacc/err_syntax22.error head/contrib/byacc/test/yacc/err_syntax23.error head/contrib/byacc/test/yacc/err_syntax24.error head/contrib/byacc/test/yacc/err_syntax25.error head/contrib/byacc/test/yacc/err_syntax26.error head/contrib/byacc/test/yacc/err_syntax27.error head/contrib/byacc/test/yacc/err_syntax3.error head/contrib/byacc/test/yacc/err_syntax4.error head/contrib/byacc/test/yacc/err_syntax5.error head/contrib/byacc/test/yacc/err_syntax6.error head/contrib/byacc/test/yacc/err_syntax7.error head/contrib/byacc/test/yacc/err_syntax7a.error head/contrib/byacc/test/yacc/err_syntax7b.error head/contrib/byacc/test/yacc/err_syntax8.error head/contrib/byacc/test/yacc/err_syntax8a.error head/contrib/byacc/test/yacc/err_syntax9.error head/contrib/byacc/test/yacc/help.output head/contrib/byacc/test/yacc/no_b_opt.output head/contrib/byacc/test/yacc/no_output2.output head/contrib/byacc/test/yacc/no_p_opt.output head/contrib/byacc/test/yacc/nostdin.output head/usr.bin/yacc/tests/Makefile head/usr.bin/yacc/tests/calc.y head/usr.bin/yacc/tests/calc1.y head/usr.bin/yacc/tests/calc2.y head/usr.bin/yacc/tests/calc3.y head/usr.bin/yacc/tests/code_calc.y head/usr.bin/yacc/tests/code_error.y head/usr.bin/yacc/tests/error.y head/usr.bin/yacc/tests/ftp.y head/usr.bin/yacc/tests/grammar.y head/usr.bin/yacc/tests/legacy_test.sh head/usr.bin/yacc/tests/pure_calc.y head/usr.bin/yacc/tests/pure_error.y head/usr.bin/yacc/tests/quote_calc.y head/usr.bin/yacc/tests/quote_calc2.y head/usr.bin/yacc/tests/quote_calc3.y head/usr.bin/yacc/tests/quote_calc4.y head/usr.bin/yacc/tests/regress.00.out head/usr.bin/yacc/tests/regress.01.out head/usr.bin/yacc/tests/regress.02.out head/usr.bin/yacc/tests/regress.03.out head/usr.bin/yacc/tests/regress.04.out head/usr.bin/yacc/tests/regress.05.out head/usr.bin/yacc/tests/regress.06.out head/usr.bin/yacc/tests/regress.07.out head/usr.bin/yacc/tests/regress.08.out head/usr.bin/yacc/tests/regress.09.out head/usr.bin/yacc/tests/regress.10.out head/usr.bin/yacc/tests/regress.11.out head/usr.bin/yacc/tests/regress.12.out head/usr.bin/yacc/tests/regress.13.out head/usr.bin/yacc/tests/regress.14.out head/usr.bin/yacc/tests/regress.sh head/usr.bin/yacc/tests/undefined.y head/usr.bin/yacc/tests/yacc_tests.sh
I made a few more adjustments to my test driver and installed files so it completely integrates into Kyua and works with the latest snapshot of byacc: https://github.com/yaneurabeya/freebsd/pull/10.diff . This is the latest and final diff for integrating the current byacc snapshot's test code into FreeBSD CURRENT. Thank you :)!
A commit references this bug: Author: ngie Date: Tue Aug 12 17:51:26 UTC 2014 New revision: 269884 URL: http://svnweb.freebsd.org/changeset/base/269884 Log: Complete the usr.bin/yacc kyua integration work I originally submitted via r268811 - Install the Kyuafile by adding FILES to FILESGROUPS - Run the testcases with an unprivileged user Some of the testcases depend upon behavior that's broken when run as root on FreeBSD because of how permissions are treated with access(2) vs eaccess(2), open(2), etc - Simplify the test driver to just inspect the exit code from run_test because it now exits with 0 if successful and exits with !0 if unsuccessful - Don't do ad hoc temporary directory creation/deletion; let Kyua handle that - Add entries for files removed in r268811 to OptionalObsoleteFiles.inc PR: 191020 X-MFC with: r268811 Approved by: jmmv (mentor) Reviewed by: bapt Sponsored by: EMC / Isilon Storage Division Changes: head/tools/build/mk/OptionalObsoleteFiles.inc head/usr.bin/yacc/tests/Makefile head/usr.bin/yacc/tests/yacc_tests.sh
A commit references this bug: Author: rodrigc Date: Sat Oct 11 19:18:25 UTC 2014 New revision: 272953 URL: https://svnweb.freebsd.org/changeset/base/272953 Log: Merge: r268811 Update to byacc 20140715 PR: 191020 Submitted by: gcooper (yaneurabeya@gmail.com) Changes: _U stable/10/ stable/10/contrib/byacc/CHANGES stable/10/contrib/byacc/MANIFEST stable/10/contrib/byacc/NOTES-btyacc-Changes stable/10/contrib/byacc/NOTES-btyacc-Disposition stable/10/contrib/byacc/VERSION stable/10/contrib/byacc/aclocal.m4 stable/10/contrib/byacc/main.c stable/10/contrib/byacc/package/byacc.spec stable/10/contrib/byacc/package/debian/changelog stable/10/contrib/byacc/package/mingw-byacc.spec stable/10/contrib/byacc/package/pkgsrc/Makefile stable/10/contrib/byacc/test/btyacc/big_b.output stable/10/contrib/byacc/test/btyacc/big_l.output stable/10/contrib/byacc/test/btyacc/err_inherit1.error stable/10/contrib/byacc/test/btyacc/err_inherit2.error stable/10/contrib/byacc/test/btyacc/err_inherit3.error stable/10/contrib/byacc/test/btyacc/err_inherit4.error stable/10/contrib/byacc/test/btyacc/err_inherit5.error stable/10/contrib/byacc/test/btyacc/err_syntax1.error stable/10/contrib/byacc/test/btyacc/err_syntax10.error stable/10/contrib/byacc/test/btyacc/err_syntax11.error stable/10/contrib/byacc/test/btyacc/err_syntax12.error stable/10/contrib/byacc/test/btyacc/err_syntax13.error stable/10/contrib/byacc/test/btyacc/err_syntax14.error stable/10/contrib/byacc/test/btyacc/err_syntax15.error stable/10/contrib/byacc/test/btyacc/err_syntax16.error stable/10/contrib/byacc/test/btyacc/err_syntax17.error stable/10/contrib/byacc/test/btyacc/err_syntax18.error stable/10/contrib/byacc/test/btyacc/err_syntax19.error stable/10/contrib/byacc/test/btyacc/err_syntax2.error stable/10/contrib/byacc/test/btyacc/err_syntax21.error stable/10/contrib/byacc/test/btyacc/err_syntax22.error stable/10/contrib/byacc/test/btyacc/err_syntax23.error stable/10/contrib/byacc/test/btyacc/err_syntax24.error stable/10/contrib/byacc/test/btyacc/err_syntax25.error stable/10/contrib/byacc/test/btyacc/err_syntax26.error stable/10/contrib/byacc/test/btyacc/err_syntax27.error stable/10/contrib/byacc/test/btyacc/err_syntax3.error stable/10/contrib/byacc/test/btyacc/err_syntax4.error stable/10/contrib/byacc/test/btyacc/err_syntax5.error stable/10/contrib/byacc/test/btyacc/err_syntax6.error stable/10/contrib/byacc/test/btyacc/err_syntax7.error stable/10/contrib/byacc/test/btyacc/err_syntax7a.error stable/10/contrib/byacc/test/btyacc/err_syntax7b.error stable/10/contrib/byacc/test/btyacc/err_syntax8.error stable/10/contrib/byacc/test/btyacc/err_syntax8a.error stable/10/contrib/byacc/test/btyacc/err_syntax9.error stable/10/contrib/byacc/test/btyacc/help.output stable/10/contrib/byacc/test/btyacc/no_b_opt.output stable/10/contrib/byacc/test/btyacc/no_output2.output stable/10/contrib/byacc/test/btyacc/no_p_opt.output stable/10/contrib/byacc/test/btyacc/nostdin.output stable/10/contrib/byacc/test/run_test.sh stable/10/contrib/byacc/test/yacc/big_b.output stable/10/contrib/byacc/test/yacc/big_l.output stable/10/contrib/byacc/test/yacc/err_syntax1.error stable/10/contrib/byacc/test/yacc/err_syntax10.error stable/10/contrib/byacc/test/yacc/err_syntax11.error stable/10/contrib/byacc/test/yacc/err_syntax12.error stable/10/contrib/byacc/test/yacc/err_syntax13.error stable/10/contrib/byacc/test/yacc/err_syntax14.error stable/10/contrib/byacc/test/yacc/err_syntax15.error stable/10/contrib/byacc/test/yacc/err_syntax16.error stable/10/contrib/byacc/test/yacc/err_syntax17.error stable/10/contrib/byacc/test/yacc/err_syntax18.error stable/10/contrib/byacc/test/yacc/err_syntax19.error stable/10/contrib/byacc/test/yacc/err_syntax2.error stable/10/contrib/byacc/test/yacc/err_syntax21.error stable/10/contrib/byacc/test/yacc/err_syntax22.error stable/10/contrib/byacc/test/yacc/err_syntax23.error stable/10/contrib/byacc/test/yacc/err_syntax24.error stable/10/contrib/byacc/test/yacc/err_syntax25.error stable/10/contrib/byacc/test/yacc/err_syntax26.error stable/10/contrib/byacc/test/yacc/err_syntax27.error stable/10/contrib/byacc/test/yacc/err_syntax3.error stable/10/contrib/byacc/test/yacc/err_syntax4.error stable/10/contrib/byacc/test/yacc/err_syntax5.error stable/10/contrib/byacc/test/yacc/err_syntax6.error stable/10/contrib/byacc/test/yacc/err_syntax7.error stable/10/contrib/byacc/test/yacc/err_syntax7a.error stable/10/contrib/byacc/test/yacc/err_syntax7b.error stable/10/contrib/byacc/test/yacc/err_syntax8.error stable/10/contrib/byacc/test/yacc/err_syntax8a.error stable/10/contrib/byacc/test/yacc/err_syntax9.error stable/10/contrib/byacc/test/yacc/help.output stable/10/contrib/byacc/test/yacc/no_b_opt.output stable/10/contrib/byacc/test/yacc/no_output2.output stable/10/contrib/byacc/test/yacc/no_p_opt.output stable/10/contrib/byacc/test/yacc/nostdin.output stable/10/usr.bin/yacc/tests/Makefile stable/10/usr.bin/yacc/tests/calc.y stable/10/usr.bin/yacc/tests/calc1.y stable/10/usr.bin/yacc/tests/calc2.y stable/10/usr.bin/yacc/tests/calc3.y stable/10/usr.bin/yacc/tests/code_calc.y stable/10/usr.bin/yacc/tests/code_error.y stable/10/usr.bin/yacc/tests/error.y stable/10/usr.bin/yacc/tests/ftp.y stable/10/usr.bin/yacc/tests/grammar.y stable/10/usr.bin/yacc/tests/legacy_test.sh stable/10/usr.bin/yacc/tests/pure_calc.y stable/10/usr.bin/yacc/tests/pure_error.y stable/10/usr.bin/yacc/tests/quote_calc.y stable/10/usr.bin/yacc/tests/quote_calc2.y stable/10/usr.bin/yacc/tests/quote_calc3.y stable/10/usr.bin/yacc/tests/quote_calc4.y stable/10/usr.bin/yacc/tests/regress.00.out stable/10/usr.bin/yacc/tests/regress.01.out stable/10/usr.bin/yacc/tests/regress.02.out stable/10/usr.bin/yacc/tests/regress.03.out stable/10/usr.bin/yacc/tests/regress.04.out stable/10/usr.bin/yacc/tests/regress.05.out stable/10/usr.bin/yacc/tests/regress.06.out stable/10/usr.bin/yacc/tests/regress.07.out stable/10/usr.bin/yacc/tests/regress.08.out stable/10/usr.bin/yacc/tests/regress.09.out stable/10/usr.bin/yacc/tests/regress.10.out stable/10/usr.bin/yacc/tests/regress.11.out stable/10/usr.bin/yacc/tests/regress.12.out stable/10/usr.bin/yacc/tests/regress.13.out stable/10/usr.bin/yacc/tests/regress.14.out stable/10/usr.bin/yacc/tests/regress.sh stable/10/usr.bin/yacc/tests/undefined.y stable/10/usr.bin/yacc/tests/yacc_tests.sh
A commit references this bug: Author: rodrigc Date: Sat Oct 11 23:31:45 UTC 2014 New revision: 272973 URL: https://svnweb.freebsd.org/changeset/base/272973 Log: Merge: r269884 Complete the usr.bin/yacc kyua integration work I originally submitted via r268811 - Install the Kyuafile by adding FILES to FILESGROUPS - Run the testcases with an unprivileged user Some of the testcases depend upon behavior that's broken when run as root on FreeBSD because of how permissions are treated with access(2) vs eaccess(2), open(2), etc - Simplify the test driver to just inspect the exit code from run_test because it now exits with 0 if successful and exits with !0 if unsuccessful - Don't do ad hoc temporary directory creation/deletion; let Kyua handle that - Add entries for files removed in r268811 to OptionalObsoleteFiles.inc PR: 191020 X-MFC with: r268811 Approved by: jmmv (mentor) Reviewed by: bapt Sponsored by: EMC / Isilon Storage Division Changes: _U stable/10/ stable/10/tools/build/mk/OptionalObsoleteFiles.inc stable/10/usr.bin/yacc/tests/Makefile stable/10/usr.bin/yacc/tests/yacc_tests.sh