Created attachment 207626 [details] Run regression tests with bash Running security/nss's regression tests reveals a number of shell errors from test scripts that use bash features like the built-in UID variable and more importantly arrays: [: -ne: unexpected operator ./ssl.sh: 1245: Syntax error: word unexpected (expecting ")") ./ectest.sh: 49: Syntax error: word unexpected (expecting ")") Running the tests with bash instead shows stunningly different results. Before (sh): Passed: 3017 Failed: 0 Failed with core: 0 ASan failures: 0 Unknown status: 0 After (bash): Passed: 14469 Failed: 0 Failed with core: 0 ASan failures: 0 Unknown status: 48 TinderboxPrint:Unknown: 48 Patch attached. We could change the shebangs of all scripts to /usr/local/bin/bash, but there is no need for this; simply calling the top-level script with bash is enough.
Comment on attachment 207626 [details] Run regression tests with bash Looks OK but see comments below. Bash'isms appear to have been introduced recently. Kinda sad as FreeBSD sh is blazingly fast while tests take a lot of time due to lack of parallelism. > +TEST_DEPENDS= bash:shells/bash For "make test" to install the dependency convert "check regression-test test" (multi-name target) to "do-test". When tinderbox was in use by the package cluster (pointyhat at the time) "regression-test" was used and TEST_DEPENDS was handled by tinderbox itself rather than the ports framework. > + ${LOCALBASE}/bin/bash ./all.sh Consistently use either "bash" or "${LOCALBASE}/bin/bash". I'd probably choose the former unless there's a good reason to ignore PATH.
A commit references this bug: Author: naddy Date: Tue Sep 24 13:45:09 UTC 2019 New revision: 512715 URL: https://svnweb.freebsd.org/changeset/ports/512715 Log: Run regression tests with bash since some test scripts unfortunately rely on bash features. PR: 240686 Reviewed by: jbeich Changes: head/security/nss/Makefile
Committed with tweaks.