Bug 148695
| Summary: | [bsd.port.mk] [patch] Easier / automatic regression-test target for ports | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Christopher Key <cjk32> | ||||
| Component: | Ports Framework | Assignee: | Steve Wills <swills> | ||||
| Status: | Closed Overcome By Events | ||||||
| Severity: | Affects Only Me | CC: | jbeich | ||||
| Priority: | Normal | ||||||
| Version: | Latest | ||||||
| Hardware: | Any | ||||||
| OS: | Any | ||||||
| Attachments: |
|
||||||
Responsible Changed From-To: freebsd-ports-bugs->portmgr bsd.port.mk patch. State Changed From-To: open->suspended This cannot be added because many packages will fail the tests (false-positive). It needs to be enabled on port-by-port basis. Responsible Changed From-To: portmgr->swills Hand over to swills who is working on a testing framework and may pick some good ideas from this PR to incorporate. regression-test isn't documented, standardised[1] and only run by tinderbox but not poudriere. And this bug maybe obsolete by the addition of option helpers in bsd.options.mk. For example, to run devel/libffi tests the following is enough
OPTIONS_DEFINE= TEST
TEST_BUILD_DEPENDS=runtest:${PORTSDIR}/misc/dejagnu
TEST_ALL_TARGET=check
and if maintainer has high confidence[1] in passing tests also add
OPTIONS_DEFAULT= TEST
while user can globally disable via make.conf to shave build time
OPTIONS_UNSET= TEST
[1] examples being archivers/lzo2 and graphics/png each requiring a different way to disable running tests
|
At the moment, many ported applications/libraries provide some form of regression testing, but very few ports actually expose this via the standard regression-test target. The regression-test target will be called by tinderbox if available, and should also make debugging misbehaving ports easier. Attached is a patch which aims to remedy this situation. Firstly, it simplifies the creation of a regression-test target, with the addition of a REGRESSION_TEST_TARGET variable. For audio/flac for example, this means that: #v+ regression-test: build @cd ${WRKSRC}; ${SETENV} ${MAKE_ENV} ${MAKE} check #v- can be replaced with: #v+ REGRESSION_TEST_TARGET= check #v- Secondly, rather more aggressively, it assumes that anything using GNU_CONFIGURE or PERL_CONFIGURE implements a check or test target respectively, and sets REGRESSION_TEST_TARGET accordingly. As far as I am aware, these target are always present, although it is not guaranteed that any tests will actually be run. Most perl modules do provide tests, but gnu software seems less likely to do so. If no useful testing will be performed, the automatic creation of a regression-test target can be overridden by setting REGRESSION_TEST_TARGET to an empty string. The attached patch probably isn't doing things in the correct places, or doing them in them in the correct way, but does work to illustrate the intent.