Bug 164896

Comment 1 Philip M. Gollucci 2012-02-08 20:13:04 UTC

Responsible Changed
From-To: freebsd-ports-bugs->portmgr

Comment 2 Mark Linimon 2012-03-24 07:54:37 UTC

State Changed
From-To: open->analyzed

Comment 3 Mark Linimon 2012-03-24 07:54:37 UTC

Responsible Changed
From-To: portmgr->linimon

Comment 4 Steve Wills 2012-04-26 03:05:37 UTC

Here's an updated patch. I believe this may be much closer to ready.

Steve

Comment 5 Steve Wills 2012-04-26 04:24:06 UTC

On 04/25/12 22:05, Steve Wills wrote:
> Here's an updated patch. I believe this may be much closer to ready.
> 
> Steve

Slight change: USE_PERL_TESTS -> WITH_PERL_TESTS

Steve

Comment 6 Steve Wills 2012-05-22 04:45:21 UTC

Here's a further updated version of the patch. It has a few minor
changes such as not including tests if TEST_DEPENDS are defined.

Would still like to get feedback from someone who knows bsd.port.mk
well. My basic question is, am I better off with this sort of "bolted
on" approach to testing, or should I take a different approach and
integrate the TEST_DEPENDS so they are treated like other DEPENDS, for
example showing up in "make describe" and creating a
"pretty-print-test-depends-list" target.

Thanks,
Steve

Comment 7 Mark Linimon 2012-06-02 04:50:29 UTC

Responsible Changed
From-To: linimon->swills

This test was completed some time ago.  Sorry to have put off doing the 
analysis until now. 

The results are in: 

http://pointyhat-west.isc.freebsd.org/errorlogs/amd64-errorlogs/e.9.20120324081752.pointyhat-west/index-reason.html 

There were no failures with the ports infrastructure. 

All 13 package errors marked 'mtree' seem to be regressions introduced 
by the patch.  Please advise.

Comment 8 Steve Wills 2012-06-11 02:57:24 UTC

On Jun 1, 2012, at 11:54 PM, linimon@FreeBSD.org wrote:

> Synopsis: [exp-run] add support for testing perl modules
> 
> Responsible-Changed-From-To: linimon->swills
> Responsible-Changed-By: linimon
> Responsible-Changed-When: Sat Jun 2 03:50:29 UTC 2012
> Responsible-Changed-Why: 
> This test was completed some time ago.  Sorry to have put off doing the
> analysis until now.
> 
> The results are in:
> 
>  http://pointyhat-west.isc.freebsd.org/errorlogs/amd64-errorlogs/e.9.20120324081752.pointyhat-west/index-reason.html
> 
> There were no failures with the ports infrastructure.
> 
> All 13 package errors marked 'mtree' seem to be regressions introduced
> by the patch.  Please advise.
> 
> http://www.freebsd.org/cgi/query-pr.cgi?pr=164896

The 13 package errors marked 'mtree' are due to the patch creating a
"regression-test" target that calls the "test" target, in the case that
the regression-test target doesn't exist. These ports have "test"
targets which create extra files, which creates the mtree issue. I've
seen that happen with many perl ports during my own testing. Since
pointyhat calls the "regression-test" target, the mtree issue is
triggered due to this patch where it wouldn't be before because the
regression-test call wasn't calling the test target.

We can solve these issues several ways. Perhaps the simplest way would
be to have the test patch changed so that the default test target is
only created if both the test and the regression-test targets do not
exist. Another would be to standardize on only "test" or
"regression-test". Or we might have pointyhat not call the test target.

Yet another way to solve the problem might be to have the ports changed
so that their tests don't leave leftover files. I'm guessing there are a
small number of ports which have regression-test targets which are
called by pointyhat and which had this "test creates mtree issue" issue
in the past which were fixed.

All but the simplest solution would require more discussion... I think
we need to standardize on the naming and decide where/when tests will be
run in any case, so perhaps now's the time....

Steve

(re-sending to get this into the PR)

Comment 9 Steve Wills 2012-06-11 03:09:37 UTC

Responsible Changed
From-To: swills->portmgr

- Back to portmgr for discussion

Comment 10 Bryan Drewery 2014-03-28 22:49:23 UTC

Responsible Changed
From-To: portmgr->bdrewery

I've been deep into bsd.port.mk deps lately, I will see this in. I will 
move bsd.test.mk into a USES as well. I agree with the "bolted on" 
approach as the general case right now is no tests"

Do note that perl.mk has a regression-test target (and a test one), look at
the end :-)

-- 
Mathieu Arnold

Comment 12 Bryan Drewery 2015-03-10 16:14:22 UTC

*** Bug 198494 has been marked as a duplicate of this bug. ***