Bug 231555 - Mk/Uses/python.mk: Add USE_PYTHON=pytest
Summary: Mk/Uses/python.mk: Add USE_PYTHON=pytest
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL:
Keywords: feature
Depends on:
Blocks:
 
Reported: 2018-09-21 17:14 UTC by Yuri Victorovich
Modified: 2018-09-24 12:59 UTC (History)
2 users (show)

See Also:
yuri: maintainer-feedback? (python)


Attachments
patch (918 bytes, patch)
2018-09-21 17:14 UTC, Yuri Victorovich
no flags Details | Diff
patch (1.51 KB, patch)
2018-09-22 06:17 UTC, Yuri Victorovich
no flags Details | Diff
patch (1.47 KB, patch)
2018-09-22 17:30 UTC, Yuri Victorovich
no flags Details | Diff
patch (947 bytes, patch)
2018-09-22 18:32 UTC, Yuri Victorovich
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer 2018-09-21 17:14:53 UTC
Created attachment 197313 [details]
patch

Many packages define 'pytest' tests.

USE_PYTHON=pytest adds the test dependency and the do-test target.
Comment 1 Matthew Seaman freebsd_committer 2018-09-22 05:57:16 UTC
Definitely +1 on this idea.  Nice one.

Could you allow for adding additional environment variables and commandline arguments?  Something like:

```
     @cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} ${PYTEST_ENV} \
         ${PYTHON_CMD} -m pytest ${PYTEST_ARGS}
```

where ${PYTEST_ENV} and ${PYTEST_ARGS} are empty by default.
Comment 2 Yuri Victorovich freebsd_committer 2018-09-22 06:17:56 UTC
Created attachment 197333 [details]
patch

Thanks Matthew,

Good improvement idea. I updated the patch.

Yuri
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2018-09-22 09:54:07 UTC
I'd like to see something like this leverage/extend the test framework bits if necessary to achieve what we want, to make it usable for cases other than just pytest (nosetests, unittest, whatever). python.mk could then setup/wrap python specific test scenarios, using generic TEST_* (TEST_COMMAND/TEST_CMD_ARGS, etc) variables wherever possible.

I see most of the value created in leveraging/extending the test framework, than is obtained from removing the need for TEST_DEPENDS and not needing to specify a test target.

Test invocations are *notoriously* non-standard in the Python world (granted, python -m pytest 'is' a common one), with various permutations of a env/commands/args combination necessary to get it just right.

Further, its very commonly the case that pytest along is not a standalone/sufficient dependency by itself, and having them split implicitly in the framework and explicitly in the port is not particularly desirable.

This idea has been an open task [1] for the Python team for a while, looking for at least a *minimally* considered/designed schema to support a plurality of test execution needs, rather than just added on an adhoc basic, and hopefully one that could be leveraged to the benefit of other softwares/language stacks in the ports tree.

Perhaps a good place to start would be to take stock of the existing test methods for existing python ports and see what scheme we can come up with to support the 'major classes' of test execution methods.
Comment 4 Yuri Victorovich freebsd_committer 2018-09-22 17:30:33 UTC
Created attachment 197357 [details]
patch

(In reply to Kubilay Kocak from comment #3)

Kubilay, you are right, thanks for your suggestion.
I plugged this test into the existing test framework. This made it simpler.

nosetests, unittest, whatever can be added later in a likewise fashion.
Comment 5 Antoine Brodin freebsd_committer 2018-09-22 18:27:11 UTC
Please don't invent new variables, there is already TEST_ENV
Comment 6 Yuri Victorovich freebsd_committer 2018-09-22 18:32:16 UTC
Created attachment 197361 [details]
patch

Use TEST_ENV/TEST_ARGS.
Comment 7 Mathieu Arnold freebsd_committer 2018-09-24 12:58:02 UTC
Does this actually work?
Comment 8 Mathieu Arnold freebsd_committer 2018-09-24 12:59:34 UTC
I'd say it lacks a:


--- Mk/bsd.port.mk
+++ Mk/bsd.port.mk
@@ -3363,3 +3363,3 @@ do-install:

-.if !target(do-test) && defined(TEST_TARGET)
+.if !target(do-test) && ( defined(TEST_TARGET) || defined(DO_MAKE_TEST) )
 DO_MAKE_TEST?= ${SETENV} ${TEST_ENV} ${MAKE_CMD} ${MAKE_FLAGS} ${MAKEFILE} ${TEST_ARGS:C,^${DESTDIRNAME}=.*,,g}