Bug 259576 - Mk/bsd.port.mk: tests are not parallelized; DO_MAKE_TEST does not contain ${_MAKE_JOBS}
Summary: Mk/bsd.port.mk: tests are not parallelized; DO_MAKE_TEST does not contain ${_...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Port Management Team
URL:
Keywords: needs-patch, performance
Depends on:
Blocks:
 
Reported: 2021-10-31 14:35 UTC by Matthias Andree
Modified: 2021-11-19 20:43 UTC (History)
3 users (show)

See Also:
koobs: maintainer-feedback? (portmgr)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Andree freebsd_committer freebsd_triage 2021-10-31 14:35:57 UTC
tests defined through TEST_TARGET and/or DO_MAKE_TEST do not contain ${_MAKE_JOBS} by default, thus tests do not run in parallel in the sub-makes.

Workaround: TEST_ARGS+=${_MAKE_JOBS}
Comment 1 Daniel Engberg freebsd_committer freebsd_triage 2021-11-02 20:30:58 UTC
I'm not saying its a bad idea but looking at other repos in general unit tests seems to be a bit sensitive to parallel jobs so this might more work than its worth and since they aren't built/run by default I don't think we need to worry too much about time spent running tests.
Comment 2 Matthias Andree freebsd_committer freebsd_triage 2021-11-04 11:51:03 UTC
Daniel,

I have yet to see a port which is either my port, or an important requisite to one of my ports, where a "make test" fails when run in parallel.  In an era of 8-or-more threads ubiquitous, this isn't an excuse. 

In the long run, we should probably also run a decent test set by default on the builders, and leave only long-runners optional.

We can of course offer a framework parameter such as TEST_JOBS_NUMBER (which would default to MAKE_JOBS_NUMBER) so it's overridable to 1 or a Boolean-logic  TEST_JOBS_UNSAFE switch, and when globally running out tests in some future point in time, this will warrant -exp runs, but none of that is a hindrance to fixing test performance now.
Comment 3 Matthias Andree freebsd_committer freebsd_triage 2021-11-04 11:51:54 UTC
(In reply to Matthias Andree from comment #2)
s/running out/rolling out/
Comment 4 Baptiste Daroussin freebsd_committer freebsd_triage 2021-11-19 20:43:05 UTC
I am all for a patch which would provide en equivalent of _MAKE_JOBS for DO_MAKE_TEST, when providing the patch, please also provide a TEST_MAKE_JOB_UNSAFE knob or equivalent.