Bug 264454 - Do not build TEST_DEPENDS in case NO_TEST is set and do not run tests
Summary: Do not build TEST_DEPENDS in case NO_TEST is set and do not run tests
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-06-04 17:12 UTC by Bjoern A. Zeeb
Modified: 2024-01-10 14:53 UTC (History)
7 users (show)

See Also:


Attachments
Honor NO_TEST (1.21 KB, patch)
2022-06-04 17:12 UTC, Bjoern A. Zeeb
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-04 17:12:14 UTC
Created attachment 234448 [details]
Honor NO_TEST

TEST_DEPENDS attributes to quite a few (often conflicting) dependencies often not wanted or needed on personal package builds.
Rather than touching every port, disable building TEST_DEPENDS in one place in bsd.port.mk if NO_TEST is set.
Likewise do not run the test: target if NO_TEST is set.

This is based on a suggestion by Tomoaki AOKI in https://lists.freebsd.org/archives/freebsd-ports/2022-May/001944.html .

It saves a lot of Joules every day ...
Comment 1 Stefan Eßer freebsd_committer freebsd_triage 2022-06-04 19:29:10 UTC
I had already commented on NO_TEST before:

Why do we need NO_TEST at all?

If TEST_DEPENDS are built without "make test" being invoked, there is a problem in bsd.port.mk or some other ports system component.

This could apply to e.g. "make package-recursive", if it did build and package TEST_DEPENDS, but it does not!

I do not know of a single port operation that recurses into the TEST_DEPENDS, without this being specifically required to support "make test".

And regarding the Joules wasted on needless port and package building: there are tools that optimize the update process (poudriere, portmaster, portupgrade, ...).

They prevent needless rebuilds of ports that did not change since the last build cycle. Always rebuilding all relevant ports at a fixed schedule without checking whether there have been any updates to the port - that's a waste of Joules!
Comment 2 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-04 20:34:59 UTC
(In reply to Stefan Eßer from comment #1)

>  If TEST_DEPENDS are built without "make test" being invoked, there is a problem in bsd.port.mk or some other ports system component.

Have you looked at the diff?
Comment 3 Stefan Eßer freebsd_committer freebsd_triage 2022-06-04 21:36:48 UTC
(In reply to Bjoern A. Zeeb from comment #2)

> Have you looked at the diff?

Yes, and in fact I had mentioned that not including TEST_DEPENDS in _UNIFIED_DEPENDS reduces the output of "make all-depends-list" by not including TEST_DEPENDS.

But I do not see why this is relevant, since no port update operation should depend on the output of that command.

Taking just one random port that has a TEST_DEPENDS entry not otherwise depended on as an example:

$ cd /usr/ports/devel/fsmtrie

$ make -V TEST_DEPENDS
checkmk:devel/check

$ make all-depends-list | grep devel/check

$ make -C fsmtrie test-depends-list | grep devel/check
/usr/git/ports/devel/check

But I see that there is an issue with "make package-recursive" not respecting the port options selected!

I have deselected DOCS and DOXYGEN, but "make package-recursive" still builds and packages devel/doxygen.

This seems to be wrong (and has nothing to do with the TEST_DEPENDS definition).

IMHO, "make package-recursive" should only create packages of the files required as run dependencies when the fsmtrie package is installed (from a package file).

But again: this has nothing to do with the suggested NO_TEST change.

It seems that nobody uses "make package-recursive" anymore, since port management tools are the much better way to install ports and build packages.
Comment 4 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-04 22:08:03 UTC
(In reply to Stefan Eßer from comment #3)

Thanks for enlightening me a bit more and finding yet another problem.

man ports still talks about package-recursive
src/tools/tools/nanobsd/dhcpd/common still seems to call it
https://wiki.freebsd.org/SummerOfCode2019Projects/PortsSeparatedBuild#Working_status and a some other (aged) wiki pages mention it.

Most of the forum/lists traffic seems to have stopped about 4-5 years ago.

If it is a problem keeping it working I'd add a deprecation notice and remove the docs first.

Should we adjust the title of the PR to "package-recursive doing more than it should"?
Comment 5 Tomoaki AOKI 2022-06-04 23:04:34 UTC
(In reply to Stefan Eßer from comment #1)

make fetch-recursive currently forces downloading of TEST_DEPENDS.
This can consume excessive disk space.

Moreover, if any of MASTER_SITES that hosts TEST_DEPENDS ports are not responsive or too slow, this causes a plenty of time to finish downloading, or failure.
This happenes at the first time and every time any TEST_DEPENDS port is updated.
Comment 6 Norikatsu Shigemura 2022-06-06 06:04:54 UTC
> Affects Only Me

No. "Affects some pepole"
I have same opinion.
Comment 7 Stefan Eßer freebsd_committer freebsd_triage 2022-06-06 10:19:14 UTC
(In reply to Tomoaki AOKI from comment #5)

> make fetch-recursive currently forces downloading of TEST_DEPENDS.
> This can consume excessive disk space.

But why are you using these *-recursive targets at all?

They are useful if you have off-line (air-gapped) systems and you want to fetch files or provide packages for use on them.

In that case you want to have all possibly relevant distribution files and packages included, just in case you want to "make test" on this system that has no access to the Internet.

I really do not understand why you are using these commands to perform port upgrades or other package management tasks.

We have tools that optimize the upgrades, e.g. portmaster or portupgrade. These do exactly what is required, and in an optimized way (e.g. fetch just the distribution files that will be required in the background, perform only those builds that are required, ...).

Portmaster has no dependencies (it is just a shell script), exists for some 20 years and has continuously been adapted to comply with changes in the ports/package system. It lets you build ports and packages in just the same way as 20 years ago, despite major changes in the ports system (e.g. the introduction of the PKGDB and of flavors).

Try "portmaster -bgw -a" to update all down-level ports (and create packages and package backups of the previous version, just in case). (The -bgw parameter can of course be set as default options in the ~/.portmasterrc file.)

Try "portmaster -F -a" to just fetch required distfiles for all ports needing updates in parallel.

Using the *-recursive targets in the way you seem top do it is a significant waste of resources, and the port management tools mentioned above implement the required logic to allow for efficient updates/upgrades.

There is so much missing in bsd.port.mk if you want to make it efficiently support your use case that adding NO_TEST will just increase its complexity, but without solving the issues you are having due to inappropriate use of this functionality. There is no functionality to selectively build just the modified ports this way, for example.

There are other port and package management tools, I'm mentioning portmaster just as an example. You may prefer portupgrade, for example.
Comment 8 Tomoaki AOKI 2022-06-06 12:00:26 UTC
(In reply to Stefan Eßer from comment #7)

I basically use portupgrade, if it fails (usually "Backing up old version" phase, due to, in many cases, symlinks), try pkg_replace, and even it fails (usually misses new indirect dependencies), fall back to oldest way.

For oldest way (make config-recursive ; make fetch-recursive && make && make deinstall reinstall clean), I use *-recursive for config and fetch not to be bothered by downloading at build phase.

And even if portupgrade works "almost" just fine, I sometimes notice fetched-but-not-installed ports. These are usually TEST_DEPENDS item.

Sometimes TEST_DEPENDS items doesn't fetch properly (too slow or inresponsive MASTER_SITES, checksum-mismatch, size-mismatch, etc.), it severly bothers me.

I want reliable way to just config and fetch actually required dependencies.
Comment 9 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-06 12:59:12 UTC
You go by assumptions of what people are doing.

I do not do port upgrades;  in fact I most often never do apart from select individual bits which I do not handle this way.
I build coherent image (tailored distribution) after FreeBSD version bumps on main at times (I am not even doing every one of them).
I do have ports which are not in FreeBSD, I have ports which in fact build bootable images bundled all together.
The only thing I noticed was that build times and built packages over time (especially in the last year) massively increased on a system which also worked ages and is documented as such and I finally started looking.

Can we get away from use whatever other tools available to actually fixing the problems at hand one way or another or otherwise close the PR as "won't fix" and save the time of repeated discussions.  I am happy to call the problem different if you have a problem that it is titled "NO_TEST" as I indicated before.

But it seems there are more people than just me using recursive targets.
Comment 10 Norikatsu Shigemura 2022-06-06 16:46:43 UTC
My case is as follows.

- At the time of the first ports installation after OS installation, after `make config-recurisve`, `make install` in parallel while `make fetch-recursive`-ing.

- When upgrading the major version of the OS, delete all ports with `pkg delete -ayf` and then execute the above.

- In some cases, the OPTIONS causes infinite loop when OPTIONS is enabled. I use `make all-depends-list` to identify the infinite loop location, but I have to check a lot of ports because of TEST_DEPENDS.
Comment 11 Stefan Eßer freebsd_committer freebsd_triage 2022-06-10 08:10:15 UTC
(In reply to Bjoern A. Zeeb from comment #9)

> You go by assumptions of what people are doing.

Yes, given no further information I do have to make assumptions.

> I do not do port upgrades;  in fact I most often never do apart from select 
> individual bits which I do not handle this way.
> I build coherent image (tailored distribution) after FreeBSD version bumps 
> on main at times (I am not even doing every one of them).

You cannot expect bsd.port.mk to provide the high level functionality for every use case. I have managed the port management tools: they do exactly that - provide user oriented and configurable functionality.

But you do not have to use a tool like portmaster. Something as simple as the following script should be sufficient to build packages for a specified set of ports without fetching or building anything not required:

----------------------------
#!/bin/sh -e

WANTED="
shells/bash
www/py-arxiv
"

make_wanted () {
    echo "$WANTED" | while read dir; do
        if [ -n "$dir" ]; then
            make -C $dir "$@"
        fi
    done
}

cd /usr/ports

make_wanted clean

REQUIRED=$(make_wanted run-depends-list | sort -u)

for dir in $REQUIRED; do
    make -C $dir package
done
----------------------------

This will create a consistent set of packages including all run dependencies. 

One possible issue with such a simple approach is that it expects all ports to be built for just the default flavor.

FLAVOR support has been implemented in the ports system a way that is very unfriendly to any tool except poudriere.

The port management tools provide the above shown functionality (and a lot more) in a way that optimizes the procedure and that can deal with non-default flavors.

Trying to get all functionality for all possible non-standard use-cases into bsd.port.mk is the completely wrong approach. This Makefile provides a set of primitive functions that can be used in scripts like the example above to implement any special case.

It is much easier to deal with such use-cases in shell scripts (or other script languages) than in a Makefile. Makefiles implements target driven functionality, scripts implement procedures.

The *-recursive function in bsd.port.mk have been implemented at a time when there were no port management tools (and not even the pkg command and PKGDB).

They do not serve your use case well and probably never will. I have shown other examples where functionality at that level is not provided by bsd.port.mk in a way that is useful for tasks like those you want it to achieve.

What you are doing is a typical use-case for poudriere. It builds packages in a way that guarantees their consistency and always in a clean environment, not influenced by any ports already present on the base system.