Bug 217284 - ports-mgmt/portupgrade and handling of shared libraries used by depending ports
Summary: ports-mgmt/portupgrade and handling of shared libraries used by depending ports
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Bryan Drewery
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-02-22 00:45 UTC by Mikhail Teterin
Modified: 2017-11-28 20:14 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (bdrewery)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mikhail Teterin freebsd_committer freebsd_triage 2017-02-22 00:45:35 UTC
Currently portupgrade will only update depending ports, when invoked with the -r flag. It should be possible (perhaps with a different option) to automatically rebuild those of them, that use a shared library provided by one of the ports being rebuild IF such a shared library's version changes.

A separate -- but related -- problem is that portupgrade relies on the dependencies explicitly-listed by each port (and put together into INDEX) -- which may be inaccurate.

Instead it should use the pkg's database -- which is more reliable since it lists the ACTUAL shared libraries used by each installed package's binaries.

This problem is currently handled by bumps to PORTREVISION -- a poor crutch, that should never have been used for the purpose.
Comment 1 Bryan Drewery freebsd_committer freebsd_triage 2017-02-22 18:28:45 UTC
PORTREVISION bumping is still the only correct thing here.  It is not
*just* shared libraries that require rebuilds.  Imagine if lang/clang is
used to build a port, and then a patch is committed to lang/clang/files/
to fix the *output* of the compiled files.  In this case we must rebuild
everything that lang/clang built.  But if we just fix a documentation bug in lang/clang, do we really need to rebuild all ports depending on it? No.  This is why PORTREVISION chasing is supposed to be done, but it is not very well.

Poudriere is far too aggressive here and assumes people don't bump PORTREVISION properly, which is pays for in longer build times.  It is seen as more acceptable since Poudriere is more a "Batch" thing and less "Interactive"
as Portupgrade can be, since it is breaking your live system as it runs.
Comment 2 Mikhail Teterin freebsd_committer freebsd_triage 2017-02-22 19:35:33 UTC
> Imagine if lang/clang is used to build a port, and then a patch is
> committed to lang/clang/files/ to fix the *output* of the compiled files

Imagine clang changing in the base... Chasing all possible reasons for why a pkg may need rebuilding is impractical and should not be undertaken at all.

A port's PORTREVISION should only go up, when the port itself changes (without PORTVERSION bump). (The discussion took place on ports-committers@ under the subject "Misuse of PORTREVISION" -- let's keep it there. So far the consensus is, yes, PORTREVISION is currently being misused, but the practice can not end until portupgrade and portsnap are fixed.)

But a change to shared libraries will prevent the port's binaries from even starting and -- instead of preserving the older libraries under compat/ -- it should be an option to rebuild all such depending ports.

You already offer the `-r' option, but that may be too aggressive for some users. Someone may not want to rebuild ALL ports depending on the ports I'm upgrading today, only those, that use shared libraries that are getting new versions.

> Poudriere is far too aggressive here and assumes people don't bump
> PORTREVISION properly, which is pays for in longer build times

I've seen people bumping PORTREVISION for the wrong reasons too -- without the excuse of being a dumb automaton :) -- resulting in gratuitous rebuilds.

And even when it is not "a mistake", there are unnecessary rebuilds. For example, if my ffmpeg is configured to not use x265, I don't need to rebuild it every time x265 is updated. But, under the current practice, I will have to rebuild it, because PORTREVISION of ffmpeg is bumped for everyone, whether they use x265 or not...
Comment 3 Bryan Drewery freebsd_committer freebsd_triage 2017-02-22 19:43:21 UTC
(In reply to Mikhail Teterin from comment #2)
> > Imagine if lang/clang is used to build a port, and then a patch is
> > committed to lang/clang/files/ to fix the *output* of the compiled files
> 
> Imagine clang changing in the base... Chasing all possible reasons for why a
> pkg may need rebuilding is impractical and should not be undertaken at all.

That's out of scope of ports, and was merely an example. Many of the dependencies
in ports may change the build result and not have anything to do with shared
libraries.

> 
> A port's PORTREVISION should only go up, when the port itself changes
> (without PORTVERSION bump). (The discussion took place on ports-committers@
> under the subject "Misuse of PORTREVISION" -- let's keep it there. So far
> the consensus is, yes, PORTREVISION is currently being misused, but the
> practice can not end until portupgrade and portsnap are fixed.)
> 

This is a simple way of ignoring the problem and blaming a few tools,
*because* of what Poudriere does.  Poudriere is not right either.

Making Portupgrade rebuild things when a shared library bump happens
is a good thing. I agree with making it do that.

However, it is not a replacement for PORTREVISION chases as shared libraries
are not the only need for rebuilds.  A system *will be broken* if we assume
only libraries are a cause for rebuild.

That thread is lacking such proper discussion of the real issues.  I will
add to it.
Comment 4 Mikhail Teterin freebsd_committer freebsd_triage 2017-02-22 20:28:30 UTC
> Making Portupgrade rebuild things when a shared library bump happens
> is a good thing. I agree with making it do that.

Thanks! Let's do that for the time being...
Comment 5 Mark Linimon freebsd_committer freebsd_triage 2017-06-22 20:38:35 UTC
Correct assignment.