Bug 220950 - Mk/Scripts/check-stagedir.sh false negatives (with %%OPTIONS%%)
Summary: Mk/Scripts/check-stagedir.sh false negatives (with %%OPTIONS%%)
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Port Management Team
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-23 19:21 UTC by Gerald Pfeifer
Modified: 2021-11-02 06:52 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gerald Pfeifer freebsd_committer freebsd_triage 2017-07-23 19:21:20 UTC
It appears that Mk/Scripts/check-stagedir.sh completely misses files that appear
 - in the staging directory, and
 - are listed in pkg-plist with an %%OPTION%% prefix when OPTION is off.

Those files should be included in packages (or when the port is installed), but
are not.

And check-stagedir.sh should diagnose that those files are in stagedir, but not
included in pkg-plist.

This is quite a problem!


How to reproduce:

 1. chdir $PORTSDIR/emulators/wine-devel
 2. svn up -r 445571  (the version before I'll fix pkg-plist in a minute)
 3. Run the following script (after adjusting STAGEDIR and PREFIX)

#!/bin/sh

STAGEDIR=$WRKDIRPREFIX/wine-devel-work/stage
PREFIX=$WRKDIRPREFIX/wine-devel-prefix

grep %%STAGING%% pkg-plist | while read f; do
  f=`echo $f | sed -e 's/%%STAGING%%//'`
  if [ -f "$STAGEDIR/$PREFIX/$f" ]; then
    printf "%s no longer contigent on %%STAGING%%?\n" "$f"
  fi
done
Comment 1 commit-hook freebsd_committer freebsd_triage 2017-07-23 20:00:13 UTC
A commit references this bug:

Author: gerald
Date: Sun Jul 23 19:59:17 UTC 2017
New revision: 446505
URL: https://svnweb.freebsd.org/changeset/ports/446505

Log:
  Move some twenty files that originally were part of the Wine Staging
  patchset when Wine 2.0 branched and have been integrated into the Wine
  development tree since then from being only included in packages / installed
  ports when option STAGING is active to the general case (i.e., omit
  the leading %%STAGING%% marker from their entries in pkg-plist).

  This points to an unfortunate deficiency in Mk/Scripts/check-stagedir.sh
  which we are relying on. [1]

  PR:		220950 [1]

Changes:
  head/emulators/wine-devel/Makefile
  head/emulators/wine-devel/pkg-plist
Comment 2 Mathieu Arnold freebsd_committer freebsd_triage 2017-07-25 22:31:36 UTC
(In reply to Gerald Pfeifer from comment #0)
> It appears that Mk/Scripts/check-stagedir.sh completely misses files that
> appear
>  - in the staging directory, and
>  - are listed in pkg-plist with an %%OPTION%% prefix when OPTION is off.
> 
> Those files should be included in packages (or when the port is installed),
> but
> are not.

No they should not.

The files should only be included if opt is enabled.

There are many, many, ports that stage all the files but only package them if some options are enabled.

> And check-stagedir.sh should diagnose that those files are in stagedir, but
> not included in pkg-plist.

Well, it might be an idea to say something, like "this file is installed but not packaged" but it must not be either a warning or an error.

> This is quite a problem!

This is a problem in *1* port, but "fixing" it in the framework would break many, many ports.

> 
> How to reproduce:
> 
>  1. chdir $PORTSDIR/emulators/wine-devel
>  2. svn up -r 445571  (the version before I'll fix pkg-plist in a minute)
>  3. Run the following script (after adjusting STAGEDIR and PREFIX)
> 
> #!/bin/sh
> 
> STAGEDIR=$WRKDIRPREFIX/wine-devel-work/stage
> PREFIX=$WRKDIRPREFIX/wine-devel-prefix
> 
> grep %%STAGING%% pkg-plist | while read f; do
>   f=`echo $f | sed -e 's/%%STAGING%%//'`
>   if [ -f "$STAGEDIR/$PREFIX/$f" ]; then
>     printf "%s no longer contigent on %%STAGING%%?\n" "$f"
>   fi
> done
Comment 3 Gerald Pfeifer freebsd_committer freebsd_triage 2017-07-26 07:02:37 UTC
(In reply to Mathieu Arnold from comment #2)
> The files should only be included if opt is enabled.

No. ;-)  What happens here is that these files only were generated due to
some patches in the Wine Staging patchset which is enabled via the STAGING
option.

Over time those patches tend to move from Wine Staging into mainline Wine,
at which point they are built by default (without the Staging patchset and
hence option STAGING) and need to be included unconditionally.

> There are many, many, ports that stage all the files but only package
> them if some options are enabled.

Hmm, in that case it becomes a little trickier indeed.  Good point.

> Well, it might be an idea to say something, like "this file is installed
> but not packaged" but it must not be either a warning or an error.

That would be lovely.  Note, this caused real user issues in case of Wine,
so is worth addressing (and we don't know which other ports may be affected).
Comment 4 commit-hook freebsd_committer freebsd_triage 2017-10-04 20:24:19 UTC
A commit references this bug:

Author: gerald
Date: Wed Oct  4 20:24:00 UTC 2017
New revision: 451245
URL: https://svnweb.freebsd.org/changeset/ports/451245

Log:
  Add a new Makefile target check-wine-devel-vs-wine-staging that works
  around a deficiency of Mk/Scripts/check-stagedir.sh that does not spot
  entries in pkg-plist that move from being enabled only with the STAGING
  option (%%STAGING%%foo/bar/file) to being there by default (foo/bar/file).

  This has caused actual issues for users, so we need to regularly check
  for such cases, which a simple `check-wine-devel-vs-wine-staging` will
  now do, provided the staging directory is accessible.

  PR:		220950

Changes:
  head/emulators/wine-devel/Makefile
Comment 5 commit-hook freebsd_committer freebsd_triage 2017-10-12 16:15:41 UTC
A commit references this bug:

Author: gerald
Date: Thu Oct 12 16:15:11 UTC 2017
New revision: 451904
URL: https://svnweb.freebsd.org/changeset/ports/451904

Log:
  include/wine/windows/d3d11_2, include/wine/windows/d3d11_3 and
  include/wine/windows/d3d11_4 actually are not contingent on the
  STAGING option, so always package them.

  PR:		220950

Changes:
  head/emulators/wine-devel/Makefile
  head/emulators/wine-devel/pkg-plist
Comment 6 Baptiste Daroussin freebsd_committer freebsd_triage 2019-09-06 08:34:27 UTC
Can we consider the issue here as resolved? or is there more actions expected?
Comment 7 Gerald Pfeifer freebsd_committer freebsd_triage 2019-09-06 15:16:36 UTC
Adding at least a warning (or if that is not acceptable then a note)
would seem important.  This has caused *real* issues for users.  

This could be controlled by an option, and then whether on by default
(my preference) or off would be a question.
Comment 8 commit-hook freebsd_committer freebsd_triage 2020-02-08 10:21:07 UTC
A commit references this bug:

Author: gerald
Date: Sat Feb  8 10:20:17 UTC 2020
New revision: 525543
URL: https://svnweb.freebsd.org/changeset/ports/525543

Log:
  $DATADIR/winebus.inf is now provided by Wine itself, no longer just by
  the Wine Staging patchset (which is contingent on the STAGING option).

  We did not detect this due to the way Mk/Scripts/check-stagedir.sh has
  false negatives in the presence of %%OPTION%%s in pkg-plist. [1]

  PR:		220950 [1]

Changes:
  head/emulators/wine-devel/Makefile
  head/emulators/wine-devel/pkg-plist
Comment 9 commit-hook freebsd_committer freebsd_triage 2020-03-11 18:51:54 UTC
A commit references this bug:

Author: gerald
Date: Wed Mar 11 18:51:07 UTC 2020
New revision: 528245
URL: https://svnweb.freebsd.org/changeset/ports/528245

Log:
  Enhance my hack to catch (more of the) false negatives of
  Mk/Scripts/check-stagedir.sh in the presence of options to
  also handle %%DATADIR%% in pkg-plist.

  This would have caught what I fixed with r525543 | gerald | 2020-02-08.

  PR:		220950

Changes:
  head/emulators/wine-devel/Makefile
Comment 10 commit-hook freebsd_committer freebsd_triage 2021-08-08 09:20:30 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=835981dbc56d281ba8cc9ac1433536f3e448491b

commit 835981dbc56d281ba8cc9ac1433536f3e448491b
Author:     Gerald Pfeifer <gerald@FreeBSD.org>
AuthorDate: 2021-08-08 09:19:40 +0000
Commit:     Gerald Pfeifer <gerald@FreeBSD.org>
CommitDate: 2021-08-08 09:19:40 +0000

    emulators/wine-devel: XAudio2 no longer depends on OpenAL

    Since upstream commit 3e390b1aafff47df63376a8ca4293c515d74f4ba on
    2019-02-20 XAudio2 uses FAudio (which already is an unconditional
    dependency) and no longer depends on OpenAL, so adjust the packing
    list accordingly. [1]

    We did not detect this via regular testing due to false negatives
    in Mk/Scripts/check-stagedir.sh in the presence of %%OPTION%%s in
    pkg-plist. [2]

    PR:             257651 [1], 220950 [2]
    Submitted by:   Alex S <iwtcex@gmail.com> [1]

 emulators/wine-devel/Makefile  |  2 +-
 emulators/wine-devel/pkg-plist | 92 +++++++++++++++++++++---------------------
 2 files changed, 47 insertions(+), 47 deletions(-)
Comment 11 commit-hook freebsd_committer freebsd_triage 2021-08-10 07:44:51 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=3f3a0267078b3438d5453b5a4bf230cf282ca0e0

commit 3f3a0267078b3438d5453b5a4bf230cf282ca0e0
Author:     Gerald Pfeifer <gerald@FreeBSD.org>
AuthorDate: 2021-08-10 07:43:27 +0000
Commit:     Gerald Pfeifer <gerald@FreeBSD.org>
CommitDate: 2021-08-10 07:43:27 +0000

    emulators/wine: XAudio2 no longer depends on OpenAL

    Since upstream commit 3e390b1aafff47df63376a8ca4293c515d74f4ba on
    2019-02-20 XAudio2 uses FAudio (which already is an unconditional
    dependency) and no longer depends on OpenAL, so adjust the packing
    list accordingly. [1]

    We did not detect this via regular testing due to false negatives
    in Mk/Scripts/check-stagedir.sh in the presence of %%OPTION%%s in
    pkg-plist. [2]

    PR:             257651 [1], 220950 [2]
    Submitted by:   Alex S <iwtcex@gmail.com> [1]

 emulators/wine/Makefile  |  2 +-
 emulators/wine/pkg-plist | 92 ++++++++++++++++++++++++------------------------
 2 files changed, 47 insertions(+), 47 deletions(-)
Comment 12 commit-hook freebsd_committer freebsd_triage 2021-11-02 06:52:50 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=e32f90cfd0b9ab6cc66e7f214e99dbc550575f0c

commit e32f90cfd0b9ab6cc66e7f214e99dbc550575f0c
Author:     Gerald Pfeifer <gerald@FreeBSD.org>
AuthorDate: 2021-11-02 06:27:04 +0000
Commit:     Gerald Pfeifer <gerald@FreeBSD.org>
CommitDate: 2021-11-02 06:27:04 +0000

    emulators/wine-devel: Account for bundled mpg123 library

    Since version 6.20 Wine bundles and unconditionally uses its own copy of
    the mpg123 library.

    Accordingly remove the MPG123 option and associated logic, unconditionally
    package associated files (Mk/Scripts/check-stagedir.sh missed those [1]),
    and bump PORTREVISION.

    PR:             220950 [1]

 emulators/wine-devel/Makefile  | 6 ++----
 emulators/wine-devel/pkg-plist | 8 ++++----
 2 files changed, 6 insertions(+), 8 deletions(-)