Bug 241227

Summary: ports-mgmt/portmaster: Portmaster reinstalls dependencies that are are flavorized and already installed
Product: Ports & Packages Reporter: Yasuhiro Kimura <yasu>
Component: Individual Port(s)Assignee: Stefan Eßer <se>
Status: Closed FIXED    
Severity: Affects Some People CC: herbert, python, rhurlin, se, tatsuki_makino, w.schwarzenfeld
Priority: --- Keywords: needs-qa
Version: LatestFlags: koobs: maintainer-feedback? (se)
koobs: merge-quarterly?
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=243432
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227347
Attachments:
Description Flags
patch proposal for devel/cmake
none
and...
none
patch proposal for portmaster
none
patch proposal for devel/cmake
none
svn-diff-portmaster-flavors none

Description Yasuhiro Kimura freebsd_committer freebsd_triage 2019-10-13 15:18:35 UTC
When installing/reinstalling/upgrading devel/cmake portmaster always reinstall textproc/py-sphinx even if it is up to date.

----------------------------------------------------------------------
root@rolling-vm-freebsd1[2001]# portmaster cmake-3.15.4                                                                    ~

===>>> Currently installed version: cmake-3.15.4
===>>> Port directory: /net/freebsd/ports/head/devel/cmake

===>>> Launching 'make checksum' for devel/cmake in background
===>>> Gathering dependency list for devel/cmake from ports
===>>> Launching child to install textproc/py-sphinx

===>>> cmake-3.15.4 >> textproc/py-sphinx (1/1)

===>>> Currently installed version: py37-sphinx-1.6.5_2,1
===>>> Port directory: /net/freebsd/ports/head/textproc/py-sphinx

===>>> Launching 'make checksum' for textproc/py-sphinx in background
===>>> Gathering dependency list for textproc/py-sphinx from ports
===>>> Initial dependency check complete for textproc/py-sphinx

===>>> Continuing initial dependency check for devel/cmake
===>>> Initial dependency check complete for devel/cmake


===>>> cmake-3.15.4 >> (1)

===>>> The following actions will be taken if you choose to proceed:
        Re-install cmake-3.15.4
        Re-install py37-sphinx-1.6.5_2,1

===>>> Proceed? y/n [y] n


===>>> If you would like to upgrade or install some, but not
       all of the above try adding '-i' to the command line.
root@rolling-vm-freebsd1[2002]#
----------------------------------------------------------------------
Comment 1 Raphael Kubo da Costa freebsd_committer freebsd_triage 2019-10-13 15:27:07 UTC
This looks like an issue in portmaster, tentatively assigning to the port maintainer.
Comment 2 Tatsuki Makino 2019-10-14 07:36:13 UTC
Created attachment 208291 [details]
patch proposal for devel/cmake

(In reply to Raphael Kubo da Costa from comment #1)

Perhaps this is a well-reported bug. I also thought it was a bug in portmaster.
However, I think this is a bug on the devel/cmake side.
Comment 3 Tatsuki Makino 2019-10-14 07:45:46 UTC
Created attachment 208292 [details]
and...

These are also likely to happen... astro/geographiclib! Really? orz
Comment 4 Yasuhiro Kimura freebsd_committer freebsd_triage 2019-10-14 14:21:07 UTC
(In reply to Tatsuki Makino from comment #2)

But if I try "cd /usr/ports/devel/cmake && make" then build completes without reinstall textproc/py-sphinx. So please tell me why this is bug of devel/cmake rather than portmaster.
Comment 5 Tatsuki Makino 2019-10-14 23:57:50 UTC
Created attachment 208319 [details]
patch proposal for portmaster

(In reply to Yasuhiro KIMURA from comment #4)

There is no problem with the canonical method, but it is unkind to automation tools.

If we fix the portmaster side, probably here.
Comment 6 Tobias C. Berner freebsd_committer freebsd_triage 2019-10-17 19:17:48 UTC
To give some context:

This line in the cmake-doc Makefile was changed in r460715 to be flavored, and backed out again in r485766 -- the consesus at the time was, that just depending on the default py-versioned sphinx binary is the correct thing. See PR 227573 .
Comment 7 Yasuhiro Kimura freebsd_committer freebsd_triage 2019-10-22 00:51:05 UTC
(In reply to Tobias C. Berner from comment #6)

Thank you for information. Then the problem is bug of portmaster.
Comment 8 Tatsuki Makino 2019-10-23 04:48:41 UTC
Created attachment 208518 [details]
patch proposal for devel/cmake

(In reply to Tobias C. Berner from comment #6)

Can bug 227573 be solved this patch?

Is it cmake side? :)
Comment 9 Yasuhiro Kimura freebsd_committer freebsd_triage 2019-11-21 03:33:25 UTC
*** Bug 242086 has been marked as a duplicate of this bug. ***
Comment 10 Yasuhiro Kimura freebsd_committer freebsd_triage 2019-11-21 03:42:45 UTC
Recently devel/binutils is flavorized. and same problem happens when a port is marked LLD_UNSAFE. Following is an example of such case.

root@rolling-vm-freebsd2[1195]# pkg info -aq | grep binutils
binutils-2.33.1,1
root@rolling-vm-freebsd2[1195]# grep LLD_UNSAFE /usr/ports/archivers/arj/Makefile
LLD_UNSAFE=     yes
root@rolling-vm-freebsd2[1196]# portmaster archivers/arj

===>>> Port directory: /net/freebsd/ports/head/archivers/arj

===>>> Launching 'make checksum' for archivers/arj in background
===>>> Gathering dependency list for archivers/arj from ports
===>>> Launching child to install devel/binutils

===>>> archivers/arj >> devel/binutils (1/1)

===>>> Currently installed version: binutils-2.33.1,1
===>>> Port directory: /net/freebsd/ports/head/devel/binutils

===>>> Launching 'make checksum' for devel/binutils in background
===>>> Gathering dependency list for devel/binutils from ports
===>>> Initial dependency check complete for devel/binutils

===>>> Continuing initial dependency check for archivers/arj
===>>> Initial dependency check complete for archivers/arj


===>>> archivers/arj >> (1)

===>>> The following actions will be taken if you choose to proceed:
        Install archivers/arj
        Re-install binutils-2.33.1,1

===>>> Proceed? y/n [y] n


===>>> If you would like to upgrade or install some, but not
       all of the above try adding '-i' to the command line.
root@rolling-vm-freebsd2[1197]#
Comment 11 Herbert J. Skuhra 2020-01-11 08:30:00 UTC
In the following case the portmaster patch does not help:

% pkg version -vPL=
binutils-2.33.1,1                  <   needs updating (port has 2.33.1_1,1)

% sudo portmaster -a
[...]
===>>> All >> (4)
                                                   
===>>> The following actions will be taken if you choose to proceed:
        Upgrade binutils-2.33.1,1 to binutils-2.33.1_1,1                   
        Re-install gettext-tools-0.20.1_1
        Re-install gmake-4.2.1_3
        Re-install mpfr-4.0.2

Using the unpatched portmaster version only binutils is updated.
Comment 12 Walter Schwarzenfeld freebsd_triage 2020-01-19 05:19:23 UTC
(In reply to Tatsuki Makino from comment #5)
try this change in your patch

 	origin=$(dir_part "$1")
+	if [ "${1}" != "${origin}" ] ; then
 	flavor=$(flavor_part "$1")
+	else
+		unset FLAVOR
+		flavor=$(pm_make -C "${pd}/${origin}" -V FLAVOR)
+		export flavor $flavor
+	fi
 	pkgname_l=$(echo $($PKG_CMD query '%n-%v' $origin)) || return 1
 	# if multiple flavors registered then fall back to $FLAVOR if no flavor has been passed in $1
 	[ "${pkgname_l}" != "${pkgname_l#* }" ] && : ${flavor:=$FLAVOR}
Comment 13 Walter Schwarzenfeld freebsd_triage 2020-01-19 06:30:13 UTC
Created attachment 210856 [details]
svn-diff-portmaster-flavors

Tested with binutils, gcc9, cmake (should tested with more ports).
Comment 14 Yasuhiro Kimura freebsd_committer freebsd_triage 2020-01-19 06:58:40 UTC
(In reply to Walter Schwarzenfeld from comment #13)

I tested your patch with following ports and the problem didn't happen.

archivers/ajr (devel/binutils)
devel/cmake (textproc/py-sphinx)
devel/libunwind (devel/binutils)
Comment 15 Tatsuki Makino 2020-01-19 22:52:23 UTC
(In reply to Walter Schwarzenfeld from comment #12)

This may have a problem with the scope of $pd.
For example, does the origin received in $1 exist in the global $pd?
I think it is enough if it is only to prevent Re-install.
Comment 16 Kubilay Kocak freebsd_committer freebsd_triage 2020-01-20 02:18:49 UTC
Are bug 243432 and bug 22747 exactly duplicates of this issue? If so, please close them as duplicates (since this bug has the most comment / patches / analysis)

Future tip: newer bugs (by ID) should be closed duplicates of the oldest/earliest issue ID.
Comment 17 Kubilay Kocak freebsd_committer freebsd_triage 2020-01-20 02:54:01 UTC
(In reply to Kubilay Kocak from comment #16)

bug 22747 should have read bug 227347
Comment 18 commit-hook freebsd_committer freebsd_triage 2020-01-21 21:06:05 UTC
A commit references this bug:

Author: se
Date: Tue Jan 21 21:05:20 UTC 2020
New revision: 523745
URL: https://svnweb.freebsd.org/changeset/ports/523745

Log:
  Attempt to fix a problem that leads to repeated building of some ports

  There are build dependencies, that were compiled again and again even
  though the latest version was already installed.

  This fix is based on a patch suggested by Tatsuki Makino, with a small
  change to better comply with the existing style used in this program.

  An extended patch has been provided by Walter Schwarzenfeld, but in my
  tests it did not make a difference. If further testing shows a need for
  the additional commands suggested by Walter, I'll add them (or a variation)
  in a follow-up commit.

  PR:		241227, 242086, 243432
  Submitted by:	Tatsuki Makino
  Approved by:	antoine (implicit)

Changes:
  head/ports-mgmt/portmaster/Makefile
  head/ports-mgmt/portmaster/files/patch-portmaster
Comment 19 Walter Schwarzenfeld freebsd_triage 2020-01-21 22:35:19 UTC
Works with cmake, arj 

does not works with binutils
===>>> binutils-2.33.1_1,1 >> (6)

===>>> The following actions will be taken if you choose to proceed:
	Re-install binutils-2.33.1_1,1
	Re-install ccache-3.7.1
	Re-install gettext-runtime-0.20.1
	Re-install gettext-tools-0.20.1_1
	Re-install gmake-4.2.1_3
	Re-install gmp-6.1.2_1
	Re-install mpfr-4.0.2

all installed and up-to-date.
Comment 20 Tatsuki Makino 2020-01-21 22:58:52 UTC
I'm sorry, I made you misunderstand.

The problem with my comment #15 is also included in the attachment 208319 [details] patch I have posted. So I made it a "proposal".

And I don't think Submitted is me :)
Comment 21 Walter Schwarzenfeld freebsd_triage 2020-01-22 23:45:06 UTC
(In reply to Walter Schwarzenfeld from comment #19)
Thanks, works after ports r523835.
Comment 22 Tatsuki Makino 2020-01-23 02:30:06 UTC
portmaster-3.19_20 seems to be working properly now.
Thank you very much.