Bug 241940 - ports-mgmt/portmaster looping on checking dependencies when using ccache
Summary: ports-mgmt/portmaster looping on checking dependencies when using ccache
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Stefan Eßer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-11-13 09:24 UTC by korsani
Modified: 2020-08-03 21:45 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (se)


Attachments
My make.conf (456 bytes, text/plain)
2019-11-17 13:47 UTC, korsani
no flags Details
pm.log (171.70 KB, application/x-xz)
2020-02-28 21:32 UTC, korsani
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description korsani 2019-11-13 09:24:59 UTC
Hi,

I recently upgraded from 12.0 to 12.1. 
When I do portmaster -af, it loops like this :

===>>> Currently installed version: ccache-3.5.dev.20180926_1                                                                                                             
===>>> Port directory: /usr/ports/devel/ccache                                                                                                                            
                                                                                                                                                                          
===>>> Starting check for build dependencies                                                                                                                              
===>>> Gathering dependency list for devel/ccache from ports                                                                                                              
===>>> Forcing update for /usr/ports/databases/libmemcached                                                                                                               
===>>> Launching child to reinstall libmemcached-1.0.18_7                                                                                                                 
                                                                                                                                                                          
===>>> exim-ldap2-4.92.3 1/21 >> libiconv-1.14_11 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.
0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_
3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 (18/83)                                  
                                                                                                                                                                          
===>>> Currently installed version: libmemcached-1.0.18_7                                                                                                                 
===>>> Port directory: /usr/ports/databases/libmemcached                                                                                                                  
                                                                                                                                                                          
===>>> Starting check for build dependencies                                                                                                                              
===>>> Gathering dependency list for databases/libmemcached from ports                                                                                                    
===>>> Forcing update for /usr/ports/devel/autoconf                                                                                                                       
===>>> Launching child to reinstall autoconf-2.69_3                                                                                                                       
                                                                                                                                                                          
===>>> exim-ldap2-4.92.3 1/21 >> libiconv-1.14_11 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.
0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_
3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 (19/83)               
                                                                                                                                                                          
===>>> Currently installed version: autoconf-2.69_3                                                                                                                       
===>>> Port directory: /usr/ports/devel/autoconf                                                                                                                          
                                                                                                                                                                          
===>>> Starting check for build dependencies                                                                                                                              
===>>> Gathering dependency list for devel/autoconf from ports                                                                                                            
===>>> Forcing update for /usr/ports/devel/ccache                                                                                                                         
===>>> Launching child to reinstall ccache-3.5.dev.20180926_1                                                                                                             
                                                                                                                                                                          
===>>> exim-ldap2-4.92.3 1/21 >> libiconv-1.14_11 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.
0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_
3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.2018
0926_1 (20/83)                                                                                                                                                            
                                                                                                                                                                          
===>>> Currently installed version: ccache-3.5.dev.20180926_1                                                                                                             
===>>> Port directory: /usr/ports/devel/ccache



And so on.

It's the case for some packages :exim-ldap2, mariadb103-server, ...

And it's also the case when, for example : portmaster -f $(pkg shlib -qR libperl.so.5.28)

The only workaround is commenting
WITH_CCACHE_BUILD=yes

in /etc/make.conf

Which ruin one of the major reason to use ccache: building to whole system :)
Comment 1 Walter Schwarzenfeld freebsd_triage 2019-11-13 16:08:30 UTC
Ccache Makefile:

.if defined(WITH_CCACHE_BUILD) && empty(OPTIONS_SLAVE:MMEMCACHED) && \
    !defined(NO_CCACHE_DEPEND)
MEMCACHED_IGNORE=               MEMCACHED cannot be combined with WITH_CCACHE_BUILD.  Use devel/ccache-memcached
# XXX: This needs more testing with Poudriere before enabling. Also bsd.options.mk support.
#MEMCACHED_DEPENDS_ARGS+= NO_CCACHE_DEPEND=1
.endif
Comment 2 Walter Schwarzenfeld freebsd_triage 2019-11-13 16:11:31 UTC
And set in /etc/make.conf:

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
.endif
Comment 3 korsani 2019-11-17 12:14:14 UTC
Hi,

I already added the part of the Makefile.

I added the make.conf stuff.

I test, I keep you informed.
Comment 4 korsani 2019-11-17 12:47:23 UTC
Hi,

It does not work.

Extract :

===>>> Starting check for build dependencies                                                                                                                              
===>>> Gathering dependency list for databases/libmemcached from ports                                                                                                    
===>>> Forcing update for /usr/ports/devel/autoconf                                                                                                                       
===>>> Launching child to reinstall autoconf-2.69_3                                                                                                                       
                                                                                                                                                                          
===>>> All >> boost-jam-1.71.0 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 >> ccache-3.5.dev.20180926_1 >> libmemcached-1.0.18_7 >> autoconf-2.69_3 (28/346)
Comment 5 Walter Schwarzenfeld freebsd_triage 2019-11-17 13:31:35 UTC
You have to build it with make NO_CCACHE=yes.
Comment 6 korsani 2019-11-17 13:43:41 UTC
Hi, 

I understand that I have to build with NO_CCACHE=yes.
But the point is that the procedure to upgrade freebsd is portmaster -af (when you have portmaster) at one point. As it could take a long time, depending on the packages you have, I don't stay in frond of my computer.
The fact is that if I don't look at the execution, it loops (and I suppose it can make freebsd crash)
Nowhere is written that when I use ccache I have to put NO_CCACHE=yes when I upgrade.
So, upgrading freebsd whith ccache enabled is not as straightforward as it seems to be.

Maybe ccache package should say "add this to make.conf" ? Or "do not portmaster -af with ccache enabled" ?
Comment 7 Walter Schwarzenfeld freebsd_triage 2019-11-17 13:45:45 UTC
Sorry, my last comment was nonsense.

Please attach your /etc/make.conf.
Comment 8 korsani 2019-11-17 13:47:50 UTC
Created attachment 209205 [details]
My make.conf

Hi,

My make.conf
Comment 9 Walter Schwarzenfeld freebsd_triage 2019-11-17 14:18:17 UTC
Nothing bad in it.

You have set in devel/ccache MEMCACHED=off. If you want memcache, you have to use devel/ccache-memcached.
Comment 10 Stefan Eßer freebsd_committer freebsd_triage 2020-01-26 18:36:02 UTC
I do not know whether this is a problem in CCACHE or in portmaster, and I have never used CCACHE myself.

I do understand that you have installed CCACHE to use it for all your compiler runs, but I do not have spare time to test why this does not work for you.

If you provide me with more debug information, I might be able to identify the source of this issue, but I do not have the spare time to reproduce your setup on my system.

If you want to provide debug traces:

bash -x /usr/local/sbin/portmaster <OPTIONS> > /tmp/pm.log 2>&1 & tail -F /tmp/pm.log

with options that reproduce the issue and make the compressed trace file "pm.log" available to me (I'd suggest to compress the file with "xz"). 

Please use "bash -x" since the trace file will contain more readable trace information than produced by "/bin/sh -x".
Comment 11 korsani 2020-02-28 21:32:46 UTC
Created attachment 212034 [details]
pm.log
Comment 12 korsani 2020-02-28 21:35:46 UTC
Hi there,

I made the pm.log (see file attached).

I let 'portmaster -af' run, and the loop exhausted all the memory !
Comment 13 korsani 2020-08-03 21:30:38 UTC
Hi there,

Upgrading from perl 5.30 to 5.32, thus invoking portmaster -f `pkg shlib -qR libperl.so.5.30` make it loop...
Comment 14 korsani 2020-08-03 21:30:57 UTC
Hi there,

Upgrading from perl 5.30 to 5.32, thus invoking portmaster -f `pkg shlib -qR libperl.so.5.30` make it loop...
Comment 15 korsani 2020-08-03 21:45:13 UTC
And adding 

.if ${.CURDIR:M*/ports/devel/ccache-memcached}
  NO_CCACHE= yes
.else
  WITH_CCACHE_BUILD=yes
.endif

Does nothing