Bug 194930

Summary: make index fails with "make_index: Circular dependency loop found: pulseaudio-0.9.23_3 depends upon itself."
Product: Ports & Packages Reporter: Thomas Schweikle <tps>
Component: Ports FrameworkAssignee: Port Management Team <portmgr>
Status: Closed Unable to Reproduce    
Severity: Affects Only Me CC: swills
Priority: ---    
Version: Latest   
Hardware: Any   
OS: Any   

Description Thomas Schweikle 2014-11-10 18:00:56 UTC
# cd /usr/ports
# make index

fails with "make_index: Circular dependency loop found: pulseaudio-0.9.23_3 depends upon itself." after "--- describe.x11-wm ---"

This is on:
# uname -a
FreeBSD test-sct-bf-muc.bfs.de 10.1-PRERELEASE FreeBSD 10.1-PRERELEASE #84 r274336: Mon Nov 10 06:09:35 CET 2014     root@test-sct-bf-muc.bfs.de:/usr/obj/usr/src/sys/TEST-SCT-BF-MUC  amd64

# svn info
Path: .
Working Copy Root Path: /usr/ports
URL: https://svn0.eu.freebsd.org/ports/head
Repository Root: https://svn0.eu.freebsd.org/ports
Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5
Revision: 372401
Node Kind: directory
Schedule: normal
Last Changed Author: robak
Last Changed Rev: 372401
Last Changed Date: 2014-11-10 16:49:38 +0100 (Mon, 10 Nov 2014)
Comment 1 John Marino freebsd_committer freebsd_triage 2014-11-14 10:31:05 UTC
still valid?  I would think there would break portsnap if it were happening universally.
Comment 2 John Marino freebsd_committer freebsd_triage 2014-11-19 09:58:02 UTC
silence is golden...
Comment 3 Thomas Schweikle 2014-11-19 13:28:15 UTC
Wasn't available for some time -- training course ...

It is the same again:
--- describe.x11-toolkits ---
--- describe.x11-wm ---
make_index: Circular dependency loop found: pulseaudio-0.9.23_3 depends upon itself.

What I did:
# cd /usr/ports
# rm -rf * .arcconfig .gitignore .svn
# svn co https://svn0.eu.freebsd.org/ports/head /usr/ports

Since this leads to an 1:1 copy if all would be OK theere should be no message like that ..., but:

# make index

leads to the above. Maybe a mirror problem with svn0.eu.freebsd.org?
Comment 4 John Marino freebsd_committer freebsd_triage 2014-11-19 13:49:16 UTC
(In reply to John Marino from comment #1)
> I would think there would break portsnap if it were happening universally.

You didn't address why we aren't seeing fallout across the board.  Doesn't everything depend on index?
Comment 5 John Marino freebsd_committer freebsd_triage 2014-11-19 14:04:24 UTC
I don't even know who I would assign this PR to.
Comment 6 John Marino freebsd_committer freebsd_triage 2014-11-19 14:05:57 UTC
my guess is you have something in make.conf that is affecting dependencies.  Something like make index should be done in clean chroot.

Just a guess though.
Comment 7 Thomas Schweikle 2014-11-19 14:17:09 UTC
To verify my assumtion it could be a mirror problem I did:
# cd /usr/ports
# rm -rf * .arcconfig .gitignore .svn
# svn co https://svn0.us-east.freebsd.org/ports/head /usr/ports
# make index
...
--- describe.x11-toolkits ---
--- describe.x11-wm ---
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

:-(

Next test:
# cd /usr/ports
# rm -rf * .arcconfig .gitignore .svn
# svn co https://svn0.us-west.freebsd.org/ports/head /usr/ports
# make index
...
--- describe.x11-toolkits ---
--- describe.x11-wm ---
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

:-(

Thus again back to svn0.eu.freebsd.org:
# cd /usr/ports
# rm -rf * .arcconfig .gitignore .svn
# svn co https://svn0.eu.freebsd.org/ports/head /usr/ports
# make index
...
--- describe.x11-toolkits ---
--- describe.x11-wm ---
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

:-(

Again.
Comment 8 Thomas Schweikle 2014-11-19 14:30:45 UTC
Since your guess:
# cat /etc/make.conf
DISABLE_SIZE=yes
#MAKE_JOBS_NUMBER=5

# PKGNG
WITH_PKGNG="YES"

#
SUBTHREADS=2
OPTIONS_UNSET=X11
BATCH=yo

#
PERL_VERSION=5.16.3

#
DEFAULT_VERSIONS=pgsql=9.3
--- eof

But this leads to an other suggestion: the machine running "make index" on has four processors. Since MAKE_JOBS_NUMBER is unset I could try to set it to one.
Did it in /etc/make.conf, then had a look at what is going on:
# ps axd
...
  887  1  Is     0:00.15 |       |-- /bin/csh
60899  1  I+     0:00.02 |       | `-- make index
60954  1  I+     0:00.02 |       |   `-- make /usr/ports/INDEX-10
60957  1  I+     0:00.00 |       |     `-- /bin/sh -e -c echo -n "Generating INDEX-10 - please wait..";  if [ "" != "" ]; then  exp
60959  1  I+     0:00.00 |       |       `-- /bin/sh -e -c echo -n "Generating INDEX-10 - please wait..";  if [ "" != "" ]; then  e
60960  1  S+     0:00.03 |       |         `-- make -j4 INDEX_TMPDIR=/tmp/index.x0rfIgkD BUILDING_INDEX=1 __MAKE_SHELL=/rescue/sh E
70208  1  I      0:00.00 |       |           |-- sh -ev
70211  1  I      0:00.02 |       |           | `-- make -B _databases.describe
70226  1  I      0:00.16 |       |           |   `-- make -B describe DIRPRFX=databases/
70297  1  S      0:00.14 |       |           |     `-- /rescue/sh -e -c for sub in R-cran-DBI R-cran-RMySQL R-cran-RSQLite R-cran-R
92800  1  S      0:00.01 |       |           |       `-- make -B describe
92817  1  RL     0:00.01 |       |           |         `-- make -V PORTVERSION -C /usr/ports/lang/python27
72828  1  I      0:00.00 |       |           |-- sh -ev
72829  1  I      0:00.02 |       |           | `-- make -B _devel.describe
72843  1  I      0:00.84 |       |           |   `-- make -B describe DIRPRFX=devel/
73219  1  S      0:00.12 |       |           |     `-- /rescue/sh -e -c for sub in 9base ChipmunkPhysics ElectricFence ORBit2 ORBit
92811  1  S      0:00.00 |       |           |       `-- make -B describe
92818  1  R      0:00.00 |       |           |         `-- /usr/local/sbin/pkg-static -v
86295  1  S      0:00.00 |       |           |-- sh -ev
86297  1  S      0:00.02 |       |           | `-- make -B _editors.describe
86313  1  S      0:00.05 |       |           |   `-- make -B describe DIRPRFX=editors/
86336  1  S      0:00.04 |       |           |     `-- /rescue/sh -e -c for sub in 2bsd-vi abiword abiword-docs aee aewan apel ased
92799  1  R      0:00.01 |       |           |       `-- make -B describe
88797  1  S      0:00.00 |       |           `-- sh -ev
88799  1  S      0:00.02 |       |             `-- make -B _emulators.describe
88813  1  S      0:00.03 |       |               `-- make -B describe DIRPRFX=emulators/
88833  1  S      0:00.02 |       |                 `-- /rescue/sh -e -c for sub in adamem advancemame advancemenu advancemess aftp
92805  1  S      0:00.00 |       |                   `-- make -B describe
92816  1  R      0:00.00 |       |                     `-- /usr/local/sbin/pkg-static version -t 1.3.8 1.3.8
...

This where the commands given:
# cd /usr/ports
# make index
...
--- describe.x11-toolkits ---
--- describe.x11-wm ---
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

Interestingly "make" ignores what I told it in "/etc/make.conf".
Comment 9 John Marino freebsd_committer freebsd_triage 2014-11-19 14:35:10 UTC
who do you want me to assign the PR to?
Obviously it's not a global issue.
Comment 10 Thomas Schweikle 2014-11-19 15:04:36 UTC
Since all tests on the quad-core system failed, I decided to have a look at a single core machine. Same setup, same OS, but only one core:

# cd /usrPorts
# rm -rf * .arcconfig .gitignore .svn
# svn co https://svn0.eu.freebsd.org/ports/head /usr/ports
# make index
...
#

:-) this works!!

Conclusion: "make index" has, since changes made to use multiple cores, problems generating the index. Can anyone confirm?
Comment 11 Thomas Schweikle 2014-11-19 15:10:34 UTC
Since it is something that has to do with general ports setup and maintenance, I'd say someone who has to do with ports setup and maintenance.

Some additional tests give a really clear result: if "make index" is called with option "-jX" with "X" set to anything else than "1" it will fail. An other try, this time with a single core system, but "-j4" (extraordinary slow) gave the awaited result: circular dependency error and no index generated.
Comment 12 John Marino freebsd_committer freebsd_triage 2014-11-19 15:13:11 UTC
try on your quad with DISABLE_MAKE_JOBS=yes set

And I wanted an actual name tbh.  I can add portmgr to the list but that will have varied results.
Comment 13 John Marino freebsd_committer freebsd_triage 2014-11-19 15:15:21 UTC
(In reply to John Marino from comment #12)
> try on your quad with DISABLE_MAKE_JOBS=yes set
> 
> And I wanted an actual name tbh.  I can add portmgr to the list but that
> will have varied results.

actually your make file commented out make jobs, your quad should be -j1 anyway.
Comment 14 John Marino freebsd_committer freebsd_triage 2014-11-19 15:22:57 UTC
(In reply to Thomas Schweikle from comment #8)
> But this leads to an other suggestion: the machine running "make index" on
> has four processors. Since MAKE_JOBS_NUMBER is unset I could try to set it
> to one.

oh, you already experimented.
make index uses multiple processors no matter what.
Comment 15 John Marino freebsd_committer freebsd_triage 2014-11-19 15:25:36 UTC
the variable is INDEX_JOBS

e.g. make index INDEX_JOBS=1
Comment 16 John Marino freebsd_committer freebsd_triage 2014-11-19 15:28:59 UTC
Bryan was the last person to have a non-prop commit.
I'll assign the PR to him.

Bryan, if this is wrong, just reassign it however you think is best.
Comment 17 Thomas Schweikle 2014-11-19 21:40:16 UTC
Tried additional tests.

Set "INDEX_JOBS=1", more than one core
-> Fails:
# make index
Generating INDEX-10 - please wait..
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

#


Set "INDEX_JOBS=1", one core
-> Fails:
# make index
Generating INDEX-10 - please wait..
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

#


Set "INDEX_JOBS=4", one core
-> Fails:
# make index
Generating INDEX-10 - please wait..
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

#


all at revision 372810!
As this is not really what I awaited for I'll do another view tests.
Comment 18 Bryan Drewery freebsd_committer freebsd_triage 2014-11-19 21:54:25 UTC
This would be a portsnap bug (if even a bug). Note I have not analyzed this at all. Assigning to portmgr@ for now.
Comment 19 Thomas Schweikle 2014-11-19 23:56:26 UTC
Tried building index for port revisions
372804
372817
372819
372821
372824
372825
372826

for all it was the same:
make_index: Circular dependency loop found: jackit-0.124.1 depends upon itself.

I've checked out the whole ports tree into a clean directory every time. This on five systems all at the same level: FreeBSD test-sct-bf-muc 10.1-STABLE FreeBSD 10.1-STABLE #93 r274683: Wed Nov 19 06:02:56 CET 2014     root@test-sct-bf-muc:/usr/obj/usr/src/sys/TEST-SCT-BF-MUC  amd64
Comment 20 Steve Wills freebsd_committer freebsd_triage 2016-04-13 03:39:47 UTC
Can't reproduce this issue and it's a bit old. Re-open if there's still a problem. Thanks!