During batch builds of ports, I observe hangs in python for ports that use "waf". The process blocks on "usem"; there seems to be a deadlock between two python processes. Example output from "ps aldxHwww" 0 91282 33575 0 52 0 6280 824 wait I 0 0:00.07 | | | `-- make package WRKDIRPREFIX=/ports-work PKGFILE=/ports-packages/talloc-2.0.7.tbz BATCH=yes NODEPENDS=yes PACKAGE_BUILDING=yes 0 91387 91282 0 52 0 14636 580 wait I 0 0:00.00 | | | `-- [sh] 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.33 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91392 91389 0 52 0 93568 8468 usem I 0 0:00.00 | | | | `-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 piperd I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | |-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 0 91389 91387 0 52 0 152704 7676 usem I 0 0:00.00 | | | `-- python ./buildtools/bin/waf configure --mandir=/usr/local/man --jobs=16 I have observed this with devel/talloc and databases/tdb. It does not happen consistently. The child process can be killed and the build fails. This seems to be the same issue as ports/160717, I'm adding this report because of the "usem" information. How-To-Repeat: Build ports until it fails.
Responsible Changed From-To: freebsd-ports-bugs->freebsd-python Assign to python maintainers like the reerenced PR
Hi Jan, if you have python built with semaphores could you reinstall without them and see if you can still reproduce the problem? If you haven't, please do show: make -C /usr/ports/lang/python27 show-config -- Mel
Hi Mel, Sorry about taking so long to get back to you. I have python installed without WITH_SEM already. The build process is = logged and I have the complete system checked into our local Perforce = server. These are the arguments to exec on the last build: Executing, with stdin from /dev/null: arg[0] =3D '/usr/sbin/chroot' arg[1] =3D '/home/janm/p4/freebsd-image-std-2011.1/work/build-chroot' arg[2] =3D '/usr/bin/env' arg[3] =3D '-' arg[4] =3D = 'PATH=3D/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/= X11R6/bin' arg[5] =3D '/bin/sh' arg[6] =3D '-cxe' arg[7] =3D ' cd /usr/ports/lang/python27 env NO_MTREE=3D"true" make package = WRKDIRPREFIX=3D/ports-work = PKGFILE=3D'/ports-packages/python27-2.7.2_3.tbz' BATCH=3Dyes = NODEPENDS=3Dyes PACKAGE_BUILDING=3Dyes=20 ' I also had the output to stdout and stderr from that build and noticed = that the configure output was still showing that Posix semaphores were = in use. Looking at the makefile, I see this: post-patch: .if defined(WITH_SEM) .if ${OSVERSION} >=3D 701106 @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < = ${PATCHDIR}/extra-patch-setup.py .endif .endif That patch only disables HAVE_SEM_OPEN and HAVE_SEM_TIMEDWAIT when = WITH_SEM is not set on FreeBSD 6 and earlier. In the build log I see = (without WITH_SEM set): checking for sem_open... yes checking for sem_timedwait... yes The interesting test would be to explicitly disable HAVE_SEM_OPEN and = HAVE_SEM_TIMEDWAIT in the python build. I'll set up a test and see if we = have any builds hang. Have you discovered anything new since your email? Thanks, Jan Mikkelsen
Hi Jan, Are you still experiencing this issue? I note that the PR you mentioned, ports/160717 was followed-up in May 2013 and closed in June.
State Changed From-To: open->feedback Requesting reconfirmation of issue from submitter
Hi, I do not see this problem in our current 9.1 and 9.2 based builds. Regards, Jan. On 24 Oct 2013, at 11:49 am, Kubilay Kocak <koobs.freebsd@gmail.com> wrote: > Hi Jan, > > Are you still experiencing this issue? I note that the PR you mentioned, > ports/160717 was followed-up in May 2013 and closed in June.
State Changed From-To: feedback->closed Submitter reports issue resolved (via email). Likely fixed with ports/160717