Recent build of devel/llvm60 on CURRENT host, poudriere CURRENT jail (1200061) fails due to the error shown below: [...] [4590/4596] cd /wrkdirs/usr/ports/devel/llvm60/work/.build/docs && /usr/local/bin/sphinx-build -b man -d /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/_doctrees-llvm-dwarfdump-man -q /wrkdirs/usr/ports/devel/llvm60/work/llvm-6.0.0.src/docs /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/man FAILED: docs/CMakeFiles/docs-llvm-dwarfdump-man cd /wrkdirs/usr/ports/devel/llvm60/work/.build/docs && /usr/local/bin/sphinx-build -b man -d /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/_doctrees-llvm-dwarfdump-man -q /wrkdirs/usr/ports/devel/llvm60/work/llvm-6.0.0.src/docs /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/man Exception occurred: File "/usr/local/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 17] File exists: '/wrkdirs/usr/ports/devel/llvm60/work/.build/docs/man' The full traceback has been saved in /tmp/sphinx-err-VHavaH.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks! [4591/4596] cd /wrkdirs/usr/ports/devel/llvm60/work/.build/docs && /usr/local/bin/sphinx-build -b man -d /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/_doctrees-dsymutil-man -q /wrkdirs/usr/ports/devel/llvm60/work/llvm-6.0.0.src/docs /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/man FAILED: docs/CMakeFiles/docs-dsymutil-man cd /wrkdirs/usr/ports/devel/llvm60/work/.build/docs && /usr/local/bin/sphinx-build -b man -d /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/_doctrees-dsymutil-man -q /wrkdirs/usr/ports/devel/llvm60/work/llvm-6.0.0.src/docs /wrkdirs/usr/ports/devel/llvm60/work/.build/docs/man Exception occurred: File "/usr/local/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 17] File exists: '/wrkdirs/usr/ports/devel/llvm60/work/.build/docs/man'
Is this reproducible?
(In reply to Brooks Davis from comment #1) No, not on all (CURRENT and 11.1-RELENG) machines I use. The problem is alos NOT present when compiling the port the only true way, via make ;-) I found out that even (on most recent CURRENT, host and poudriere jail) when the whole set of packages of our individual repository is deleted and recompiled due to a bump of the revision number of the jail, the problem persists. But the problem vanish into thin air when performing a "poudriere options ... -c devel/llvm60". Sometimes I wish I would have a knob which could provide such a thing like cleaning up the ports configuration and restore it as it has been set. I think this problem affects only a very small set of people, it might be random and seems to be a flaw in the bigger picture of how ports work. I close the PR ... Thanks for responding.
There have been races in the llcm document build infrastructure before. I wouldn't surprise if someone reintroduced one.
Created attachment 193077 [details] poudriere log ${WRKSRC}/docs/CMakeLists.txt: 113 if (${SPHINX_OUTPUT_MAN}) 114 add_sphinx_target(man llvm) 115 add_sphinx_target(man llvm-dwarfdump) 116 add_sphinx_target(man dsymutil) 117 endif() It generates three targes with the same output directory /usr/ports/devel/llvm60/work/.build/docs/man ${WRKDIR}/.build/build.ninja: build docs/CMakeFiles/docs-dsymutil-man: CUSTOM_COMMAND COMMAND = cd /usr/ports/devel/llvm60/work/.build/docs && /usr/local/bin/sphinx-build -b man -d /usr/ports/devel/llvm60/work/.build/docs/_doctrees-dsymutil-man -q /usr/ports/devel/llvm60/work/llvm-6.0.0.src/docs /usr/ports/devel/llvm60/work/.build/docs/man -- build docs/CMakeFiles/docs-llvm-man: CUSTOM_COMMAND COMMAND = cd /usr/ports/devel/llvm60/work/.build/docs && /usr/local/bin/sphinx-build -b man -d /usr/ports/devel/llvm60/work/.build/docs/_doctrees-llvm-man -q /usr/ports/devel/llvm60/work/llvm-6.0.0.src/docs /usr/ports/devel/llvm60/work/.build/docs/man -- build docs/CMakeFiles/docs-llvm-dwarfdump-man: CUSTOM_COMMAND COMMAND = cd /usr/ports/devel/llvm60/work/.build/docs && /usr/local/bin/sphinx-build -b man -d /usr/ports/devel/llvm60/work/.build/docs/_doctrees-llvm-dwarfdump-man -q /usr/ports/devel/llvm60/work/llvm-6.0.0.src/docs /usr/ports/devel/llvm60/work/.build/docs/man all these sphinx-build processes check/create the output directory: https://github.com/sphinx-doc/sphinx/blob/33fd46b2a5c994fd450da9b48e1d359eea34daf4/sphinx/application.py#L161 161 if not path.isdir(outdir): 162 logger.info('making output directory...') 163 os.makedirs(outdir) a workaround maybe Index: Makefile =================================================================== --- Makefile (revision 469090) +++ Makefile (working copy) @@ -358,6 +358,9 @@ ${PYTHON_PKGNAMEPREFIX}enum34>0:devel/py-enum34 .endif +post-configure-DOCS-on: + ${MKDIR} ${WRKDIR}/.build/docs/man + post-extract-CLANG-on: ${MV} ${WRKSRC_clang} ${PATCH_WRKSRC}/tools/clang
I believe this analysis is correct. The workaround might work, but isn't right. I've got a patch that I think does the right that that I'll commit shortly.
A commit references this bug: Author: brooks Date: Sun May 6 16:00:01 UTC 2018 New revision: 469235 URL: https://svnweb.freebsd.org/changeset/ports/469235 Log: Work around a race in sphinx. Create the target directory before calling sphinx to render content into it. Sphinx suffers from a race where it crashes with an exception if it tries to make a directory and someone else does first. PR: 227460 Reported by: ohartmann@walstatt.org MFH: 2018Q2 Changes: head/devel/llvm60/files/patch-cmake_modules_AddSphinxTarget.cmake
Thanks brooks@ :)
Hopefully fixed in llvm60 and llvm-devel. I've submitted the patch upstream as well https://reviews.llvm.org/D46503.