With the use of machines with multiple cores, being able to use "make -j" for ports becomes more and more desirable. Work has been done to allow for individual ports to be built with "make -j", however making multiple ports at once can cause problems. Problems arise when multiple ports try to build/install the same dependency at the same time. So I've worked on a patch to Mk/bsd.port.mk that will use the lockf(1) command to lock the port's Makefile before doing any of the seven major targets (fetch, extract, patch, configure, build, install and package). This method will cause the 2nd make (or 3rd, etc) to block it's make while another is running. It does cause the two to swap between being blocked and doing the work, but at least it's safe in that only one of them is doing the work. It adds "WITH_PORT_LOCKING" and "WITHOUT_PORT_LOCKING" variables to control whether to do locking and "LOCKF_CMD" that does the actual locking. I've chosen the port's Makefile to control the locking of the port as every port has one and it allows for multiple logins/xterms to do makes with locking. Fix: The attached patch to Mk/bsd.port.mk contains the magic for port locking. Include it if its decided it's good enough. Patch attached with submission follows:
Responsible Changed From-To: freebsd-ports-bugs->portmgr portmgr territory.
Sorry, but the diff I uploaded is missing a character... The last hunk is the offending one. I don't know how it slipped through, as I made sure it was working before making the patch file. The line if [ -d "$$dir" -a X"${WITH_PORT_LOCKING}" != "X" -a X"${WITHOUT_PORT_LOCKING" = "X" ]; then \ should be if [ -d "$$dir" -a X"${WITH_PORT_LOCKING}" != "X" -a X"${WITHOUT_PORT_LOCKING}" = "X" ]; then \ ie, there was a missing '}' character at the end of WITHOUT_PORT_LOCKING. ivan. -- Ivan Brawley
Infrastructure PR.
This locking mechanism is not good enough, first it locks the Makefile which may be readonly, while it should lock the target, imho it will need a plain lock file in the WRKDIR directory.
Two points... (1) The lockf command appears to still work if the Makefile is read-only. I've just tried two tests. One was using the lockf command as a regular user that can't write to the Makefile and by having my /usr/ports as a ZFS filesystem and setting the readonly flag to on. Both of these test cases, the lockf command was able to be paused until the other completed. (2) This patch was made 3.5 years ago, and a lot has changed in the way that ports operates. I would be surprised if the patch can still be applied without pain, so I'll need to revisit the whole patch anyway if it would be worth while.
I do like the general idea of this patch, I'm sorry it took so long for one to actually have a look at it, if you refresh your patch I'll try to make sure this time it does not take that long :)
OK, I'll take another look at the way ports are now built and come up with a new patch. May take a little while. I'm open to suggestions to how this should work (inc. using a lock file in WRKDIR as that may help out on other edge cases). Any other suggestions?
Is this still relevant? Shall we perhaps close this PR?