Created attachment 221201 [details] Patch for biology/pooler/Makefile Hi, I got an email from pkg-fallout showing make attempted to create two copies of 32-64.h at the same time, which failed. Perhaps USES=gmake might avoid this, but I suppose the safest thing to set is MAKE_JOBS_UNSAFE=yes (diff attached).
Maintainer informed via mail
The only failures I'm seeing in pkg-fallout are on armv[67], e.g. https://docs.freebsd.org/cgi/getmsg.cgi?fetch=173847123+0+current/freebsd-pkg-fallout That's showing a failure to find omp.h. Did you get something different?
Ah, that's different from the email I got from pkg-fallout, which showed a permissions error after it tried to write to 32-64.h a second time when the first process had already marked it read-only. The make rule chmods it read-write, updates it, then chmods it read-only again, and the log showed that two identical copies of this same make rule were being run in parallel, with obvious race-conditions that wouldn't be reproduced every time. Since 32-64.h only needs to be made once, I don't know how make ended up deciding to run it in duplicate on two separate cores, but it's not suitable for that (even if we didn't have the read-only thing, there might still be issues on some filesystems if two processes try to write the same file simultaneously). omp.h is included only if _OPENMP is defined. For single-core machines we could drop the -fopenmp flag and it should then compile without OpenMP libraries, but it would be nice if OpenMP can work on armv6/7 as this architecture is supposed to support SMP and the later Raspberry Pi machines are quad-core. I don't know why omp.h is missing on that architecture; maybe this isn't something we can fix yet. But it would still be nice to eliminate the chance of having two copies of 32-64.h generated at the same time. thanks.
I see the problem since you sent the pkg-fallout log (off-list). I'm still not seeing it on the pkg-fallout web sites, which seems odd. It's an issue of how make handles multiple targets in the same rule: 128.h 32.h 32-64.h: 64.h 64-128.h chmod +w 128.h 32.h 32-64.h 2>/dev/null || true sed -e s/64/128/g < 64.h > 128.h sed -e s/64/32/g < 64.h > 32.h sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h chmod -w 128.h 32.h 32-64.h Both BSD make and gmake try to build them all in parallel by simply running the same build commands for each target, so switching to gmake won't help: BSD make: ===> Building for pooler-1.75 --- 32.h --- --- 32-64.h --- --- 128.h --- --- 32.h --- chmod +w 128.h 32.h 32-64.h 2>/dev/null || true --- 128.h --- chmod +w 128.h 32.h 32-64.h 2>/dev/null || true --- 32-64.h --- chmod +w 128.h 32.h 32-64.h 2>/dev/null || true sed -e s/64/128/g < 64.h > 128.h --- 32.h --- sed -e s/64/128/g < 64.h > 128.h --- 128.h --- sed -e s/64/128/g < 64.h > 128.h --- 32-64.h --- sed -e s/64/32/g < 64.h > 32.h --- 128.h --- sed -e s/64/32/g < 64.h > 32.h --- 32.h --- sed -e s/64/32/g < 64.h > 32.h --- 32-64.h --- sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h --- 32.h --- sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h --- 128.h --- sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h --- 32-64.h --- chmod -w 128.h 32.h 32-64.h --- 32.h --- chmod -w 128.h 32.h 32-64.h --- 128.h --- chmod -w 128.h 32.h 32-64.h --- pooler --- gcc9 -O2 -pipe -fopenmp -fstack-protector-strong -Wl,-rpath=/usr/local/lib/gcc9 -fno-strict-aliasing *.c -o pooler -lm gmake: ===> Building for pooler-1.75 gmake[2]: Entering directory '/usr/ports/wip/pooler/work/PrimerPooler-1.75/pooler' chmod +w 128.h 32.h 32-64.h 2>/dev/null || true chmod +w 128.h 32.h 32-64.h 2>/dev/null || true chmod +w 128.h 32.h 32-64.h 2>/dev/null || true sed -e s/64/128/g < 64.h > 128.h sed -e s/64/128/g < 64.h > 128.h sed -e s/64/128/g < 64.h > 128.h sed -e s/64/32/g < 64.h > 32.h sed -e s/64/32/g < 64.h > 32.h sed -e s/64/32/g < 64.h > 32.h sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h chmod -w 128.h 32.h 32-64.h chmod -w 128.h 32.h 32-64.h chmod -w 128.h 32.h 32-64.h gcc9 -O2 -pipe -fopenmp -fstack-protector-strong -Wl,-rpath=/usr/local/lib/gcc9 -fno-strict-aliasing -fopenmp *.c -o pooler -lm gmake[2]: Leaving directory '/usr/ports/wip/pooler/work/PrimerPooler-1.75/pooler' Would you consider the following patch as a permanent upstream fix? This will eliminate the need to disable make jobs while ensuring that each command is run only once. --- Makefile.orig 2021-01-07 19:49:26 UTC +++ Makefile @@ -61,12 +61,20 @@ clean: # For the generated 128.h etc, we keep them read-only as # an extra reminder that they're auto-generated (as well # as the explanatory comment at the top) -128.h 32.h 32-64.h: 64.h 64-128.h - chmod +w 128.h 32.h 32-64.h 2>/dev/null || true +128.h: 64.h 64-128.h + chmod +w 128.h 2>/dev/null || true sed -e s/64/128/g < 64.h > 128.h + chmod -w 128.h + +32.h: 64.h 64-128.h + chmod +w 32.h 2>/dev/null || true sed -e s/64/32/g < 64.h > 32.h + chmod -w 32.h + +32-64.h: 64.h 64-128.h + chmod +w 32-64.h 2>/dev/null || true sed -e s/64/32/g -e s/128/64/g < 64-128.h > 32-64.h - chmod -w 128.h 32.h 32-64.h + chmod -w 32-64.h # ----------------------------------------------------- # Rules you won't need unless releasing new versions:
Created attachment 221387 [details] Revised patch for biology/pooler Thanks, I've done that upstream (with minor dependency cleanup) now tagged as v1.76 and attached a revised patch.
A commit references this bug: Author: jwb Date: Fri Jan 8 22:44:14 UTC 2021 New revision: 560817 URL: https://svnweb.freebsd.org/changeset/ports/560817 Log: biology/pooler: Upgrade to 1.76 Fixes race condition for make jobs in upstream Makefile PR: 252352 Submitted by: silas-freebsd@flatline.org.uk Changes: head/biology/pooler/Makefile head/biology/pooler/distinfo
Thanks!