Bug 162709

Summary:	DISABLE_MAKE_JOBS should be the default
Product:	Ports & Packages	Reporter:	sgk
Component:	Individual Port(s)	Assignee:	Chris Rees <crees>
Status:	Closed FIXED
Severity:	Affects Only Me
Priority:	Normal
Version:	Latest
Hardware:	Any
OS:	Any

Description sgk 2011-11-21 01:30:13 UTC

It seems that many ports set MAKE_JOBS_SAFE without regards
to the amount of memory or the underlying compiler being
used.

By default, all ports should use DISABLE this feature and a
user should have to specifically enable this use multiple
processor.

While building libxul, I've hit this following behavior:

last pid: 54447;  load averages:  1.10,  1.12,  0.93    up 0+03:14:46  17:16:50
75 processes:  2 running, 73 sleeping
CPU:  0.0% user, 50.0% nice,  3.3% system,  6.9% interrupt, 39.8% idle
Mem: 656M Active, 66M Inact, 236M Wired, 27M Cache, 110M Buf, 2140K Free
Swap: 1024M Total, 884M Used, 140M Free, 86% Inuse, 4780K In, 4100K Out

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
54404 root          1  71   10   107M 77792K CPU1    1   3:36 97.17% cc1plus
54369 root          1 -20   10   978M   606M swread  0   0:20  0.34% cc1plus
 1598 kargl         1  40    0   377M   261M select  0   4:31  0.00% Xorg

because g++ is a memory pig.

Comment 1 vsjcfm 2011-11-21 16:42:47 UTC

Just set MAKE_JOBS_NUMBER to 1 and forget about it.

Comment 2 sgk 2011-11-21 17:11:09 UTC

On Mon, Nov 21, 2011 at 06:42:47PM +0200, Sayetsky Anton wrote:
>
> Just set MAKE_JOBS_NUMBER to 1 and forget about it.
>

Where is MAKE_JOBS_NUMBER documented?  Not that this
is the important isssue.  The important issue is that
the default is a rather stupid choice for any application
that uses g++.  Given that many laptops have 2 cores
and limited memory, g++ can easily make the system
unusable while building the application.  This behavior
leads to swapping (for systems that actually are
confgured with a swap partition) and is certainly not
battery friend.

-- 
Steve

Comment 3 vsjcfm 2011-11-21 17:16:39 UTC

2011/11/21 Steve Kargl <sgk@troutmask.apl.washington.edu>:
> Where is MAKE_JOBS_NUMBER documented?
http://www.freebsd.org/releases/7.2R/relnotes-detailed.html
2.5 Ports/Packages Collection Infrastructure
The FreeBSD Ports Collection now supports multiple make(1) jobs in
some supported ports. This is automatically enabled when a port is
marked as MAKE_JOBS_SAFE and improves CPU utilization at the build
stage by passing an option -jX to the top level Makefile from the
vendor. The number X is set to the number of CPUs by default, and can
be set by users via a make(1) variable MAKE_JOBS_NUMBER. For more
details, see ports/Mk/bsd.port.mk.

So i think this in unusable PR.

PS. If you want to apply jobs number to specific ports, just use
.if ${.CURDIR
...
...
...
.endif
as documented in /usr/share/examples/etc/make.conf.

Comment 4 sgk 2011-11-21 17:39:45 UTC

On Mon, Nov 21, 2011 at 07:16:39PM +0200, Sayetsky Anton wrote:
> 2011/11/21 Steve Kargl <sgk@troutmask.apl.washington.edu>:
> > Where is MAKE_JOBS_NUMBER documented?
> http://www.freebsd.org/releases/7.2R/relnotes-detailed.html

That's hardly acceptable documentation (given that
I'm running FreeBSD-current) ...  

> 2.5 Ports/Packages Collection Infrastructure
> The FreeBSD Ports Collection now supports multiple make(1) jobs in
> some supported ports. This is automatically enabled when a port is
> marked as MAKE_JOBS_SAFE and improves CPU utilization at the build
> stage by passing an option -jX to the top level Makefile from the
> vendor.

This last sentence is flat out wrong if the system starts to swap.

> The number X is set to the number of CPUs by default, and can
> be set by users via a make(1) variable MAKE_JOBS_NUMBER. For more
> details, see ports/Mk/bsd.port.mk.

.., and bsd.port.mk isn't acceptable documentation either.

troutmask:kargl[204] zgrep JOB /usr/share/man/man7/ports.7.gz
troutmask:kargl[205] pwd
/usr/share/doc/en_US.ISO8859-1/books/handbook
troutmask:kargl[206] grep JOBS *.html
book.html:<dt><a id="ERESOURCES-CHARTERS-JOBS" name="ERESOURCES-CHARTERS-JOBS"></a><a
eresources.html:<dt><a id="ERESOURCES-CHARTERS-JOBS" name="ERESOURCES-CHARTERS-JOBS"></a><a

So, all of the JOB knobs are undocumented.

-- 
Steve

Comment 5 Chris Rees freebsd_committer

2011-11-21 18:27:23 UTC

Responsible Changed
From-To: freebsd-ports-bugs->crees

I'll take it.

Comment 6 Chris Rees freebsd_committer

2011-11-21 18:30:41 UTC

State Changed
From-To: open->feedback

Where would you suggest to best put documentation for MAKE_JOBS_NUMBER? 
Personally I'd report it as a bug if the port fails to build, but I'm 
not quite sure I understand why it's such a big problem in your 
example...  Of course, compiling on a laptop is hardly normal use as 
well.

Comment 7 vsjcfm 2011-11-21 18:38:31 UTC

If DISABLE_MAKE_JOBS will be switched on, how we can enable parallel
make? I think many users doesn't know about this "undocumented"
option, and will never switch it on. So the presence of make jobs
becomes almost totally useless, isn't it?

Comment 8 Chris Rees freebsd_committer

2011-11-21 20:12:31 UTC

State Changed
From-To: feedback->closed

OK, you have several valid points here.  I would respectfully submit 
however, that a much better place for this discussion would be the 
ports@ mailing list; what you are describing is more of a 
wishlist/large-scale problem rather than a small problem with a specific 
solution.  I look forward to participating in the conversation; feel 
free to reference this PR in it.