Bug 76515

Summary: [patch] misleading use of make -j flag in handbook
Product: Documentation Reporter: pete
Component: Books & ArticlesAssignee: Tom Rhodes <trhodes>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Latest   
Hardware: Any   
OS: Any   

Description pete 2005-01-20 23:20:01 UTC 
At:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html
It states that:
"It is now possible to specify a -j option to make which will cause it to spawn several simultaneous processes. This is most useful on multi-CPU machines. However, since much of the compiling process is IO bound rather than CPU bound it is also useful on single CPU machines."

After testing this out on a SMP system doing a:
$ make -jN buildworld
(when "N" ranges from 1 to 8) I found that compile times do *not* decrease after starting two make jobs (make -j2 buildworld).  This was also tested by others on Uniproc machines and they did not find a decrease in time after starting one make job (make -j1 buildworld).  Here are edited results on my SMP system:

building world with 1 jobs...

real    95m20.543s
user    83m18.550s
sys     13m20.120s

cleaning /usr/src....
building world with 2 jobs...

real    54m15.856s
user    84m42.337s
sys     16m39.216s

cleaning /usr/src....
building world with 3 jobs...

real    53m53.239s
user    85m12.189s
sys     17m13.039s

cleaning /usr/src....
building world with 4 jobs...

real    53m56.539s
user    85m22.767s
sys     17m22.433s

Fix: 

It seems that the -j flag is useful only in direct proportion to the amount
of CPUs available to the system.  Maybe the doc should be corrected to
relate this.
How-To-Repeat: Here is the script I used to produce these results:
#!/usr/local/bin/bash
MAX=8
for ((i=1; i <= MAX ; i++))
do
        echo "cleaning /usr/src...."
        sudo make clean > /home/pete/tmp/clean
        echo "building world with $i jobs..."
        time make -j$i buildworld > /home/pete/tmp/build_world_$i
        echo ""
        sleep 90
done
Comment 1 Giorgos Keramidas freebsd_committer freebsd_triage 2005-01-21 12:38:27 UTC 
On 2005-01-20 23:11, pete wright <pete@nomadlogic.org> wrote:
> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html
> states that:

> "It is now possible to specify a -j option to make which will cause
> it to spawn several simultaneous processes. This is most useful on
> multi-CPU machines. However, since much of the compiling process is
> IO bound rather than CPU bound it is also useful on single CPU
> machines."
>
> After testing this out on a SMP system doing a:
> $ make -jN buildworld
> (when "N" ranges from 1 to 8) I found that compile times do *not*
> decrease after starting two make jobs (make -j2 buildworld).
> This was also tested by others on Uniproc machines and they did not
> find a decrease in time after starting one make job [...]

There have been problems with running multiple make instances in the
past.  I'd probably argue for entirely removing the recommendation
for -j from the Handbook.

New users should never use it, because they don't be able to easily
troubleshoot broken builds.  Experienced users will probably know if
it's safe to use -j, by experimenting and reading relevant material in
the make(1) manpage.

- Giorgos
Comment 2 Tom Rhodes freebsd_committer freebsd_triage 2005-02-24 03:41:52 UTC 
Giorgos,

I'm going to remove that line unless you provide me a valid
reason not to.  :)

-- 
Tom Rhodes
Comment 3 Giorgos Keramidas freebsd_committer freebsd_triage 2005-03-02 01:08:12 UTC 
Tom Rhodes <trhodes@FreeBSD.org> wrote:
>  Giorgos,
>  I'm going to remove that line unless you provide me a valid
>  reason not to.  :)

Please, do.  Removing the suggestion for "make -j" is something I wouldn't
object to any day.  People who are experienced enough with the build of
FreeBSD will have no problem using -j, since they are already acquainted
with Make a lot.

What I would remove is shown below.  Feel free to edit/adopt this as needed:

%%%
Index: chapter.sgml
===================================================================
RCS file: /home/ncvs/doc/en_US.ISO8859-1/books/handbook/cutting-edge/chapter.sgml,v
retrieving revision 1.215
diff -u -r1.215 chapter.sgml
--- chapter.sgml	12 Jan 2005 23:56:24 -0000	1.215
+++ chapter.sgml	2 Mar 2005 01:05:02 -0000
@@ -960,29 +960,6 @@
 	<para>Run</para>
 
 	<screen>&prompt.root; <userinput>make buildworld</userinput></screen>
- 
-        <para>It is now possible to specify a <option>-j</option> option to
-          <command>make</command> which will cause it to spawn several
-          simultaneous processes.  This is most useful on multi-CPU machines.
-          However, since much of the compiling process is IO bound rather
-          than CPU bound it is also useful on single CPU machines.</para>
-
-	<para>On a typical single-CPU machine you would run:</para>
-	  
-	  <screen>&prompt.root; <userinput>make -j4 buildworld</userinput></screen>
-
-	<para>&man.make.1; will then have up to 4 processes running at any one
-	  time.  Empirical evidence posted to the mailing lists shows this
-	  generally gives the best performance benefit.</para>
-
-	<para>If you have a multi-CPU machine and you are using an SMP
-	  configured kernel try values between 6 and 10 and see how they speed
-	  things up.</para>
-
-	<para>Be aware that this is still somewhat experimental, and commits
-	  to the source tree may occasionally break this feature.  If the
-	  world fails to compile using this parameter try again without it
-	  before you report any problems.</para>
       </sect3>
       
       <sect3>
%%%
Comment 4 Tom Rhodes freebsd_committer freebsd_triage 2006-11-05 13:24:03 UTC 
State Changed
From-To: open->closed

Wording in the handbook has been changed to make the use of 
-j less dangerous.  Thanks!
Comment 5 Tom Rhodes freebsd_committer freebsd_triage 2006-11-05 13:24:03 UTC 
Responsible Changed
From-To: freebsd-doc->trhodes

Over to me.