Bug 43941

Summary:	document the Rationale for Upgrade Sequence (e.g. why the "historical" build sequence is deprecated)
Product:	Documentation	Reporter:	kientzle
Component:	Books & Articles	Assignee:	Giorgos Keramidas <keramida>
Status:	Closed FIXED
Severity:	Affects Only Me
Priority:	Normal
Version:	Latest
Hardware:	Any
OS:	Any

Description kientzle 2002-10-11 22:00:07 UTC

Mark Linimon suggested that I "get this exact reasoning added to the official
docs, it's the clearest explanation I've seen yet."  This was in response to
a posting I made to freebsd-stable explaining some of the rationale behind the
buildworld/buildkernel/installkernel/installworld sequence.  I've attached the
text of my posting below.  I hope you find it useful.

Fix: 

The following is an edited version of
my post to FreeBSD-stable mailing
list on 11 Oct, 2002, in response to a question
about why the "old" build sequence was no longer
considered sufficient:

------------------------------------------

Problem:  You're currently running an old system, consisting
of old compiler, old kernel, old world, and old configuration files.
(By "world" here, I mean the core system binaries, libraries,
and programming files.  The compiler is part of "world", but
has a few special concerns, which is why I've named it separately
here.)  You have source code for a new system and want to compile
and install it over the old system.  How should this be done?

This problem is a bit more subtle than it might seem, and the
FreeBSD developers have found it necessary over the years to
change the recommended approach fairly dramatically as new
kinds of unavoidable dependencies come to light.  The following
describes the rationale behind the current update sequence
(being used by 5.0, for example).

Any successful update sequence must deal with the
following issues:

* The old compiler might not be able to compile
  the new kernel.  (Old compilers sometimes have bugs.)
  So, the new kernel should be built with the new compiler.
  In particular, the new compiler must be built before
  the new kernel is built.  This does not necessarily
  mean that the new compiler must be _installed_ before
  building the new kernel, however.

* The new world might rely on new kernel features.
  So, the new kernel must be installed before the new world
  is installed.

These first two issues are the basis for the
core buildworld/buildkernel/installkernel/installworld
sequence that I'll describe in a moment.  They
are not the only concerns, however:

* The old world might not run correctly on the new kernel,
  so you must install the new world immediately
  upon installing the new kernel.

* Some configuration changes must be done before
  the new world is installed, but others might break
  the old world.  Hence, two different configuration
  upgrade steps are generally needed.

* For the most part, the update process only replaces
  or adds files; existing old files are not deleted.
  In a few cases, this can cause problems.  As a result,
  the update procedure will sometimes specify certain files
  that should be manually deleted at certain steps.
  This may or may not be automated in the future.

These concerns have led to the following recommended
sequence.  Note that the detailed sequence for
particular updates may require additional steps,
but this core process should remain unchanged for some
time:

1) make buildworld

   This first compiles the new compiler and
   a few related tools, then uses the new compiler
   to compile the rest of the new world.  The result
   ends up in /usr/obj.

2) make buildkernel

   Unlike the older config/make approach,
   this uses the _new_ compiler residing in
   /usr/obj.  This protects you against compiler/kernel
   mismatches.

3) make installkernel

   Place the new kernel and kernel modules onto the disk.

4) Reboot to single user mode.

   Single user mode minimizes problems from
   updating software that's already running.
   It also minimizes any problems from
   running the old world on a new kernel.
   (Curiously, when moving from 4.x to 5.0,
   you're still running the old kernel here,
   because the new kernel is in a different
   place and the boot loader hasn't been
   updated yet.)

5) mergemaster -p

   This does some initial configuration file updates
   in preparation for the new world.

6) make installworld

   Copies the world from /usr/obj.
   You now have a new kernel and new world on disk.

7) mergemaster

   Now you can update the remaining configuration files,
   since you have a new world on disk.

8) reboot

   A full machine reboot is needed now
   to load the new kernel and new world
   with new configuration files.


Note that if you're upgrading from, say,
4.6.2 to 4.7, then this procedure may not
be absolutely necessary, since you're
unlikely to run into serious mismatches
between compiler/kernel/world/configuration.
The older approach of "make world" followed
by building and installing a new kernel
might work well enough for minor updates.

But, when upgrading across major releases,
people who don't follow this procedure
should expect some problems.

Also, note that many upgrades (4.x to 5.0,
for instance) may require specific additional
steps (renaming or deleting specific files
prior to installworld, for instance).  Read
the UPDATING file carefully, especially at
the end, where the currently recommended upgrade
sequence is spelled out.

This procedure has evolved over time
as the developers have found it impossible
to completely prevent certain kinds of
mismatch problems.  Hopefully, the current
procedure will remain stable for a long time.

Note that upgrading from 3.x or earlier
is a bit trickier; read UPDATING carefully.

Comment 1 Chris Pepper 2002-10-11 22:12:30 UTC

At 1:55 PM -0700 2002/10/11, Tim Kientzle wrote:

>  >Description:
>Mark Linimon suggested that I "get this exact reasoning added to the 
>official docs, it's the clearest explanation I've seen yet."  This 
>was in response to a posting I made to freebsd-stable explaining 
>some of the rationale behind the 
>buildworld/buildkernel/installkernel/installworld sequence.  I've 
>attached the text of my posting below.  I hope you find it useful.

>* The new world might rely on new kernel features.
>   So, the new kernel must be installed before the new world
>   is installed.

	It's worth mentioning that installing a new broken kernel is 
testable and fixable, while installing a new world is much uglier. 
This is another reason to installkernel, reboot, and installworld if 
nothing blows up.

>These concerns have led to the following recommended
>sequence.  Note that the detailed sequence for
>particular updates may require additional steps,
>but this core process should remain unchanged for some
>time:
>
>1) make buildworld
>
>    This first compiles the new compiler and
>    a few related tools, then uses the new compiler
>    to compile the rest of the new world.  The result
>    ends up in /usr/obj.
>
>2) make buildkernel
>
>    Unlike the older config/make approach,
>    this uses the _new_ compiler residing in
>    /usr/obj.  This protects you against compiler/kernel
>    mismatches.
>
>3) make installkernel
>
>    Place the new kernel and kernel modules onto the disk.
>
>4) Reboot to single user mode.
>
>    Single user mode minimizes problems from
>    updating software that's already running.
>    It also minimizes any problems from
>    running the old world on a new kernel.
>    (Curiously, when moving from 4.x to 5.0,
>    you're still running the old kernel here,
>    because the new kernel is in a different
>    place and the boot loader hasn't been
>    updated yet.)
>
>5) mergemaster -p
>
>    This does some initial configuration file updates
>    in preparation for the new world.

	mergemaster's documentation says it should run with -p before 
buildworld, so if this is intended to be "official", mergemaster's 
docco should be updated.

	From mergemaster(8):
>      -p          Pre-buildworld mode.  Compares only files known to be essen-
>                  tial to the success of {build|install}world, including
>                  /etc/make.conf.

>[www:~] root# mergemaster --help
>mergemaster version 1.6.2.14
>Usage: mergemaster [-scrvahipC] [-m /path]
>          [-t /path] [-d] [-u N] [-w N] [-D /path]
>Options:
>   -s  Strict comparison (diff every pair of files)
>   -c  Use context diff instead of unified diff
>   -r  Re-run on a previously cleaned directory (skip temproot creation)
>   -v  Be more verbose about the process, include additional checks
>   -a  Leave all files that differ to merge by hand
>   -h  Display more complete help
>   -i  Automatically install files that do not exist in destination directory
>   -p  Pre-buildworld mode, only compares crucial files
>   -C  Compare local rc.conf variables to the defaults
>   -m /path/directory  Specify location of source to do the make in
>   -t /path/directory  Specify temp root directory
>   -d  Add date and time to directory name (e.g., /var/tmp/temproot.1011.17.11)
>   -u N  Specify a numeric umask
>   -w N  Specify a screen width in columns to sdiff
>   -D /path/directory  Specify the destination directory to install files to


						Chris Pepper
-- 
Chris Pepper:               <http://www.reppep.com/~pepper/>
Rockefeller University:     <http://www.rockefeller.edu/>

Comment 2 kientzle 2002-10-11 23:00:30 UTC

The current UPDATING file

puts the 'mergemaster -p' step
after installkernel/reboot and before
installworld.

Warner Losh (who maintains UPDATING)
should be able to say whether UPDATING
or mergemaster(8) is correct.

Tim Kientzle

Re <http://www.freebsd.org/cgi/query-pr.cgi?pr=43941>,
Chris Pepper observes:

>     mergemaster's documentation says it should run with -p before 
> buildworld, so if this is intended to be "official", mergemaster's docco 
> should be updated.
> 
>     From mergemaster(8):
> 
>>      -p          Pre-buildworld mode.  Compares only files known to be 
>> essen-
>>                  tial to the success of {build|install}world, including
>>                  /etc/make.conf.

Comment 3 M. Warner Losh 2002-10-11 23:15:23 UTC

In message: <3DA749FE.8010407@acm.org>
            Tim Kientzle <kientzle@acm.org> writes:
: The current UPDATING file
: 
: puts the 'mergemaster -p' step
: after installkernel/reboot and before
: installworld.
: 
: Warner Losh (who maintains UPDATING)
: should be able to say whether UPDATING
: or mergemaster(8) is correct.

mergemaster -p isn't needed for buildworld, but is needed for
installworld.  One can run it before buildworld without ill effect, as
far as I know, but it isn't required.  /etc/make.conf is empty except
for comments by default, so updating it with mergemaster likely won't
make any difference.

Warner

Comment 4 Giorgos Keramidas freebsd_committer

2008-01-16 07:08:50 UTC

Responsible Changed
From-To: freebsd-doc->keramida

Grab PR; I'll try to integrate the text of Tim to the Handbook

Comment 5 dfilter service freebsd_committer

2009-02-01 04:25:31 UTC

keramida    2009-02-01 04:25:22 UTC

  FreeBSD doc repository

  Modified files:
    en_US.ISO8859-1/books/handbook/cutting-edge chapter.sgml 
  Log:
  Add an explanation of the reasons that buildworld, buildkernel,
  installkernel, reboot, ... is a good idea. Before describing
  every step in detail, we let the reader know "why" so that they
  can later appreciate the "how" a bit more, and feel less confused
  about the whole dance.
  
  PR:             docs/43941
  Submitted by:   kientzle
  Reviewed by:    trhodes
  
  Revision  Changes    Path
  1.240     +184 -3    doc/en_US.ISO8859-1/books/handbook/cutting-edge/chapter.sgml
_______________________________________________
cvs-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/cvs-all
To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"

Comment 6 Giorgos Keramidas freebsd_committer

2009-02-01 04:37:33 UTC

State Changed
From-To: open->closed

It took me a while, but I finally managed to SGMLify this. 
Committed as revision 1.240 of cutting-edge/chapter.sgml. 

Thanks! :)