Bug 22256

Summary: [RARE] cross-compiled static bins in /usr/obj/usr/src/i386 can cause Signal 4 during make installworld
Product: Base System Reporter: Keith Jones <keith>
Component: binAssignee: Marcel Moolenaar <marcel>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: 4.1.1-STABLE   
Hardware: Any   
OS: Any   

Description Keith Jones 2000-10-23 20:10:01 UTC
	I'm not really sure which category this belongs in; 'bin' seems
	to be the likeliest bet. Apologies if this is in error.

	To remake world on #2, it is first built on #1 as compliation time
	is far too long and available memory probably isn't up to the job.

	'make world' was previously run on machine #1 with CFLAGS options
	'-march=i686 -mcpu=i686', for obvious performance reasons.

	Before performing 'make buildworld' with machine #2 as the intended
	target system, these options were changed to '-march=i386 -mcpu=i386'.

	The installation is performed in the following stages (assume
	single-user mode for machine #2):

	1.  a 'make buildworld buildkernel KERNEL=DINO' is run on machine #1.
	2.  /usr/src and /usr/obj are NFS-mounted onto machine #2.
	3.  'make installkernel KERNEL=DINO' is run on machine #2.
	4.  'make installworld' is run on machine #2.

	In the case above (which is admittedly rare), this results in a
	Signal 4 (Illegal Instruction) during 'make installworld' when the
	'strip' command is called (notably by 'install with the '-s' option)
	as 'make installworld' will attempt to use the broken binaries in the
	/usr/obj/usr/src/i386 tree.

	I suspect that that is happening during build time is this: a number
	of static binaries in /usr/obj/usr/src/i386 are being linked with the
	(possibly older, or even incompatible) libraries resident in /usr/lib
	at the time the build took place, rather than the newly-compiled
	libraries in /usr/obj. I suspect therefore that this is a bug in the
	'make buildworld' process.

	This may affect other binaries used during the install process (though
	I must confess I didn't notice any problems with any of the others).
	It is not expected to affect any binaries on the target system once
	installation is complete, as the new binaries will be (dynamically
	or statically) linked with the correct libraries during the
	installation process (they are not statically linked during the
	build process).

Fix: 

The workaround is to copy the existing (static) 'strip' command
	from Machine #2's /usr/bin to /usr/obj/usr/src/i386/usr/bin prior
	to performing 'make installworld'.

	A permanent fix may be to include 'strip' in the list of files to be
	copied to '/tmp/install.NNN' (where NNN is the PID) during the install
	phase, and remove it from the list of static binaries built in
	/usr/obj/usr/src/i386 during the build phase.
How-To-Repeat: 
	See above. (You'll need two machines, I suspect.)
Comment 1 Johan Karlsson freebsd_committer freebsd_triage 2000-10-25 19:29:58 UTC
Responsible Changed
From-To: freebsd-bugs->marcel

Marcel, can you have a quick look at this and maybe add 'strip' 
to the copied programs in installworld.
Comment 2 marcel 2000-10-25 20:30:11 UTC
johan@FreeBSD.org wrote:
> 
> Synopsis: [RARE] cross-compiled static bins in /usr/obj/usr/src/i386 can cause Signal 4 during make installworld
> 
> Responsible-Changed-From-To: freebsd-bugs->marcel
> Responsible-Changed-By: johan
> Responsible-Changed-When: Wed Oct 25 11:29:58 PDT 2000
> Responsible-Changed-Why:
> Marcel, can you have a quick look at this and maybe add 'strip'
> to the copied programs in installworld.
> 
> http://www.freebsd.org/cgi/query-pr.cgi?pr=22256

There's no bug in the build process that I know of. The tools in
/usr/obj that are specifically compiled to run on the build machine need
to be linked against the libraries in /usr/lib. In this case the build
machine is incompatible with the install machine, simply because the
binaries and libraries were optimized for the build machine and by that
unusable on the install machine. This is just as bad as building on
Alpha and installing on IA-32.

To make the build process a bit more resistant, we can force bootstrap,
build and cross tools to be dynamicly linked instead of staticly linked.

We should not add strip to installworld. Think about what would happen
if we did a source upgrade on FreeBSD 2.2.5 and basicly install ELF over
aout. We would be using an aout strip on ELF files...

-- 
Marcel Moolenaar
  mail: marcel@cup.hp.com / marcel@FreeBSD.org
  tel:  (408) 447-4222
Comment 3 Marcel Moolenaar freebsd_committer freebsd_triage 2000-11-21 08:10:11 UTC
State Changed
From-To: open->closed

Mixing libraries and binaries on different processor models 
within the same architecture is only supported if the 
libraries and binaries are not optimized for the specific 
processors. Future enhancements that rebuilt tools necessary 
for installation should allow this to certain extend. No 
plans currently exist to add such enhancements. 

This PR is closed to avoid confusion. The underlying idea is 
good, but the suggested fix is not. Alternatives exist to 
build on machine A and install on machine B, without running 
anything on machine B (ie set DESTDIR to a NFS mounted FS and 
run the install on machine A). 

If you think this PR is closed in error, let me know.