Bug 20312

The problem is due to a incomplete fix for panics on machines 
where the BIOS doesn't set the physical IO APIC ids, cf. PR 18919.

msmith      2000/05/31 14:37:29 PDT

  Revision  Changes    Path
  1.117     +3 -8      src/sys/i386/i386/mp_machdep.c
  1.39      +19 -1     src/sys/i386/i386/mpapic.c
  1.52      +2 -1      src/sys/i386/include/smp.h


Section 3.6.6 of the Intel MP spec describes the operating system as
responsible for verifying that the IO APIC ids are unique and
acceptable.  If also contains a statement restricting the acceptable
ids to the lowest possible ids left over after local apic id assignment.
Thus the following patch might be applicable.

- Tor Egge

Index: mp_machdep.c
===================================================================
RCS file: /home/ncvs/src/sys/i386/i386/mp_machdep.c,v
retrieving revision 1.117
diff -u -r1.117 mp_machdep.c
--- mp_machdep.c	2000/05/31 21:37:24	1.117
+++ mp_machdep.c	2000/08/05 05:45:50
@@ -1013,6 +1029,42 @@
 	}
 }
 
+
+static void
+swap_apic_id(int oldid, int newid)
+{
+	int x;
+	int apic;
+	int oapic;
+
+
+	if (oldid == newid)
+		return;
+
+	printf("IOAPIC#%d assigned physical APIC ID %d\n",
+	       ID_TO_IO(oldid), newid);
+
+	/* Swap IOAPIC physical IDs in interrupt entries */
+	for (x = 0; x < nintrs; x++) {
+		if (io_apic_ints[x].dst_apic_id == oldid)
+			io_apic_ints[x].dst_apic_id = newid;
+		else if (io_apic_ints[x].dst_apic_id == newid)
+			io_apic_ints[x].dst_apic_id = oldid;
+	}
+
+	/* Swap IOAPIC physical IDs in IO_TO_ID and ID_TO_IO mappings */
+	apic = ID_TO_IO(oldid);
+	oapic = ID_TO_IO(newid);
+	if (oapic >= 0 && IO_TO_ID(oapic) == newid) {
+		ID_TO_IO(oldid) = oapic;
+		IO_TO_ID(oapic) = oldid;
+	} else {
+		ID_TO_IO(oldid) = -1;
+	}
+	ID_TO_IO(newid) = apic;
+	IO_TO_ID(apic) = newid;
+}
+
 /*
  * parse an Intel MP specification table
  */
@@ -1024,6 +1076,9 @@
 	int	bus_0 = 0;	/* Stop GCC warning */
 	int	bus_pci = 0;	/* Stop GCC warning */
 	int	num_pci_bus;
+	int	apic;		/* logical IO APIC ID */
+	int     new_physid;	/* Free physical IO APIC ID */
+	int	max_physid;	/* Max physical IO APIC ID */
 
 	/*
 	 * Fix mis-numbering of the PCI bus and its INT entries if the BIOS
@@ -1076,6 +1131,21 @@
 			}
 		}
 	}
+	/* Assign IOAPIC ids */
+	new_physid = 0;		/* Start with first possible APIC ID */
+	max_physid = 15;	/* physical APIC IDs are in range 0..15 */
+	for (apic = 0; apic < mp_napics; ++apic) { /* For all IO APICs */
+		/* Find next free APIC ID (not used by any CPU) */
+		while (new_physid <= max_physid &&
+		       ID_TO_CPU(new_physid) >= 0 &&
+		       CPU_TO_ID(ID_TO_CPU(new_physid)) == new_physid)
+			new_physid++;
+		
+		if (new_physid < max_physid) {
+			swap_apic_id(IO_TO_ID(apic), new_physid);
+			new_physid++;
+		}
+	}
 }
 
 
@@ -1602,7 +1672,6 @@
 {
 	int     ap_cpu_id;
 #if defined(APIC_IO)
-	u_int32_t ux;
 	int     io_apic_id;
 	int     pin;
 #endif	/* APIC_IO */

Comment 2 Sheldon Hearn 2000-08-07 09:41:37 UTC

State Changed
From-To: open->analyzed

Fixed in HEAD by tegge: 
rev 1.53 of src/sys/i386/include/smp.h 
rev 1.118 of src/sys/i386/i386/mp_machdep.c 
rev 1.40 of src/sys/i386/i386/mpapic.c 

Comment 3 Sheldon Hearn 2000-08-07 09:41:37 UTC

Responsible Changed
From-To: freebsd-bugs->tegge

Tor's reminder.

On Mon, Aug 07, 2000 at 01:43:17AM -0700, sheldonh@FreeBSD.org wrote:
> Synopsis: IO APIC problems
> 
> State-Changed-From-To: open->analyzed
> State-Changed-By: sheldonh
> State-Changed-When: Mon Aug 7 01:41:37 PDT 2000
> State-Changed-Why: 
> Fixed in HEAD by tegge:
> rev 1.53 of src/sys/i386/include/smp.h
> rev 1.118 of src/sys/i386/i386/mp_machdep.c
> rev 1.40 of src/sys/i386/i386/mpapic.c
> 
> 
> Responsible-Changed-From-To: freebsd-bugs->tegge
> Responsible-Changed-By: sheldonh
> Responsible-Changed-When: Mon Aug 7 01:41:37 PDT 2000
> Responsible-Changed-Why: 
> Tor's reminder.
> 
> http://www.freebsd.org/cgi/query-pr.cgi?pr=20312

Firstly, thanks again for taking a look at this, still doesn't work
though =(

I grabbed the newer versions from these files and stuck them in my
source tree, recompiled and it still panics - so I'm back to a non-smp
kernel again.

Just to check the versions.

steve@moose 5 0 % fgrep "\$FreeBSD" /sys/i386/include/smp.h /sys/i386/i386/mp_machdep.c /sys/i386/i386/mpapic.c    ~/  3:26pm
/sys/i386/include/smp.h: * $FreeBSD: src/sys/i386/include/smp.h,v 1.53 2000/08/06 00:04:02 tegge Exp $
/sys/i386/i386/mp_machdep.c: * $FreeBSD: src/sys/i386/i386/mp_machdep.c,v 1.118 2000/08/06 00:04:02 tegge Exp $
/sys/i386/i386/mpapic.c: * $FreeBSD: src/sys/i386/i386/mpapic.c,v 1.40 2000/08/06 00:04:03 tegge Exp $

I've just copied the smp.h into /usr/include/machine as well, but now I can't seem to
build anything at all, so it's possible I screwed up somewhere (although the newer
panic below does indicate I managed to build a kernel with the new versions at
least because the Changing APIC ID line only appears in the newer files I think..)

=-= Panic Information :

Changing APIC ID for IO APIC #0 from 0 to 16 on chip
panic: can't control IO APIC #0 ID, reg: 0x00000000
mp_lock = 0000000a; cpuid = 0; lapic.id = 01000000
Uptime: 0s

[ Machine freezed ]

Anyway, I'm just about to cvsup down a completely clean source tree. Any ideas
which I should go for - i.e. which might give me working SMP ? 

So... which should I try: HEAD, RELENG_4 (again) or wherever RELENG_3 is now ?

[I'm open to all ideas on this one, as I find using one CPU on this box for
what I need to do a little too slow for most things.]

Tia,

	Steve

Comment 5 Doug Barton 2000-10-31 07:18:47 UTC

State Changed
From-To: analyzed->closed

Originator confirms that that this problem is solved in 
RELENG_4/5.