Bug 85346

Summary: PREEMPTION causes unstability in Alpha4000 SMP kernel
Product: Base System Reporter: Pegasus McCleaft <ken>
Component: alphaAssignee: freebsd-alpha (Nobody) <alpha>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: 6.0-BETA3   
Hardware: Any   
OS: Any   

Description Pegasus McCleaft 2005-08-27 13:00:43 UTC
	The default option and build of a kernel for the 6.0 BETA3 is
to have preemption enabled. This option seems to still cause problems on my
machine after a few hours of running. The problems range between kernel traping
to hard locking. If a kernel trap is presented, the usual message is
a illegal address load within the syncer. (normally 0xffff....ff)

Fix: 

Remove the "options PREEMPTION" from the config file, rebuild kernel.
I found that running the below script in another xterm keeps the machine
running long enough to build the new kernel:

	while true
	do
	sync
	echo -n "Sleeping.."
	sleep 5
	done
How-To-Repeat: 	Build and load any kernel with PREEMPTION enabled. Do large amounts
of disk IO and ethernet, wait for the trap message (about 1 hour)
Comment 1 Kris Kennaway freebsd_committer freebsd_triage 2005-09-21 07:46:32 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-alpha

Sounds alpha-specific
Comment 2 John Baldwin freebsd_committer freebsd_triage 2005-09-21 20:17:01 UTC
Hmm, I still don't run with PREEMPTION enabled on my DS20 on HEAD.  In my 
experience I haven't gotten any panics, just hard locks when I have enabled 
PREEMPTION and SMP on Alpha.  You can try testing this patch to see if it 
helps things at all though:

--- //depot/projects/smpng/sys/alpha/alpha/interrupt.c	2005/04/14 18:55:16
+++ //depot/user/jhb/preemption/alpha/alpha/interrupt.c	2005/04/14 19:32:16
@@ -427,6 +427,13 @@
 	atomic_add_long(i->cntp, 1);
 
 	/*
+	 * It seems that we need to return from an interrupt back to PAL
+	 * on the same CPU that received the interrupt, so pin the interrupted
+	 * thread to the current CPU until we return from the interrupt.
+	 */
+	sched_pin();
+
+	/*
 	 * Handle a fast interrupt if there is no actual thread for this
 	 * interrupt by calling the handler directly without Giant.  Note
 	 * that this means that any fast interrupt handler must be MP safe.
@@ -435,26 +442,18 @@
 	if ((ih->ih_flags & IH_FAST) != 0) {
 		critical_enter();
 		ih->ih_handler(ih->ih_argument);
-		/* XXX */
-		curthread->td_owepreempt = 0;
 		critical_exit();
-		return;
-	}
+	} else {
+		if (ithd->it_disable) {
+			CTR1(KTR_INTR,
+			    "alpha_dispatch_intr: disabling vector 0x%x",
+			    i->vector);
+			ithd->it_disable(ithd->it_vector);
+		}
 
-	if (ithd->it_disable) {
-		CTR1(KTR_INTR,
-		    "alpha_dispatch_intr: disabling vector 0x%x", i->vector);
-		ithd->it_disable(ithd->it_vector);
+		error = ithread_schedule(ithd);
+		KASSERT(error == 0, ("got an impossible stray interrupt"));
 	}
-
-	/*
-	 * It seems that we need to return from an interrupt back to PAL
-	 * on the same CPU that received the interrupt, so pin the interrupted
-	 * thread to the current CPU until we return from the interrupt.
-	 */
-	sched_pin();
-	error = ithread_schedule(ithd);
-	KASSERT(error == 0, ("got an impossible stray interrupt"));
 	sched_unpin();
 }
 
--- //depot/projects/smpng/sys/kern/kern_thread.c	2005/05/27 14:58:46
+++ //depot/user/jhb/preemption/kern/kern_thread.c	2005/05/27 19:03:12
@@ -955,9 +957,11 @@
 	mtx_assert(&sched_lock, MA_OWNED);
 	PROC_LOCK_ASSERT(p, MA_OWNED);
 	if (!P_SHOULDSTOP(p)) {
+		critical_enter();
 		while ((td = TAILQ_FIRST(&p->p_suspended))) {
 			thread_unsuspend_one(td);
 		}
+		critical_exit();
 	} else if ((P_SHOULDSTOP(p) == P_STOPPED_SINGLE) &&
 	    (p->p_numthreads == p->p_suspcount)) {
 		/*
@@ -992,9 +996,11 @@
 	 * to continue however as this is a bad place to stop.
 	 */
 	if ((p->p_numthreads != 1) && (!P_SHOULDSTOP(p))) {
+		critical_enter();
 		while ((td = TAILQ_FIRST(&p->p_suspended))) {
 			thread_unsuspend_one(td);
 		}
+		critical_exit();
 	}
 	mtx_unlock_spin(&sched_lock);
 }
--- //depot/projects/smpng/sys/kern/subr_sleepqueue.c	2005/09/15 19:40:43
+++ //depot/user/jhb/preemption/kern/subr_sleepqueue.c	2005/09/15 20:09:55
@@ -410,9 +410,10 @@
 	 * just return.
 	 */
 	if (td->td_sleepqueue != NULL) {
-		MPASS(!TD_ON_SLEEPQ(td));
 		mtx_unlock_spin(&sc->sc_lock);
 		mtx_lock_spin(&sched_lock);
+		MPASS(!TD_ON_SLEEPQ(td));
+		MPASS(!TD_IS_SLEEPING(td));
 		return;
 	}
 
--- //depot/projects/smpng/sys/vm/vm_glue.c	2005/05/27 14:58:46
+++ //depot/user/jhb/preemption/vm/vm_glue.c	2005/05/27 19:03:12
@@ -556,6 +556,7 @@
 			vm_thread_swapin(td);
 
 		PROC_LOCK(p);
+		critical_enter();
 		mtx_lock_spin(&sched_lock);
 		p->p_sflag &= ~PS_SWAPPINGIN;
 		p->p_sflag |= PS_INMEM;
@@ -570,6 +571,7 @@
 
 		/* Allow other threads to swap p out now. */
 		--p->p_lock;
+		critical_exit();
 	}
 #endif /* NO_SWAPPING */
 }

-- 
John Baldwin <jhb@FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org
Comment 3 John Baldwin freebsd_committer freebsd_triage 2010-11-03 13:28:05 UTC
State Changed
From-To: open->closed

To the best of my knowledge, the source of this was Alpha specific and not 
MI.  Since development of Alpha has ceased, this will not be fixed.