Bug 150581

Summary: [irq] Unknown error generates IRQ address decoding error
Product: Base System Reporter: Tyler S <tyler>
Component: armAssignee: freebsd-arm (Nobody) <freebsd-arm>
Status: Open ---    
Severity: Affects Only Me CC: imp
Priority: Normal    
Version: Unspecified   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
arm150581.patch none

Description Tyler S 2010-09-15 03:10:01 UTC
When executing "portsnap fetch", the machine (a guruplug server plus) floods its local console with:
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 46% of   63 MB  214 kBps 02m43s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 46% of   63 MB  213 kBps 02m43s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 46% of   63 MB  213 kBps 02m43s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 47% of   63 MB  213 kBps 02m41s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 47% of   63 MB  213 kBps 02m40s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 47% of   63 MB  213 kBps 02m39s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 47% of   63 MB  213 kBps 02m38s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 48% of   63 MB  212 kBps 02m38s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 48% of   63 MB  213 kBps 02m37s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 48% of   63 MB  213 kBps 02m35s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 49% of   63 MB  212 kBps 02m35s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 49% of   63 MB  212 kBps 02m34s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 49% of   63 MB  213 kBps 02m32s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 50% of   63 MB  214 kBps 02m30s
c86a862ec3012622d1fc4d0d450dcb2469828ffec71fd7 50% of   63 MB  214 kBps 02m28s
c86a862ec301262IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
IRQ ERR: Address decoding error
IRQ ERR: cause: 0x00000001
- snip -

and the machine becomes unresponsive, prompting a cold boot.

Kernel was built with the following patches applied to it:

--- /usr/src/sys/kern/vfs_mount.c.orig	2010-07-04 22:50:00.613726077 -0400
+++ /usr/src/sys/kern/vfs_mount.c	2010-07-05 12:11:09.986561693 -0400
@@ -1651,6 +1651,9 @@
 	int error, i, asked = 0;

 	options = NULL;
+
+	/* NASTY HACK: wait for USB sticks to appear */
+	pause("usbhack", hz * 10);

 	root_mount_prepare();

--- /usr/src/etc/rc.d/fsck.orig	2010-07-07 13:02:41.765255856 -0400
+++ /usr/src/etc/rc.d/fsck	2010-07-07 13:02:46.286575144 -0400
@@ -27,7 +27,16 @@
 		if checkyesno background_fsck; then
 			fsck -F -p
 		else
-			fsck -p
+			if checkyesno force_fsck; then
+				echo "Force fsck enabled"
+				for filesystem in ${force_fsck_list}
+				do
+					echo "Force check $filesystem"
+					fsck -y $filesystem
+				done
+			else
+				fsck -p
+			fi
 		fi

 		case $? in
--- /usr/src/contrib/bind9/lib/isc/arm/include/isc/atomic.h.orig	2010-08-04 02:02:01.194401084 -0400
+++ /usr/src/contrib/bind9/lib/isc/arm/include/isc/atomic.h	2010-08-04 02:04:53.462379414 -0400
@@ -49,26 +49,22 @@
 static inline isc_int32_t
 isc_atomic_cmpxchg(isc_int32_t *p, isc_int32_t cmpval, isc_int32_t val)
 {
-	register int done, ras_start;
+	register int done, ras_start  = 0xffff1004;

 	__asm __volatile("1:\n"
 	    "adr	%1, 1b\n"
-	    "mov	%0, #0xe0000004\n"
 	    "str	%1, [%0]\n"
-	    "mov	%0, #0xe0000008\n"
 	    "adr	%1, 2f\n"
-	    "str	%1, [%0]\n"
+	    "str	%1, [%0, #4]\n"
 	    "ldr	%1, [%2]\n"
 	    "cmp	%1, %3\n"
 	    "streq	%4, [%2]\n"
 	    "2:\n"
 	    "mov	%3, #0\n"
-	    "mov	%0, #0xe0000004\n"
 	    "str	%3, [%0]\n"
 	    "mov	%3, #0xffffffff\n"
-	    "mov	%0, #0xe0000008\n"
-	    "str	%3, [%0]\n"
-	    : "=r" (ras_start), "=r" (done)
+	    "str	%3, [%0, #4]\n"
+	    : "+r" (ras_start), "=r" (done)
 	    ,"+r" (p), "+r" (cmpval), "+r" (val) : : "memory");
 	return (done);
and, the following kernel configuration:

#
# Custom kernel for Marvell SheevaPlug devices.
#
# $FreeBSD: src/sys/arm/conf/SHEEVAPLUG,v 1.3 2010/06/13 13:28:53 raj Exp $
#

ident           SHEEVAPLUG
include         "../mv/kirkwood/std.sheevaplug"

options         SOC_MV_KIRKWOOD
makeoptions     MODULES_OVERRIDE=""

#makeoptions    DEBUG=-g                #Build kernel with gdb(1) debug symbols
makeoptions     WERROR="-Werror"

options         SCHED_4BSD              #4BSD scheduler
options         INET                    #InterNETworking
options         INET6                   #IPv6 communications protocols
options         FFS                     #Berkeley Fast Filesystem
options         NFSCLIENT               #Network Filesystem Client
options         NFSLOCKD                #Network Lock Manager
options         NFS_ROOT                #NFS usable as /, requires NFSCLIENT
#options        BOOTP
#options        BOOTP_NFSROOT
#options        BOOTP_NFSV3
#options        BOOTP_WIRED_TO=mge0

options         GEOM_PART_BSD
options         GEOM_PART_GPT
options         GEOM_PART_MBR
options         GEOM_LABEL

options         SOFTUPDATES     # Enable FF Soft updates support
options         MSDOSFS         # Enable MSDOS Filesystems
options         PROCFS          # Process Filesystem
options         PSEUDOFS        # Psuedo-filesystem support
#options                ZFS             # zfs module
#options                opensolaris     # support module for zfs
# Root fs on USB device
#options        ROOTDEVNAME=\"ufs:/dev/da0a\"

options         SYSVSHM                 #SYSV-style shared memory
options         SYSVMSG                 #SYSV-style message queues
options         SYSVSEM                 #SYSV-style semaphores
options         _KPOSIX_PRIORITY_SCHEDULING #Posix P1003_1B real-time extensions
options         MUTEX_NOINLINE
options         RWLOCK_NOINLINE
#options        NO_FFS_SNAPSHOT
#options        NO_SWAPPING

# extra
options SW_WATCHDOG
options IPFIREWALL
options IPFIREWALL_FORWARD
options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPFIREWALL_VERBOSE
options IPFIREWALL_VERBOSE_LIMIT=80
device ata
device atadisk # ATA disk drives
options ATA_STATIC_ID # Static device numbering

# Debugging
options         ALT_BREAK_TO_DEBUGGER
options         DDB
options         KDB

# Pseudo devices
device          random
device          pty
device          loop
device          md                      # Memory Disks
# Serial ports
device          uart

# Networking
device          ether
device          mge                     # Marvell Gigabit Ethernet controller
device          mii
device          e1000phy
device          bpf
options         HZ=1000
options         DEVICE_POLLING
device          vlan

# USB
options         USB_DEBUG       # enable debug msgs
device          usb
device          ehci
device          umass
device          scbus
device          pass
device          da

# Flattened Device Tree
options         FDT
options         FDT_DTB_STATIC
makeoptions     FDT_DTS_FILE=sheevaplug.dts

How-To-Repeat: run "portsnap fetch" on a guruplug server+ with the above configuration.
Comment 1 wthww 2010-09-15 10:17:14 UTC
Actually, this may not be an issue. I realized I applied patches meant for
-stable to -current. I will checkout the source again and rebuild to see
what happens.

On Tue, Sep 14, 2010 at 10:10 PM, <FreeBSD-gnats-submit@freebsd.org> wrote:

> Thank you very much for your problem report.
> It has the internal identification `arm/150581'.
> The individual assigned to look at your
> report is: freebsd-arm.
>
> You can access the state of your problem report at any time
> via this link:
>
> http://www.freebsd.org/cgi/query-pr.cgi?pr=150581
>
> >Category:       arm
> >Responsible:    freebsd-arm
> >Synopsis:       Unknown error generates IRQ address decoding error
> >Arrival-Date:   Wed Sep 15 02:10:01 UTC 2010
>
Comment 2 johny.mattsson+fbsd 2010-09-17 01:44:58 UTC
Hi Tyler,

The issue could be the same that I'm having on my PogoPlug (stripped down
version of the Sheevaplug essentially). I don't have a serial console on it,
so I haven't been able to get any actual error messages, but I do experience
the same hard hang when I have heavy USB I/O. I'm still working on resolving
this issue properly, but I have found a work-around. If rolling back those
patches didn't fix the problem for you, try limiting the amount of RAM the
kernel sees down to 128MiB. For me, going from the actual 256MiB down to
128MiB prevents the I/O lock-up and I have a stable system (I can in fact
push it up to 131MiB, but above that it's bad news).

To limit the RAM, open /usr/src/sys/boot/fdt/dts/sheevaplug.dts, find the
"memory" entry and reduce it to 0x08000000, then rebuild your kernel.

I'd be very interested in hearing whether this fixes your issue, as it's
been quite frustrating to troubleshoot this without seeing what goes on the
console when the hang occurs!

Regards,
/Johny
Comment 3 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:49 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped
Comment 4 Warner Losh freebsd_committer freebsd_triage 2019-01-07 05:31:38 UTC
The patch appears to be required to work around a quirk in the Marvell hardware. It has not been applied yet.