Bug 25632

Summary: USB modem (umodem) may destroy the cfreelist queue
Product: Base System Reporter: kawai <kawai>
Component: kernAssignee: njl
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Unspecified   
Hardware: Any   
OS: Any   

Description kawai 2001-03-09 14:20:01 UTC
The cblock_alloc() function is a critical section. The priority level is
raised by spltty() before executing cblock_alloc().
The umodemreadcb() function in /usr/src/sys/dev/usb/umodem.c calls
cblock_alloc() via ttyinput() and putc().
Because the interrupt priority level of USB is defined as INTR_TYPE_BIO in
/usr/src/sys/pci/ohci_pci.c and /usr/src/sys/pci/uhci_pci.c, the interrupt
which calls umodemreadcb() isn't blocked when another process is executing
cblock_alloc() routine. This may destroy the cfreelist queue.

To reconfirm this problem, I put a simple detector into cblock_alloc() and
got a core dump. Because I used umodem by KLD module, we can't see the symbol
name of umodemreadcb() in the trace. The anonymous function on #13 should be
umodemreadcb(). Also, we can't see cblock_alloc() in the trace because
cblock_alloc() is a inline function.

--- kern/tty_subr.c-	Sat Oct  9 15:30:35 1999
+++ kern/tty_subr.c	Fri Mar  9 10:48:13 2001
@@ -93,17 +93,23 @@
  * Remove a cblock from the cfreelist queue and return a pointer
  * to it.
  */
+static int someone_here = 0;
+
 static __inline struct cblock *
 cblock_alloc()
 {
 	struct cblock *cblockp;
 
+	if (someone_here)
+		panic("cblock_alloc: Already someone is here\n");
+	someone_here = 1;
 	cblockp = cfreelist;
 	if (cblockp == NULL)
 		panic("clist reservation botch");
 	cfreelist = cblockp->c_next;
 	cblockp->c_next = NULL;
 	cfreecount -= CBSIZE;
+	someone_here = 0;
 	return (cblockp);
 }

-------- trace
#0  dumpsys () at ../../kern/kern_shutdown.c:469
469		if (dumping++) {
(kgdb) where
#0  dumpsys () at ../../kern/kern_shutdown.c:469
#1  0xc013062d in boot (howto=260) at ../../kern/kern_shutdown.c:309
#2  0xc01309c5 in panic (fmt=0xc01fbd54 "from debugger")
    at ../../kern/kern_shutdown.c:556
#3  0xc011c9b5 in db_panic (addr=-1071763772, have_addr=0, count=-1, 
    modif=0xc7198aa4 "") at ../../ddb/db_command.c:433
#4  0xc011c954 in db_command (last_cmdp=0xc0224278, cmd_table=0xc02240d8, 
    aux_cmd_tablep=0xc025e160) at ../../ddb/db_command.c:333
#5  0xc011ca1a in db_command_loop () at ../../ddb/db_command.c:455
#6  0xc011eb3f in db_trap (type=3, code=0) at ../../ddb/db_trap.c:71
#7  0xc01e2c6e in kdb_trap (type=3, code=0, regs=0xc7198bc0)
    at ../../i386/i386/db_interface.c:158
#8  0xc01eec3c in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = 16, 
      tf_edi = 7034930, tf_esi = 256, tf_ebp = -954627064, 
      tf_isp = -954627092, tf_ebx = -1071641856, tf_edx = 0, tf_ecx = 0, 
      tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1071763772, tf_cs = 8, 
      tf_eflags = 582, tf_esp = -1071532641, tf_ss = -1071650845})
    at ../../i386/i386/trap.c:569
#9  0xc01e2ec4 in Debugger (msg=0xc01fe7e3 "panic") at machine/cpufunc.h:64
#10 0xc01309bc in panic (fmt=0xc0200b00 "cblock_alloc: Already someone is here\n")
    at ../../kern/kern_shutdown.c:554
#11 0xc014974f in putc (chr=96, clistp=0xc087e900) at ../../kern/tty_subr.c:104
#12 0xc0144c3f in ttyinput (c=96, tp=0xc087e900) at ../../kern/tty.c:563
#13 0xc02cad1e in ?? ()
#14 0xc01a4cf9 in usb_transfer_complete (xfer=0xc0881580)
    at ../../dev/usb/usbdi.c:839
#15 0xc01a1e30 in ohci_process_done (sc=0xc0678000, done=97748944)
    at ../../dev/usb/ohci.c:1207
#16 0xc01a1c4a in ohci_intr1 (sc=0xc0678000) at ../../dev/usb/ohci.c:1065
#17 0xc01a1b41 in ohci_intr (p=0xc0678000) at ../../dev/usb/ohci.c:994
#18 0xc014994a in b_to_q (src=0xc7198d94 "./usr/local/X11R6/share\n\002", 
    amount=23, clistp=0xc0270a38) at ../../kern/tty_subr.c:109
#19 0xc01469a4 in ttwrite (tp=0xc0270a00, uio=0xc7198ed4, flag=8323073)
    at ../../kern/tty.c:1962
#20 0xc0147517 in ttywrite (dev=0xc02612d8, uio=0xc7198ed4, flag=8323073)
    at ../../kern/tty.c:2584
#21 0xc0166fb1 in spec_write (ap=0xc7198e8c)
    at ../../miscfs/specfs/spec_vnops.c:283
#22 0xc01916c4 in ufsspec_write (ap=0xc7198e8c)
    at ../../ufs/ufs/ufs_vnops.c:1863
#23 0xc0191c6d in ufs_vnoperatespec (ap=0xc7198e8c)
    at ../../ufs/ufs/ufs_vnops.c:2391
#24 0xc016308f in vn_write (fp=0xc095b3c0, uio=0xc7198ed4, cred=0xc0680600, 
    flags=0, p=0xc69dbd40) at vnode_if.h:363
#25 0xc013e5ed in dofilewrite (p=0xc69dbd40, fp=0xc095b3c0, fd=1, 
    buf=0x8052000, nbyte=24, offset=-1, flags=0) at ../../sys/file.h:163
#26 0xc013e4e2 in write (p=0xc69dbd40, uap=0xc7198f80)
    at ../../kern/sys_generic.c:328
#27 0xc01ef52a in syscall2 (frame={tf_fs = 47, tf_es = 47, tf_ds = 47, 
      tf_edi = 134553600, tf_esi = 134535856, tf_ebp = -1077937568, 
      tf_isp = -954626092, tf_ebx = 672043880, tf_edx = 134535856, 
      tf_ecx = 134535856, tf_eax = 4, tf_trapno = 0, tf_err = 2, 
      tf_eip = 672003256, tf_cs = 31, tf_eflags = 663, tf_esp = -1077937612, 
      tf_ss = 47}) at ../../i386/i386/trap.c:1150
#28 0xc01e35b5 in Xint0x80_syscall ()
#29 0x280dd132 in ?? ()
#30 0x280dd0a1 in ?? ()
#31 0x280da0d8 in ?? ()
#32 0x28085710 in ?? ()
#33 0x804aabc in ?? ()
#34 0x8049424 in ?? ()
#35 0x804b594 in ?? ()
#36 0x80491ab in ?? ()
(kgdb) quit

How-To-Repeat: 
For example, use umodem and sio at the same time.
Comment 1 Nick Hibma freebsd_committer freebsd_triage 2001-04-09 19:09:05 UTC
Responsible Changed
From-To: freebsd-bugs->n_hibma

USB devices will be the most heavy users of this I guess.:wq 
.
Comment 2 Joe Ondrechen 2002-04-02 08:55:59 UTC
I have been experiencing frequent "clist reservation botch" panics while 
using my USB modem, so I decided to try a simple extension of Kawai's 
detector -- i.e., a simple wait queue:

+static int someone_here = 0;

static __inline struct cblock *
cblock_alloc()
{
+	int i;
	struct cblock *cblockp;

+	while(someone_here) {
+	/* waste some time */
+	for (i = 0;i = 34223; i++)
+		;
+	}
+	someone_here = 1;
	cblockp = cfreelist;
	if (cblockp == NULL)
		panic("clist reservation botch");
	cfreelist = cblockp->c_next;
	cblockp->c_next = NULL;
	cfreecount -= CBSIZE;
+	someone_here = 0;
	return (cblockp);
}

This has worked for two hours under conditions that usually caused my system 
panic, i.e. surfing busy sites, pop-up, mutlitple browsers, and console 
windows. I will continue "stress-testing" for a few days, then perhaps look 
for a more elegant delay loop or timing funciton.
Comment 3 Joe Ondrechen 2002-04-02 08:56:58 UTC
I have been experiencing frequent "clist reservation botch" panics while 
using my USB modem, so I decided to try a simple extension of Kawai's 
detector -- i.e., a simple wait queue:

+static int someone_here = 0;

static __inline struct cblock *
cblock_alloc()
{
+	int i;
	struct cblock *cblockp;

+	while(someone_here) {
+	/* waste some time */
+	for (i = 0;i = 34223; i++)
+		;
+	}
+	someone_here = 1;
	cblockp = cfreelist;
	if (cblockp == NULL)
		panic("clist reservation botch");
	cfreelist = cblockp->c_next;
	cblockp->c_next = NULL;
	cfreecount -= CBSIZE;
+	someone_here = 0;
	return (cblockp);
}

This has worked for two hours under conditions that usually caused my system 
panic, i.e. surfing busy sites, pop-up, mutlitple browsers, and console 
windows. I will continue "stress-testing" for a few days, then perhaps look 
for a more elegant delay loop or timing funciton.
Comment 4 Joe Ondrechen 2002-04-02 11:00:23 UTC
Sorry about the bad code and double post. When my for loop logic is fixed, 
the failure mode simply shifts from the clist reservation botch to a page 
fault. I apologize for not realizing the obvious -- it if were simple to fix, 
it would have been fixed a long time ago.
Comment 5 njl freebsd_committer freebsd_triage 2003-08-22 05:12:05 UTC
Responsible Changed
From-To: n_hibma->njl

I'm interested in tracking this down and I do have a USB modem.  This is 
very similar to kern/37015.
Comment 6 njl freebsd_committer freebsd_triage 2003-10-27 06:31:33 UTC
State Changed
From-To: open->closed

I believe this has been fixed in all versions.