Bug 193367

Summary: [panic] sleeping thread
Product: Base System Reporter: sasamotikomi
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me CC: eugen, jhb, kib, rea, rene, sasamotikomi, stable
Priority: Normal    
Version: 9.1-RELEASE   
Hardware: i386   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195165

Description sasamotikomi 2014-09-06 08:14:35 UTC
panic: sleeping thread
Sleeping thread (tid 102916, pid 16913) owns a non-sleepable lock
KDB: stack backtrace of thread 102916:
#0 0xc0ac8ca9 at mi_switch+0x139
#1 0xc0afddee at sleepq_switch+0xce
#2 0xc0afe95e at sleepq_timedwait+0x3e
#3 0xc0ac92a2 at _sleep+0x2a2
#4 0xc0d1c306 at swap_pager_getpages+0x3e6
#5 0xc0d28291 at vm_fault_hold+0x1031
#6 0xc0d2993b at vm_fault+0x8b
#7 0xc0e2525f at trap_pfault+0x1df
#8 0xc0e260ea at trap+0x44a
#9 0xc0e0f6cc at calltrap+0x6
#10 0xc9fb0b5a at drm_ioctl+0x2ca
#11 0xc09a41aa at devfs_ioctl_f+0x10a
#12 0xc0b05140 at kern_ioctl+0x2a0
#13 0xc0b052b4 at sys_ioctl+0x134
#14 0xc0e257fa at syscall+0x34a
#15 0xc0e0f731 at Xint0x80_syscall+0x21
panic: sleeping thread
cpuid = 0
KDB: stack backtrace:
#0 0xc0af3b1f at kdb_backtrace+0x4f
#1 0xc0ac054f at panic+0x16f
#2 0xc0b01b46 at propagate_priority+0x106
#3 0xc0b02958 at turnstile_wait+0x2c8
#4 0xc0aae71c at _mtx_lock_sleep+0x1ec
#5 0xc0aaeaec at _mtx_lock_flags+0x4c
#6 0xc9fb3f8d at drm_lock+0xad
#7 0xc9fb0bfb at drm_ioctl+0x36b
#8 0xc09a41aa at devfs_ioctl_f+0x10a
#9 0xc0b05140 at kern_ioctl+0x2a0
#10 0xc0b052b4 at sys_ioctl+0x134
#11 0xc0e257fa at syscall+0x34a
#12 0xc0e0f731 at Xint0x80_syscall+0x21
Comment 1 sasamotikomi 2014-09-06 08:17:21 UTC
core.txt.12
FreeBSD 9.1-RELEASE-p4
Sleeping thread (tid 104117, pid 95818) owns a non-sleepable lock
KDB: stack backtrace of thread 104117:
#0 0xc0ac8ca9 at mi_switch+0x139
#1 0xc0afddee at sleepq_switch+0xce
#2 0xc0afe95e at sleepq_timedwait+0x3e
#3 0xc0ac92a2 at _sleep+0x2a2
#4 0xc0d1c306 at swap_pager_getpages+0x3e6
#5 0xc0d28291 at vm_fault_hold+0x1031
#6 0xc0d2993b at vm_fault+0x8b
#7 0xc0e2525f at trap_pfault+0x1df
#8 0xc0e260ea at trap+0x44a
#9 0xc0e0f6cc at calltrap+0x6
#10 0xcaaacb5a at drm_ioctl+0x2ca
#11 0xc09a41aa at devfs_ioctl_f+0x10a
#12 0xc0b05140 at kern_ioctl+0x2a0
#13 0xc0b052b4 at sys_ioctl+0x134
#14 0xc0e257fa at syscall+0x34a
#15 0xc0e0f731 at Xint0x80_syscall+0x21
panic: sleeping thread
cpuid = 0
KDB: stack backtrace:
#0 0xc0af3b1f at kdb_backtrace+0x4f
#1 0xc0ac054f at panic+0x16f
#2 0xc0b01b46 at propagate_priority+0x106
#3 0xc0b02958 at turnstile_wait+0x2c8
#4 0xc0aae71c at _mtx_lock_sleep+0x1ec
#5 0xc0aaeaec at _mtx_lock_flags+0x4c
#6 0xcaaaff8d at drm_lock+0xad
#7 0xcaaacbfb at drm_ioctl+0x36b
#8 0xc09a41aa at devfs_ioctl_f+0x10a
#9 0xc0b05140 at kern_ioctl+0x2a0
#10 0xc0b052b4 at sys_ioctl+0x134
#11 0xc0e257fa at syscall+0x34a
#12 0xc0e0f731 at Xint0x80_syscall+0x21
Comment 2 sasamotikomi 2014-09-06 08:28:57 UTC
core.txt.15
FreeBSD 9.1-RELEASE-p6
panic: sleeping thread
Unread portion of the kernel message buffer:
Sleeping thread (tid 100889, pid 3535) owns a non-sleepable lock
KDB: stack backtrace of thread 100889:
#0 0xc0ac8fe9 at mi_switch+0x139
#1 0xc0afe12e at sleepq_switch+0xce
#2 0xc0afec9e at sleepq_timedwait+0x3e
#3 0xc0ac95e2 at _sleep+0x2a2
#4 0xc0d1c686 at swap_pager_getpages+0x3e6
#5 0xc0d28611 at vm_fault_hold+0x1031
#6 0xc0d29cbb at vm_fault+0x8b
#7 0xc0e255df at trap_pfault+0x1df
#8 0xc0e2646a at trap+0x44a
#9 0xc0e0fa4c at calltrap+0x6
#10 0xc7fe7b5a at drm_ioctl+0x2ca
#11 0xc09a44ea at devfs_ioctl_f+0x10a
#12 0xc0b05480 at kern_ioctl+0x2a0
#13 0xc0b055f4 at sys_ioctl+0x134
#14 0xc0e25b7a at syscall+0x34a
#15 0xc0e0fab1 at Xint0x80_syscall+0x21
panic: sleeping thread
cpuid = 0
KDB: stack backtrace:
#0 0xc0af3e5f at kdb_backtrace+0x4f
#1 0xc0ac088f at panic+0x16f
#2 0xc0b01e86 at propagate_priority+0x106
#3 0xc0b02c98 at turnstile_wait+0x2c8
#4 0xc0aaea5c at _mtx_lock_sleep+0x1ec
#5 0xc0aaee2c at _mtx_lock_flags+0x4c
#6 0xc7feaf8d at drm_lock+0xad
#7 0xc7fe7bfb at drm_ioctl+0x36b
#8 0xc09a44ea at devfs_ioctl_f+0x10a
#9 0xc0b05480 at kern_ioctl+0x2a0
#10 0xc0b055f4 at sys_ioctl+0x134
#11 0xc0e25b7a at syscall+0x34a
#12 0xc0e0fab1 at Xint0x80_syscall+0x21
Comment 3 sasamotikomi 2014-09-06 08:33:54 UTC
core.txt.18
FreeBSD 9.1-RELEASE-p7
panic: sleeping thread
Sleeping thread (tid 100214, pid 93700) owns a non-sleepable lock
KDB: stack backtrace of thread 100214:
#0 0xc0ac8fe9 at mi_switch+0x139
#1 0xc0afe12e at sleepq_switch+0xce
#2 0xc0afec9e at sleepq_timedwait+0x3e
#3 0xc0ac95e2 at _sleep+0x2a2
#4 0xc0d1c726 at swap_pager_getpages+0x3e6
#5 0xc0d286b1 at vm_fault_hold+0x1031
#6 0xc0d29d5b at vm_fault+0x8b
#7 0xc0e2567f at trap_pfault+0x1df
#8 0xc0e2650a at trap+0x44a
#9 0xc0e0faec at calltrap+0x6
#10 0xc7fcbb5a at drm_ioctl+0x2ca
#11 0xc09a44ea at devfs_ioctl_f+0x10a
#12 0xc0b05480 at kern_ioctl+0x2a0
#13 0xc0b055f4 at sys_ioctl+0x134
#14 0xc0e25c1a at syscall+0x34a
#15 0xc0e0fb51 at Xint0x80_syscall+0x21
Comment 4 sasamotikomi 2014-09-06 08:48:21 UTC
*** Bug 182498 has been marked as a duplicate of this bug. ***
Comment 5 John Baldwin freebsd_committer freebsd_triage 2014-09-08 16:49:49 UTC
In all these crashes, the root bug is a NULL pointer dereference (probably) at drm_ioctl+0x2ca.  Can you provide the kgdb backtrace from a single core.txt file?  It would be good to see what source line corresponds to that address.
Comment 6 sasamotikomi 2014-09-11 03:40:39 UTC
(In reply to John Baldwin from comment #5)
> In all these crashes, the root bug is a NULL pointer dereference (probably)
> at drm_ioctl+0x2ca.  Can you provide the kgdb backtrace from a single
> core.txt file?  It would be good to see what source line corresponds to that
> address.
I still have old GENERIC kernel (/boot/kernel.old.9.1/kernel) how to do that?
I new for kernel debugging.
Comment 7 John Baldwin freebsd_committer freebsd_triage 2014-09-12 17:43:21 UTC
Do you have core.txt.N files in /var/crash?  They will contain the kgdb backtrace if so.
Comment 8 Eygene Ryabinkin freebsd_committer freebsd_triage 2014-11-27 21:17:00 UTC
Looks like there is at least off-by-one access to the driver-specific ioctl array, http://codelabs.ru/fbsd/patches/drm2/fix-drm_drv-off-by-one.diff, since max_ioctl is not a number of a highest supported ioctl number, but rather a size of driver's ioctl array.
Comment 9 Eygene Ryabinkin freebsd_committer freebsd_triage 2014-11-27 21:20:49 UTC
(In reply to sasamotikomi from comment #6)
> (In reply to John Baldwin from comment #5)
> > In all these crashes, the root bug is a NULL pointer dereference (probably)
> > at drm_ioctl+0x2ca.  Can you provide the kgdb backtrace from a single
> > core.txt file?  It would be good to see what source line corresponds to that
> > address.
> I still have old GENERIC kernel (/boot/kernel.old.9.1/kernel) how to do that?
> I new for kernel debugging.

Try the following:
{{{
cd /boot/kernel.old.9.1
kgdb drm2.ko
list *(drm_ioctl+0x2ca)
}}}
This assumes that you have drm2.ko and its symbolic information (drm2.ko.symbols) is present too.
Comment 10 Konstantin Belousov freebsd_committer freebsd_triage 2014-11-28 10:29:02 UTC
(In reply to Eygene Ryabinkin from comment #8)
> Looks like there is at least off-by-one access to the driver-specific ioctl
> array, http://codelabs.ru/fbsd/patches/drm2/fix-drm_drv-off-by-one.diff,
> since max_ioctl is not a number of a highest supported ioctl number, but
> rather a size of driver's ioctl array.

I think that the patch is wrong, and there is no off-by-one bug.  At least in i915 case, max_ioctl is initialized as nitems(i915_ioctls), which is the number of array elements.  Applying your patch, the last ioctl in the driver-specific array becomes inaccessible.  As an easier test, consider what would happen after your change if array consists of single element.
Comment 11 Eygene Ryabinkin freebsd_committer freebsd_triage 2014-11-28 10:37:32 UTC
./i915/i915_dma.c:	.max_ioctl	= DRM_ARRAY_SIZE(i915_ioctls),
and drmP.h reads
#define DRM_ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0]))
so for the case when nr = dev->driver->max_ioctl, with dev->driver->ioctls[nr] you will be doing the same thing as for
{{{
<sometype> array[N];

value = array[N];
}}}
that is off-by-one.

Am I missing something?
Comment 12 Konstantin Belousov freebsd_committer freebsd_triage 2014-11-28 10:49:25 UTC
(In reply to Eygene Ryabinkin from comment #11)
> ./i915/i915_dma.c:	.max_ioctl	= DRM_ARRAY_SIZE(i915_ioctls),
> and drmP.h reads
> #define DRM_ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0]))
> so for the case when nr = dev->driver->max_ioctl, with
> dev->driver->ioctls[nr] you will be doing the same thing as for
> {{{
> <sometype> array[N];
> 
> value = array[N];
> }}}
> that is off-by-one.
> 
> Am I missing something?

No, you are right. Sorry for the confusion.

Feel free to commit the patch with reasonable MFC period (say, 1 week).
Approved by: kib.
Comment 13 Eygene Ryabinkin freebsd_committer freebsd_triage 2014-11-28 11:10:23 UTC
Perhaps the more proper thing will be to say
.max_ioctl	= (DRM_ARRAY_SIZE(i915_ioctls) - 1)
since this will preserve the semantics of the "max_ioctl" name that should keep the index of the last ioctl and not the number of driver-specific ioctls.

I'd go this route, though the change will be slightly more intrusive; namely http://codelabs.ru/fbsd/patches/drm2/fix-drm_drv-off-by-one-v2.diff.

What do you think?
Comment 14 Konstantin Belousov freebsd_committer freebsd_triage 2014-11-28 11:35:07 UTC
(In reply to Eygene Ryabinkin from comment #13)
> Perhaps the more proper thing will be to say
> .max_ioctl	= (DRM_ARRAY_SIZE(i915_ioctls) - 1)
> since this will preserve the semantics of the "max_ioctl" name that should
> keep the index of the last ioctl and not the number of driver-specific
> ioctls.
> 
> I'd go this route, though the change will be slightly more intrusive; namely
> http://codelabs.ru/fbsd/patches/drm2/fix-drm_drv-off-by-one-v2.diff.
> 
> What do you think?

I think that the original one-liner is fine, and this one is overcomplication.
Comment 15 commit-hook freebsd_committer freebsd_triage 2014-11-28 12:15:40 UTC
A commit references this bug:

Author: rea
Date: Fri Nov 28 12:15:00 UTC 2014
New revision: 275209
URL: https://svnweb.freebsd.org/changeset/base/275209

Log:
  DRM2: fix off-by-one overflow in ioctl processing

  Call to the driver-specific ioctl used to process ioctl number
  that will lead to the out-of-bounds access to the ioctl handler
  array.

  PR:		193367
  Approved by:	kib
  MFC after:	1 week

Changes:
  head/sys/dev/drm2/drm_drv.c
Comment 16 Eygene Ryabinkin freebsd_committer freebsd_triage 2014-11-28 12:31:59 UTC
I wonder if the bug manifests itself in the production system for cases where kernel has no KMS support, but Xorg or other userland utility wants to issue DRM_RADEON_GEM_INFO that will trigger the patched bug.

Can original submitter say if he runs Xorg with KMS support or some another tool that wants to query DRM infrastructure about graphics capabilities?

Thanks!
Comment 17 sasamotikomi 2015-01-19 00:47:47 UTC
(In reply to Eygene Ryabinkin from comment #16)
Yes, I use 10.1-RELEASE now and KMS+VT (also I soon switch to STABLE because it's has AGP support).
Comment 18 commit-hook freebsd_committer freebsd_triage 2015-04-28 12:38:12 UTC
A commit references this bug:

Author: dumbbell
Date: Tue Apr 28 12:37:10 UTC 2015
New revision: 282141
URL: https://svnweb.freebsd.org/changeset/base/282141

Log:
  DRM2: fix off-by-one overflow in ioctl processing

  Call to the driver-specific ioctl used to process ioctl number
  that will lead to the out-of-bounds access to the ioctl handler
  array.

  PR:             193367
  Approved by:    kib
  MFC of:		r275209 (original commit by rea)

Changes:
_U  stable/10/
  stable/10/sys/dev/drm2/drm_drv.c
Comment 19 Eitan Adler freebsd_committer freebsd_triage 2018-05-28 19:40:54 UTC
batch change:

For bugs that match the following
-  Status Is In progress 
AND
- Untouched since 2018-01-01.
AND
- Affects Base System OR Documentation

DO:

Reset to open status.


Note:
I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.
Comment 20 Eugene Grosbein freebsd_committer freebsd_triage 2018-05-28 20:07:41 UTC
The fix seems to be present in all of head, stable/10 and stable/11 branches. If so, this should be closed.
Comment 21 Rene Ladan freebsd_committer freebsd_triage 2018-09-03 19:08:21 UTC
Maintainer reset.
Comment 22 Eugene Grosbein freebsd_committer freebsd_triage 2018-09-03 19:15:04 UTC
Feedback timeout. Also, this is believed to be fixed in all supported branches.