Bug 211895 - [11.0-RC2, 12.0] `usbconfig` hangs after suspend/resume, other USB devices don't work
Summary: [11.0-RC2, 12.0] `usbconfig` hangs after suspend/resume, other USB devices do...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: usb (show other bugs)
Version: 11.0-RC1
Hardware: amd64 Any
: --- Affects Some People
Assignee: FreeBSD USB mailing list
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2016-08-16 09:55 UTC by Aleksander Alekseev
Modified: 2016-09-25 13:53 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Aleksander Alekseev 2016-08-16 09:55:08 UTC
Functionality mentioned below worked flawlessly in 11.0-BETA4 but was broken in 11.0-RC1.

I have Logitech USB mouse and keyboard. When I hibernate a system for a long time (a few hours) after resume these devices don't work. Reconnecting doesn't help. When I execute 'sudo usbconfig' it hangs for a few minutes before printing anything.

Here are truss output (hanging start is marked by two empty lines) and /var/log/messages:

http://afiskon.ru/s/ed/ab93d78930_logitech-bug.txt

Shutting a system down after this shows such an error:

http://afiskon.ru/s/22/8c7d9a480f_freebsd110rc1bug.jpg

(As a side note it could be two separate issues.)

This problem reproduces every day so I would be happy to provide any additional debug information you might need.

Could be related to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202551
Comment 1 Aleksander Alekseev 2016-08-17 04:38:20 UTC
For the record - bug doesn't reproduce if you disconnect Logitech adapter before suspend and connect it back after resume.
Comment 2 Aleksander Alekseev 2016-08-26 05:42:28 UTC
I've found very easy way to reproduce this issue. You need a laptop with working suspend/resume and Logitech K360 keyboard.

1. Reboot the system
2. Login as root
3. Suspend: acpiconf -s 3
4. Turn off Logitech keyboard (there is a switch on it)
5a. Resume...
5b. ...quickly turn the keyboard on...
5c. ... and start typing.
6. If keyboard doesn't work and `usbconfig` hangs as described above, you reproduced an issue
7. Otherwise repeat steps 3-5 a few more times. I usually need 2-3 attempts.

I didn't see "Exception: AE_ERROR ..." for a while, in fact I saw it only once. Thus I think it's an unrelated problem. I'm also pretty sure that this problem could exist before 11.0-RC1.
Comment 3 Hans Petter Selasky freebsd_committer 2016-08-26 07:42:00 UTC
Hi,

There hasn't been any USB changes between 11.0-BETA4 and 11.0-RC1, so I suspect the problem is somewhere else.

When usbconfig hangs, you might be able to enter kgdb, and show the backtrace of the "USB process" threads.

--HPS
Comment 4 Aleksander Alekseev 2016-09-04 12:31:39 UTC
I'm sorry but I'm having some problems getting a backtrace.

There are:

```
dumpdev="AUTO"
ddb_enable="YES"
```

... lines in /etc/rc.conf

I can't enter debugger using sysctl:

```
$ sudo sysctl debug.kdb.enter=1
Password:
debug.kdb.enter: 0 -> 0
```

Ctr+Alt+Escape hotkey doesn't work either.

I can crash a kernel using:

```
sudo sysctl debug.kdb.panic=1
```

... command but after reboot there is no dump in /var/crash directory.

I tried to reboot without loading i915kms.ko in case it causes these problems. But without this driver screen doesn't turn on after suspend/resume so even if I enter a debugger I can't read a backtrace.

In dmesg -a I see:

```
No suitable dump device was found.
/etc/rc: WARNING: failed precmd routine for ddb
```

Full `dmesg -a` after crash and reboot:

http://afiskon.ru/s/67/6c0d4868c6_dmesg_a_after_crash.txt

I have a single disk partition, no swap:

```
# Device	Mountpoint	FStype	Options	Dump	Pass#
/dev/ada0p2	/		ufs	rw	1	1
```

`savecore -v` tells me `savecore: no dumps found`.


uname -a:

``
FreeBSD e733 11.0-RC2 FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
```

Any advice?
Comment 5 Hans Petter Selasky freebsd_committer 2016-09-04 16:34:10 UTC
Bruce Evans has recently fixed some issues using the USB keyboard from the debugger. Are you running the latest 12-current?
Comment 6 karl 2016-09-04 16:40:55 UTC
Uh.... you might want to look here:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211884

Try reverting that change and see if the problem goes away.  This change appears to impact a lot of different people with the common element being USB keyboards (or mice) being completely screwed from the time the kernel gets control until the system has booted and is running normally.  My IPKVM systems attach both a keyboard and mouse, and exhibit the problem -- but reverting that change restores normal functionality.
Comment 7 Aleksander Alekseev 2016-09-05 08:44:26 UTC
hselasky@,

> Are you running the latest 12-current?

No, I'm running 11.0-RC2
Comment 8 Hans Petter Selasky freebsd_committer 2016-09-05 09:06:31 UTC
Can you test a 12-current kernel and see if this problem is fixed?

--HPS
Comment 9 Aleksander Alekseev 2016-09-08 08:31:54 UTC
Here is some additional debug information, just for the record (still on 11.0-RC2).

dmesg right after resume:

http://afiskon.ru/s/c1/e5cf238331_usb1.jpg

dmesg after reconnecting USB adapter:

http://afiskon.ru/s/73/9dbafeef9a_usb2.jpg

If I disconnect an adapter everything works just fine, usbconfig doesn't hang, etc. If I connect it or (!) any other USB device (a flash drive for instance) back the problem returns. Apparently when I do it code enters an infinite loop somewhere in usub_reattach_port and usb_alloc_device procedures. Explains why usbconfig hangs.

I will try to reproduce this issue on 12.0 and add a bit more debug output. I have no experience in kernel development so any suggestions on how else I could debug this issue would be much appreciated.
Comment 10 Aleksander Alekseev 2016-09-12 17:18:30 UTC
Ok, here are a few updates on this issue.

1. I solved all issues regarding entering KDB and writing kernel core dumps. Now I can provide really any debug information you might need. In case anyone is interested the problem with KDB was that for some reason RC and RELEASE builds of FreeBSD don't have DDB option in GENERIC kernel by default. To write kernel dumps one needs a swap partition and dumpdev value in /etc/rc.conf pointing to this partition. A good explanation of how UFS partition could be shrinked to create a swap partition: https://www.reddit.com/r/freebsd/comments/4ph4xu/shrink_ufs_root_fs/

2. The problem _does_ exist in 12.0 as well (I tested on r305736). usbconfig doesn't hang, but keyboard still doesn't work. Neither does any other USB device (flash drives, etc). Here is how dmesg looks like:

http://afiskon.ru/s/67/ba7648265f_usb120.jpg

As you may see there is no more infinite loop that calls usb_alloc_device and uhub_reattach_port - they are called only one. But no other USB device could be connected afterwards (see "giving up" errors).

3. Here are core.txt.0 and vmcore.0 created manually after issue was reproduced:

http://afiskon.ru/s/aa/0e02bb563b_core.txt.0.txt
http://afiskon.ru/s/7a/d529b2c2d7_vmcore.0.tgz

Is there anything else I can do to help?
Comment 11 Hans Petter Selasky freebsd_committer 2016-09-12 18:08:20 UTC
Hi,

Are there any settings in the BIOS for USB, like Legacy support. Can you turn them off? Does it make any difference?

Maybe you could just "diff -ur" the RC1 and 11-0-BETA4 sources and look at what areas where touched.

--HPS
Comment 12 Hans Petter Selasky freebsd_committer 2016-09-12 18:13:44 UTC
Just guessing, that some PCI hotplug support was merged to 11.0:

https://svnweb.freebsd.org/changeset/base/304434

Can it be the cause?

--HPS
Comment 14 Aleksander Alekseev 2016-09-25 13:53:37 UTC
Latest discovery - apparently USB stack stops to work as described above after few suspend/resume even if you don't connect any usb devices at all.

Reproduces like this:
1. Use laptop as usual, suspend, resume, but don't connect any USB devices!
2. After some time try to connect USB microphone
3. Observe a bug as described above