Bug 222646

Summary: ukbd(4): support Consumer Control based multimedia keys
Product: Base System Reporter: Greg V <greg>
Component: kernAssignee: freebsd-usb mailing list <usb>
Status: Closed Overcome By Events    
Severity: Affects Some People CC: 0mp, cem, emaste, henry.hu.sh, hselasky, marek, mason, wulf
Priority: --- Keywords: feature, patch
Version: CURRENT   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
ukbd_consumer_ctl.patch
none
ukbd_consumer_ctl.patch v2 none

Description Greg V 2017-09-27 15:58:44 UTC
Created attachment 186765 [details]
ukbd_consumer_ctl.patch

This patch adds support for media keys (such as play/pause, next/prev track, volume up/down) on USB keyboards where they're implemented as HID Consumer Control reports by converting them into plain old key presses of matching keys.

Background: my mouse (Logitech G402) pretends to also be a keyboard, to allow two programmable extra buttons to press any keyboard key. One of the options for these keys is volume control, and I really like that. These key presses were not recognized by the FreeBSD kernel. With this patch, libinput-debug-events shows them as KEY_VOLUMEDOWN (114) and KEY_VOLUMEUP (115), just like on Linux, and my XF86AudioRaise(Lower)Volume bindings in Xorg work fine! Nothing in this patch is evdev specific though, it literally translates these events to key presses.

Also: I have a wireless keyboard that pretends to also be a mouse (…yeah), and for the media keys, it sends identical packets, but through a ums device. Tried to do a similar thing (but evdev-only, since the normal mouse interface can't press keys I guess) in the ums driver, wasted several hours only to encounter weird problems… hid_locate couldn't even find the stuff, though it's clearly there (uhidd recognizes it fine!). After removing the "Only copy HID item, increment position and return if correct kindset" check from hid_get_item, hid_locate "found" it but returned wrong data… So screw that for now, hopefully most input devices use ukbd for media keys :D
Comment 1 Hans Petter Selasky freebsd_committer 2017-09-28 07:42:34 UTC
Hi,

I think it is clever to add a check:

if (data != 0) ->>>

if (sc->sc_ndata.keycode[0] == KEY_ERROR && data != 0)

So that you don't override valid key input.

Or else scan the keycode array and place data last.

--HPS
Comment 2 Vladimir Kondratyev freebsd_committer 2017-09-29 17:18:39 UTC
(In reply to Greg V from comment #0)
Did you check events generated by your patched ukbd when two HID consumer-page keys are pressed and then released simultaneously? I think you will get press events for both keys but release event only for a one of them.
Comment 3 Greg V 2017-09-29 19:33:59 UTC
(In reply to Hans Petter Selasky from comment #1)
(In reply to Vladimir Kondratyev from comment #2)

From what I understand, consumer control packets are completely separate packets that cannot contain keyboard keypresses (totally different usage page than keyboard keys, 0xC vs 0x7). Just confirmed at https://deskthority.net/wiki/Media_key#USB

Pressing both volume buttons on my mouse results in the two keys pressed-released in order. Some devices *might* support simultaneous presses of multiple media keys, but I think in that case, we would just lose the extra key presses.
Comment 4 Vladimir Kondratyev freebsd_committer 2017-09-29 22:04:27 UTC
(In reply to Greg V from comment #3)
> Pressing both volume buttons on my mouse results in the two keys pressed-released in order.
Did you observed it with evemu or with libinput? Please use former as it connects directly to input device and gives raw data.

> Some devices *might* support simultaneous presses of multiple media keys, but I think in that case, we would just lose the extra key presses.
I recently implemented evdev support for consumer page for bluetooth keyboards (not publicly available yet) and found out that my bt-keyboard loses *release* events on simultaneous presses as it is unable to report "which key has been depressed" at HID-protocol level. It is capable only to say that "last/all key has been depressed" so my driver has to generate release event at next interrupt at his own. I do not see how your patch does such "autorelease". Maybe ukbd does this automagically? I should look deeper in its internals. Unfortunately I cant do hardware testing now
Comment 5 Greg V 2017-09-30 11:40:41 UTC
(In reply to Vladimir Kondratyev from comment #4)
Didn't know about evemu, nice tool. Just tried it, events are always sequential: 1, 0, 1, 0…

My mouse does not support simultaneous media key presses, it always returns only one key.

ukbd gets the report with all zeros and generates the release event. My patch only allows one media key to be pressed, others are lost, so the release event will always release that key.
Comment 6 Hans Petter Selasky freebsd_committer 2017-09-30 12:15:47 UTC
Can you make the patch so that no other keys are lost. For example by extending the key array?

--HPS
Comment 7 Greg V 2017-09-30 15:30:26 UTC
Created attachment 186814 [details]
ukbd_consumer_ctl.patch v2

ok here's a version that supports multiple keys.
Comment 8 Henry Hu 2020-03-04 04:57:25 UTC
Hi Greg,

I'm trying this patch with my WASD keyboard (https://www.wasdkeyboards.com/). It's not working, here are the symptoms:

* The keyboard shows as a ums and a ukbd
* When keyboard is attached, I don't see "found consumer control".
* When the Fn key is pressed, I see

Mar  3 23:55:28 goldpeak kernel: ukbd_intr_callback: actlen=8 bytes
Mar  3 23:55:28 goldpeak kernel: ukbd_intr_callback: modifiers = 0x0000

* Similarly, if I then press the media keys (they are sent by combining Fn + other keys, such as PgUp/PgDn), I see the above 2 lines again.
* When I release the keys, each one triggers the same lines.

I'll try to grab the HID descriptor of the keyboard.
Comment 9 Henry Hu 2020-03-04 05:30:19 UTC
Decoded HID report descriptor:

0x05, 0x01,        // Usage Page (Generic Desktop Ctrls)
0x09, 0x06,        // Usage (Keyboard)
0xA1, 0x01,        // Collection (Application)
0x05, 0x07,        //   Usage Page (Kbrd/Keypad)

0x19, 0xE0,        //   Usage Minimum (0xE0)
0x29, 0xE7,        //   Usage Maximum (0xE7)
0x15, 0x00,        //   Logical Minimum (0)
0x25, 0x01,        //   Logical Maximum (1)
0x75, 0x01,        //   Report Size (1)
0x95, 0x08,        //   Report Count (8)
0x81, 0x02,        //   Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)

0x95, 0x01,        //   Report Count (1)
0x75, 0x08,        //   Report Size (8)
0x81, 0x01,        //   Input (Const,Array,Abs,No Wrap,Linear,Preferred State,No Null Position)

0x95, 0x03,        //   Report Count (3)
0x75, 0x01,        //   Report Size (1)
0x05, 0x08,        //   Usage Page (LEDs)
0x19, 0x01,        //   Usage Minimum (Num Lock)
0x29, 0x03,        //   Usage Maximum (Scroll Lock)
0x91, 0x02,        //   Output (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position,Non-volatile)

0x95, 0x05,        //   Report Count (5)
0x75, 0x01,        //   Report Size (1)
0x91, 0x01,        //   Output (Const,Array,Abs,No Wrap,Linear,Preferred State,No Null Position,Non-volatile)

0x95, 0x06,        //   Report Count (6)
0x75, 0x08,        //   Report Size (8)
0x26, 0xFF, 0x00,  //   Logical Maximum (255)
0x05, 0x07,        //   Usage Page (Kbrd/Keypad)
0x19, 0x00,        //   Usage Minimum (0x00)
0x29, 0x91,        //   Usage Maximum (0x91)
0x81, 0x00,        //   Input (Data,Array,Abs,No Wrap,Linear,Preferred State,No Null Position)

0xC0,              // End Collection

// 62 bytes
Comment 10 Hans Petter Selasky freebsd_committer 2020-03-04 09:01:25 UTC
Henry:

Try to run:

usbdump -i usbusX -f Y -s 65536 -vvv

where X.Y are the numbers after ugenX.Y for your keyboard.

It will show the actual event data packet.

--HPS
Comment 11 Hans Petter Selasky freebsd_committer 2020-03-04 09:03:12 UTC
Greg: You need to rebase your patch again.

After the gamers keyboard support, keycode's are gone and we now have a bitmap where we set all the input keys!
Comment 12 Vladimir Kondratyev freebsd_committer 2020-03-04 12:36:50 UTC
The patch will not work for WASD keyboard as it's report descriptor does not includes consumer page input usages. Most probably, consumer page is accessible as separate USB HID device and is accessible with usbhidctl/usbhidaction utils.
Comment 13 Greg V 2020-03-04 17:05:04 UTC
There is consumer page support in iichid now. (also I use a different mouse) so I'm abandoning this patch.
Comment 14 Henry Hu 2020-03-05 00:42:15 UTC
(In reply to Vladimir Kondratyev from comment #12)
Hans, yes, every key press there seem to be 2 reports:

19:36:22.021230 usbus0.2 SUBM-INTR-EP=00000081,SPD=LOW,NFR=1,SLEN=0,IVAL=10
 frame[0] READ 8 bytes
19:36:22.021231 usbus0.2 DONE-INTR-EP=00000081,SPD=LOW,NFR=1,SLEN=8,IVAL=10,ERR=0
 frame[0] READ 8 bytes
 0000  00 00 00 00 00 00 00 00  -- -- -- -- -- -- -- --  |........        |
19:36:22.021353 usbus0.2 DONE-INTR-EP=00000082,SPD=LOW,NFR=1,SLEN=8,IVAL=10,ERR=0
 frame[0] READ 7 bytes
 0000  01 08 00 00 00 00 00 --  -- -- -- -- -- -- -- --  |.......         |
19:36:22.021356 usbus0.2 SUBM-INTR-EP=00000082,SPD=LOW,NFR=1,SLEN=0,IVAL=10
 frame[0] READ 8 bytes

19:36:22.101224 usbus0.2 SUBM-INTR-EP=00000081,SPD=LOW,NFR=1,SLEN=0,IVAL=10
 frame[0] READ 8 bytes
19:36:22.101225 usbus0.2 DONE-INTR-EP=00000081,SPD=LOW,NFR=1,SLEN=8,IVAL=10,ERR=0
 frame[0] READ 8 bytes
 0000  00 00 00 00 00 00 00 00  -- -- -- -- -- -- -- --  |........        |
19:36:22.101348 usbus0.2 DONE-INTR-EP=00000082,SPD=LOW,NFR=1,SLEN=8,IVAL=10,ERR=0
 frame[0] READ 7 bytes
 0000  01 00 00 00 00 00 00 --  -- -- -- -- -- -- -- --  |.......         |
19:36:22.101349 usbus0.2 SUBM-INTR-EP=00000082,SPD=LOW,NFR=1,SLEN=0,IVAL=10
 frame[0] READ 8 bytes

Vladimir: There is another device, which is attached by ums driver, but interestingly no button/axis is reported.

Greg: I may try iichid as well.

Let's open another PR to track this.
Comment 15 Vladimir Kondratyev freebsd_committer 2020-03-06 00:57:29 UTC
(In reply to Henry Hu from comment #14)
> I may try iichid as well.

You should install latest iichid from github rather than from ports in that case. I left instructions how to replace ukbd, ums et al with usbhid in project's readme so you can try.

P.S. Really there are at least 3 ways to report consumer key states in HID format. And only one has been tested by me and one another does not work due to deficiencies in our system HID parser. So, do not expect too much.
Comment 16 Henry Hu 2020-03-06 18:25:42 UTC
(In reply to Vladimir Kondratyev from comment #15)
Thanks Vladimir, it seems to be working fine:

Mar  6 13:24:07 goldpeak kernel: ugen0.2: <vendor 0x04d9 USB Keyboard> at usbus0 (disconnected)
Mar  6 13:24:07 goldpeak kernel: ukbd0: at uhub2, port 3, addr 1 (disconnected)
Mar  6 13:24:07 goldpeak kernel: ukbd0: detached
Mar  6 13:24:07 goldpeak kernel: uhid0: at uhub2, port 3, addr 1 (disconnected)
Mar  6 13:24:07 goldpeak kernel: uhid0: detached
Mar  6 13:24:09 goldpeak kernel: ugen0.2: <vendor 0x04d9 USB Keyboard> at usbus0
Mar  6 13:24:09 goldpeak kernel: usbhid0 on uhub2
Mar  6 13:24:09 goldpeak kernel: usbhid0: <vendor 0x04d9 USB Keyboard, class 0/0, rev 1.10/2.09, addr 7> on usbus0
Mar  6 13:24:09 goldpeak kernel: hidbus0: <HID bus> on usbhid0
Mar  6 13:24:09 goldpeak kernel: hkbd0 on hidbus0
Mar  6 13:24:09 goldpeak kernel: usbhid1 on uhub2
Mar  6 13:24:09 goldpeak kernel: usbhid1: <vendor 0x04d9 USB Keyboard, class 0/0, rev 1.10/2.09, addr 7> on usbus0
Mar  6 13:24:09 goldpeak kernel: hidbus1: <HID bus> on usbhid1
Mar  6 13:24:09 goldpeak kernel: hcons0: <vendor 0x04d9 USB Keyboard Consumer Control> on hidbus1
Mar  6 13:24:09 goldpeak kernel: hsctrl0: <vendor 0x04d9 USB Keyboard System Control> on hidbus1

I tried the media keys and they're working!