Bug 259230 - Touching the touchpad on a frame.work laptop causes reboot or poweroff
Summary: Touching the touchpad on a frame.work laptop causes reboot or poweroff
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.0-STABLE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks: frameworklaptop
  Show dependency treegraph
 
Reported: 2021-10-17 16:14 UTC by Jose Quinteiro
Modified: 2022-03-14 13:57 UTC (History)
10 users (show)

See Also:


Attachments
Dmesg from the failing system (10.75 KB, text/plain)
2021-10-17 16:14 UTC, Jose Quinteiro
no flags Details
Demsg from 14-CURRENT (10.86 KB, text/plain)
2021-10-17 17:32 UTC, Jose Quinteiro
no flags Details
iichid.patch (566 bytes, patch)
2021-11-04 23:08 UTC, Vladimir Kondratyev
no flags Details | Diff
iichid debug output (4.62 KB, text/plain)
2021-12-01 21:36 UTC, Mark Johnston
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jose Quinteiro 2021-10-17 16:14:50 UTC
Created attachment 228780 [details]
Dmesg from the failing system

New i7-1165G7 Frame.work laptop. Even grazing the touchpad causes the system to power off or reboot. Not reproducible on 12.2. Dmesg is attached.
Comment 1 Jose Quinteiro 2021-10-17 17:31:33 UTC
Problem is reproducible on 14-CURRENT. Dmesg attached.
Comment 2 Jose Quinteiro 2021-10-17 17:32:11 UTC
Created attachment 228782 [details]
Demsg from 14-CURRENT
Comment 3 georg.lastname 2021-10-21 16:48:27 UTC
The good thing is that this bug is easily reproducible.

dmesg shows hmt driver attaching to the touchpad.

Do you have a crash dump?

It's kinda weird that sometimes it powers off, and sometimes it reboots. I think on a kernel panic it defaults to rebooting.

If it works on 12.2, then it is probably not a hardware issue.
Comment 4 Jose Quinteiro 2021-10-21 18:15:56 UTC
I'll have to look into how to get a crash dump.

Problem goes away if I load the drm-devel-kmod on 14-CURRENT. Laptop doesn't power off or reboot if I touch the touchpad even before I start X.

I'm currently working around troubles with the USB Ethernet adapter I'm using that I suspect are not specific to Freebsd. I've ordered some replacements, but they haven't arrived yet.
Comment 5 Mark Johnston freebsd_committer freebsd_triage 2021-10-22 01:42:14 UTC
I'm seeing this as well on the same laptop.  Seems related to the iichid driver, if I disable autoloading (e.g., with devmatch_blocklist="iichid" in /etc/rc.conf).  The touchpad appears to be usable without it.

I see the same messages when iichid is loaded:

iichid0: <FRMW0001:01 32AC:0006 I2C HID device> at addr 0x50 on iicbus1
iichid0: Interrupt setup failed. Fallback to sampling
hidbus0: <HID bus> on iichid0
iichid1: <PIXA3854:01 093A:0274 I2C HID device> at addr 0x2c on iicbus2
iichid1: Interrupt setup failed. Fallback to sampling
hidbus1: <HID bus> on iichid1
Comment 6 Vincent Milum Jr 2021-10-24 20:00:42 UTC
Can confirm this also happens for me, and it appears everyone we've talked to with the Framework Laptop so far. I personally tested on 14-CURRENT.

Another temporary work-around is to disable PS/2 emulation in the BIOS, which I've gone ahead and noted as a quirk on the wiki page until this issue is resolved. With this option changed in the BIOS, touchpad works as expected on both FreeBSD and Windows. With emulation enabled, touchpad works just fine on Windows, but reboots as described in this thread on FreeBSD.

https://wiki.freebsd.org/Laptops/Framework_Laptop
Comment 7 Kyle Evans freebsd_committer freebsd_triage 2021-10-24 20:34:01 UTC
I noted earlier that it's specifically the i2c reset in iichid_attach that eventually causes it. I tried disabling atkbdc/psm* to no avail, and I was preventing the iichid attachment so not entirely sure what's going on there. I went to look at acpi bits and discovered that iasl dies, so I haven't poked around much yet.
Comment 8 Kyle Evans freebsd_committer freebsd_triage 2021-11-04 06:49:58 UTC
I had contacted frame.work support after my last comment because I suspected a firmware bug, and I did get a response from them tonight; quoted below. The short version is that we are triggering a firmware bug (the crashing) as it falls back to legacy mode, and the root cause is likely that we're sending the reset command but can't setup the interrupt because it's an ACPI GPIO interrupt that we don't currently support.

--

Hi Kyle, 

I wanted to follow up on the touchpad issue you are seeing in freebsd. I can reproduce the issue, and i suspect that with PS2 emulation enabled freebsd may be holding the i2c bus to the touchpad low for too long or during startup. It does not seem to reproduce 100% of the time. 

I suspect what is happening is the OS is sending the reset command to the touchpad and then not processing interrupts from the touchpad soon enough, causing the EC to fallback to ps2 mouse emulation as it does not detect the i2c-hid interface as being active. 
I suspect that the freebsd driver may be holding the i2c lines low in this state, causing the EC to fail communication with the touchpad. 

Unexpectedly this causes the EC to crash, which is what is causing the system to shutdown. 

I will take a look at exactly what is causing the crash and we can role a fix into a future firmware update, as that should not happen! But for now I would suggest disabling ps2 emulation in the bios. 

Thanks for the report! 

--
Comment 9 Vladimir Kondratyev freebsd_committer freebsd_triage 2021-11-04 11:27:03 UTC
You may try my hacked iichid driver: https://github.com/wulf7/iichid_gpio
But TigerLake is not tested yet.
Comment 10 Vladimir Kondratyev freebsd_committer freebsd_triage 2021-11-04 23:08:56 UTC
Created attachment 229277 [details]
iichid.patch

Try attached patch. pause() value may require adjustment
Comment 11 Kyle Evans freebsd_committer freebsd_triage 2021-11-05 04:09:06 UTC
(In reply to Vladimir Kondratyev from comment #10)

With the attached patch, default iichid with sampling seems to work here.

With iichid_gpio, it ended up rebooting before I had a chance to touch the touchpad or catch the console output.
Comment 12 Mark Johnston freebsd_committer freebsd_triage 2021-11-24 20:58:54 UTC
Sorry for the delayed follow up.  The patch works for me.  iichid_gpio appears to work as well.  One unrelated problem is that I can't seem to trigger a right button down event.  Clicking anywhere on the touchpad results in a left click, as does a two finger click.  I see 

event 9  - PIXA3854:01 093A:0274 TouchPad: kernel bug: clickpad with right button, assuming it is not a clickpad

in Xorg.log, not sure yet if that's relevant.
Comment 13 commit-hook freebsd_committer freebsd_triage 2021-11-30 21:32:00 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=c508b0818b245ea136d6538d68868cb3541f8f8f

commit c508b0818b245ea136d6538d68868cb3541f8f8f
Author:     Vladimir Kondratyev <wulf@FreeBSD.org>
AuthorDate: 2021-11-30 21:29:50 +0000
Commit:     Vladimir Kondratyev <wulf@FreeBSD.org>
CommitDate: 2021-11-30 21:29:50 +0000

    iichid(4): Perform acknowledgement of I2C device interrupt after RESET command

    in sampling mode to workaround firmware bug.

    This fixes reboot or poweroff on frame.work laptops after first touch.

    Reported by:    many
    PR:             259230
    MFC after:      1 week
    Tested by:      kevans, markj

 sys/dev/iicbus/iichid.c | 5 +++++
 1 file changed, 5 insertions(+)
Comment 14 Vladimir Kondratyev freebsd_committer freebsd_triage 2021-11-30 21:37:28 UTC
(In reply to Mark Johnston from comment #12)
> The patch works for me.  iichid_gpio appears to work as well.  

Thanks!

> One unrelated problem is that I can't seem to trigger a right button down event.
>  Clicking anywhere on the touchpad results in a left click, as does a two finger click.

Please, add `hw.iichid.debug=1` to /boot/loader.conf and share iichid`s probe/attach output.
Comment 15 Mark Johnston freebsd_committer freebsd_triage 2021-12-01 21:36:34 UTC
Created attachment 229834 [details]
iichid debug output

Debug output from iichid.
Comment 16 Vladimir Kondratyev freebsd_committer freebsd_triage 2021-12-02 10:16:22 UTC
(In reply to Mark Johnston from comment #15)
HID report descriptor appeared to be broken. It contains records for external left and right buttons which is wrong for clickpads.

Linux seems to "fix" it at libinput level with placing a quirk in to /usr/local/share/libinput/50-framework.quirks file. But it does not work in your case due to slightly different device name: "PIXA3854:00 093A:0274 Touchpad" vs "PIXA3854:01 093A:0274 TouchPad". You may try to add another quirk with fixed name.
Comment 17 Mark Johnston freebsd_committer freebsd_triage 2021-12-02 16:12:46 UTC
(In reply to Vladimir Kondratyev from comment #16)
This works.  Thank you!  I guess I should try and get this quirk submitted upstream.

To anyone else cc'ed here who has a framework laptop, it'd be useful to see the device list printed by "evtest", in case there are other variants.  In that case, it would also be useful to know if the right clickpad button works out of the box or not.
Comment 18 Vladimir Kondratyev freebsd_committer freebsd_triage 2021-12-02 20:37:46 UTC
(In reply to Mark Johnston from comment #17)
libinput has changed clickpad detection mechanism several hours ago:
https://gitlab.freedesktop.org/libinput/libinput/-/commit/fbe5d35dca88ae9a751623cbfa3b6ed2f012ec83

For now the touchpad should work out of box and only produce a warnings in Xorg log.
Comment 19 Vincent Milum Jr 2021-12-06 05:45:08 UTC
Just tested Frame.Work BIOS Firmware build 3.06, and it appears they've fixed the issue on their end at this point. :)

https://community.frame.work/t/public-beta-test-bios-v3-06-driver-bundle-2021-10-29/10167
Comment 20 Vladimir Kondratyev freebsd_committer freebsd_triage 2022-03-01 21:54:22 UTC
MFC-ed to 13-STABLE: https://cgit.freebsd.org/src/commit/?id=77ec8dd61cb77ce7471dc0009bc8ec48b5e2cf81
Comment 21 parv 2022-03-12 13:36:26 UTC
(In reply to Mark Johnston from comment #17)
What is "evtest", or rather how do I start it & which package would include it?
Comment 22 Mark Johnston freebsd_committer freebsd_triage 2022-03-14 13:57:51 UTC
(In reply to parv from comment #21)
It's a debugging tool for input devices.  It comes with the evtest package, so "pkg install evtest" should install it.