Unresponsive keyboard on Lenovo X1 — possible ACPI error

Ask Fedora
, , , ,
1

I’ve encountered a catastrophic problem with my laptop that is beyond my abilities to resolve. Thank you sincerely to anyone who can point me towards a fix!

Problem: After a recent reboot, my laptop’s inbuilt keyboard (excluding several top row keys) is unresponsive (including on every successive reboot).

System information: Running Fedora 39 with X11 on a Lenovo X1 Carbon Gen 8. Fedora came pre-loaded on my laptop and I have no history of serious problems like this.

Additional information:

  • The keyboard still works in the BIOS/Lenovo menus before Fedora is loaded.

  • The keyboard does not work if I boot any alternative/earlier version of Fedora offered in my boot menu.

  • All other hardware seems to work: trackpad, power button, some top row keys, etc. An external/USB keyboard also works.

  • Running “journalctl -k” I found the following error messages:

kernel: ACPI Error: Needed [Integer/String/Buffer], found [Package] 0000000083b095a5 (20230628/exresop-469)
kernel: ACPI Error: AE_AML_OPERAND_TYPE, While resolving operands for [OpcodeName unavailable] (20230628/dswexec-433)
kernel: ACPI Error: Aborting method \ADBG due to previous error (AE_AML_OPERAND_TYPE) (20230628/psparse-529)
kernel: ACPI Error: Aborting method _SB.HIDD._DSM due to previous error (AE_AML_OPERAND_TYPE) (20230628/psparse-529)

This appears to indicate some connection with ACPI.

  • My system software is fully up to date. Furthermore, between my last successful reboot (where there were no keyboard problems) and the start of these problems, I only ran “dnf update” once. The package groups that were updated included (in order of possible relevance): mesa, llvm-libs, pipewire, grubby, hwdata, LibRaw, Perl, Python, Zathura.

  • My firmware is up to date. At least, running “fwupdmgr update” does not result in any new updates.

2

What happens if you boot an older kernel from the grub menu? Can you boot a Fedora 38 live system and see whether the problem persists?
Maybe this is a regression that was introduced in newer kernels …

3

An update: the keyboard started functioning again after a few days. Not sure why it started working again; the only thing I did was to remove a few of the suspicious packages that I had installed, though nothing that seemed it would affect the keyboard.

Here’s something that might help others with similar problems: the ACPI kernel error I mentioned above is still present in my log. So I think it was unrelated to the keyboard failure.

4

Glad your keyboard is working. If you have time, it would be helpful to try the “suspicious” packages one-by-one to see if you can identify the one causing the problem so it can be improved.

Many systems have ACPI “errors” that are either “harmless” or handled by workarounds. It can be tricky to link ACPI errors to a specific issue.

5

Out of Curiosity, Have you reached out to Lenovo about the issue? Seems like there could be cause of them to intervene if a hardware issue is present on a Product they should still be supporting, especially with a distro of Linux it shipped with.

6

Unfortunately, after my keyboard functioned normally for a few reboot cycles, it stopped working again yesterday. To try to resolve it, I uninstalled the packages kernel-devel and akmod-acpi_call. This did not make my keyboard work again, but when I further ran “dnf autoremove”, which removed 77 other packages, the keyboard worked again on reboot.

At this point I think there are too many involved packages (~80) for me to check them one-by-one. In any case, the fact that the problem repeated after I thought I had solved it by uninstalling some packages makes me uncertain that packages caused the problem at all.

7

Journalctl probably has some useful information, but finding the one line in 1000’s that is relevant can be a challenge. You could start by comparing high priority messages for a previous boot where the keyboard was unresponsive to the current boot.

You mentioned that you are using Xorg. Does the problem occur using Wayland?

If the issue returns and you have time to go down the rabbit-hole: The evtest “Input device event monitor and query tool” and evemu utilities are used debug issues with input devices. Baeldung.com and bec-systems have examples. My experience with input device debugging is limited to to Wacom gear, so I always had a working keyboard. I think you will need your USB keyboard.

Once some bug has caused runtime problems, a power cycle may reconfigure devices (keyboard, mouse, network, storage) that are active a boot. You don’t mention if any of the updates and package removals were accompanied by a power cycle.

8

Thanks for the helpful tips! I will have to become better acquainted with journalctl.

I have not tested if the problem happens on Wayland, good point.

I was not aware of evemu, but evtest did not register any input from the keyboard, which I took to be a very bad sign.

In my earlier message when I said “rebooting” I meant “turning my computer off and on again”. Is that what you meant by “power cycling”? If so, what is the difference between power cycling and rebooting?

9

The key difference between power cycling and rebooting is that power cycling forces the “BIOS” to reload.

I googled this question and found contradictory answers with about 99% containing nonsense like “rebooting requires turning off the power” – amazing how fast AI-generated misinformation has come to dominate search results.

Power cycling restarts the pre-boot “BIOS” software which means starting keyboard, display, pointing device, and networking (if you haven’t disabled it in the “BIOS” configuration). Modern systems generally have “smart” power buttons that can enter “sleep” mode, attempt a clean shutdown, or force “power off”. Many laptop systems run in low power mode but don’t fully turn off so battery management software can run. Desktops often have a separate switch on the power supply that just turns off power. Forcing power off should be avoided as it can lead to a corrupt filesystem.

If your keyboard isn’t working, the power button may still shutdown with a long press (and on a desktop is better than switching the system off at the power supply).

Some laptops require removing the battery to fully restart “BIOS” software.

Edit: I checked wikipedia, which uses the terms “warm boot” (e.g., power stays on) and “cold boot” (power is turned off and then on).

10

Unfortunately, this problem continues to happen sporadically.

I reached out to Lenovo (through the standard service request line) and they could not offer any help. Apparently they do not support Fedora even though they sold me the computer with Fedora on it.

Do you know of any other method of contacting Lenovo?

11

Thanks for the idea of trying an earlier kernel! Unfortunately, even when I booted Fedora 38 the keyboard was unresponsive. I find that really mystifying because my keyboard had worked fine last year when my laptop was running Fedora 38.

Do you have any other ideas for troubleshooting tactics to try? There must be some log I can look at or debugging flag I can turn on or boot/kernel parameters I can play with . . .

12

Ridiculous. Tell them it’s a hardware issue.

Boot Ubuntu live disk and see if it works. Boot windows and see if it works (maybe installed on external usb drive).
Try external keyboard.

Check logs for suspicious messages.

13

I have to echo the previous statement, This is a Hardware issue. If the machine was sold with Fedora on it, and they do not support Fedora that (depending on your jurisdiction) can violate consumer protection laws. It’s clear the machine needs to be serviced.

Check dmesg for Device Detection:

  • Open a terminal and use the dmesg command to check for any messages related to the keyboard: dmesg | grep -i keyboard

Check Input Devices:

  • Use the lsinput command to list input devices and check if your keyboard is recognized:

Restart the Input Service:

  • Restart the input service. The command may vary based on your distribution. For example, on systems using systemd:

sudo systemctl restart systemd-udevd

Check for Key Events:

  • Use the evtest command to check for key events from your keyboard:

sudo evtest

  • Select your keyboard device and press some keys to see if events are registered.

Kernel Modules:

  • Check if the necessary kernel modules for your keyboard are loaded. Use the lsmod command to list loaded modules.

if You are using X11 :

Check Xorg Log (if using X11):

  • If you are using Xorg, check the Xorg log for any keyboard-related issues:

cat /var/log/Xorg.0.log | grep -i keyboard

14

They offered for me to ship my laptop to them and have them take a look, but since it’s out of warranty I would potentially have to pay a lot for this. The fact that they told me Fedora was “unsupported” and the general tone of the person I spoke to did not give me much confidence that this method would help.

I agree it’s completely ridiculous, but I understand that there’s not enough of an economic incentive for them to support Linux and it’s not clear to me yet that this is the right hill to die on for my particular problem.

So in the spirit of Linux, I will see how far I can get using the wonderful community here and the tools available to me!

Good idea, I’ll try Ubuntu next if I can’t get any information from the logs. An external keyboard works okay (indeed, that’s how I’m typing this post).

This has already yielded some helpful information (guided by hammerhead corvette’s answer), see below.

15

Wow, thank you for your detailed list of tests to run! Here’s the information I could gather (in decreasing order of helpfulness).

Let me know if any of this jumps out or gives you further ideas.

dmesg: There are three relevant lines:
[ 1.976394] atkbd serio0: Failed to deactivate keyboard on isa0060/serio0

[ 2.392612] atkbd serio0: Failed to enable keyboard on isa0060/serio0

[ 2.392744] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3

According to sudo evtest, the AT Translated Set 2 keyboard is the inbuilt laptop keyboard, and according to lsinput, its physical address (?) is isa0060/serio0/input0. It looks like we’ve finally identified a concrete error, this is promising!

I googled this error and did not immediately find anything helpful, though it has come up for others, so that’s also promising.

Another interesting comment: my keyboard is actually mapped to two devices: there are also ThinkPad Extra Buttons in the top row. These also do not work, but I was not able to find this reflected in the logs anywhere.

lsinput: This helped identify the physical address (?) of the keyboard, see above.

evtest: This helped identify the name and device (/dev/input/event3) for the keyboard. It did not register any input from the inbuilt keyboard.

restarting input service: I tried this, no effect.

kernel modules: I was able to list the kernel modules but I don’t really understand how to parse this, more tips are appreciated if we think this is a helpful direction.

Xorg log: Sadly, I’m using Wayland. Previously I used X11 and switched to see if it would resolve the keyboard issue, but it didn’t.

Are you so certain that it is a hardware issue? The keyboard always works in the BIOS etc menus before Fedora is loaded. It also sometimes starts working again after I reboot my laptop, and whether it works or not only changes after powering off the system.

I guess I’m hoping it’s not a hardware issue because then I’ll have to fight with Lenovo.

Could this be a firmware problem of some kind? When I loaded an older version of Fedora from my usb drive, would that have used different firmware, or is the firmware somehow “inbuilt”?

16

To filter the output to show only lines containing specific keywords, you can use the grep command. For example:

lsmod | grep -E 'kbd|input|hid'

If you see modules related to keyboards or input devices, it indicates that the necessary modules are loaded. If not, you might need to load the appropriate modules manually or troubleshoot why they are not being loaded.

sudo modprobe kbd to manually load a module.

In an extreme case ( maybe this one. . . ) I would go out of my way to disconnect the keyboard from the main board. Boot with an external keyboard do some things, and shutdown. reconnect the keyboard , unplug the external one and see if that helps.

I will go through your post again, I’m quite busy this a.m, but will check in soon.

17

Keyboard issues on Lenovo Yoga Slim 7i Pro suggest some workarounds for the widely used i8042 PS/2 controllers:

~% modinfo i8042
name:           i8042
filename:       (builtin)
license:        GPL
file:           drivers/input/serio/i8042
description:    i8042 keyboard and mouse controller driver
author:         Vojtech Pavlik <vojtech@suse.cz>
parm:           nokbd:Do not probe or use KBD port. (bool)
parm:           noaux:Do not probe or use AUX (mouse) port. (bool)
parm:           nomux:Do not check whether an active multiplexing controller is present. (bool)
parm:           unlock:Ignore keyboard lock. (bool)
parm:           probe_defer:Allow deferred probing. (bool)
parm:           reset:Reset controller on resume, cleanup or both (reset_param)
parm:           direct:Put keyboard port into non-translated mode. (bool)
parm:           dumbkbd:Pretend that controller can only read data from keyboard (bool)
parm:           noloop:Disable the AUX Loopback command while probing for the AUX port (bool)
parm:           notimeout:Ignore timeouts signalled by i8042 (bool)
parm:           kbdreset:Reset device connected to KBD port (bool)
parm:           dritek:Force enable the Dritek keyboard extension (bool)
parm:           nopnp:Do not use PNP to detect controller settings (bool)
parm:           debug:Turn i8042 debugging mode on and off (bool)
parm:           unmask_kbd_data:Unconditional enable (may reveal sensitive data) of normally sanitize-filtered kbd data traffic debug log [pre-condition: i8042.debug=1 enabled] (bool)

First verify that your system is running this module: lsmod | grep i8042. If so, it would be worth trying the suggested workarounds, and add the results to your bug report.

18

Thank you for your quick reply!

Thanks for these instructions—I know how to filter terminal output but am unfamiliar with many of the acronyms for kernel modules.

I used your suggested search parameters and also modinfo to generate a list of potentially relevant kernel modules. (Sadly, I might have missed something because for some modules the “description” field is empty.) They are:

  • uinput
  • intel_hid
  • sparse_keymap
  • hid_multitouch
  • i2c_hid_acpi
  • i2c_hid
  • thinkpad_acpi
  • serio_raw

Based on gnwiii’s answer below, I suspect that some kernel modules relevant to the keyboard might not be loaded. Does that seem possible to you?

Is there a way to generate a list of kernel modules that were loaded on previous boots? This could help indicate if there’s been a change.

I tried loading kbd and it failed because apparently my system does not have this module installed.

I also tried loading i8042 using the suggestion below, but it seemed to fail: I don’t see it loaded when I run lsmod.

Let me see how far we can get by playing with these kernel modules, but if all software approaches fail I might just be desperate enough to do surgery on my computer. (Maybe what you proposed is actually really easy, but I don’t have much experience working with hardware at this level.)

19

Interestingly, my system is not running this module (see my answer to hammerhead corvette above) and it seems that I am unable to load it. Could this be a clue?

Since my last message, I rebooted my computer several times. The keyboard is still not working, but I noticed that the kernel logs have changed: the atkbd error I mentioned above is no longer present. Moreover, the kernel module atkbd is not loaded at all (and I cannot seem to load it using modprobe).

Perhaps there are two problems here: first, some kernel module is interacting in a bad way with my keyboard, and second, some kernel modules are not automatically loading. Could that be?

20

Run modprobe -v i8042 and then journalctl -b -g i8042. On my Dell desktop, modprobe -v .. gives no output, but journalctl has:

... kernel: i8042: PNP: No PS/2 controller found.