I’ve gotten quite a bit further this time. I first removed the quiet
boot parameter to get more information. After a couple of reboots, I got the crash again. It is a kernel panic, seemingly due to a null pointer dereference.
Because the display was all glitchy and sending images wasn’t ideal, I decided to try to get Kdump working. After finding out that you need to allocate more RAM to crashkernel due to running out of memory during LUKS decryption (I settled for 2GB for now), I got it working.
The crash happens before Kdump has the chance to initialize. Therefore, I needed to set up “early kdump” according to this Red Hat guide. Thankfully, it all worked, and I successfully captured a kernel panic!
Snippet of kernel panic dmesg
[ 15.358886] systemd[1]: Started systemd-journald.service - Journal Service.
[ 15.360430] audit: type=1130 audit(1723967809.810:7): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-journald comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.363306] audit: type=1130 audit(1723967809.813:8): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=kmod-static-nodes comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.366135] audit: type=1130 audit(1723967809.816:9): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=modprobe@configfs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.368992] audit: type=1131 audit(1723967809.816:10): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=modprobe@configfs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.370551] audit: type=1130 audit(1723967809.820:11): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=modprobe@dm_mod comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.370562] audit: type=1131 audit(1723967809.820:12): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=modprobe@dm_mod comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.374420] audit: type=1130 audit(1723967809.824:13): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=modprobe@drm comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.374430] audit: type=1131 audit(1723967809.824:14): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=modprobe@drm comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 15.428941] systemd-journald[984]: Received client request to flush runtime journal.
[ 15.475164] systemd-journald[984]: /var/log/journal/7d79d8fa0ad3463fa2abea9e44e9e4f7/system.journal: Journal file uses a different sequence number ID, rotating.
[ 15.476026] systemd-journald[984]: Rotating system journal.
[ 15.760318] zram0: detected capacity change from 0 to 16777216
[ 15.769513] BTRFS info: devid 1 device path /dev/mapper/luks-e55b5050-373b-43a5-a42b-e799a2320ec7 changed to /dev/dm-0 scanned by (udev-worker) (1050)
[ 15.778909] BTRFS info: devid 1 device path /dev/dm-0 changed to /dev/mapper/luks-e55b5050-373b-43a5-a42b-e799a2320ec7 scanned by (udev-worker) (1050)
[ 15.863420] Adding 8388604k swap on /dev/zram0. Priority:100 extents:1 across:8388604k SSDsc
[ 15.913499] EXT4-fs (nvme1n1p2): mounted filesystem a35be0ba-1087-4d51-92be-830f3b57d209 r/w with ordered data mode. Quota mode: none.
[ 15.937655] Consider using thermal netlink events interface
[ 15.986415] idma64 idma64.0: Found Intel integrated DMA 64-bit
[ 16.024011] mei_me 0000:00:16.0: enabling device (0004 -> 0006)
[ 16.042645] thinkpad_acpi: ThinkPad ACPI Extras v0.26
[ 16.044153] thinkpad_acpi: http://ibm-acpi.sf.net/
[ 16.045618] thinkpad_acpi: ThinkPad BIOS N24ET76W (1.51 ), EC N24HT37W
[ 16.047065] thinkpad_acpi: Lenovo ThinkPad T480, model 20L60034MX
[ 16.052235] e1000e: Intel(R) PRO/1000 Network Driver
[ 16.052237] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 16.052505] e1000e 0000:00:1f.6: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 16.065970] thinkpad_acpi: radio switch found; radios are enabled
[ 16.065980] thinkpad_acpi: This ThinkPad has standard ACPI backlight brightness control, supported by the ACPI video driver
[ 16.065981] thinkpad_acpi: Disabling thinkpad-acpi brightness events by default...
[ 16.083431] thinkpad_acpi: rfkill switch tpacpi_bluetooth_sw: radio is unblocked
[ 16.089898] i801_smbus 0000:00:1f.4: enabling device (0000 -> 0003)
[ 16.090028] i801_smbus 0000:00:1f.4: SPD Write Disable is set
[ 16.090049] i801_smbus 0000:00:1f.4: SMBus using PCI interrupt
[ 16.092639] i2c i2c-7: Successfully instantiated SPD at 0x50
[ 16.094929] i2c i2c-7: Successfully instantiated SPD at 0x51
[ 16.128386] thinkpad_acpi: battery 2 registered (start 0, stop 100, behaviours: 0x7)
[ 16.138552] thinkpad_acpi: battery 1 registered (start 0, stop 100, behaviours: 0x7)
[ 16.145928] ACPI: battery: new extension: ThinkPad Battery Extension
[ 16.203920] BUG: kernel NULL pointer dereference, address: 0000000000000108
[ 16.204798] #PF: supervisor read access in kernel mode
[ 16.205542] #PF: error_code(0x0000) - not-present page
[ 16.206282] PGD 0 P4D 0
[ 16.206982] Oops: Oops: 0000 [#1] PREEMPT SMP PTI
[ 16.207649] CPU: 2 PID: 1059 Comm: (udev-worker) Kdump: loaded Not tainted 6.10.4-200.fc40.x86_64 #1
[ 16.208364] Hardware name: LENOVO 20L60034MX/20L60034MX, BIOS N24ET76W (1.51 ) 02/27/2024
[ 16.209053] RIP: 0010:synaptics_process_byte+0x4b0/0xdf0
[ 16.209743] Code: ff 48 8b 6d 48 48 85 ed 0f 84 b3 fe ff ff 48 89 ef e8 44 d9 ff ff 0f b6 b3 e9 00 00 00 31 d2 48 89 ef 49 89 c4 48 85 c0 74 0d <83> b8 18 01 00 00 04 0f 84 11 08 00 00 e8 8e 91 fe ff e9 80 fe ff
[ 16.210476] RSP: 0000:ffffac4b40a27d80 EFLAGS: 00010086
[ 16.211197] RAX: fffffffffffffff0 RBX: ffff961682d8f200 RCX: 0000000000000010
[ 16.211918] RDX: 0000000000000000 RSI: 00000000000000aa RDI: ffff96168ca5d800
[ 16.212641] RBP: ffff96168ca5d800 R08: 0000000000000084 R09: 00000000000000c4
[ 16.213354] R10: 0000000000000000 R11: 0000000000000000 R12: fffffffffffffff0
[ 16.214069] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 16.214580] proc_thermal 0000:00:04.0: enabling device (0000 -> 0002)
[ 16.214792] FS: 00007fa0d88b4980(0000) GS:ffff9619ef300000(0000) knlGS:0000000000000000
[ 16.216219] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 16.217166] CR2: 0000000000000108 CR3: 000000010b6fc006 CR4: 00000000003706f0
[ 16.217168] Call Trace:
[ 16.217170] <TASK>
[ 16.217172] ? __die_body.cold+0x19/0x27
[ 16.217177] ? page_fault_oops+0x15a/0x2f0
[ 16.217182] ? exc_page_fault+0x7e/0x180
[ 16.217185] ? asm_exc_page_fault+0x26/0x30
[ 16.217189] ? synaptics_process_byte+0x4b0/0xdf0
[ 16.217193] ? pm_wakeup_dev_event+0x2d/0x60
[ 16.217197] psmouse_handle_byte+0x12/0x60
[ 16.217200] ps2_interrupt+0x9a/0x1a0
[ 16.217203] serio_interrupt+0x47/0x90
[ 16.217206] i8042_interrupt+0x12c/0x290
[ 16.217209] ? __count_memcg_events+0x75/0x130
[ 16.217212] __handle_irq_event_percpu+0x47/0x190
[ 16.227594] handle_irq_event+0x38/0x80
[ 16.227598] handle_edge_irq+0x8b/0x230
[ 16.228951] __common_interrupt+0x49/0xd0
[ 16.228955] common_interrupt+0x42/0xa0
[ 16.230230] asm_common_interrupt+0x26/0x40
[ 16.230233] RIP: 0033:0x7fa0d8947533
[ 16.231507] Code: 0d 00 00 ba 00 08 00 00 f7 d7 21 7d 9c 41 89 c7 29 f2 c1 ea 05 01 f2 48 8b 85 58 ff ff ff d1 65 c8 66 89 10 8b 45 9c 8b 75 c8 <89> c7 44 01 c8 21 f7 01 f8 41 81 ff ff ff ff 00 77 1e 4d 39 dc 0f
[ 16.231509] RSP: 002b:00007fff404a2730 EFLAGS: 00000202
[ 16.231511] RAX: 0000000000000100 RBX: 0000563dee8e37d0 RCX: 0000000000000088
[ 16.232853] input: ThinkPad Extra Buttons as /devices/platform/thinkpad_acpi/input/input8
[ 16.232882] RDX: 0000000000000297 RSI: 0000000000000220 RDI: 0000000000000100
[ 16.232884] RBP: 00007fff404a2810 R08: 00000000154793b1 R09: 0000000000000003
[ 16.235930] R10: 000000000002dfb0 R11: 00000000000002ea R12: 0000000000000440
[ 16.235932] R13: 00007fff404a66e0 R14: 00007fff404a2ac0 R15: 000000008467e257
[ 16.235936] </TASK>
[ 16.235936] Modules linked in: snd_seq(+) mc(+) snd_ctl_led think_lmi(+) bluetooth(+) processor_thermal_device_pci_legacy(+) thunderbolt(+) processor_thermal_device snd_seq_device firmware_attributes_class cfg80211(+) i2c_i801 processor_thermal_wt_hint wmi_bmof snd_pcm processor_thermal_rfim e1000e(+) thinkpad_acpi(+) i2c_smbus sparse_keymap snd_timer processor_thermal_rapl intel_rapl_common mei_me platform_profile processor_thermal_wt_req rfkill processor_thermal_power_floor mei processor_thermal_mbox int3403_thermal snd idma64 intel_xhci_usb_role_switch intel_soc_dts_iosf intel_pch_thermal soundcore int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad joydev loop nfnetlink zram dm_crypt i915 serio_raw crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic i2c_algo_bit ghash_clmulni_intel nvme drm_buddy ttm sha512_ssse3 uas drm_display_helper nvme_core sha256_ssse3 usb_storage sha1_ssse3 ucsi_acpi typec_ucsi typec nvme_auth cec video wmi fuse i2c_dev
[ 16.235984] CR2: 0000000000000108
Snippet of kexec dmesg
Aug 18 10:57:10 Astelpaju systemd[1]: Mounting sysroot.mount - /sysroot...
Aug 18 10:57:10 Astelpaju systemd[1]: Mounted sysroot.mount - /sysroot.
Aug 18 10:57:10 Astelpaju systemd[1]: Reached target initrd-root-fs.target - Initrd Root File System.
Aug 18 10:57:10 Astelpaju systemd[1]: initrd-parse-etc.service - Mountpoints Configured in the Real Root was skipped because of an unmet condition check (ConditionPathExists=!/proc/vmcore).
Aug 18 10:57:10 Astelpaju systemd[1]: Reached target initrd-fs.target - Initrd File Systems.
Aug 18 10:57:10 Astelpaju systemd[1]: Reached target initrd.target - Initrd Default Target.
Aug 18 10:57:10 Astelpaju systemd[1]: dracut-mount.service - dracut mount hook was skipped because no trigger condition checks were met.
Aug 18 10:57:10 Astelpaju systemd[1]: Starting dracut-pre-pivot.service - dracut pre-pivot and cleanup hook...
Aug 18 10:57:10 Astelpaju systemd[1]: Finished dracut-pre-pivot.service - dracut pre-pivot and cleanup hook.
Aug 18 10:57:10 Astelpaju systemd[1]: Starting kdump-capture.service - Kdump Vmcore Save Service...
Aug 18 10:57:10 Astelpaju kdump[830]: Kdump is using the default log level(3).
Aug 18 10:57:10 Astelpaju kdump[865]: saving to /sysroot/var/crash/127.0.0.1-2024-08-18-10:57:10/
Aug 18 10:57:10 Astelpaju kdump[870]: saving vmcore-dmesg.txt to /sysroot/var/crash/127.0.0.1-2024-08-18-10:57:10/
Aug 18 10:57:10 Astelpaju kdump[877]: saving vmcore-dmesg.txt complete
Aug 18 10:57:10 Astelpaju kdump[879]: saving vmcore
Aug 18 10:57:11 Astelpaju kdump.sh[880]:
Checking for memory holes : [ 0.0 %] /
Checking for memory holes : [100.0 %] |
Excluding unnecessary pages : [100.0 %] \
Copying data : [ 7.7 %] - eta: 1s
Copying data : [100.0 %] / eta: 0s
Copying data : [100.0 %] | eta: 0s
Aug 18 10:57:11 Astelpaju kdump.sh[880]: The kernel version is not supported.
Aug 18 10:57:11 Astelpaju kdump.sh[880]: The makedumpfile operation may be incomplete.
Aug 18 10:57:11 Astelpaju kdump.sh[880]: The dumpfile is saved to /sysroot/var/crash/127.0.0.1-2024-08-18-10:57:10//vmcore-incomplete.
Aug 18 10:57:11 Astelpaju kdump.sh[880]: makedumpfile Completed.
Aug 18 10:57:11 Astelpaju kdump[888]: saving vmcore complete
Edit: Screencapture of another kernel panic
Kdump warned that The kernel version is not supported
and The makedumpfile operation may be incomplete
, so there might still be some more configuration required.
The crash might have something to do with thinkpad_acpi
, but that’s just a guess.
Please let me know if you need any more help!