Consistent Kernel Null Pointer Dereference with 6.7.4-200.fc39.x86_64 on Fedora 39

Ask Fedora
, , ,
1

Hi,

I’m looking for how to debug this / where to go from here. After updating from 6.6.13-200.fc39.x86_64 to 6.7.4-200.fc39.x86_64, my system boots up OK, but then gets into a weird state where sudo hangs, most systemctl commands hang, and only some graphical applications launch.

I found a kernel panic in the dmesg:

[   58.212355] BUG: kernel NULL pointer dereference, address: 000000000000001b
[   58.212359] #PF: supervisor read access in kernel mode
[   58.212361] #PF: error_code(0x0000) - not-present page
[   58.212362] PGD 0 P4D 0
[   58.212365] Oops: 0000 [#1] PREEMPT SMP NOPTI
[   58.212367] CPU: 10 PID: 2865 Comm: NetworkManager Not tainted 6.7.4-200.fc39.x86_64 #1
[   58.212369] Hardware name: Gigabyte Technology Co., Ltd. B550 AORUS PRO AC/B550 AORUS PRO AC, BIOS F13j 04/23/2021
[   58.212371] RIP: 0010:iwl_mvm_vif_dbgfs_add_link+0x7e/0xc0 [iwlmvm]
[   58.212386] Code: 03 f3 48 ab 4d 85 ed 74 34 48 89 f3 4c 89 e9 48 c7 c2 b8 af ef c1 4c 89 e7 be 64 00 00 00 e8 a9 bd 1b c0 48 8b b5 c8 1c 00 00 <49> 8b 7d 28 4c 89 e2 e8 d6 3f 7a bf 48 89 83 a8 05 00 00 48 8b 44
[   58.212388] RSP: 0018:ffffb6fc4295b5e8 EFLAGS: 00010246
[   58.212390] RAX: 0000000000000018 RBX: ffff9e5b52aedc38 RCX: 0000000000000007
[   58.212391] RDX: 0000000000000000 RSI: ffff9e5b07b46d80 RDI: 000000004295b5ec
[   58.212393] RBP: ffff9e5b56bd1fc8 R08: ffffffffc1efafcb R09: 0000000000000000
[   58.212394] R10: 0000000000000001 R11: 0000000000000100 R12: ffffb6fc4295b5ec
[   58.212395] R13: fffffffffffffff3 R14: ffff9e5b52aedc38 R15: 0000000000000000
[   58.212397] FS:  00007f1749d9e540(0000) GS:ffff9e69fec80000(0000) knlGS:0000000000000000
[   58.212398] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   58.212400] CR2: 000000000000001b CR3: 0000000105024000 CR4: 0000000000f50ef0
[   58.212401] PKRU: 55555554
[   58.212403] Call Trace:
[   58.212405]  <TASK>
[   58.212407]  ? __die+0x23/0x70
[   58.212411]  ? page_fault_oops+0x171/0x4e0
[   58.212415]  ? srso_alias_return_thunk+0x5/0xfbef5
[   58.212419]  ? iwl_trans_txq_send_hcmd+0x378/0x450 [iwlwifi]
[   58.212432]  ? exc_page_fault+0x7f/0x180
[   58.212435]  ? asm_exc_page_fault+0x26/0x30
[   58.212440]  ? iwl_mvm_vif_dbgfs_add_link+0x7e/0xc0 [iwlmvm]
[   58.212451]  ? iwl_mvm_vif_dbgfs_add_link+0x77/0xc0 [iwlmvm]
[   58.212462]  iwl_mvm_mld_mac_add_interface+0x1ff/0x270 [iwlmvm]
[   58.212473]  drv_add_interface+0x52/0x240 [mac80211]
[   58.212500]  ieee80211_do_open+0x499/0x790 [mac80211]
[   58.212527]  ieee80211_open+0x66/0x90 [mac80211]
[   58.212551]  __dev_open+0xf4/0x1a0
[   58.212556]  __dev_change_flags+0x1d6/0x240
[   58.212559]  dev_change_flags+0x26/0x70
[   58.212562]  do_setlink+0x375/0x12d0
[   58.212566]  ? srso_alias_return_thunk+0x5/0xfbef5
[   58.212569]  ? kmem_cache_alloc+0x186/0x310
[   58.212571]  ? avc_alloc_node+0x24/0x170
[   58.212574]  ? srso_alias_return_thunk+0x5/0xfbef5
[   58.212576]  ? __nla_validate_parse+0x61/0xcf0
[   58.212580]  ? srso_alias_return_thunk+0x5/0xfbef5
[   58.212582]  ? avc_alloc_node+0x24/0x170
[   58.212586]  __rtnl_newlink+0x651/0xa10
[   58.212590]  ? __kmem_cache_alloc_node+0x198/0x2e0
[   58.212593]  ? rtnl_newlink+0x2e/0x70
[   58.212596]  rtnl_newlink+0x47/0x70
[   58.212599]  rtnetlink_rcv_msg+0x152/0x3c0
[   58.212602]  ? avc_alloc_node+0x24/0x170
[   58.212604]  ? srso_alias_return_thunk+0x5/0xfbef5
[   58.212606]  ? avc_has_perm_noaudit+0x6b/0xf0
[   58.212609]  ? __pfx_rtnetlink_rcv_msg+0x10/0x10
[   58.212612]  netlink_rcv_skb+0x5b/0x110
[   58.212617]  netlink_unicast+0x1a6/0x290
[   58.212620]  netlink_sendmsg+0x254/0x4d0
[   58.212623]  ____sys_sendmsg+0x399/0x3d0
[   58.212627]  ? copy_msghdr_from_user+0x7d/0xc0
[   58.212630]  ___sys_sendmsg+0x9a/0xe0
[   58.212637]  __sys_sendmsg+0x7a/0xd0
[   58.212641]  do_syscall_64+0x64/0xe0
[   58.212644]  ? srso_alias_return_thunk+0x5/0xfbef5
[   58.212646]  ? syscall_exit_to_user_mode+0x2b/0x40
[   58.212648]  ? srso_alias_return_thunk+0x5/0xfbef5
[   58.212650]  ? do_syscall_64+0x70/0xe0
[   58.212652]  entry_SYSCALL_64_after_hwframe+0x6e/0x76
[   58.212654] RIP: 0033:0x7f174aeddb6b
[   58.212672] Code: 48 89 e5 48 83 ec 20 89 55 ec 48 89 75 f0 89 7d f8 e8 b9 2e f7 ff 8b 55 ec 48 8b 75 f0 41 89 c0 8b 7d f8 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2d 44 89 c7 48 89 45 f8 e8 11 2f f7 ff 48 8b
[   58.212674] RSP: 002b:00007ffcd0b00cb0 EFLAGS: 00000293 ORIG_RAX: 000000000000002e
[   58.212676] RAX: ffffffffffffffda RBX: 000055b5d08a36b0 RCX: 00007f174aeddb6b
[   58.212677] RDX: 0000000000000000 RSI: 00007ffcd0b00cf0 RDI: 000000000000000d
[   58.212679] RBP: 00007ffcd0b00cd0 R08: 0000000000000000 R09: 0000000000000000
[   58.212680] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000052
[   58.212681] R13: 000055b5d08a36b0 R14: 0000000000000001 R15: 0000000000000000
[   58.212685]  </TASK>
[   58.212686] Modules linked in: rfcomm uinput uhid snd_seq_dummy snd_hrtimer xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nf_nat_tftp nf_conntrack_tftp bridge stp llc nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw iptable_security ip_set nf_tables nfnetlink ip6table_filter iptable_filter qrtr bnep nct6775_core sunrpc binfmt_misc iwlmvm snd_hda_codec_realtek snd_hda_codec_generic mac80211 ledtrig_audio snd_hda_codec_hdmi uvcvideo vfat fat intel_rapl_msr snd_hda_intel libarc4 uvc btusb snd_intel_dspcfg videobuf2_vmalloc intel_rapl_common snd_intel_sdw_acpi videobuf2_memops snd_usb_audio btrtl videobuf2_v4l2 snd_hda_codec btintel edac_mce_amd videobuf2_common btbcm snd_usbmidi_lib btmtk snd_hda_core snd_ump joydev videodev kvm_amd snd_hwdep
[   58.212737]  bluetooth snd_rawmidi mc apple_mfi_fastcharge snd_seq iwlwifi snd_seq_device kvm snd_pcm irqbypass snd_timer cfg80211 it87 hwmon_vid snd rapl rfkill gigabyte_wmi wmi_bmof soundcore i2c_piix4 k10temp pcspkr acpi_cpufreq gpio_amdpt gpio_generic loop dm_crypt hid_logitech_hidpp hid_logitech_dj amdgpu crct10dif_pclmul video crc32_pclmul crc32c_intel amdxcp i2c_algo_bit polyval_clmulni drm_ttm_helper polyval_generic ttm drm_exec gpu_sched ghash_clmulni_intel sha512_ssse3 drm_suballoc_helper drm_buddy nvme sha256_ssse3 drm_display_helper sha1_ssse3 hid_apple ccp nvme_core r8169 sp5100_tco cec realtek nvme_auth wmi scsi_dh_rdac scsi_dh_emc scsi_dh_alua ip6_tables ip_tables dm_multipath i2c_dev fuse
[   58.212780] CR2: 000000000000001b
[   58.212782] ---[ end trace 0000000000000000 ]---
[   58.212783] RIP: 0010:iwl_mvm_vif_dbgfs_add_link+0x7e/0xc0 [iwlmvm]
[   58.212794] Code: 03 f3 48 ab 4d 85 ed 74 34 48 89 f3 4c 89 e9 48 c7 c2 b8 af ef c1 4c 89 e7 be 64 00 00 00 e8 a9 bd 1b c0 48 8b b5 c8 1c 00 00 <49> 8b 7d 28 4c 89 e2 e8 d6 3f 7a bf 48 89 83 a8 05 00 00 48 8b 44
[   58.212796] RSP: 0018:ffffb6fc4295b5e8 EFLAGS: 00010246
[   58.212797] RAX: 0000000000000018 RBX: ffff9e5b52aedc38 RCX: 0000000000000007
[   58.212799] RDX: 0000000000000000 RSI: ffff9e5b07b46d80 RDI: 000000004295b5ec
[   58.212800] RBP: ffff9e5b56bd1fc8 R08: ffffffffc1efafcb R09: 0000000000000000
[   58.212801] R10: 0000000000000001 R11: 0000000000000100 R12: ffffb6fc4295b5ec
[   58.212802] R13: fffffffffffffff3 R14: ffff9e5b52aedc38 R15: 0000000000000000
[   58.212804] FS:  00007f1749d9e540(0000) GS:ffff9e69fec80000(0000) knlGS:0000000000000000
[   58.212805] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   58.212807] CR2: 000000000000001b CR3: 0000000105024000 CR4: 0000000000f50ef0
[   58.212808] PKRU: 55555554
[   58.212809] note: NetworkManager[2865] exited with irqs disabled

Full dmesg on the bad kernel: UNTITLED - Pastebin Service

Working kernel:

$ uname -av
Linux zenhat 6.6.13-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Jan 20 18:03:28 UTC 2024 x86_64 GNU/Linux

Bad kernel:

$ uname -av
Linux zenhat 6.7.4-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Feb  5 22:21:14 UTC 2024 x86_64 GNU/Linux
2

Suggest you report on the fedora bugzilla.
When you file the bug report against the kernel it will suggest trying the rawhide kernel.
You might what to do that and report here first.

dnf install fedora-repos-rawhide
dnf update --enablerepo=rawhide kernel
reboot

See if the 6.8.0-rc4 kernel works better

3

I just want to interject here for a quick question. On the forum, we typically ask people to “File a Bug” most people don’t, or don’t know how to. When you say Fedora Bugzilla, are you referencing the way to file a bug on Gnome ? Via Problem Reporting ? I believe it pushes it to Bugzilla ?

Sorry to hijack/interject.

4

There is a similar report with different hardware on reddit. Do you have the current iwlwifi-mvm-firmware.noarch (MVM Firmware for Intel(R) Wireless WiFi adapters)?

5

Good point, you would suggest this link in future when ask8ng for a bug report? Using Bugzilla :: Fedora Docs

6

I kept procrastinating on this, and it seems someone else must have either submitted this properly or found the issue. Thank you to them!

This no longer occurs with kernel 6.8.5-201.fc39.x86_64. (Up to current day, it was still occurring on every 6.7.* kernel - I just happened to restart for the first time in a few weeks today, and saw there was a 6.8 kernel installed so I thought I’d try it rather than continuing to run 6.6.13)

1 Like