Fedora hangs on boot after upgrading to kernel 6.3.4

in my case of 6.3.4 boot hang, it was just to grub menu construction failure, presumably during install of kernel package.

menuentry 'Fedora Linux (6.3.4-201.fc38.x86_64) 38 (Server Edition)' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-6.3.4-201.fc38.x86_64-advanced-9773eea1-a504-47c6-8c0d-8dbd4bb3022d' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod xfs
        search --no-floppy --fs-uuid --set=root f275b3b2-05fe-4450-844e-534180d20a40
        echo    'Loading Linux 6.3.4-201.fc38.x86_64 ...'
        linux   /vmlinuz-6.3.4-201.fc38.x86_64 root=/dev/mapper/fedora_fedora-root ro rd.lvm.lv=fedora_fedora/root rhgb quiet 
}
menuentry 'Fedora Linux (6.2.15-300.fc38.x86_64) 38 (Server Edition)' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-6.2.15-300.fc38.x86_64-advanced-9773eea1-a504-47c6-8c0d-8dbd4bb3022d' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod xfs
        search --no-floppy --fs-uuid --set=root f275b3b2-05fe-4450-844e-534180d20a40
        echo    'Loading Linux 6.2.15-300.fc38.x86_64 ...'
        linux   /vmlinuz-6.2.15-300.fc38.x86_64 root=/dev/mapper/fedora_fedora-root ro rd.lvm.lv=fedora_fedora/root rhgb quiet 
        echo    'Loading initial ramdisk ...'
        initrd  /initramfs-6.2.15-300.fc38.x86_64.img
}

The older kernel has it’s ramdisk directive, but the new one has the kernel directive with no initrd directive.

manually running sudo grub2-mkconfig -o /boot/grub2/grub.cfg while booted in to the old kernel then correctly regenerated the grub config.
I can now boot the new kernel fine.

Sadly my autohide was enabled. Your post made me realise from diagnostic point of view that may not be a good idea as may grub.cfg does not contain the menuentry items you have.

But I will rework my grub2-mkconfig when a I get a chance. Either nvme driver load was a problem or as you point out grub construction issue which my grub.cfg as setup may not show.

How can I check where my taints are?

I do yum ‘priorities’ to prevent this but F38 may have moved on that this does not work. It should only be nvidia drivers but some of my audio and other multimedia codecs (workstation after all) from rpmfusion maybe culprits.

In infostat I did notice odd Nvidia messages I wasn’t aware off that I need to investigate. A curious impact of F38 upgrade a month ago was that Login is impossible with Plasma xorg setting. Can only login on Plasma wayland. But still find wayland a bit unstable with nvidia getting occasional freezes with F36 as well. That’s why xorg and proprietary drivers. That’s rock solid.

I filed a bug report with my fpaste output and the output from cat /proc/sys/kernel/tainted (which returned a value of 0). I also added the link to this discussion as well as the bodhi page. I will keep you updated with any developments. Thanks!

I actually also raised a bug report with sadly my tainted kernel. We’ll see what happens.

https://bugzilla.redhat.com/show_bug.cgi?id=2211784

RH review suggest problem with nvidia driver?

I will try grub update and then nvidia kernel update when I get a chance.

@rairai9 Please provide a link to your bug report.

We should link the two reports to avoid additional work and to enable
developers to have all related data together. That fosters problem solving.

Here is the link to my bug report - 2212012 – Fedora 38 workstation hangs on boot after upgrading to kernel 6.3.4

When filing the report, I wasn’t asked to provide any specific logs, but I included my system’s output from fpaste --sysinfo --printonly and cat /proc/sys/kernel/tainted as an attachment.

I would be happy to provide any other logs or other system info if needed. Thanks!

See link above/previous post

The kernel provided in 2211784 – fedora 38 kernel 6.3.4 boot fail upon upgrade seems to solve the issue for the users that tested it so far.

Please review the ticket first to check if your problem is the same. If so, and if the kernel works out for you, feel free to provide feedback in the bug report. This fosters to get the related fix asap in the next official kernel that will end up in your updates.

Hi can you pls provide a reference, link on how to do that. Never had to install kernel outside package system before as a test. \Don’t want to stuff that up.

If the issue of 2211784 – fedora 38 kernel 6.3.4 boot fail upon upgrade applies to you and if you want to test the kernel that Justin provided, you can install that one by
sudo dnf update https://kojipkgs.fedoraproject.org//work/tasks/3237/101713237/kernel-6.3.5-201.fc38.x86_64.rpm https://kojipkgs.fedoraproject.org//work/tasks/3237/101713237/kernel-core-6.3.5-201.fc38.x86_64.rpm https://kojipkgs.fedoraproject.org//work/tasks/3237/101713237/kernel-modules-6.3.5-201.fc38.x86_64.rpm https://kojipkgs.fedoraproject.org//work/tasks/3237/101713237/kernel-modules-core-6.3.5-201.fc38.x86_64.rpm https://kojipkgs.fedoraproject.org//work/tasks/3237/101713237/kernel-modules-extra-6.3.5-201.fc38.x86_64.rpm
→ these are from https://koji.fedoraproject.org/koji/taskinfo?taskID=101713237 , which is the descendant of https://koji.fedoraproject.org/koji/taskinfo?taskID=101713227

Be aware: This kernel is only for x86_64. dnf will check if the packages fit your system. This should look like a normal kernel update: dnf downloads the packages, shows you what it will install and what it removes, then you can chose Y/N and then it installs. Do not skip any errors or warnings! If any come up, let us know here.

Please check in advance that another kernel that works remains installed! E.g., if the last kernel that works for you is 6.2.15., check if dnf will remove 6.2.15 before chosing Y (this can happen if you have deployed already earlier 6.3.X kernels or so). By default, Fedora keeps only the most recent 3 kernels. If that issue happens, I suggest to change (in advance to the dnf command) the option installonly_limit=3 to installonly_limit=4 in /etc/dnf/dnf.conf, which increases the number of kernels that remain installed to 4.

Also, it can happen on your systems that dnf does not install all packages that I have put in the dnf command. If dnf chooses itself to ignore some, this is ok. You can proceed as long as no warnings/errors come up.

Also, be aware that this kernel is experimental. This is why it should be used for testing the bug fix only. Once some people confirmed that it works, the fix will be pushed through the whole testing processes of Fedora until it ends up in one of the next kernels to be deployed in “production”.

@rairai9 I know you have tainted = 0, which indicates that you have no nvidia. But given the correlations of the occurrences we have so far, it might be still worth a try if this kernel helps you too → Fedora hangs on boot after upgrading to kernel 6.3.4 - #24 by py0xc3

Alternatively, if it does not help, you can check if that is indicative for you: Can't boot after update to kernel v6.3.4 - #5 by computersavvy

Got work to do now. End of today, Australian time, Monday, I’ll have a go at testing this. I hope after my timeshift upgrades that it will take me back if ‘shit hits the fan’.

installonly limits is already at 10.

Bit anxious though - never done this before.

Thanks for the info.

@py0xc3 I tried both the experimental kernel and the kernel command line options suggested in Can't boot after update to kernel v6.3.4 - #5 by computersavvy and unfortunately neither worked for me. The experimental kernel still boots to the same black screen with a single underscore (_) that I see when attempting to boot 6.3.4. I also just installed 6.3.5 as an update via GNOME software but that kernel does not boot either and results in the same black screen with an underscore that I see when trying to boot 6.3.4.

I still have kernel 6.2.15 installed and will keep using it for the time being since none of the newer kernels boot on my system for now. I also added a comment to my bug report explaining that kernel 6.3.5 hangs on boot the same way that 6.3.4 does.

Thanks for all of the suggestions so far and please let me know if there is something else I can try.

@rairai9 sad to hear. But be aware that the experimental 6.3.5 kernel provided by Justin is not equal to the one in the “stable” update repository → in the official updates, you have 6.3.5-200, the
experimental is 6.3.5-201 → just to be on the same page: you tested both?

In case both 6.3.5 kernels didn’t work for you, let’s assume you have maybe something completely different and start from the beginning.

First, even if your screen is black immediately after grub, I would like to check if the root file system is maybe already mounted when the system breaks. That would be very helpful, so let’s give it a try.

Therefore, please start your system with the newest kernel that does not work (I assume this is 6.3.5-200; but do not use the experimental 6.3.5-201 kernel for this!). Now you experience the black screen. Well, let’s just wait a minute at this point. Then, force turn off your machine. Then, boot immediately next the working 6.2.15 kernel. Once up, please get the output of sudo journalctl -r --boot=-1 → this gives us the journal from the respectively last boot. If you want, you can
already compare the system time of the journal output with the time you tried the broken kernel. If the time fits, the root file system was mounted and provides us (hopefully valuable) logs about what happened. In either case, please provide the journal logs. Even if it is only the logs of the last 6.2.15 kernel boot (in case the system breaks with 6.3.5 before the root file system is mounted), it could contain errors that indicate why later kernels break completely. Otherwise, we have to play with grub, but I would be glad if we could avoid this.

Feel free to replace data you consider private in the logs (usernames, IP/MAC addresses, …).

I am about to try this 'experiemntal kernel too this morning. Need to do backups first. Will advise.

Please note that some refer to removing from grub command line kvm_ignore_msrs=1 to solve the boot problem. My /etc/default/grub

GRUB_TIMEOUT=“5”
GRUB_DISTRIBUTOR=“$(sed ‘s, release .*$,g’ /etc/system-release)”
GRUB_DEFAULT=“saved”
GRUB_DISABLE_SUBMENU=“true”
GRUB_TERMINAL_OUTPUT=“console”
GRUB_CMDLINE_LINUX=“rd.lvm.lv=VG01_nvme_pcie/rootfs rd.luks.uuid=luks-160cee22-ab53-47b2-a48a-382fca72928a rhgb rd.driver.blacklist=nouveau modprobe.blacklist=nouveau nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init”
GRUB_DISABLE_RECOVERY=“true”
GRUB_ENABLE_BLSCFG=“true”

suggest kvm_ignore_msrs=1 it’s not a default setting. About to reboot to check grub menu.

I can confirm my grub2 menu on boot does NOT have kvm_ignore_msrs=1

As per above in this thread just for the hell of it with 6.3.4-201 in place did
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
[sudo] password for robertk:
Generating grub configuration file …
Adding boot menu entry for UEFI Firmware Settings …
done

Trying boot again with rebuild grub.cfg into 6.3.4-201
This made no difference. Still does not boot and gets stuck if auto start at
“Boot Fedora Linux 6.3.4-201 … 38 Workstation Edition”

Did a ctl-alt-del - reboots but post generates a non-normal beep but inspecting UEFI/BIOS no apparent error codes listed.

Reboots fine selecting 6.2.15-300

Created just for this orginal bad kernel sudo journalctl -r --boot=-1 and uploading results to

As per bug report experimental 6.3.5-201 kernel worked re boot. See bug report for detail.