Fedora hangs on boot after upgrading to kernel 6.3.4

As per above in this thread just for the hell of it with 6.3.4-201 in place did
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
[sudo] password for robertk:
Generating grub configuration file …
Adding boot menu entry for UEFI Firmware Settings …
done

Trying boot again with rebuild grub.cfg into 6.3.4-201
This made no difference. Still does not boot and gets stuck if auto start at
“Boot Fedora Linux 6.3.4-201 … 38 Workstation Edition”

Did a ctl-alt-del - reboots but post generates a non-normal beep but inspecting UEFI/BIOS no apparent error codes listed.

Reboots fine selecting 6.2.15-300

Created just for this orginal bad kernel sudo journalctl -r --boot=-1 and uploading results to

As per bug report experimental 6.3.5-201 kernel worked re boot. See bug report for detail.

@py0xc3 yes, I tested both the official 6.3.5-200 kernel and the experimental 6.3.5-201 kernel and sadly neither one booted for me.

I followed your instructions by attempting to boot with 6.3.5-200, waiting a few minutes on the black screen, and force turning off my system. I then booted immediately into the working 6.2.15 kernel and pulled up sudo journalctl -r --boot=-1 but the logs only described the previous time kernel 6.2.15 booted earlier and no logs were present from the time I just tried to boot 6.3.5-200.

I am linking to the output from sudo journalctl -r --boot=0 to show what happened for my current boot on kernel 6.2.15. Please let me know if you notice anything that I should address, and thanks again for your help.

Password - Pv&#4FoQ20%1

@rairai9 , your issue is definitely different from that of Robert. I do not split the topic at this point because it is already linked on several other pages, while some comments also refer to both issues we have here (that would be confusing).

I have added comment to your bug report with some minor points for the people there. I hope someone there can make something out of the existing data. After skimming your logs, I cannot see issues that could explain what happens in later kernels. So I assume the issue did not exist in 6.2.X. If the issue has risen with 6.3.X, it should be in good hands in the bugzilla.

However, there is something you could try to maybe get some more data: Currently, the grub “options” you boot your kernel with include “rhgb quiet” → remove these two options from the file of the 6.3.5 kernel, and then try it again. Maybe this leads to some more output. If so, just make a screenshot (with a camera or so). Then let us know here.

Alternatively to changing the options in the grub file in advance, you can also press “E” on the “6.3.5-200” option within the grub menu at startup and then modify the options line at boot time (this is not permanent).

Created attachment 1969412 [details] Journalctl for 6.3.5-test-201 Previous attachment F38-kernel-6.3.5-test-201-Success.txt uploaded the log for journalctl -r boot=-1 which effectively was 6.2.5.

This upload is for the booting $ uname -a Linux earth 6.3.5-201.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jun 1 15:13:15 UTC 2023 x86_64 GNU/Linux

Sorry!

@rkoppelh It seems that the fix for your issue has been already put into the new kernel 6.3.6. There are no further reports necessary. It was only necessary to know if the issue you experienced has been solved by the experimental kernel, which it has if I understood you right. Also, with regards to the bug report, please keep focused on the very issue.

If you want, you can contribute in bodhi to test the new kernel (6.3.6) to ensure that the issue is gone once you update next with dnf:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-ed3bcae7e8
→ if you want to help verifying that the issue is solved, keep watching this page. Once all automated tests are successfully conducted, the status will be set to testing. Then, below the “Reboot required” comment in the top, there will be another line with a dnf command, which will be sudo dnf upgrade --refresh --advisory=<hash value> → then, with this command, you can install the 6.3.6 kernel and test if it boots properly and if the issue is solved as expected. It will work as usual with updates.

If so, you can login to bodhi with your normal Fedora credentials, and at the bottom of the page, you can add your results. You can then see there three options to add results (positive & negative each): Karma, BZ#2211784, regression. Unless you want to enter into regression testing, please leave the regression status neutral. If you “play” with the kernel just by working with it in order to see if everything works fine with it, you can add Karma if you want (Karma = generally working). Most important in your case: BZ#2211784 is the bug you reported. With this option, you confirm that the bug is solved or not. If the kernel works for you without causing the issue you reported, then add here a thumbs up. If it does not solve the issue, thumbs down. Obviously, in your case with BZ#2211784, a positive Karma would imply a positive answer to BZ#2211784 :wink:

Once sufficient people have tested the new kernel (and at the best if some confirmed that the issue BZ#2211784 is solved within that kernel), it will be pushed to stable to then end up in the usual daily updates.

Bodhi - Fedora Project Wiki

You do not need to paste your results here, only in bodhi.

1 Like

@py0xc3 Yes, I can confirm that the issue did not exist in any 6.2.X kernels but has only appeared in 6.3.X kernels. I tried booting into 6.3.5-200 with the “rhgb quiet” options removed from the kernel file and here is the output I saw when attempting to boot the kernel:

 Booting a command list

EFI stub: UEFI Secure Boot is enabled.

When booting the kernel with secure boot disabled, the output is:

 Booting a command list

In both cases, the system displays one of the two messages listed above and hangs indefinitely.

@rkoppelh Glad you found a solution to your issue!

I didn’t do much. Seems that if nvidia-drm.modeset=1 the new kernel does not install drm something or other. 6.3.5 forgot to incorporate that known change.

Sadly looks like your system is very different with same symptoms though I do not run UEFI secure boot. Namely because my MOBO was manf’d during BIOS → UEFI transition period (2012), while Intel UEFI, the secure boot and nvme is it’s weak point. I am willing to bet if I turned on secure boot (not willing to go there) I would have with 6.3.5 the same issue text as you. See nvidia comment in Understanding nvidia-drm.modeset=1 (NVIDIA Linux driver modesetting) - #2 by generix - Linux - NVIDIA Developer Forums

Maybe just for the hell of it on command line try setting to 0 or removing nvidia-drm.modeset=1 if you have it as a wild arse guess? Or if you don’t have it put it in?

My experience with Centos 7 to F36, now 38, is that whenever the system does not boot or has black screen it was always related to nvidia issues. I’ve never seen boot problems due to something else since using linux from 2008. Usually the next kernel fixed it. Though this time was the worst as no dmesg or journal logs. Literally no boot.

I was hopping opensource nvidia, nuveau, wayland would fix these nvidia issues. But I find KDE and wayland are a bit iffy. Xorg with KDE & Nvidia rocksolid. Don’t know about Gnome. I hate that desktop. I wish FEDORA/REDHAT/IBM just bit the bullet and work with Nvidia to fix these persistent annoying niggles permanently. Yes I get the Free software arguments. But you also have to pragmatic, nvidia makes good GPUs, they have a monopoly, i.e. they’ve won and unless someone comes up with competition (AMD tried) which I now doubt, the Linux world is going to have to kiss the ring. Herassy I here you say! Maybe, but it’s the reality.

Thanks for the suggestions, my command line didn’t have nvidia-drm.modeset=1 so I tried adding it and booting into 6.3.5-200 and then I tried adding nvidia-drm.modeset=0 and booting into the kernel. Unfortunately neither option changed the result since I still saw the same output as before and the system hung indefinitely:

 Booting a command list

Sad to hear clearly something unique about your system.

In the bugzilla report re my bug, look through journalctl outputs therein and compare my 6.2.15 and 6.3.6 with your availble journal 6.2.15. Maybe you can see something that is uniquie in first 1000 lines. After that I’d say our system will wildly go different to mine. Just the boot part even first lines would give maybe a clue.

Again wild arse guess.

Please all, keep focused on the very problem we still have here, which is the one without nvidia.

@rairai9 You have no nvidia card. This is why nvidia-drm.modeset=1 does not suit your kernel.

Concerning 6.2.X kernels, this was about potential issues that are already logged at 6.2.X but that did not cause the 6.2.X kernels to break (so that it feels for you like there is nothing). So, I was seeking a hint that indicates why 6.3.X broke on your system. It was unlikely that 6.2.X already logged something that later caused another kernel to break, but it was worth a try.

 Booting a command list
EFI stub: UEFI Secure Boot is enabled.

Unfortunately, this is not indicative for us.


Since kernel 6.3.6 is already on its way, it is worth to hope for it to solve the issue. If you want to check it earlier, feel free to use bodhi to test it already now. For that, see: Fedora hangs on boot after upgrading to kernel 6.3.4 - #37 by py0xc3

If you do so, and if you want to give feedback in bodhi, please keep the “BZ#2211784” option neutral since your issue is a different one. But if 6.3.6 really solves your issue, you can add a comment to bodhi that 6.3.6 solves “BZ#2212012” (which is your bug report). Let’s hope…

If that does not work, I think we have to wait for the bug report to be processed. Unless someone else has time to play with your grub (which would end up in longer time investments). But I am not sure if that would make a difference. It any case, keep watching your bug report, and offer the information that people there ask for. Also, if 6.3.6 solves your issue, please let us know.

Also, you can keep watching F37 - 6.3.4-101.fc37 and 6.3.5-100.fc37 black screen after GRUB on AMD desktop, Nvidia laptop works and its related bug report (if that user adds one) → given that this user’s system is able to mount root, I assume this is a different bug. But since the hardware is completely different while using F37 (you F38), it is also possible that the same bug just behaves different on his system. So it is worth to keep checking if something comes up that solves their issue, and if so, let’s see if that helps you too. If that user finds a solution, feel free to trigger me here with @py0xc3 so that I can transfer his solution to your machine (do not just do what they do! It’s F37).

@py0xc3 I’ll just wait until 6.3.6 becomes a regular update and then test it out, thanks! I’ll let everyone know if that kernel works for me.

Sounds like a plan :wink:

@py0xc3 I just upgraded to kernel 6.3.6 and I still have the same issue as I did with kernels 6.3.4 and 6.3.5, sadly. I updated my bug report with this new info.

Hi.

I think, I have similar problem. Kerenl 6.2.15 is the last one, what works fine.

My computer doesn’t have any NVIDIA and AMD parts.

Having the same issues as @rairai9 and @glewik . The output of cat /proc/sys/kernel/tainted is 0 for me as well. There are no logs present in journalctl for the unbootable kernels (6.3.4 and above) once I boot with a working one (6.2.15)
Fortunately I was able to grab some logs with journalctl while booting with 6.3.6 and removing the rhgb quiet part from the command.
Please refer to the attached imgur album : Imgur: The magic of the Internet

Failed to mount sysroot.mount
Dependency failed for initrd-rootfs.target
Dependency failed for initrd-parse-etc.service

@rairai9 we became just aware that the template of the kernel bug report has been broken (it seems to no longer ask for some information). I assume the bug report did not ask you for dmesg. Could you add this now? So get the output of dmesg and add the file/link to the bug report (please do not paste the content as comment). Also note that the dmesg is taken from 6.2.15.

@ all others:
IF YOU FULFILL ALL OF THE FOLLOWING CONDITIONS:

  • output of cat /proc/sys/kernel/tainted returns 0.
  • the “symptoms” / behaviors are the same as that of Sean / @rairai9 when you boot the current kernel 6.3.6
  • You use F38 on x86_64 (you can find out both with uname -r)
  • 6.2.15 is the newest kernel that works.

In this case, add a short note to the bug report of Sean and note that you have the same issue, so that we can consolidate a little if and how far this is a wider phenomenon. But please do only add a short note without journals, logs or so: Just inform that cat /proc/sys/kernel/tainted returns 0 and that you have the same issue as Sean with 6.3.6 on F38/x86_64.


Those with F37: please check F37 - 6.3.4-101.fc37 and 6.3.5-100.fc37 black screen after GRUB on AMD desktop, Nvidia laptop works

I am not able to check content of this file when i am trying using kernel higher than 6.2.15. When I using kernel 6.2.15 result cat /proc/sys/kernel/tainted is 0

3 posts were split to a new topic: Fedora enters emergency mode with all kernels after 6.2.14-300.fc38.x86_64