Hi Team ( @computersavvy , @grumpey , @alan-jelaska , @operkh , @gnwiii , @j-pow , @serg1990io ),
I need your help and assistance on following.
How can I emulate a nvme disk when I have QEMU/KVM installed on my assembled PC ?
Can I get the same level of flexibility to try and replicate this issue on one or more public cloud providers with Fedora cloud-images ?
Please review and assist!
Hello!
Here is the result:
15775 lrwxrwxrwx 1 root root 7 Sep 5 10:30 /bin -> usr/bin/
About the request of hold off upgrade/update yesterday I’ve been tempted to update so I did. I can confirm that the problem is present with kernel 6.4.13 in the exact same way… Sorry for rushing things!
PS: the output I’ve posted is from the emergency shell from kernel 6.4.12
I got the same error. I noticed the file for this kernel in /boot/loaders/entries had “$Kernelopt” on the option line and not the options that were included in the previous kernel. Editing the file solved the problem for now.
I have no idea whre this $kernelopts was comming from; I did not see that on anothner system that updated “out of the box”
Please clarify – is your system is booting to emergency mode after an upgrade or do you sometimes boot to the desktop but then nvme drives stop working? Which kernel is having problems?
I rarely encounter emergency mode – the last time was when an update added text to the kernel command-line with misplaced quotes.
Typos?: /boot/loaders/entries should be /boot/loader/entries, and should$Kernelopt be $kernelopts ( RHEL8 mentions use of $kernelopts with grubby. In Fedora 38, /boot/grub2/grubenv is quite different from the RHEL8 example.
Running Fedora 38
Update kernel 6.4.7 → 6.4.13
After reboot I get emergency mode. Old kernel works fine
Fedora grub2 stopped using kernelopts a quite a while ago, as it was found out to be error prone. grub.cfg still sets it, but does not use it for anything.
Does you system boot from an nvme drive?
Please compare output from journalctl -xe with the first post in this thread. Look for the “failed to switch to root” message. You can also boot without “rhgb quiet” and look for messages similar to those in the bug report. You may want to add yourself to teh CC list.
Just a quick update for everyone else who experienced my same issue (and all the people who helped trying to diagnose it): on bugzilla (2232838 – Some NVME controllers fail to initialize with kernel 6.4.11 or later (nvme0: controller is down; will reset: CSTS:0xffffffff, PCI_STATUS=0xffff)) they have identified the root cause of the problem and solved it.
I’m on Fedora 38 and I’ve tested the proposed version of kernel 6.4.16 using the command sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-0ae1162af6 and I can confirm that I can successfully complete the boot process on kernel 6.4.16.
So I guess that once this solution is fully confirmed will be released for general public.
Thank you for your support!
This appears to be a new issue, so you should create a new topic.
The image mentions drm early in the call trace, which suggests a problem with display drivers. Please mention whether you using the internal display or external display with or without docking station.
Hello, I just wanted to say that I’ve been experiencing this issue myself on my own XPS 15 9560 laptop. This happens to me on both kernels 6.4.11 and 6.4.15 (those are the two affected kernels which GRUB offers to me; I have not tested the kernels in between). In my case though the results are more inconsistent; I’ve tried booting on these kernels many times but have only entered emergency mode once. The other times my computer either gets stuck during the booting process or becomes unresponsive shortly after booting, so I have to hold the power button to shut down the laptop.
I know that I’m late to this discussion; I’m somewhat of a Linux newbie and only just found this thread. It seems that this bug is going to be fixed in the upcoming kernel release, so I don’t know whether anybody wants me to report further details on my case at this point. I’ve put myself on the CC list for the linked Bugzilla report in case there is anything I can help with there.
Are you getting the same nvme controller is down... messages?
This suggests that you have a different problem, probably hardware. You should run Dell’s diagnostics and also test the memory with memtest86+.
Kernel 6.5.8 still can’t boot
Rarely, I can boot to desktop, but in a some time it freezes and if I’m lucky to switch to tty2 I can see the same I/O errors
After all these errors I switch to 6.3.13 and the system boots and works without any problems
Your issue is not the same as the one addressed successfully with a kernel update to 6.4.16 in this thread. It is difficult to follow multiple problems in one thread, so you are more likely to get help if you open a new thread with a more specific title like “Kernel 6.4.14–6.5.8 nvme I/O error or timeout”.
