After the 6.15.3-200 update Fedora now cannot boot. It first shows a screen with this:
Then it kernel panics with the following message: unable to mount root fs on unknown-block(0 0)
I can still boot fine on the older 6.14 kernel. Is there something I can do? Or some kind of log I can extract? How could I do it? Thanks
Oh, I managed to fix it. If anyone gets the same problem. First, boot into an older kernel, then uninstall the offending kernel with (use the exact version, below an example):
sudo dnf remove kernel-core-0:6.15.3-200.fc42.x86_64
Update again:
sudo dnf update
Then regenerate the initramfs manually:
sudo dracut --kver 6.15.3-200.fc42.x86_64 --force
I still wonder though what happened this time. I usually use offline updates and never had any problems. Why wouldn’t it generate an initramfs image? Is that a bug?
People were reporting the same problem in Fedora does not boot after upgrade to kernel 6.15.3 with amdgpu drm error
this is dangerous as you can end up with non booting kernels if there is a general issue with newly created initramfs images.
It should h have been enough to create the missing initramfs with
sudo dracut --kver 6.15.3-200.fc42.x86_64
Hm, thanks for the reply! I believe the problem I hit is different, since I use an Intel system with an integrated GPU. With some investigation, it appears I hit bug 2333757 – initrd is not created if a dkms build fails leaving system boot to fail - The system couldn’t build my v4l2-loopback dkms module and thus didn’t regenerate the initramfs image automatically. Maybe your issue is similar, somehow?
Thanks. I’ll update the post with the solution so if anyone sees it they run the right command.
You don’t even need to reinstall the kernel. By doing so it would trigger dracut automatically.
booting an older kernel and the dracut command could have resolved the issue.
what dkms modules are installed on your system?
I guess bug 2333757 – initrd is not created if a dkms build fails leaving system boot to fail would prevent the automatic process from happening, that’s why I think running the command manually is needed. I think the culprit was the v4l2loopback module. It was outdated. I’m now trying to update it to version 0.15 from the oficial page GitHub - v4l2loopback/v4l2loopback: v4l2-loopback device
Not sure if this has anything to do with what happened, but trying to install a new version of v4l2loopback with dkms shows
guilherme@fedora:/usr/src$ sudo dkms install -m v4l2loopback -v ${version}
Found pre-existing /lib/modules/6.15.3-200.fc42.x86_64/extra/v4l2loopback.ko.xz, archiving for uninstallation
Installing /lib/modules/6.15.3-200.fc42.x86_64/extra/v4l2loopback.ko.xz
Running depmod...... done.
Executing post-transaction command..............(bad exit status: 5)
Failed command:
dracut --regenerate-all --force
For some reason dkms is having trouble updating the initramfs, but I’m unsure why it only started happening now.
It’s strange that running dracut --regenerate-all --force by dkms fails but when you run it yourself it succeeded.
dkms has --verbose option with a description Enable verbose output of external commands executed in DKMS.
Could you try running dkms with it and see if there is information reported by dracut?
(Hopefully dkms runs dracut with --debug and/or --stdlog options when run with --verbose option itself.)
Unfortunately I think dkms doesn’t use --debug or --stdlog, the only thing I get is:
guilherme@fedora:/usr/src$ sudo dkms install -m v4l2loopback -v ${version} --force --verbose
Module v4l2loopback/main for kernel 6.15.3-200.fc42.x86_64 (x86_64):
Before uninstall, this module version was ACTIVE on this kernel.
Deleting /lib/modules/6.15.3-200.fc42.x86_64/extra/v4l2loopback.ko.xz
Restoring archived original module /lib/modules/6.15.3-200.fc42.x86_64/extra/v4l2loopback.ko.xz
do_depmod 6.15.3-200.fc42.x86_64
done.
Found pre-existing /lib/modules/6.15.3-200.fc42.x86_64/extra/v4l2loopback.ko.xz, archiving for uninstallation
Installing /lib/modules/6.15.3-200.fc42.x86_64/extra/v4l2loopback.ko.xz
do_depmod 6.15.3-200.fc42.x86_64
done.
dracut --regenerate-all --force
(bad exit status: 5)
With a brief search online I couldn’t find anything about exit status 5.
When I run dracut --regenerate-all --force by itself I do get some error messages for some old uninstalled kernels, but the command seems to work otherwise (because it fixed the original issue):
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.4-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.4-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.5-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.5-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.6-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.6-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.8-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.8-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.9-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.9-300.fc42.x86_64 does not exist or is not accessible.
what was the RC of the above command? Was it 5?
next time check with echo $?.
you could also check the log files in /var/log/dkms for clues
i.e /var/log/post_transaction.log
Have you removed the previous version of the dkms module and deleted the old source code in /usr/src before extracting the new version?
pls share output of ls /usr/src/ , find /var/lib/dkms/v4l2loopback/ and dkms status
EDIT: check ls /lib/modules/ for stale directories of previously installed kernel versions. It should be save to remove them.
what was the RC of the above command? Was it 5?
Yes, the return code was 5
you could also check the log files in /var/log/dkms for clues
i.e/var/log/post_transaction.log
/var/log/dkms doesn’t exist, but I can find a post_transaction.log in /var/lib/dkms/, which basically outputs the same as manually running dracut in the terminal:
Executing post-transaction command
# command: dracut --regenerate-all --force
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.4-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.4-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.5-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.5-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.6-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.6-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.8-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.8-300.fc42.x86_64 does not exist or is not accessible.
dracut[F]: Can't write to /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.9-300.fc42.x86_64: Directory /boot/efi/c63ecb94c27f4207aa6884663c007f89/6.14.9-300.fc42.x86_64 does not exist or is not accessible.
# exit code: 5
# elapsed time: 00:00:38
Have you removed the previous version of the dkms module and deleted the old source code in /usr/src before extracting the new version?
Yes I did delete the old sources. When the problem described in this topic happened, I was using the version 0.13 of v4l2loopback. After managing to fix the issue, I tried 0.15 and the same dracut error happened, then I tried the main branch and the same happened, but since I manually ran dracut --regenerate-all --force I didn’t have problems. So now my /usr/src shows:
guilherme@fedora:~$ ls /usr/src/
debug kernels v4l2loopback-main vboxhost-7.1.8
find /var/lib/dkms/v4l2loopback/ shows:
/var/lib/dkms/v4l2loopback/
/var/lib/dkms/v4l2loopback/main
/var/lib/dkms/v4l2loopback/main/source
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64/x86_64
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64/x86_64/log
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64/x86_64/log/auto.conf
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64/x86_64/log/make.log
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64/x86_64/module
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64/x86_64/module/Module.symvers
/var/lib/dkms/v4l2loopback/main/6.15.3-200.fc42.x86_64/x86_64/module/v4l2loopback.ko.xz
/var/lib/dkms/v4l2loopback/original_module
/var/lib/dkms/v4l2loopback/original_module/6.15.3-200.fc42.x86_64
/var/lib/dkms/v4l2loopback/original_module/6.15.3-200.fc42.x86_64/x86_64
/var/lib/dkms/v4l2loopback/original_module/6.15.3-200.fc42.x86_64/x86_64/v4l2loopback.ko.xz
/var/lib/dkms/v4l2loopback/original_module/6.15.3-200.fc42.x86_64/x86_64/v4l2loopback.ko.xz.origin
/var/lib/dkms/v4l2loopback/kernel-6.15.3-200.fc42.x86_64-x86_64
And dkms status shows:
v4l2loopback/main, 6.15.3-200.fc42.x86_64, x86_64: installed (Original modules exist)
ls /lib/modules/ does show stale versions
6.14.11-300.fc42.x86_64 6.14.4-300.fc42.x86_64 6.14.6-300.fc42.x86_64 6.14.9-300.fc42.x86_64
6.14.2-300.fc42.x86_64 6.14.5-300.fc42.x86_64 6.14.8-300.fc42.x86_64 6.15.3-200.fc42.x86_64
The real problem just seems to be dkms failing to run dracut --regenerate-all --force which is a reported bug 2333757 – initrd is not created if a dkms build fails leaving system boot to fail I think there’s not much I, as a user, can do until that gets fixed other than always manually running dracut --regenerate-all --force after a kernel upgrade
Oh, WAIT! It worked now after removing the stale modules in /lib/modules! I tried reinstalling the module again and this time there were no errors.
/var/lib/dkms/post_transaction.log shows:
Executing post-transaction command
# command: dracut --regenerate-all --force
# exit code: 0
# elapsed time: 00:00:34
This doesn’t necessarily fix 2333757 – initrd is not created if a dkms build fails leaving system boot to fail but at least a successful dkms build means I don’t hit the bug. I’ll only be sure next time the kernel upgrades, but it shouldn’t be a problem anymore.
Experienced the exact same problem. This was enough to fix the issue for me, from a working kernel.
I had this, too.
sudo dracut --kver 6.15.3-200.fc42.x86_64 fixed it enough to boot. But GPU was falling back to Nova_Core(?), and I was only getting one display. Blacklisted that and nouveau and it’s good for now.
Just updated to kernel 6.15.4-200 today and got the same problem. Probably fixed it again in the same way, but it is starting to get a bit annoying
Can someone please tell me if my problem is related to this?
The next time this happens do this:
sudo dracut --kver 6.15.3-200.fc42.x86_64 where 6.15.3-200.fc42.x86_64 is the version of the newest kernel.ps.
Most things in life are hard and IT is super hard.
Try to stay cool the next time something bad happens as our lives are the never ending chains of bad things…