Fedora 42 upgrade failed, kernel panic on boot

After updating fedora and running sudo dnf5 offline reboot system got stuck, i’ve tried to reboot it but cannot boot any of the kernels(including recovery) because of kernel pani exitcode=0x00007f00

Is there a way to rerun upgrade from livecd? should it be a F41 or F42 livecd?

Tried to mount root, boot, boot/efi partitions to run a distro-sync against them.
but couldn’t chroot /mnt

liveuser@localhost-live:~$ sudo chroot /mnt
grep: error while loading shared libraries: /lib64/libpcre2-8.so.0: file too short
grep: error while loading shared libraries: /lib64/libpcre2-8.so.0: file too short
grep: error while loading shared libraries: /lib64/libpcre2-8.so.0: file too short
flatpak: error while loading shared libraries: /lib64/libglib-2.0.so.0: file too short
/usr/bin/sed: error while loading shared libraries: /lib64/libpcre2-8.so.0: file too short

I assume you mean “upgrading”. During an offline upgrade there can be long periods with little visible activity. This has led others to reboot while the upgrade was in progress, but you could also have a hardware issue such as out of space on a filesystem or a failed storage device. For individual users I recommend removing the rhgb quiet from the kernel command line as this will let you see issues as they occur.

The file too short errors suggest either a full or failed storage device. I suggest booting the Live Installer environment and using Gnome Disks to check “drive health” and make sure the Fedora partitions have free space. Note that btrfs requires use of btrfs fi df /mnt for accurate free space values.

Yes, upgrading for sure, thank you!

Before reading your comment i’ve noticed that libraries in /lib64 were corrupted(0 bytes files), i’ve copied them from livecd and was able to disto-sync and boot previous kernel.

After i did check the disks with manufacturer’s utility and can confirm that they are healthy.
Fedora partition also has free space.

liveuser@localhost-live:~$ btrfs fi df /mnt
Data, RAID0: total=46.00GiB, used=25.02GiB
System, RAID1: total=8.00MiB, used=16.00KiB
Metadata, RAID1: total=2.00GiB, used=849.52MiB
GlobalReserve, single: total=82.66MiB, used=0.00B

i’ve tried to boot newest kernel with removing rhgb quiet.
The boot is stuck in infinite job dev-disk-by..... start

LIkely that means you may have a bad entry in /etc/fstab or you have may have a resume= on the kernel command line that names a partition that does not exist.

/etc/fstab UUID’s seems to match relevant partitions i’m getting from lsblk -o +UUID

so, more likely it is indeed a resume= on kernel command.
Is it safe to just remove it?

Yes it’s safe to remove it.

No resume= in my kernel command.
What else could be the reason?

Command for the previous kernel is pretty much the same, just the image version is different.

you could try to reinstall the kernel

sudo dnf reinstall kernel*6.14.9*

You also need to verify that system is fully upgraded to fc42
e.g. check dnf repolist for old f41 repositories
and probably clean up old packages
see optional post upgrade steps

it may be also a good idea to do a rpm --verify on all installed packages.

sudo rpm -V $( rpm -qa   --qf "%{name} ")

Reinstalled the kernel, done the post-upgrade stuff and verified packages.

Now seems like the issue is with Nvidia driver, as it’s trying to fallback to nouveau and getting stuck on this.

Will try to reinstall it from livecd with mounted volumes and chroot, as now neither of the kernels is bootable again.

The system was probably compiling the nvidia kernel modules, most likely because you did not wait long enough and restarted the system right away after dnf has finished.

Yeah more likely, after reinstalling it I got a step further.
After entering a LUKS password darcut-initque timed out and I got into emergency shell

no this is something else.
I would consider instaling new into this system and keeping /home untouched.

To which filesystem or partition does the UUID belong? Is this the root fs / btrfs device?

It’s one of the LUKS partitions containing root btrfs

shot in the dark:

liveimage, chroot
dnf reinstall dracut\* cryptsetup\*
dracut --force --kver 6.14.9-300.fc42.x86_64

Thanks! after doing it i was able to boot to the system with the latest kernel and F42.
Boot took a little more time than usually(extra ~90 sec for starting plymouth-quit-wait and akmods services).

The only side effect i see right now is that only one display(identified as None-1) with resolution of 1280x1024 is visible, but at least the system is bootable and files aren’t corrupted.

Thanks for the help everybody involved!

1 Like