F39 kernels fail to install when /boot/efi/$machineid is present

Problem

On certain Fedora systems, which have been installed a long time ago (multiple years) and continuously upgraded, new Fedora 39 kernel packages may fail to install (and then fail to appear in the GRUB bootloader menu, when displayed).

The error during kernel installation might look like this:

  Running scriptlet: kernel-core-6.5.9-300.fc39.x86_64                      2/2 
/usr/lib/kernel/install.d/51-dracut-rescue.install: line 81: /boot/efi/loader/entries/1604a63a532947d99d18bf17a754e833-0-rescue.conf: No such file or directory
/usr/lib/kernel/install.d/51-dracut-rescue.install failed with exit status 1.

Note: This problem only affects UEFI mode installations. If you installed your system in the legacy BIOS mode, it is not subject to this problem.

Cause

When /boot/efi/$machineid exists (where $machineid value can be determined by looking into /etc/machine-id), the systemd in F39 now considers this machine to be using sd-boot bootloader instead of the usual GRUB. However, this directory might exist, if the system is old and upgraded over time, even if it actually uses GRUB and not sd-boot. In that case, this heuristic fails and the new kernel fails to install.

Related Issues

Bugzilla report: #2248624

Workarounds

Figure out if you’re affected by testing for presence of /boot/efi/$machineid. If you get an error, you’re not affected by this issue:

$ sudo ls -d "/boot/efi/$(cat /etc/machine-id)"
ls: cannot access '/boot/efi/<machine-id hash here>': No such file or directory

If it prints the directory, it is present, and you might be affected by this problem:

$ sudo ls -d "/boot/efi/$(cat /etc/machine-id)"
/boot/efi/<machine-id hash here>

If your system uses the default GRUB bootloader (if you haven’t done any custom modifications in this area, it does), rename the directory:

sudo mv "/boot/efi/$(cat /etc/machine-id)" "/boot/efi/$(cat /etc/machine-id)_disabled"

Now, reinstall the kernel that failed to install (example, adjust your version):

sudo dnf reinstall 'kernel*-6.5.9-300.fc39.x86_64'

Or, alternatively, update to the latest kernel:

sudo dnf update 'kernel*'

It should now install properly, you should see it in the startup GRUB boot menu, and after booting the system you can verify that you’re running it like this:

$ uname -r
6.5.9-300.fc39.x86_64

You can discuss this issue here.