Grub Menu not updating after installing newer Kernels

Hi,

I installed Fedora 36 about a month ago and it’s been working great mostly. Only major issue i’ve had is this one:

After running a sudo dnf update a couple of weeks ago and upgrading from kernel 5.18.16 to 5.18.17, the grub boot menu doesn’t show the updated kernel despite it being installed correctly (afaik). I’ve since run a couple more updates (5.18.18 and 5.18.19) but still the same issue, grub menu only shows 5.18.16 and a broken 5.17.5 (was removed during update i think). Kernel updates prior to this were working fine

Not sure if it’s related but I also ‘fixed’ the BTRFS subvolumes created on OS Installation… They were initially called root and home, I changed these to @ and @home respectively in order to get Timeshift working. BTRFS backups using Timeshift now work great but the kernel updating issue started around the same time… Don’t think these should be related but I’m a btrfs novice, any idea?

Steps taken to fix:

  • Tried fixes suggested in these threads for similar issues (Thread 1, Thread 2, Thread 3):
    • Compared args in 5.18.16 to 5.18.19 sudo grubby --info=ALL
    • Confirmed that ro rootflags=subvol=root exists as ro rootflags=subvol=@in the 5.18.19 entry (same as working kernel 5.18.16)
    • Tried sudo cat /etc/kernel/cmdline but failed as cmdline file didn’t exist
    • Removed 5.18.19 and reinstalled it via sudo dnf remove kernel-core-5.18.19 then sudo dnf upgrade
  • Tried uninstalling and reinstalling my nvidia driver
  • Tried bug fix from: https://bodhi.fedoraproject.org/updates/FEDORA-2022-a3480ad0d3
    • sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-a3480ad0d3
    • Didn’t work so rolled back via snapshot

Not sure what outputs will be helpful, included some below (please prod me for anything else):

~/ sudo grubby --info=ALL                                                                                      
[sudo] password for sunny:
index=0
kernel="/boot/vmlinuz-5.18.19-200.fc36.x86_64"
args="ro rootflags=subvol=@ rd.driver.blacklist=nouveau modprobe.blacklist=nouveau nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init resume=UUID=21ff6547-2923-4d53-a505-b972b5e51ecf rhgb quiet rd.driver.blacklist=nouveau modprobe.blacklist=nouveau nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init rootflags=subvol=@"
root="UUID=f6e56409-da8e-43b1-b238-d041419bbce4"
initrd="/boot/initramfs-5.18.19-200.fc36.x86_64.img"
title="Fedora Linux (5.18.19-200.fc36.x86_64) 36 (Thirty Six)"
id="ab2b085a1785456792f05fbe6626b188-5.18.19-200.fc36.x86_64"
index=1
kernel="/boot/vmlinuz-5.18.16-200.fc36.x86_64"
args="ro rootflags=subvol=@ resume=UUID=21ff6547-2923-4d53-a505-b972b5e51ecf rhgb quiet rd.driver.blacklist=nouveau modprobe.blacklist=nouveau nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init rootflags=subvol=@"
root="UUID=f6e56409-da8e-43b1-b238-d041419bbce4"
initrd="/boot/initramfs-5.18.16-200.fc36.x86_64.img"
title="Fedora Linux (5.18.16-200.fc36.x86_64) 36 (Thirty Six)"
id="ab2b085a1785456792f05fbe6626b188-5.18.16-200.fc36.x86_64"


~/ ls -l /boot                                                                                           [130]
.rw-r--r--@ 247k root  3 Aug 17:18 config-5.18.16-200.fc36.x86_64
.rw-r--r--@ 247k root 21 Aug 17:26 config-5.18.19-200.fc36.x86_64
drwx------@    - root  1 Jan  1970 efi
drwxr-xr-x@    - root  4 May 22:26 extlinux
drwx------@    - root 26 Aug 12:43 grub2
.rw-------@ 104M root  9 Aug 15:34 initramfs-0-rescue-ab2b085a1785456792f05fbe6626b188.img
.rw-------@  34M root 13 Aug 12:54 initramfs-5.18.16-200.fc36.x86_64.img
.rw-------@  34M root 26 Aug 12:13 initramfs-5.18.19-200.fc36.x86_64.img
drwxrwxr-x@    - root  9 Aug 15:33 loader
drwx------@    - root  9 Aug 15:32 lost+found
lrwxrwxrwx@   47 root  9 Aug 16:17 symvers-5.18.16-200.fc36.x86_64.gz -> /lib/modules/5.18.16-200.fc36.x86_64/symvers.gz
lrwxrwxrwx@   47 root 26 Aug 12:13 symvers-5.18.19-200.fc36.x86_64.gz -> /lib/modules/5.18.19-200.fc36.x86_64/symvers.gz
.rw-------@ 6.3M root  3 Aug 17:18 System.map-5.18.16-200.fc36.x86_64
.rw-------@ 6.3M root 21 Aug 17:26 System.map-5.18.19-200.fc36.x86_64
.rwxr-xr-x@  12M root  9 Aug 15:33 vmlinuz-0-rescue-ab2b085a1785456792f05fbe6626b188
.rwxr-xr-x@  12M root  3 Aug 17:18 vmlinuz-5.18.16-200.fc36.x86_64
.rwxr-xr-x@  12M root 21 Aug 17:26 vmlinuz-5.18.19-200.fc36.x86_64


~/ find /etc -name cmdline                                                                            
~/ 

Thanks for any help!
Sunny

Quick update…

Just found a cmdline file in /proc/cmdline. This is the output from it:

~/ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.18.16-200.fc36.x86_64 root=UUID=f6e56409-da8e-43b1-b238-d041419bbce4 ro rootflags=subvol=@ rd.driver.blacklist=nouveau modprobe.blacklist=nouveau nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init resume=UUID=21ff6547-2923-4d53-a505-b972b5e51ecf rhgb quiet rd.driver.blacklist=nouveau modprobe.blacklist=nouveau nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init rootflags=subvol=@

Looks like 5.18.16 is missing from it. Is this expected?

Thanks
S

Have you tried a grub fix? sudo grub2-mkconfig -o /boot/grub2/grub.cfg.

What is the content of ls /boot and sudo ls /boot/loader/entries/
Both of those should show files for each installed kernel.

Also, what is the output of cat /boot/efi/EFI/fedora/grub.cfg. Do not change it simply show the output. If that file has more than 4 lines output then a recovery will likely be needed.

Hi Jeff, thanks for your help.

Think I did a sudo grub2-mkconfig -o /boot/grub2/grub.cfg but didn’t work. Going to try again and will post findings shortly…

~/ ls /boot                                                                                                                        [130]
config-5.18.16-200.fc36.x86_64                           lost+found
config-5.18.19-200.fc36.x86_64                           symvers-5.18.16-200.fc36.x86_64.gz
efi                                                      symvers-5.18.19-200.fc36.x86_64.gz
extlinux                                                 System.map-5.18.16-200.fc36.x86_64
grub2                                                    System.map-5.18.19-200.fc36.x86_64
initramfs-0-rescue-ab2b085a1785456792f05fbe6626b188.img  vmlinuz-0-rescue-ab2b085a1785456792f05fbe6626b188
initramfs-5.18.16-200.fc36.x86_64.img                    vmlinuz-5.18.16-200.fc36.x86_64
initramfs-5.18.19-200.fc36.x86_64.img                    vmlinuz-5.18.19-200.fc36.x86_64
loader

~/ sudo ls /boot/loader/entries/
[sudo] password for sunil:
ab2b085a1785456792f05fbe6626b188-0-rescue.conf		       ab2b085a1785456792f05fbe6626b188-5.18.19-200.fc36.x86_64.conf
ab2b085a1785456792f05fbe6626b188-5.18.16-200.fc36.x86_64.conf

~/ sudo cat /boot/efi/EFI/fedora/grub.cfg                                                                                            
search --no-floppy --fs-uuid --set=dev cd151c3e-597e-4474-95b2-34c6709e7e96
set prefix=($dev)/grub2
export $prefix
configfile $prefix/grub.cfg

Cheers
S

grub2-mkconfig found 5.18.19

~/  sudo grub2-mkconfig -o /boot/grub2/grub.cfg                                                                                    [130]
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.18.19-200.fc36.x86_64
Found initrd image: /boot/initramfs-5.18.19-200.fc36.x86_64.img
Found linux image: /boot/vmlinuz-5.18.16-200.fc36.x86_64
Found initrd image: /boot/initramfs-5.18.16-200.fc36.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-ab2b085a1785456792f05fbe6626b188
Found initrd image: /boot/initramfs-0-rescue-ab2b085a1785456792f05fbe6626b188.img
Found Windows Boot Manager on /dev/nvme0n1p1@/efi/Microsoft/Boot/bootmgfw.efi
Found Manjaro Linux (21.3.6) on /dev/nvme0n1p3
Found Fedora Linux 36 (Thirty Six) on /dev/nvme1n1p3
Adding boot menu entry for UEFI Firmware Settings ...
done

Will update after a reboot

Cheers
S

Cheers Jeff… The grub2-mkconfig worked… I’m new to Fedora, I think I must have run a sudo grub2-mkconfig -o /boot/grub/grub.cfg as per other dists!!

Thanks again bud. All good now:

~/ uname -r
5.18.19-200.fc36.x86_64

S

2 Likes

You should never need to run the grub2-mkconfig command. Dnf automatically updates the grub.cfg info with each kernel update.

It seems you may have been hit with the small hiccup that has occurred for many with recent kernels (5.18.17 and newer) that caused the upgrade to complete without actually updating grub for booting the new kernel properly.

I myself have seen it once and newer updates seem to have fixed it.

Anyway, glad to assist and good luck going forward. :nerd_face:

3 Likes

IIRC, it does that automatically if blsfg is enabled (which should be the default), otherwise you have to update grub manually

1 Like

Thanks Jeff, Lorenzo.

Looks like the 5.19.4 update recently required a grub2-mkconfig again.

I followed a guide on how to install Fedora to make it compliant with Timeshift requirements for BTRFS backups. One of the steps in the guide was to disable BLSCFG in /etc/default/grub. Think BLSCFG was supposed to be buggy at the time the guide was made. Any idea if it’s safe to re-enable it now?

Do I just edit /etc/default/grub and swap out false with true on the GRUB_ENABLE_BLSCFG=false entry, before the next kernel update?

Thanks,
Sunny

I don’t remember the details, but I think it’s not enough…

Ok no worries, thanks. I will try to reverse the original build instructions and post them here before actioning

Looks like this may be system specific somehow. I did not have that issue myself.

I did however have to do a removal and reinstall of the nvidia drivers when I did a system upgrade from 35 to 36 yesterday on one of my machines

There is a grub2-switch-to-blscfg that should help you to swithc to bls configuration. It was called during updates on version f30 and f31 and from f32 no more. I don’t know if this script works properly as it is expected that everybody has converted by now. Non-bls configuration depends on grubby-deprecated to update grub.cfg with a new kernel. With f37 there is no grubby-deprecated anymore, so you would need to run grub2-mkconfig after every kernel updates. I guess that would still work, probably.

Could be. I’ll see if GRUB_ENABLE_BLSCFG=true fixes the problem or not. Not had a chance to do this yet

Villy. Thanks for the info. I’ve got 5.19.3 installed, that’s part of f36 still right? Hopefully grub2-switch-to-blscfg would solve this for me still

You might make the change in /etc/default/grub then do an update to install the newer kernel and see if it works. BLS has been active for quite some time and that setting in the default config enables or disables it when the grub.cfg file is generated.

1 Like

Quick update, hopefully this will help someone else

I changed /etc/default/grub and installed 5.19.6 today but the grub menu didn’t update on reboot. I ran a grub2-mkconfig and it fixed the issue again…

i did a bit more research here and looks like after editing /etc/default/grub I probably need to run a mkconfig anyway. So this will hopefully fix it permanently now

I think they are two separate issues. In general, you have to run mk-config each time you change default grub or any of its files in etc. On the contrary, when you update a kernel or install a new one, it should run mk-config automatically.

It does not on Fedora. Instead it creates a new file in /boot/loader/entries containing the configuration for the new kernel.

Hi Lorenzo. I see. There are issues around grub being reported on Fedora Silverblue and also Arch. Maybe it’s behind my issue too

@vekruse I may see what happens after the next kernel update and try the grub2-switch-to-blscfg command you mentioned.

Btw. I’ve got 2 grubby packages installed. I guess the newer grubby version must be running instead of deprecated after I run a kernel update. Could that explain why my kernel may not be updating?

~/ sudo dnf list |grep -i grubby
[sudo] password for sunny:
grubby.x86_64             8.40-67.fc36      @updates
grubby-deprecated.x86_64          8.40-67.fc36    updates

Cheers
S

In fact, that’s why I said before that if you don’t use blsfg you have to update grub yourself.