Boot entries gone after upgrade

After running rpm-ostree upgrade on SB 31 (30.20191001), no boot entries are found (my computer starts the UEFI instead). I have a live usb stick at hand, but I could not find any straightforward approach to repair my system. As far as I experienced, grub2-mkconfig and the likes don’t work. Anyone ever had this problem, or has an idea how I can get my boot entries back?

Ugh, you may have hit
https://github.com/ostreedev/ostree/pull/1929#issuecomment-535579644
Is this installation upgraded from one with an old grub?

I think so. I installed SB 28 or 29 (don’t remember exactly) and never changed anything of grub. Updated to SB 30, and recently to SB 31 beta.

@mzimmermann can you please share your /etc/default/grub, /boot/efi/EFI/fedora/grub.cfg and /boot/efi/EFI/fedora/grubenv files?

@mzimmermann you may also try to update your grub2 EFI binary in the ESP to the latest version, since that’s not done by ostree.

That should be located in /usr/lib/ostree-boot/efi/EFI/fedora/grubx64.efi of your latest deployment and needs to be copied to /boot/efi/EFI/fedora/grubx64.efi

But make sure to backup your current /boot/efi/EFI/fedora/grubx64.efi just in case.

Below is the content of /etc/default/grub. Unfortunately, grub.cfg and grubenv did not exist. Although I think I remember that the boot directory was empty when looking into it from the live stick, I also fiddled around with it afterwards, when trying to fix my installation.

GRUB_TIMEOUT=5

GRUB_DISTRIBUTOR=“$(sed ‘s, release .*$,g’ /etc/system-release)”
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT=“console”
GRUB_CMDLINE_LINUX=“resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet”

GRUB_DISABLE_RECOVERY=“true”

GRUB_ENABLE_BLSCFG=true

I had this same problem and was discussing it with @refi64

The computer ends up going into the firmware settings as only “System setup” is left on GRUB.

I ended up reinstalling as I tried reinstalling GRUB from a chroot (using the newest ostree deployment) and it generated the same configurations.

1 Like

I ran into this too a few days ago and ended up reinstalling. I thought I did something wrong trying to add newer Podman and Crun from Koji to get toolbox working. How does chrooting into Silverblue work? It sounds like you don’t just mount the root partition and chroot into it , and when I tried that things were missing (like /usr/bin).

@mzimmermann thanks for the information. I wonder how you had GRUB_ENABLE_BLSCFG=true since BLS wasn’t the default in Fedora 28 and 29 (one of the versions you said that installed) and according to @jlebon the /etc/default/grub file shouldn’t be updated on a ostree deployment.

I also wonder why you didn’t have a grubenv and grub.cfg in your ESP.

@pluto and @tpb was this on an EFI or legacy BIOS install? Also, do you remember what Silverblue version was initially installed?

First of all, thank you very much for your help!

I wonder how you had GRUB_ENABLE_BLSCFG=true since BLS wasn’t the default in Fedora 28 and 29 (one of the versions you said that installed)

This is not my main machine, and I started with SB 28. I guess then I did a clean reinstall of SB 30 and forgot about it…

I also wonder why you didn’t have a grubenv and grub.cfg in your ESP.

I am also not 100% sure, because I fiddled around with it quickly after the error occured. I am sorry that I can only provide such unreliable and incomplete information… =/

BTW: I have also decided to go with a clean reinstall to fix this issue.

EFI. I started using SB on said computer when Nvidia support got added so a late 29 build or beta 30 build.

This too was in my config. If it’s any help, I did have osprober disabled as it prevents ostree-finalize-staged from taking forever on systems that have a Bitlockered Windows on another drive.

Instead of using /mnt as a chroot directory, use /mnt/ostree/deploy/fedora/deploy/<deploy id>. You’ll need to mount the dev, proc, /boot etc there too.

rpm-ostree won’t work, but grub-install does.

I thought my computer was already using BLS, based on the /boot/loader/entries

@mzimmermann you are welcome and sorry for the inconvenience.

yes, my guess is that you did a SB 30 install (and that’s why you had GRUB_ENABLE_BLSCFG=true) and got hit by a grub2 bug in Fedora 30 that was preventing the menu entry to be populated from BLS when /boot/loader was a symlink (this got fixed quickly but since ostree doesn’t update the bootloader, the user needs to update it).

That would explain it.

I see, probably 30 if you had that config option set.

Kind of, the BLS snippets are used as the source of truth but were not used directly. Instead, they were parsed and a grub.cfg was generated adding a menuentry command for each BLS in /boot/loader/entries.

Now that grub2 has BLS support, this step was not needed anymore but changing that is what probably caused an issue for you due not having the latest version of grub2 with all the fixes for the blscfg module.

I have the same problem and can’t use my computer (at work).
I’ve launched a Fedora live from USB but I can’t figure out how to access the files of my installation. I mean, I’ve opened the file manager and I could mount both the home and the root partition, but I can’t understand the root partition and find files. I’m new to OSTree… Can you give me some hint?

Also, I didn’t understand what’s the recommended fix: disabling BLS or updating the grub2 binary?

Quite annoyed by this bug… I thought OSTree was more stable than regular Fedora…

I suppose a potential fix would be to copy the efi files from sysroot/ostree/deploy/fedora/deploy/*/usr/lib/ostree-boot/efi/EFI/fedora to the boot/efi/EFI/fedora folder.

You’ll have to mount the folders from your install.

This is a bug resulting from SB not updating the bootloader files (yet).

Thanks but didn’t work. I can’t wait further, I’ll have to reinstall…

Hi.
Have you checked the UEFI entries order? Maybe it just changed. You can check from a live media with efibootmgr or the boot menu key from your system. With efibootmgr you can change the order.

For users hitting this issue, could you please check if your grubenv (in /boot/grub2/grubenv for legacy BIOS installs or /boot/efi/EFI/fedora/grubenv for EFI installs) has a blsdir variable set? If that’s the case, you should disable it with ‘sudo grub2-editenv - unset blsdir’