Switching to systemd-boot from GRUB

[pramodvu1502 (Pramod V. U.)]

Fedora atomic desktops are f41 onwards using a static GRUB config rather than the previously used grub2-mkconfig-generated dynamic config.

It is apparently to transition to the new composefs-based system using bootc+dnf5_plugins ditching ostree and rpm-ostree [I don’t understand the motive, sorry… the current seems to work excellent; And simple].

In f41, grub2-mkconfig just overwrites the static config, and still there aren’t issues. [/etc/grub.d/15_ostree takes care of the ostree boot menu]

In the new static config, theming isn’t supported without overwriting the config.
/boot/grub2/user.cfg file sourced by the static config doesn’t support including themes well. [You have to manually snip off many parts to get a non-conflicting config, and still there is spectacular failure of a ASCII undersized windows-only GRUB menu]
Additionally, you have to use insmod, and many unknown commands and variables in /boot/grub2/user.cfg rather than the well-known /etc/default/grub. For trivial changes like the timeout, resolution, etc…

• systemd-boot has a decent theme which doesn’t look like a hacker’s console.
• It even has more stability, which is the main point of atomic distros.
• It automatically detects Windows, other distros etc…
• It is easier to maintain /boot of systemd-boot than grub. [No permissions etc… as FAT32 is supported; No superfluous config]

Some myths:

• It doesn’t support shim: It does support being called as grub${ARCH}.efi, chainloaded by shim, and all this is fully integrated into bootctl… except the installation part.
• It is complicated: It is dead simple
• It is only about UKIs: While UKIs are better, systemd-boot does support traditional kernels and initrds
• It uses a new systemd-specific standard: It uses the standard BLS scheme.

I would recommend it be manually copied as the grub executable, or use shim’s config [the EFI cmdline to shim is what it executes if it exists and works]. And skip boot${ARCH}.efi if it already exists.

Why not make bootc use systemd-boot instead of GRUB, on UEFI systems?
It is much better.

GRUB is needed anyway on BIOS systems their limited MBR partition support means the kernel can’t usually have a separate partition.
On UEFI, anyways everything is embedded into the executable, and systemd-boot supports things more breakproof and better.

It’s config too is much simpler, and you don’t need to write a pseudo-shscript or use commands to “generate” a config[script-code /boot/grub2/grub.cfg] based on config[/etc/default/grub] and scripts[/etc/grub.d ] [“generate” = grub2-mkconfig].

Yes, it is better for traditional workstation systems too, but those anyways can switch. Immutable distros need support from the maintainer for such low-level changes.

[siosm (Timothée Ravier)]

Partially replied in /etc/default/grub is missing on Silverblue 41 fresh install - #25 by siosm

[pramodvu1502 (Pramod V. U.)]

Atomic distros use a much restricted approach. There is no protected.d. The ostree underlying image manager only supports GRUB.

In normal fedora though, you need an additional sdubby package to fully support systemd-boot [In package hooks and misc pieces], also dnf removeing grubby.

kernel-install add-all takes care of re-installing all installed kernel versions. No need for the $VRA in your script.

And when you are at it, why not just install systemd-ukify and use UKIs… They offer quite a few advantages.

[vekruse (Villy Kruse)]

@pramodvu1502

In normal fedora though, you need an additional sdubby package to fully support systemd-boot [In package hooks and misc pieces], also dnf removeing grubby.

sdubby provides the grubby command for sd-boot. It does assume that ESP is mounted on /boot/efi, as per Fedora standard.
grubby and sdubby conflicts each other so use the swap subcommand.

dnf swap grubby sdubby

[pramodvu1502 (Pramod V. U.)]

@steppybug

I cannot tell to what lengths you would be willing to go to get this working. It may be a while before I put my hand to it but I am confident I can succeed with it. If you would like to take this further I would be willing to help. Maybe some response to your points could be useful. If you are not interested in this path that is great too.

As a user, you already showed the right things to do.

On Fedora’s side, it’s enough if a systemd-boot${ARCH}.efi.signed is provided.
AND maybe support for it at install time.

Additionally, /etc/dnf/protected.d snippets will be needed for systemd-boot… you can understand why.

@steppybug

Removing packages really is optional. It will not hurt anything to have unused code sitting around though I prefer to only install what will actually get used.

I mentioned grubby because it isn’t anymore a working piece of software. Replacing it with sdubby will provide a compatible shim for the time being. They are conflicting, so one needs to be removed.

@steppybug

This is a big reason for looking at bootc base image building.

Could you elaborate a bit please…

@steppybug

For now anaconda will not install properly if /boot and /boot/efi are combined like this. Also anaconda will not allow /boot to be vfat. So switching to sd-boot after anaconda completes is my answer.

I don’t think it would be difficult to fix a simple artificial requirement like this…
And though systemd-boot recommends /efi for the ESP, it works perfectly fine with /boot/efi too, but nested. bootctl -p gets the EFI mountpoint and bootctl -x the xbootldr.

@steppybug

The sdubby package is not useful at all in this single os scenario. I’ve successfully used this script for everything from c9s to rawhide. Really only the ESP needs to be modified and everything else could be left alone. Some included tooling may no longer work so managing the ESP going forward may be a manual effort.

Huh! I didn’t understand what you are trying to say here. I just myself want to tell that sdubby is just a shim for grubby, for hooks and scripts needing it.

An alternative manager like kernelstub, sdbootutil and blsforme could also work great…
For a good “boot manager” than kernel-install and bootctl commands.

@steppybug

kernel-install is part of systemd and knows how to install kernels to the ESP supporting a wide variety of ESP mount points. In the kickstart workflow the running kernel is possibly different than the installed kernel. So specifying the kernel version in the kernel-core package becomes necessary.

I told the add-all command in view of the users switching to systemd-boot on a GRUB installation.

@steppybug

The finer points for when both grub and sd-boot are installed may require command line parameters but I am sure it can be done.

kernel-install just executes a bunch of helper scripts passing environment variables for the location of the kernel, esp, xbootldr etc… and the scripts do the work. It’s upto them whther to do something or not, to be self-contained or to follow some configuration.
If both are hooked up to GRUB, unless they honour some config or environment variable, both will run.
In case of conflict, the script run later in order will prevail.

The best thing to do is to settle on systemd-boot for UEFI and GRUB for BIOS [Due to MBR table’s 4-partition limit, dual-booters will need to keep the kernel in the rootfs itself; Not a problem on UEFI].

@steppybug

Going the additional step of systemd-ukify is on my radar

Great to know…

@steppybug

I still have a bit more to do in deploying the local certificate infrastructure to support this. My current experiments revolve arount a bare-metal single-node kubernetes cluster hosting the certificate infrastructure, docker registry, unbound dns server, AWX and any other containerized services. The redhat microshift product on a rhel edge server is my current solution.

I have no idea of all this; I am not a distro developer; sorry. Anyway, thanks for telling
Hopefully this means that systemd-boot too will be signed…

@steppybug

Wish me luck.

Wishing you all the best luck…

[vekruse (Villy Kruse)]

The simple way to switch to using sd-boot is the following 4 steps:

Step 1

Turn off secure boot in the UEFI setup.

The following steps should be run as root or with sudo.

Step 2

Create the file /etc/kernel/install.conf with the following contents

BOOT_ROOT=/boot/efi
layout=bls

Step 3

Install the sd-boot package

dnf install systemd-boot-unsigned

Step 4
Install sd-boot into the ESP

bootctl install --make-machine-id-directory

Step 5

Install the kernels into the ESP

kernel-install add-all

With this setup, you can still boot the system with grub if you wish.

[pramodvu1502 (Pramod V. U.)]

The best experience with full secureboot will need signed binaries of systemd-boot, just like how grub is currently shipped.

shim supports loading a custom EFI executable than ./grub${ARCH}.efi when the absolute executable path is passed to the shim EFI cmdline in the EFI format [\ not /].

But this doesn’t work in all systems… [Not in mine when I was using gentoo…]

Additionally, systemd-boot isn’t more break-prone or unstable than GRUB [understood by common sense]… So if GRUB works users better shift to systemd-boot.

If systemd-boot needs to by fully integrated into the OS like GRUB, dnf swap grubby sdubby is required to provide a grubby which works with systemd-boot…
And grubby and sdubby packages conflict…

Anyways, thanks… and plz include the sdubby “if systemd-boot boots well”.
[If kernel-install is sufficient and grubby is needed for grub entries, then fine…
I presume grub2-mkconfig+“misc scripts” handle GRUB and kernel-install handles systemd-boot,
grubby and sdubby are just shims for some other package hooks and for a CLI to modify kargs etc…]

[vekruse (Villy Kruse)]

@pramodvu1502

shim supports loading a custom EFI executable than ./grub${ARCH}.efi when the absolute executable path is passed to the shim EFI cmdline in the EFI format [\ not /].

The entry would look like that

BootCurrent: 0002
Timeout: 0 seconds
BootOrder: 0002,0003,0001,0000
Boot0000* UiApp	FvVol(7cb8bdc9-f8eb-4f34-aaea-3ee4af6516a1)/FvFile(462caa21-7614-4503-836e-8ab6f4662331)
Boot0001* UEFI Misc Device	PciRoot(0x0)/Pci(0x2,0x3)/Pci(0x0,0x0){auto_created_boot_option}
Boot0002* shimloader	HD(1,GPT,e801eacc-bb68-443a-8947-195f5ce7b628,0x800,0x200000)/EFI\fedora\shimx64.efi5c004500460049005c00730079007300740065006d0064005c00730079007300740065006d0064002d0062006f006f0074007800360034002e006500660069000000
Boot0003* Fedora	HD(1,GPT,e801eacc-bb68-443a-8947-195f5ce7b628,0x800,0x200000)/\EFI\systemd\systemd-bootx64.efi

[pramodvu1502 (Pramod V. U.)]

Exactly; the Boot0002.

But reportedly, and in my experience, it doesn’t work in Dell laptops.
[Even EFISTUB setups without a bootloader at all aren’t supported on Dell regarding the kargs… in ArchWiki]

It could be fixed with various things, different in each computer, like appending a \0 or \n, or even doing some bootorder trickery…

A milestone for the next SHIM version [mostly 16] is to support a configfile [mostly options.csv]… To fix the above…

But for now, it would be better if systemd-boot would be installed as grub${ARCH}.efi in EFI/Fedora… [bootctl fully supports it once it is installed beforehand onto the ESP… kernel-install and misc other tools don’t care about it…]

[jrredho (john r red-horse)]

Just a quick general question: How large are you folks making your EFI partitions to allow all of this? Can you give some recommendations for the cases of with or without UKI?

Thanks!

[vekruse (Villy Kruse)]

@jrredho

Just a quick general question: How large are you folks making your EFI partitions to allow all of this?

Just consider that what usually went into /boot now goes into /boot/efi. 500 Meg should be enough, but I usually make it one Gig, and it usually have about 75% free space.

[glb (Gregory Lee Bartholomew)]

You might also want to be mindful that the ESP is shared among all the OSs that you install on that drive. If you think you might want to, for example, clone your rootfs at some point (e.g. one for development, another for testing, maybe another with i686 packages for wine/steam games, etc.) all those instances would be configured with different machine IDs and have separate kernel+initramfs images stored on the ESP. If you have plenty of space, I’d bump the size to 2 or even 4 GiB.

BTW, VFAT supports trim/discard, so if you are using a SSD, the unallocated sectors won’t be completely “wasted”. The SSD will cycle them in to internal queues to improve wear-leveling and optimize sector allocation.

Just FYI.

[vekruse (Villy Kruse)]

@pramodvu1502

But reportedly, and in my experience, it doesn’t work in Dell laptops.
[Even EFISTUB setups without a bootloader at all aren’t supported on Dell regarding the kargs… in ArchWiki]

It doesn’t work on my ASUS based desktop either. The UEFI boot menu won’t show the entry with the shim arguments although the efibootmgr shows the entry to be present.

That would also affect the “No More Boot Loader” project done at RedHat.

[pramodvu1502 (Pramod V. U.)]

I myself would use the default Windows size of about 100MB, and an “XBOOTLDR” of 512MB-2GB…

The XBOOTLDR is a partition which holds the kernel+initrd and UKIs instead of the ESP, in the same layout. [/boot for this and /efi for ESP, and then /boot/EFI/Linux is where UKIs go.]

Just that the XBOOTLDR partition needs to be a certain type… bls_boot flag to be turned on in parted/GParted

EXTRA INFO:
kernel-install and almost all other tools, including all external tools like blsforme, sdbootutil etc… except sdboot-manage fully support this XBOOTLDR format…

bootctl -p gets the ESP mounpoint and bootctl -x the XBOOTLDR… Scripts should use bootctl -x || bootctl -p to retrieve the XBOOTLDR; Just in case there isn’t one, the ESP is used.

[pramodvu1502 (Pramod V. U.)]

That isn’t a good thing to do… unless you are expert enough… How do you pass kargs?

That file is meant to be a backup bootloader… incase the bootloader fails… Put systemd-boot in there and let it load the UKI… Unless you know what you are doing, and it’s consequences…

[pramodvu1502 (Pramod V. U.)]

An ESP is at max 1 GB, don’t overload it.

SD Cards aren’t as reliable as you think… Don’t do this unless you have a working LIVECD/LIVEUSB ready at hand.

You can do an ESP+XBOOTLDR combination on your system disk itself… At Max 1GB-2GB…

[pramodvu1502 (Pramod V. U.)]

For that very reason that the ESP is shared, it is better to avoid stuffing the whole boot stack into it.

100MB ESP + 1-4GB XBOOTLDR where kernels, initrds and UKIs go…

[pramodvu1502 (Pramod V. U.)]

Yours seems to be a slightly different issue…

In Dell systems efibootmgr syncs well with the system UEFI, but the arguments passed to any EFI executable are not passed to it by the UEFI…

[steppybug]

Part of the scenario in using the ESP as the place for the combined storage of what fedora by default uses /boot and /boot/efi for is that it would be updated atomically in the use cases I am aiming for. A new version of the ESP, or making a second copy of the current version, can be accomplished at will.

In the sd-boot case with a freshly formatted ESP, running bootctl init copies systemd-bootx64.efi to both ESP/efi/systemd/systemd-bootx64.efi and ESP/efi/EFI/BOOT/BOOTX64.EFI. There are also some loader files and random number quality files. Also efivars are written.

find /efi -ls
        1      4 drwx------   5 root     root         4096 Jan  1  1970 /efi
        3      4 drwx------   5 root     root         4096 Mar 13 14:40 /efi/EFI
        4      4 drwx------   2 root     root         4096 Mar 13 14:40 /efi/EFI/systemd
        9    140 -rwx------   1 root     root       142255 Sep 10  2024 /efi/EFI/systemd/systemd-bootx64.efi
       10      4 drwx------   2 root     root         4096 Mar 13 14:40 /efi/EFI/BOOT
       11    140 -rwx------   1 root     root       142255 Sep 10  2024 /efi/EFI/BOOT/BOOTX64.EFI
       15      4 drwx------   2 root     root         4096 Mar 13 14:40 /efi/EFI/Linux
        6      4 drwx------   3 root     root         4096 Mar 15 10:12 /efi/loader
       18      4 drwx------   2 root     root         4096 Mar 13 14:41 /efi/loader/entries
       27      4 -rwx------   1 root     root          439 Mar 13 14:41 /efi/loader/entries/31a5da3655ff49cb9ea16dffa3e61fd0-5.14.0-503.11.1.el9_5.x86_64.conf
       28      4 -rwx------   1 root     root           30 Mar 13 14:40 /efi/loader/loader.conf
        8      4 -rwx------   1 root     root           32 Mar 15 10:12 /efi/loader/random-seed
       12      4 -rwx------   1 root     root            6 Mar 13 14:40 /efi/loader/entries.srel
       20      4 drwx------   3 root     root         4096 Mar 13 14:40 /efi/31a5da3655ff49cb9ea16dffa3e61fd0
       22      4 drwx------   2 root     root         4096 Mar 13 14:41 /efi/31a5da3655ff49cb9ea16dffa3e61fd0/5.14.0-503.11.1.el9_5.x86_64
       31  35808 -rwx------   1 root     root     36667026 Mar 13 14:41 /efi/31a5da3655ff49cb9ea16dffa3e61fd0/5.14.0-503.11.1.el9_5.x86_64/initrd
       32  14120 -rwx------   1 root     root     14456952 Mar 13 14:41 /efi/31a5da3655ff49cb9ea16dffa3e61fd0/5.14.0-503.11.1.el9_5.x86_64/linux

There is no shim and no parameters to shim and no parameters to systemd-bootx64. To secureboot, signing systemd-bootx64 myself gets the job done without shim.

With uki, again signed myself thus no shim, there is only one file that needs to get written to the ESP and if that file is placed at ESP/EFI/BOOT/BOOTX64.EFI no efivars need to be touched. Being able to accomplish this by mkfs followed by a file copy without ever running bootctl is an advantage for the automated provisioning process I am using.

This works well in the atomically updated ESP scenario.

The ESP is rarely written to and only needs to be mounted on the running system during update. The storage being an sdcard is not a concern from an mtbf perspective. Being removable is important where when combined with actual full disk encryption the sdcard can be kept seperate from the computer and only attached during boot and removed before login. This is an improvement over the default fedora luks encrypted disk configuration.

The ESP being shared does not come into play for the most part. The Dell UEFI built-in diagnostics uses the ESP for storage of reports. And fwupd uses the ESP for the firmware and it’s own bootloader. I also find it advantageous to use the mmc driver Dell built into their signed UEFI as I had problems with the one grub includes.

So far I have not had any problems using more than 1GB of the sdcard’s capacity. They come from the manufacturer pre-formatted FAT32 and reformatting them in linux has never been an issue. The UEFI specification requires FAT32 and more UEFI implementations support FAT32 even when they do not support other FATs. FAT32 has a maximum drive size of 2TB.

I agree with you that this is not a general solution for all use cases. Hopefully this explanation is of benefit and does not come across as argumentative as I am trying to be helpful and not necessarily right.

[pramodvu1502 (Pramod V. U.)]

If it works for you fine, but it isn’t a good idea for all…

The “extra files” are the files using which systemd-boot allows basic configuration, the BLS layout for automated installation and management [kernel+initrd+cmdline etc…], as well as a RNG seed passed to the kernel at boot. [It noticeably improves the speed of userspace boot in certain computers like mine…]

Signing a file without SHIM involves enrolling the key into the UEFI directly.
SHIM uses a dedicated section in the UEFI called MOK, which is meant for custom-enrolled keys for an intermediate verifier in the secureboot chain…
SHIM comes signed with the default M$-UEFI keys, and no need to tinker with UEFI…
[I am talking in average user’s perspective]

A removable ESP like yours, again, is only a thing to boast.
It offers no additional protection over a LUKS encrypted system with TPM sealing, or with a manually entered password…
It may have the advantage of an inaccessible manually-signed EFI file, but standard files like SHIM+fedoraca-signed which most average users use, will not have any advantage.

@steppybug

I also find it advantageous to use the mmc driver Dell built into their signed UEFI as I had problems with the one grub includes.

GRUB reimplements too many things half-baked…

@steppybug

I agree with you that this is not a general solution for all use cases. Hopefully this explanation is of benefit and does not come across as argumentative as I am trying to be helpful and not necessarily right.

No issues here

@steppybug

So far I have not had any problems using more than 1GB of the sdcard’s capacity. They come from the manufacturer pre-formatted FAT32 and reformatting them in linux has never been an issue. The UEFI specification requires FAT32 and more UEFI implementations support FAT32 even when they do not support other FATs. FAT32 has a maximum drive size of 2TB.

Still, not for the average faint-hearted user…