F40 Change Proposal: Unified Kernel Support Phase 2 (System-Wide)

Unified Kernel Support Phase 2

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Wiki
Announced

:link: Summary

Improve support for unified kernels in Fedora.

:link: Owner

:link: Phase 2 goals

  • Add support for booting UKIs directly.
    • Boot path is shim.efi → UKI, without any boot loader (grub, sd-boot) involved.
    • The UEFI boot configuration will get an entry for each kernel installed.
    • Newly installed kernels are configured to be booted once (via BootNext).
    • Successful boot of the system will make the kernel update permanent (update BootOrder).
  • Enable UKIs for aarch64.
    • Should be just flipping the switch, dependencies such as kernel zboot support are merged.
  • Add a UEFI-only cloud image variant which uses UKIs.
    • Also suitable for being used in confidential VMs.
    • Cover both x86_64 and aarch64.

:link: Related bugs

:link: Feedback

:link: Benefit to Fedora

  • Better secure boot support: the UKI initrd is covered by the signature.
  • Better support for tpm measurements and confidential computing.
    • measurements are more useful if we know what hashes to expect for the initrd.
    • measurements are more useful without grub.efi in the boot path (which measures each grub.cfg line processed).
  • More robust boot process
    • generating the initrd on the installed system is fragile

:link: Scope

  • Proposal owners:

    • updates for virt-firmware and uki-direct packages.
    • enable UKIs on aarch64 (MR 2818).
    • prepare kickstart (Fedora kickstarts) changes for generating UKI enabled images.
  • Other developers:

    • installer/anaconda: implement discoverable partition support.
    • bootloader/shim: fix bugs.
    • Fedora Cloud SIG: Add UKI enabled images as an option to Download Fedora Cloud
    • See also: Related Bugs section.
  • Release engineering: #Releng issue number

  • Policies and guidelines: N/A (not needed for this Change)

  • Trademark approval: N/A (not needed for this Change)

  • Alignment with Objectives:

:link: Upgrade/compatibility impact

None, it’s opt-in. Also the uefi cloud image is an additional image and will not replace the current bios/uefi hybrid image.

:link: How To Test

:link: Switch an existing install to use UKIs.

Needs up-to-date Fedora 39 or Rawhide install in a virtual machine. Bare metal hardware with standard storage (ahci / nvme) should work too.

Needs an big enough ESP to store UKI images there (minimum 200M, recommended 500M).

  1. dnf install virt-firmware uki-direct
  • The uki-direct package contains the kernel-install plugin and systemd unit needed to automatically manage kernel updates.
  • You should have version 23.10 or newer.
  1. sh /usr/share/doc/python3-virt-firmware/experimental/fixup-partitions-for-uki.sh
  • Workaround for bug 2160074 (anaconda not setting up discoverable partitions).
  • UKIs need this to find the root filesystem without root=… on the kernel command line.
  1. dnf install kernel-uki-virt

  2. kernel-bootcfg --show

  • optional step, shows UEFI boot configuration, the new UKI should be added as BootNext

$ kernel-bootcfg --show # C - BootCurrent, N - BootNext, O - BootOrder # -------------------------------------------- # N - 0008 - 6.5.7-300.fc39.x86_64 <= entry for the the new kernel # C O - 0007 - 6.5.6-300.fc39.x86_64 <= currently running kernel # O - 0006 - Fedora <= grub2 entry # O - 0001 - UEFI QEMU QEMU HARDDISK [ … ]

  1. reboot

  2. kernel-bootcfg --show

  • optional again, after successful boot the new kernel should be first in BootOrder.

$ kernel-bootcfg --show # C - BootCurrent, N - BootNext, O - BootOrder # -------------------------------------------- # C O - 0008 - 6.5.7-300.fc39.x86_64 # O - 0007 - 6.5.6-300.fc39.x86_64 # O - 0006 - Fedora # O - 0001 - UEFI QEMU QEMU HARDDISK [ … ]

:link: Test UKI cloud images

Repo with kickstart files and scripts: Gerd Hoffmann / fedora-uki ¡ GitLab

Images for download: Index of /fedora-uki

  • fedora-uki-cloud: uki-based cloud image, use cloud-init to configure this.
  • fedora-uki-direct: minimal uki-based image, root password is ‘root’.
  • fedora-classic: minimal non-uki image, root password is ‘root’.

Known problems:

  • images can fail to boot on the first attempt
    • should that happen reset the guest once, the second and all following boots will work fine.
    • root cause is a shim bug (github 554).
    • known workaround: add a vTPM to the guest configuration.

:link: Booting another kernel

From the booted system:

  • uefi-boot-menu --reboot

From the firmware:

If your UEFI firmware offers an boot menu you should be able to use that to select the kernel to boot. Unfortunately this is not standardized so there is no standard procedure to do so.

  • Virtual machines (OVMF): Enter the firmware setup by pressing ESC when you see the tianocore splash screen. Select “Boot Manager” in the toplevel menu.
  • Thinkpad laptops: Interupt normal boot (just ‘Enter’ on recent hardware, or using the special key on older models), then press F12 (“choose a temporary startup device”).

:link: User Experience

:link: Dependencies

:link: Contingency Plan

  • Contingency mechanism:
    • drop kickstart file for the uefi-only cloud image.
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? No

:link: Documentation

N/A (not a System Wide Change)

5 Likes

This all seems reasonable, but could you possibly add (to the change or just release notes later) what the use cases/reasons to use this is?

I guess its all just incremental progress to a goal, but if we are making these images and testing them and hoping our users help test them and provide feedback, it might be nice to explain when someone might want to use them?

Might be there’s something I missed here reading back after the holidays. :wink:

i got an error in red font after step 2

sh /usr/share/doc/python3-virt-firmware/experimental/fixup-partitions-for-uki.sh

but i forgot what it was. is it safe to run it the second time, just to paste it here?

also, the kernel-bootcfg-boot-successful.service fails with the following errors in journalctl

May 17 10:23:04 fedora kernel-bootcfg[1375]: INFO: No update needed, BootCurrent is already in BootOrder.
May 17 10:23:04 fedora kernel-bootcfg[1375]: Traceback (most recent call last):
May 17 10:23:04 fedora kernel-bootcfg[1375]:   File "/usr/bin/kernel-bootcfg", line 33, in <module>
May 17 10:23:04 fedora kernel-bootcfg[1375]:     sys.exit(load_entry_point('virt-firmware==24.4', 'console_scripts', 'kernel-bootcfg')())
May 17 10:23:04 fedora kernel-bootcfg[1375]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 17 10:23:04 fedora kernel-bootcfg[1375]:   File "/usr/lib/python3.12/site-packages/virt/firmware/bootcfg/main.py", line 305, in main
May 17 10:23:04 fedora kernel-bootcfg[1375]:     update_boot_csv(cfg, options)
May 17 10:23:04 fedora kernel-bootcfg[1375]:   File "/usr/lib/python3.12/site-packages/virt/firmware/bootcfg/main.py", line 168, in update_boot_csv
May 17 10:23:04 fedora kernel-bootcfg[1375]:     if entry.devicepath != shimpath:
May 17 10:23:04 fedora kernel-bootcfg[1375]:        ^^^^^^^^^^^^^^^^
May 17 10:23:04 fedora kernel-bootcfg[1375]: AttributeError: 'NoneType' object has no attribute 'devicepath'
May 17 10:23:04 fedora systemd[1]: kernel-bootcfg-boot-successful.service: Main process exited, code=exited, status=1/FAILURE

but everything seems to be working correctly, even TPM based unlocking of the root partition (i configured it with all 10 PCRs 0-9)

Yes, it’s safe to run it a second time.
Can you also add the output of kernel-bootcfg --verbose?

sorry for the huge delay, somehow i wasn’t notified of your message (and i’m back on fedora)

here’s the output of the sh command

 ~ sudo sh /usr/share/doc/python3-virt-firmware/experimental/fixup-partitions-for-uki.sh
# /dev/nvme0n1 2 -> BC13C2FF-59E6-4262-A352-B275FD6F7172
+ sfdisk --part-type /dev/nvme0n1 2 BC13C2FF-59E6-4262-A352-B275FD6F7172

The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Device or resource busy
The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or partx(8).
Syncing disks.
# unknown device: /dev/mapper/fedora-root

journalctl -xeu kernel-bootcfg-boot-successful.service

May 31 23:09:50 fedora kernel-bootcfg[2559]: INFO: No update needed, BootCurrent is already in BootOrder.
May 31 23:09:50 fedora kernel-bootcfg[2559]: Traceback (most recent call last):
May 31 23:09:50 fedora kernel-bootcfg[2559]:   File "/usr/bin/kernel-bootcfg", line 33, in <module>
May 31 23:09:50 fedora kernel-bootcfg[2559]:     sys.exit(load_entry_point('virt-firmware==24.4', 'console_scripts', 'kernel-bootcfg')())
May 31 23:09:50 fedora kernel-bootcfg[2559]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 31 23:09:50 fedora kernel-bootcfg[2559]:   File "/usr/lib/python3.12/site-packages/virt/firmware/bootcfg/main.py", line 305, in main
May 31 23:09:50 fedora kernel-bootcfg[2559]:     update_boot_csv(cfg, options)
May 31 23:09:50 fedora kernel-bootcfg[2559]:   File "/usr/lib/python3.12/site-packages/virt/firmware/bootcfg/main.py", line 168, in update_boot_csv
May 31 23:09:50 fedora kernel-bootcfg[2559]:     if entry.devicepath != shimpath:
May 31 23:09:50 fedora kernel-bootcfg[2559]:        ^^^^^^^^^^^^^^^^
May 31 23:09:50 fedora kernel-bootcfg[2559]: AttributeError: 'NoneType' object has no attribute 'devicepath'
May 31 23:09:50 fedora systemd[1]: kernel-bootcfg-boot-successful.service: Main process exited, code=exited, status=1/FAILURE

kernel-bootcfg --verbose

# C - BootCurrent, N - BootNext, O - BootOrder
# --------------------------------------------
# C   O  -  0000  -  Fedora
#                    -> path: Partition(nr=1)/FilePath(\EFI\fedora\shim.efi)
#                    -> opt/hex: 5243
#
#     O  -  2001  -  EFI USB Device
#                    -> path: 
#                    -> opt/hex: 5243
#
#     O  -  3000  -  Internal Hard Disk or Solid State Disk
#                    -> path: 
#                    -> opt/hex: 5243
#
#     O  -  2002  -  [ missing ]
#     O  -  2004  -  [ missing ]
#        -  0001  -  6.8.10-300.fc40.x86_64
#                    -> path: Partition(nr=1)/FilePath(\EFI\fedora\shimx64.efi)
#                    -> opt/ucs16: \EFI\Linux\46ec5efd86754b4aafddfbb77f6e3ef9-6.8.10-300.fc40.x86_64.efi root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rd.luks.uuid=luks-5514491e-4c37-4a55-9c85-2d47f3b887a3 rhgb quiet loglevel=3 systemd.show_status=auto amdgpu.sg_display=0 amdgpu.dcdebugmask=0x10 vconsole.font=ter-v28n 
#
#        -  0004  -  Fedora
#                    -> path: Partition(nr=1)/FilePath(\EFI\fedora\shimx64.efi)
#

this is on fedora 40 btw

@kraxel

Ok, the script doesn’t find the root filesystem partition because the root filesystem is on lvm.
The error is harmless, but you need to pass the root filesystem location on the kernel command line because the systemd autodetection works for partitions only. But apparently you figured that already :wink:

The kernel-bootcfg-boot-successful.service error is triggered by the missing boot entries. Fixed that in the git repo, with the next python3-virt-firmware update the error should go away.

A few potential problems I can see:

  1. If shim is always there, what’s the story for custom signatures? Would it make sense to distinguish the default boot path from generally supported boot paths? I think great care should be put towards not hardcoding shim as a dependency anywhere (though it does make sense to have it in the default boot path).
  2. The systemd folk have often warned system designers that BIOSes are buggy and that the NAND used to store information like boot configuration are often not of the greatest quality. When the NAND wears out or you EFI is buggy and your system does not boot as expected, what’s the contingency plan?
  3. There is a bootloader spec that provides automatic rollbacks and file-drop-based configuration and both grub and sd-boot follow it. What’s the story for these use cases if we exclusively rely on EFI boot menu?
  4. Currently, any UEFI-compliant system will boot, without any manual configuration, a disk containing a valid ESP. With a bootloader, I can guarantee that what is booted is the latest known-working UKI that was put on there. Can something similar be achieved with shim+uki?

I’m not arguing that bootloader support should be baked in and hardcoded. Just like shim, it should be an element that should be easily discardable if one does not need it. But there is merit in using one by default.

(4) This magic is done by /boot/efi/EFI/BOOT/fbx64.efi (part of shim-x64.rpm) which creates the boot configuration in case it does not exist. Works for UKIs too (otherwise the cloud image wouldn’t boot …).
(1) Not using shim is not really an option due to (4). Nevertheless the tools do not insist on shim.efi being present, if the firmware can boot the UKI (for example if secure boot is disabled) they happily create boot entries without shim in the loop.
(2) There is no fallback for broken UEFI implementations.
(3) New kernels are booted once via BootNext, only of that works the change is made permanent in the UEFI configuration.

The workflow is modeled largely after bootloaderspec type 2 entries. Once we have a secure boot support story for sd-boot it should be relatively simple to switch over.

1 Like