Silverblue .iso made via Lorax fails installation

Project Discussion
1

For a while now, I have been trying to create a pre-customized .iso of Fedora Silverblue by composing a repo based on workstation-ostree-config and passing that to Lorax using --add-template-var arguments.

While I have been able to build a custom .iso using this method, I have yet to succeed in installing the end product either on real hardware or in KVM, due to an error that happens at the end of the install process (“The command ‘cp -r -r /mnt/sysroot/usr/lib/ostree-boot/loader /mnt/sysimage/boot’ exited with the code 1.”, also screenshot below). I was unable to find any information online concerning this error, nor did I find the folder “…/ostree-boot/loader” present in any official build of Silverblue.

In an attempt to rectify this, I tried multiple changes to the script adding and subtracting different arguments, including building with and without lorax-embed-flatpaks.tmpl, trying different versions of Silverblue without any customization (basically using the official config .yaml’s for the Gnome and Kinoite editions of F34, F35 and Rawhide), though to no avail. My .iso’s still always fail at the same point during the install process.

SELinux was always set to permissive during the build process and I am always using a fresh install of Fedora Workstation with rpm-ostree and its dependencies installed. The Lorax log files also appear to indicate that everything works as intended, as there is no visible error present.

I honestly don’t know where the issue lies, so I am hoping that you may point me in the right direction.

What follows are the arguments I use to call Lorax, plus how I build the repo beforehand (here I am just trying to build the .iso based on the standard fedora-silverblue.yaml for F35 without any modifications):

mkdir iso && cd iso
git clone -b f35 https://pagure.io/workstation-ostree-config.git
git clone -b f35 https://pagure.io/fedora-lorax-templates.git
mkdir ostree_repo
ostree init --repo=ostree_repo
rpm-ostree compose --unified-core tree --repo=$(pwd)/ostree_repo $(pwd)/workstation-ostree-config/fedora-silverblue.yaml

exec lorax  --product=Fedora \
                --version=35 \
                --release=20211104 \
                --source=https://kojipkgs.fedoraproject.org/compose/35/latest-Fedora-35/compose/Everything/x86_64/os/ \
                --variant=Silverblue \
                --nomacboot \
                --volid=Fedora-SB-ostree-x86_64-35 \
                --add-template=$(pwd)/fedora-lorax-templates/ostree-based-installer/lorax-configure-repo.tmpl \
                --add-template=$(pwd)/fedora-lorax-templates/ostree-based-installer/lorax-embed-repo.tmpl \
                --add-template=$(pwd)/fedora-lorax-templates/ostree-based-installer/lorax-embed-flatpaks.tmpl \
                --add-template-var=ostree_install_repo=file://$(pwd)/ostree_repo \
                --add-template-var=ostree_update_repo=file://$(pwd)/ostree_repo \
                --add-template-var=ostree_osname=fedora \
                --add-template-var=ostree_oskey=fedora-35-primary \
                --add-template-var=ostree_contenturl=mirrorlist=https://ostree.fedoraproject.org/mirrorlist \
                --add-template-var=ostree_install_ref=fedora/35/x86_64/silverblue \
                --add-template-var=ostree_update_ref=fedora/35/x86_64/silverblue \
                --add-template-var=flatpak_remote_name=fedora \
                --add-template-var=flatpak_remote_url=oci+https://registry.fedoraproject.org \
                --add-template-var=flatpak_remote_refs="runtime/org.fedoraproject.Platform/x86_64/f35 app/org.gnome.gedit/x86_64/stable" \
                --logfile=$(pwd)/lorax.log \
                --tmp=$(pwd)/temp \
                --rootfs-size=8 \
                $(pwd)/finished

Screenshots:

Bildschirmfoto von 2021-11-04 11-52-48
Bildschirmfoto von 2021-11-04 11-52-481172×531 19.4 KB

Bildschirmfoto von 2021-11-04 11-54-04
Bildschirmfoto von 2021-11-04 11-54-041761×1275 212 KB

tl;dr : Trying to build Silverblue .iso with Lorax, result cannot be installed due to Anaconda error connected to the bootloader (“The command ‘cp -r -r /mnt/sysroot/usr/lib/ostree-boot/loader /mnt/sysimage/boot’ exited with the code 1.”).

2

Has no one encountered this issue before? Or should I rather file a bug report on the Anaconda/Lorax repository in this case?

3

Hi decent,

Sorry, I do not have a solution to your problem.
I am responding because I have the exact same error, but with Fedora IoT.

I have created a customized OSTree commit, based on Fedora IoT 35 plus some extra’s. I followed various instructions (one of which is: Build your own Fedora IoT Remix - Fedora Magazine). That OSTree commit built fine and is served via a webserver in the network.
I wrte a custom Kickstart file and used Lorax mkksiso to create an ISO image based on the original Fedora IoT 35 ISO combining it with my kickstart file.

Installation seems to procede just fine until it runs into this same error:

The command 'cp -r -p /mnt/sysroot/usr/lib/ostree-boot/loader /mnt/sysimage/boot' exited with the code 1.

Just like yours does.
Have you been able to find a solution just yet?
Or does anybody here have a clue on how to procede?

2 Likes
4

Sadly, I haven’t been able to find a solution as of yet. I did however file a bug report in the hope that someone at the Anaconda dev team may be able to help. Previously also filed one at the Lorax repo, though they told me that the issue is not connected to Lorax.
As stated in the bug report, for some reason during installation with an official .iso, Anaconda does not try to copy the folders causing these issues. I really don’t know what causes this behavior in Anaconda…

5

hey @decently - I’m not 100% sure what’s going on here but it seems that your composed tree has extra files under /usr/lib/ostree-boot/ that the official composes don’t have. Here’s the latest F35 Silverblue right now:

$ ostree ls fedora/35/x86_64/silverblue /usr/lib/ostree-boot
d00555 0 0      0 /usr/lib/ostree-boot
d00700 0 0      0 /usr/lib/ostree-boot/efi
d00700 0 0      0 /usr/lib/ostree-boot/grub2

What does your local compose show when you run that command against your local ref?

FWIW the code that Fedora releng uses to generate the OSTree for Silverblue is located here. One notable difference is that it’s not using --unified-core but I doubt that’s the root cause, though it is something to investigate.

2 Likes
6

Thank you very much. I just ran the command pointed at the local repo and, as expected, the mysterious loader folder is present:

$ ostree --repo='/home/user/build/iso/ostree_repo'  ls fedora/35/x86_64/silverblue /usr/lib/ostree-boot
d00755 0 0      0 /usr/lib/ostree-boot
d00700 0 0      0 /usr/lib/ostree-boot/efi
d00700 0 0      0 /usr/lib/ostree-boot/grub2
d00755 0 0      0 /usr/lib/ostree-boot/loader

I tried building the repo without --unified-core and:

$ ostree --repo='/home/user/build/iso/ostree_repo2'  ls fedora/35/x86_64/silverblue /usr/lib/ostree-boot
d00555 0 0      0 /usr/lib/ostree-boot
d00700 0 0      0 /usr/lib/ostree-boot/efi
d00700 0 0      0 /usr/lib/ostree-boot/grub2

So yeah, turns out it is caused by --unified-core. Feel honestly a bit embarrassed now, I seriously was under the illusion that I had tried every change to the commands one could make. I even looked at the pungi repo for guidance. Seems that I had missed the one thing that could actually make a difference…

This leads us to two questions though:

First, where does --unified-core get the “loader” directory from. I was under the impression that this argument was just supposed to make the whole rpm-ostree compose operation behave like it does on client side and that it could serve, basically, as a drop-in replacment…

Anyways, thank you so much to you and jlebon over at the rpm-ostree Github, you both pointed me in this direction today.

1 Like
7

Yes. The lorax templates where this Scripts to create custom fedora silverblue iso · GitHub was put together from did not have unified core flag. I’ve added it as it suggested to try and see if it works… It did a year ago :wink:

One thing worth mentioning about this is that while ISO will contain your changes from your manifest file. It will still need to be re-pointed at different remote. Otherwise you will install your changes via ISO and on next rpm-ostree it will rebase itself back to the upstream silverblue remote.

1 Like
8

Yeah, thanks for that script. Alongside dustymabe’s blog and some other sources, that was very helpful for getting a simple script that I could be fairly confident in actually working, rather than just me making some mistake. Sorry for not linking it beforehand, I honestly had looked through so many sources at that point that I lost track where I first found that.

You are absolutely right on the need to host a repo online for deployment, but I honestly just did this as a learning exercise. I wanted to find out how different immutable distros are built and what kind of changes one could easily make.

9

No worries. I don’t mind.
Its stolen from bunch of other sources anyway :wink:

There is another cool feature upcoming possibly in F36 which will allow you to use Containers to modify and create ostree remotes and rebase from them.

10

So I have built my first x86_64 iso and it works. Now I want to know how I can build an aarch64 iso this is what I have so far.
`mkdir isobuild && cd isobuild
sudo dnf install rpm-ostree lorax -y
git clone -b f36 Overview - workstation-ostree-config - Pagure.io
git clone -b f36 Overview - fedora-lorax-templates - Pagure.io
mkdir repo
ostree init --repo=repo
rpm-ostree compose tree --repo=$(pwd)/repo
$(pwd)/workstation-ostree-config/fedora-silverblue.yaml

lorax --product=Fedora
–version=36
–release=20220808
–source=Index of /compose/36/latest-Fedora-36/compose/Everything/aarch64/os
–variant=Silverblue
–nomacboot
–volid=Fedora-SB-ostree-aarch64-36
–add-template=$(pwd)/fedora-lorax-templates/ostree-based-installer/lorax-configure-repo.tmpl
–add-template=$(pwd)/fedora-lorax-templates/ostree-based-installer/lorax-embed-repo.tmpl
–add-template-var=ostree_install_repo=file://$(pwd)/repo
–add-template-var=ostree_update_repo=file://$(pwd)/repo
–add-template-var=ostree_osname=fedora
–add-template-var=ostree_oskey=fedora-36-primary
–add-template-var=ostree_contenturl=mirrorlist=https://ostree.fedoraproject.org/mirrorlist
–add-template-var=ostree_install_ref=fedora/36/aarch64/silverblue
–add-template-var=ostree_update_ref=fedora/36/aarch64/silverblue
–logfile=$(pwd)/lorax.log
–tmp=$(pwd)/tmp
–rootfs-size=8
$(pwd)/ostree_installer`