F40 Change Proposal: Build Fedora Cloud Edition Images Using Kiwi in Koji (System-Wide)

Project Discussion Change Proposals
,
21

When we migrated the fedora-toolbox images from Container/Dockerfile to Pungi and Kickstart, my only complaint was the significant difficulty in building them locally. It’s a lot easier to do podman build ... than koji image-build ..., which has implications for Toolbx contributors trying to alter the images. From what Neal said, it sounds like it’s easier to do that with KIWI, so that’s wonderful.

So, it sounds like you might not be opposed to moving to this?

Yes, I am not opposed to moving to KIWI.

22

Could we possibly make a image via kiwi and run the tests against it?

Sure! We can have it done both ways: Toolbox can build the image in CI upstream to validate stuff there, and we can downstream also fetch the tests and run things too, provided the harness on their end is easy to execute in TMT/Zuul.

The upstream Toolbx Bats tests are also available through the toolbox-tests package. So, I suppose, one way or another, they can be added to the fedora-kiwi-descriptions CI.

Right now, the upstream Toolbx CI, will only invoke the Bats tests against the fedora-toolbox images that have been already built and published through registry.fedoraproject.org.

There’s another set of tests that are currently in the Container/Dockerfile and Kickstart. These are a bit more rudimentary and ensure that certain files are present (eg., documentation, locales, translations, etc.) and are as they are expected to be (eg., links versus regular files). If necessary, we could rewrite them as Bats tests or some other form.

We also keep a copy of the Fedora and RHEL image sources in the upstream Toolbx Git repository. This is to make the full sources for the RHEL images publicly available, because, it seems, otherwise, only the Container/Dockerfiles would be available without the secondary files. For the Fedora images, it provides a known place for contributors outside the Fedora universe from where they can see the sources and make changes to them if necessary.

To exploit this, I have forever wanted to run the upstream tests a second time on images built on-the-fly with podman build ... and such, over and above the already published official images. It could flag changes in package dependencies and such that might break future image builds. I was close to adding this, but then we switched to Image Factory, and it wasn’t as easy to use as podman build, so I set it aside.

Currently the kiwi description does not generate a Docker/OCI formatted image directly because I don’t think it would work in the existing image build+publish process our compose tooling uses, but I could change that or add an alternative target that generates it so it can be loaded into Podman for testing right away.

If KIWI generates a Docker/OCI archive tarball, then it would just be a skopeo copy ... away from being visible to Podman, right? Or am I missing something.

23

That would be correct. KIWI in Fedora essentially orchestrates buildah for the process of creating such an archive.

24

ok. Great.

Can we update the change with this info?
I am not sure if FESCo would like more comment on it after that, but at
least we should update the devel list thread explaining the expanding
scope.

It might be good to make a test image and run it through the toolbox ci
tests if possible to know that the porting and different tooling didn’t
break anything…

25

Kevin Fenzi kevin Stroopwafel (Cookie VII)
January 11

ok. Great.
Can we update the change with this info?

I can update the change.

26

ok. So, I looked and I was misremembering. There’s a number of other things that are using ImageFactory: vagrant images, all the aarch64 raw spins, the server kvm qcow2.

Now, we could replace all those with kiwi too, but that makes the scope of this… really big IMHO.

Also, I re-looked at the change and… perhaps I misread it. I understood it to be just moving Cloud images that we currently produce to kiwi. But Perhaps you all intended it to also add new artifacts? The change has only "We will use Toddler and Ansible to deliver images to the various public cloud targets (GCP, AWS, Azure, OCI, etc.) "
We already make GCP and Azure images, but are there going to be other net new images here? what exactly are they?

How about this: For this change / f40, just switch existing cloud artifacts to kiwi. In f41, look again at osbuild and if it’s looking not ready for us by then, move the rest of the imagefactory items over to kiwi so we can at least stop using imagefactory. Thats also going to possibly mean replacing livemedia-creator possibly? Would need to get buyin from spins and workstation about that.

27

I guess we could consider dropping vagrant images…

28

If we are going to consider dropping vagrant images, we probably missed the window for that. What say we just produce them (we can produce them with Kiwi) for 40 and consider dropping them in 41. I actually considered them part of this cloud change originally. Is there a reason we are distinguishing them?

29

nothing in particular, just Kevin mentioned them and I thought, hmm, at some point we might want to not do those any more, since Vagrant isn’t F/OSS and no F/OSS replacement has emerged…

30

Yes, I think we should just switch what we’ve committed to for now (Cloud-owned images and the container images) this cycle, then Fedora Cloud WG can work on porting the rest of them afterward and reach out to all affected stakeholders for switch for Fedora 41 to a migration plan.

That means that after the completion of this change, ImageFactory will be deprecated, slated to be retired for Fedora 41. At the minimum, this will ensure that the images that we regularly respin and build post-GA are moved over to kiwi now, so at the end of the F41 development cycle, we can pull the plug on ImageFactory as F39 will also be EOL afterward and we will stop respinning cloud and container images for that release then too.

Fedora Cloud WG will handle the deprecation notices and contacting everyone about this.

31

Yes, I think we should just switch what we’ve committed to for now (Cloud-owned images and the container images) this cycle,

Sounds reasonable.

then Fedora Cloud WG can work on porting the rest of them afterward and reach out to all affected stakeholders for switch for Fedora 41 to a migration plan.

That means that after the completion of this change, ImageFactory will be deprecated, slated to be retired for Fedora 41. At the minimum, this will ensure that the images that we regularly respin and build post-GA are moved over to kiwi now, so at the end of the F41 development cycle, we can pull the plug on ImageFactory as F39 will also be EOL afterward and we will stop respinning cloud and container images for that release then too.

Fedora Cloud WG will handle the deprecation notices and contacting everyone about this.

I think we need to see if we can move to osbuild by f41 time.

Either way, this needs buyin from spins folks, workstation working
group, server working group and possibly others.

So I would just leave f41 open for now, and as we get closer and have
more input a f41 change can be put in place?

32

"Also, I re-looked at the change and… perhaps I misread it. I understood it to be just moving Cloud images that we currently produce to kiwi. But Perhaps you all intended it to also add new artifacts? The change has only “We will use Toddler and Ansible to deliver images to the various public cloud targets (GCP, AWS, Azure, OCI, etc.) "
We already make GCP and Azure images, but are there going to be other net new images here? what exactly are they?”

can someone answer this also? :wink:

33

AArch64 images for all the clouds, and AArch64 images for Vagrant too are the new images that will be published.

34

I think we need to see if we can move to osbuild by f41 time.

To reiterate what @davdunc said above, nothing in this Change prevents moving to osbuild in the future. But in the meantime, this will allow us to get rid of ImageFactory and enable Cloud to move forward and fulfill its goals.

2 Likes
35

So, currently we produce:

Cloud-Base → qcow2
Cloud-Base → raw-xz
(for all 4 arches we have)

Fedora-Cloud-Base-GCP → tar.gz
(x86_64 only)

Fedora-Cloud-Base-Azure → vpc
(aarch64 and x86_64)

Fedora-Cloud-Base-Vagrant
(x86_64 only)

So, this would add aarch64 for GCP and Vagrant and thats it?

kevin

36

Yes. The original plan was to also add a separate Change to introduce a WSL image this cycle, but the approval for this Change has taken too long, so there’s no time. It will be introduced in 41 instead.

1 Like
37

Catching up on this. Is this the correct example for generating an image?

https://pagure.io/fedora-kiwi-descriptions/blob/rawhide/f/components/boot.xml

39

Well, no, that’s not an example, it’s just part of the configuration.

Tree - fedora-kiwi-descriptions - Pagure.io is maybe more of an example. Install those packages, run a command something like that. --kiwi-description-dir should point to a checkout of fedora-kiwi-descriptions.

1 Like
40

Closing change proposals for F40. The ship has sailed. Of course we can still discuss if needed, but please open new topics for that.