OCI based host provisioning (baremetal/virt)

Project Discussion
#1

Hi All,
I’ve been working on a host provisioning strategy based on OCI artifacts that I refer to as OHP, which I’ve begun documenting here: GitHub - afflom/ohp: OCI Host Provisioning

I’ve also recently become aware of Changes/OstreeNativeContainer - Fedora Project Wiki and I’m curious if/how these concepts can intersect.

The biggest difference between these referenced topics that I can see; is that the OSTree Native Container is an OCI image which would require extraction from its OCI format prior to provisioning/installation. This might be a benefit because traditional container image workflows/tooling would support this strategy, while OHP leverages OCI artifacts that would store the layered contents within the OCI artifact in their original (raw) format (consumable by UEFI chainloading) which would make OHP reliant on an ORAS based builder/client.

Any discourse/feedback on the subject is welcome and much appreciated.

#2

This is really cool! Thanks so much for posting this here.

There’s also another intersection with GitHub - cgwalters/coreos-diskimage-rehydrator: Part of implementing https://github.com/openshift/enhancements/pull/201 - did you see that? I’d set that project aside in favor of the ostree-native-container work, but they are definitely all related.

Offhand, it seems like the way these projects could be chained together is that one would follow something like these steps Applying a new package using CoreOS layering - coreos/rpm-ostree to make a new coreos-derived container image with custom in-OS content. You’d push that to a registry, then craft an Ignition config (or use via butane sugar) that switches to your custom OS via container image on firstboot. To be hermetic, that image should be pulled via @sha256 digest perhaps?

That Ignition config then gets embedded as proposed now in OHP. These things are orthogonal - what we’re gaining by combining them is just that one can now customize the OS, but the customized OS state is now captured along with its “provisioning prerequisites” as containers (or OCI artifacts really as you propose).

There’s also a strong intersection with a discussion around shipping the FCOS stream metadata as a container/OCI itself. That came up in container-native CoreOS release engineering · Issue #828 · coreos/fedora-coreos-tracker · GitHub I think.

Does this all seem right? BTW, how are you seeing support for OCI artifacts today? I guess we can probably design new things today that require it, but I worry we’d at least need fallbacks to ship via OCI containers, not artifacts in some cases.

#3

The intersection between these two gets stronger if we support taking a FCOS-derived container image, and wrapping it in an ISO, AMI, etc. We have all that code in coreos-assembler today. If we do that, then the user has created fully custom disk images, and then their OHP configs reference them.

A random question…is this mainly focused on PXE? And/or is this project mainly trying to define a standard schema for the provisioning chain, and then a separate tool handles “rendering” it for a given infrastructure? Would it ever be in scope to e.g. have an AMI/Azure/OpenStack/etc image reference in this chain?