CoreOS Ignition file location - Cockpit VM Installation

Hi guys,

I am coming back to Fedora after a very long break and have a lot of catching up to do. Not quite a newbie, but close enough.

Anyway, I created a VM CoreOS instance through Cockpit (hosted on Fedora Server 38), but I am struggling to find details on where the ignition file needs to reside in this setup. When CoreOS boots, it is looking for an Ignition file, but I’m not clear where that would need to reside to be accessible during the initial boot.

I still have to create an Ignition file, but once created, where does that need to sit to be accessible?

The CoreOS iso is in /var/lib/libvirt/boot and the qcow2 in /var/lib/libvirt/images.

Cheers, Innes (NZ)

I honestly am not really sure (haven’t done a VM installation through cockpit before), but our docs show you exactly what to do using either libvirt or plain qemu, which might help you along your way. In both of those cases you are booting a qcow2 disk image (you don’t need to start from the ISO).

Hey Dusty, thanks for the reply. Apparently, Cockpit will recognise VMs created via libvirt or qemu, so that’s what I’ll try. I’ll post back once I’ve tried that route. Cheers, Innes

It’s probably easiest to do it via the virt-install commands from the Fedora CoreOS docs as Dusty suggested.

If you want to do it purely via the Cockpit UI, you’ll likely have to upload your Ignition config file somewhere on the server beforehand, and then use the “Create and edit” button to add the additional XML bits needed to reference the Ignition config via the QEMU command line argument --qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=..." (libvirt: QEMU command-line passthrough).

Hi Dusty, happy to report that I managed to stand up a CoreOS VM using virt-install with an associated Ignition file and which then appeared in Cockpit automatically. Happy days.

Thanks for your help. I didn’t realise I’d get someone so close to CoreOS reaching out, but after watching the discussion with Mark Pusey from Columbia, I found out your connection. My last experience with Linux was RHEL 3/4 when I built a virtual prod environment on top of VMWare ESX. How things have improved.

Cheers, Innes

1 Like

Hey Timothée,

Dusty’s suggested worked fine, but I’ll try the Cockpit UI method. I was sure there had to be a solution, I just wasn’t looking in the right place.

Thanks for the approach that was specific to my original issue.

Cheers, Innes

It’s all but undocumented, but I recently had to figure this out as well. You end up having to read the Ignition code directly, and the distro-specific (Fedora) dracut ignition module code. There’s basically no documentation from any project about anything but a few narrow use-cases.

The Ignition file processing is all handled by the Ignition project’s tool, which runs as part of your initramfs. The Ignition project provides a basic dracut module covering the basics of setup for, and calls to, the actual Ignition executable, but expects the individual distros to customize it for their needs. Fedora and OpenSuse are the only two I’ve seen using it so far and have different customizations.

A lot of the specific logic is hardcoded into the Ignition binary itself, and the provided dracut module is primarily for making the series of calls to the Ignition binary. You’ll have to end up reading the Ignition Golang code itself to figure out what files it’s loading when, how it’s merging, etc.


The Ignition code currently has 2 different types of Ignition configs it loads, a “base system” config, and a “user” config. These are merged together before being applied, with “user” settings taking precedence over “base system” settings (i.e. loaded later). The “base system” config itself is also split into multiple files that get merged together.

Where the “user” config comes from is dependent on your “platform”, which must either be specified to the Ignition binary as part of the dracut module, or will get auto-detected by the binary itself. The possible platform types are a hardcoded list you have to pick from, though they’re almost all different cloud providers, VM tools, or the bare metal value “metal”.

The “base system” config also makes use of the platform value, but only if it’s passed on the command-line. If not, it doesn’t do any platform related logic (currently). The Fedora distro version of the dracut ignition module only sets the platform on the command-line if you set the kernel argument ignition.platform_id=. The OpenSuse distro version of the dracut ignition module auto-detects if you’re running in a VM by using systemd-detect-virt and specifying “metal” otherwise, but only if the kernel argument isn’t present. You can optionally add this auto-detect logic to the Fedora version if you modify the /usr/lib/dracut/modules.d/30ignition/ignition-generator file to add in the equivalent logic from the OpenSuse code. In either case you can only get cloud-provider-specific “base system” config fragments if you set the cloud provider in the ignition.platform_id= kernel argument (OpenSuse just makes it possible to use the VM-engine and “metal” platform fragments without setting the kernel argument).

The current Ignition binary will load the “base system” ignition config fragments from /usr/lib/ignition/base.d/*.ign within the initramfs. If the platform type is set on the command line, it will also load ignition fragments from /usr/lib/ignition/base.platform.d/{platform_type}/*.ign. These are all witihin the initramfs, so you usually need to add dracut modules to write these files to the generated initramfs.

You have to read the code for the specific platform (called “provider” classes, with not all provider classes being available to pick or detect as a platform type) to determine where the “user” config comes from, but for VMs it is read from a VM config variable. For “metal” it can come from a /usr/lib/ignition/user.ign file in the initramfs, but the distro-specific dracut modules generate this file in some cases, or where the file comes from can be changed with kernel arguments or other distro-specific dracut-module-specified conditions.

Be aware, the default dracut ignition module, and both the Fedora and OpenSuse distro versions of it will delete your ignition files from the initramfs after they’ve been consumed. This is a feature to ensure changes are only made once, the first time its booted. Direct modification of the dracut ignition module is needed if you don’t want this to happen.

There is documentation on how to pass an Ignition config to a system: Supported Platforms | Ignition. See also all the examples for each platform supported by Fedora CoreOS: Installing CoreOS on Bare Metal :: Fedora Docs.

How Ignition handles the passed config internally isn’t really documented indeed but that’s not expected to be a user facing interface.

That’s actually the documentation on the very narrow use-cases I was referring to. It’s a one sentence explanation for each platform, and is useful to know basically what you need to do, but it’s by no means complete. For example, in the VirtualBox case it just says to “provide the config” via a property, there’s no mention of the format. It’s either a URL to download it from (but only for specific set of protocols), or the raw content itself.

The bare metal description is a bit more useful in covering some of the basics, coreos-installer is very useful but also is not very well documented. But neither ignition nor coreos-installer explain what is done during th e programming, which is a problem since the ignition file being supplied has to transform whatever it setup into what is needed without wiping out the boot or root partitions.

Similarly, there’s very little mention of the 'built in" files being loaded, which means it’s necessary to dig into the Ignition code to figure out where to look in the prebuilt FCOS image to find the base config that’s always loaded and affects the config being supplied (arguably an FCOS/OpenSUSE documentation gap).

Admittedly my personal use case was in extending FCOS images to provide prebuilt baseline images that could be further adapted, not exactly what most people are doing, and I was personally trying to make the images runnable on bare metal or VMs dynamically with slightly different built in base configs. It’s more of an expectation my development case would require digging into the Ignition code a bit. But none of the users I was supplying the images to were able to independently figure out how to add their own config based on the documentation you linked without a lot of struggle, and often deciding to use the live ISO bare metal method to create a VM instance rather than the VM property.

I’d be happy to supply some doc PRs to help clarify some of this if they’d be of interest, but I usually get mixed or lukewarm responses with PRs often going stale before they’re ever looked at from Fedora tool owners when I do.

How to set things up for Virtualbox is documented in Provisioning Fedora CoreOS on VirtualBox :: Fedora Docs.

coreos-installer usage is documented in CoreOS Installer | CoreOS Installer documentation.

Documentation updates are always welcomed.

Please link to those PRs so that we can take a look.