Failed attempt to provide ignite config to CoreOS (libvirt)

Dear Blue Hatters,

I have problems with providing ignite configuration to my CoreOS instance running on KVM/qemu.

I’m simply using official docs as provided here: Provisioning Fedora CoreOS on libvirt

My Fedora 41 is running with @virtualization group installed, I can create and play around with VMs through virt-manager without any issues.

I’m able to boot CoreOS using the script from “Provisioning Fedora CoreOS on libvirt” up to the point where I get login prompt and this unfortunate message:

Ignition: no config provided by user
No SSH authorized keys provided by Ignition or Afterburn

So, simply speaking, I’m unable to access my CoreOS through SSH.

Reproduced steps of how I’m doing all this stuff:

fcos.bu (butane config) file:

variant: fcos
version: 1.5.0
passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-ed25519 MY_SSH_PUBKEY[...]

fcos.ign (transformed butane config butane -p -s fcos.bu > fcos.ign) file:

{
  "ignition": {
    "version": "3.4.0"
  },
  "passwd": {
    "users": [
      {
        "name": "core",
        "sshAuthorizedKeys": [
          "ssh-ed25519 MY_SSH_PUBKEY[...]"
        ]
      }
    ]
  }
}

My exact script for initializing fcos VM:

#!/bin/bash

IGNITION_CONFIG="/var/lib/libvirt/ignite/fcos.ign"
IMAGE="/var/lib/libvirt/images/fedora-coreos-41.20241215.3.0-qemu.x86_64.qcow2"
VM_NAME="fcos"
VCPUS="8"
RAM_MB="24576"
STREAM="stable"
DISK_GB="10"

IGNITION_DEVICE_ARG=(--qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}")

# Setup the correct SELinux label to allow access to the config
#chcon --verbose --type svirt_home_t ${IGNITION_CONFIG}

virt-install --connect="qemu:///system" --name="${VM_NAME}" --vcpus="${VCPUS}" --memory="${RAM_MB}" \
  --os-variant="fedora-coreos-$STREAM" --import --graphics=none \
  --disk="size=${DISK_GB},backing_store=${IMAGE}" \
  --network bridge=virbr0 "${IGNITION_DEVICE_ARG[@]}"

Permissions for /var/lib/libvirt/ignite directory:

/var/lib/libvirt $ ls -l
[...]
drwxr-xr-x. 1 root root  24 Jan 25 12:52 ignite

SELinux context for /var/lib/libvirt/ignite directory:

/var/lib/libvirt $ ls -Z
[...]
unconfined_u:object_r:virt_var_lib_t:s0 ignite

SELinux context for /var/lib/libvirt/ignite/fcos.ign file:

/var/lib/libvirt/ignite $ ls -Z fcos.ign 
unconfined_u:object_r:virt_var_lib_t:s0 fcos.ign

I’d appreciate all help with tracking down where is the problem…

Thank you in advance!

Works here. Putting ignition file and qcow2 imange in~/.local/share/libvirt
The only difference from the linked documentation is that I don’t use --connect="qemu:///system"
Otherwise putting all in /var/lib/libvirt/ as you do, it works as well.
I don’t know, double check all things.

Thanks for checking that on your side.

So, how did you manage to end up with no Permission denied shenanigans when placing image in ~/.local/share/libvirt? (I suppose you have meant ~/.local/share/libvirt/images)

Can you show regular rwx permissions of your home directory, as ACL permissions by showing output of the $ sudo getfacl -e $HOME command?

If you don’t pass the --connect option to virt-install, it still defaults to --connect qemu:///system so I suppose that’s not a big deal here, but I may be wrong.

Yes, without passing --connect qemu:///system, otherwise yes, I get permission denied.

ERROR    Cannot access backing file '/home/user/.local/share/libvirt/images/fedora-coreos-41.20250105.3.0-qemu.x86_64.qcow2' of storage file '/var/lib/libvirt/images/fcos-test-01.qcow2' (as uid:107, gid:107): Permission denied
user@host:~$ sudo getfacl -e $HOME
[sudo] password for user: 
getfacl: Removing leading '/' from absolute path names
# file: home/user
# owner: user
# group: user
user::rwx
group::---
other::---

I confirm that’s the same behaviour on my side.

Where do you place your ignition config?

I like to do that in ~/.config/libvirt/ignition which inherits default SELinux context policy, but I am still getting permission denied here.

Yes, sorry ~/.local/share/libvirt/images/

Again, in ~/.local/share/libvirt/images/

If I use ~/.config/libvirt/ignition

ERROR    internal error: QEMU unexpectedly closed the monitor (vm='fcos-test-01'): 2025-01-25T20:19:53.467871Z qemu-system-x86_64: -fw_cfg name=opt/com.coreos/config,file=/home/user/.local/share/libvirt/ignition/fcos.ign: can't load /home/user/.local/share/libvirt/ignition/fcos.ign: Failed to open file “/home/user/.local/share/libvirt/ignition/fcos.ign”: Permission denied

Because

ll -Z /home/user/.local/share/libvirt/ignition/fcos.ign
-rw-r--r--. 1 user user unconfined_u:object_r:data_home_t:s0 211 Jan 25 21:19 /home/user/.local/share/libvirt/ignition/fcos.ign

Btw,
chcon --verbose --type svirt_home_t /home/alessio/.local/share/libvirt/ignition etc. solves the issue.

OK I think I got past permissions problems…

CoreOS boots but still with

Ignition: no config provided by user
No SSH authorized keys provided by Ignition or Afterburn

Do you have the same message, or config was loaded?

In my case, the ignition configuration is correctly applied.

I don’t understand it. I’m doing everything according to the guide. I verified that executed script indeed provides ignite config to qemu:

Screenshot From 2025-01-26 20-30-31

But I get this:

Is anyone here who could help with this?

Since it’s not entirely clear from the picture what exactly is written on this line, can you post the entire script you use to start the VM as pre-formatted text?

Hi Hristo,

This is the exact script to populate CoreOS VM:

#!/bin/bash

IGNITION_CONFIG="${HOME}/.config/libvirt/ignition/fcos-k8s.ign"
#IGNITION_CONFIG="${HOME}/.local/share/libvirt/images/fcos-k8s.ign"
IMAGE="${HOME}/.local/share/images/fedora-coreos-41.20241215.3.0-qemu.x86_64.qcow2"
VM_NAME="fcos-k8s"
VCPUS="8"
RAM_MB="24576"
STREAM="stable"
DISK_GB="10"

IGNITION_DEVICE_ARG=(--qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}")

# setup the correct selinux label to allow access to the config
#chcon --verbose --type svirt_home_t ${IGNITION_CONFIG}

virt-install --connect "qemu:///session" --name "${VM_NAME}" --vcpus "${VCPUS}" --memory "${RAM_MB}" \
  --os-variant "fedora-coreos-$STREAM" --import --graphics none \
  --disk "size=${DISK_GB},backing_store=${IMAGE}" \
  --network bridge=virbr0 "${IGNITION_DEVICE_ARG[@]}"

Providing the Ignition config from the same directory as yours works on my end. Can you share the contents of the Ignition config you are using?

My ignition config:

 ~ $ cd ~/.config/libvirt/ignition/
 ~/.config/libvirt/ignition $ cat fcos-k8s.ign 
{
  "ignition": {
    "version": "3.4.0"
  },
  "passwd": {
    "users": [
      {
        "name": "core",
        "sshAuthorizedKeys": [
          "ssh-ed25519 MY_PUBKEY user@hostname"
        ]
      }
    ]
  }
}
 ~/.config/libvirt/ignition $ 

The same script and Ignition config works for me. It shouldn’t matter, but what is the exact path to your script?

1 Like

I keep it in /home/ixoc/Projects/fcos-k8s/init-fcos-k8s file:

 ~/Projects/fcos-k8s $ ls -al init-fcos-k8s 
-rwxr-xr-x. 1 ixoc ixoc 802 Jan 25 22:12 init-fcos-k8s
 ~/Projects/fcos-k8s $ file init-fcos-k8s 
init-fcos-k8s: Bourne-Again shell script, ASCII text executable

Although it doesn’t matter, I tested the script from the same directory and it works for me. Since we can’t reproduce your issue, I’m not sure how we can assist further.

Is libvirt group enough for my user to be in order to use virt-install? Or qemu and kvm are also neccesary?

A few things:

  • Make sure to completely remove the VM between each try as Ignition only runs on the first boot
  • Make sure that the unprivileged user (qemu) used by the system session has access to your images in you home dir: chmod o+x /home/foo
  • Either use the user libvirtd session and place all files in your home dir or the system one and place all files in /var/lib/libvirt`
1 Like