Running a Fedora CoreOS physical computer as VM host. Possible? Good/bad idea?

I thought it would be interesting to try out to create a few VM:s of Fedora CoreOS with
virt-install. As VM host I could use a computer (i.e. physical machine)
that currently has an empty HDD so I would first need to install an operating system on that computer.

As VM host operating system, I guess Fedora 32 would be a good choice.
But I was wondering if it would be possible to instead use Fedora CoreOS as the VM host operating system?

Maybe it would be possible to create an Ignition file that automates the installation of the VM host and the virt-install of the FCOS VM:s?

What do you think?
Are there any tricky things that you foresee?

Thanks,
Erik Sjölund

I don’t think you will run into any particular issue with running VMs on Fedora CoreOS. You will probably have to overlay the libvirt and qemu packages as I don’t know how well libvirt work inside of a container.

If you choose to keep auto-updates enabled by default, you should be aware that the VMs will be stopped for the system to reboot. You might want to take a look at setting up maintenance windows with zincati to make this happen only when appropriate for your users.

1 Like

We currently deploy VMs of CoreOS. The way we do it is to use packer to create a VMX from the ISO and then convert that to an OVA using VMware’s ovftool.

@dans Why not use the official Fedora CoreOS VMware OVA image?

Looking back at the code, I was incorrect. We don’t use the ISO. We start with the OVA, but we use packer because we customize the image.

1 Like

I don’t believe that FCOS would be a good virtualization host. That’s not what it is made for. It doesn’t have that technology available by design. It is made to host containers, which are not virtualization.

I use libvirt/qemu/kvm on several rather beefy raw iron machines to host CoreOS virtual machines that boot from iPXE and CoreOS automatically mounts /var (and persists containers) on an iSCSI disk assigned to the CoreOS virtual machine. Everything else is immutable and transient. To upgrade, I just change the images on the configuration server (that provides the kernel and initrd) and reboot the virtual machines. I don’t ever install CoreOS to disk manually. I just tell the DHCP server that a mac address should be a CoreOS machine, and then my ignition provisioning on a configuration server picks up from there after the kernel finishes bootstrapping itself.

The best part about it is that the virtualization grants live mobility of the CoreOS virtual machines. I can shift all of the machines between VM hosts in minutes because I don’t have to move the disk, and the memory is copied over 10GbE in seconds.