Directories to delete to enforce kvm/qemu/libvirt configurations to be recreated with current defaults (managed by cockpit-machines and virt-machines, that shall be reset too), and without investing much time :)

Ask Fedora
, , , , , ,
1

I have migrated my virtual machines from one instance to the next for a long time, and what I have configured on my system is thus far away from the current defaults. This led already to issues when I had to file bugs, and especially with current SELinux policies (concerning SELinux confined user accounts), my configurations are no longer compatible and cause trouble (especially with SELinux policies in confined accounts, it is not always easy to identify the origin, its not always the VM configuration/setup itself, of something that is broken and thus identify how to mitigate).

I thus decided to delete “my” defaults, assuming the applications will create new ones that are (from today’s point of view) not customized.

I just wanted to get some review about the directories I have chosen to delete:

/var/lib/libvirt/
/etc/libvirt/
/etc/cockpit
/etc/qemu
/etc/qemu-ga/
~/.local/share/libvirt
~/.config/virt-viewer
~/.config/libvirt
/home/<cockpit user account>/

Any more directories that might be worth deleting to keep it short and simple and ensuring the outcome?

→ I aim to not delete /var/lib/libvirt/images to keep the disks of existing VMs, assuming this does not have potential for any negative impact given the mentioned goal.

I do not care if the new default is based on user sessions or system. On this machine, I just need to occasionally run VMs from inside my libvirt-group user account with GUIs. I also do not care if my virtual networks are lost.

I am aware that some of these directories are likely not related to the relevant VM configuration, but I just want to save time and delete to get defaults as easy and reliable as possible tbh, and with the highest possible chance to be compatible with each other and other parts of the system in the current condition of F42. No longer bug reports that cost my time and that of maintainers :classic_smiley: Including issues around the interfaces and so on :expressionless_face:

At the moment, it is a good time for that as this approach does not take much time, and at the moment I have no critical virtual machines or much that needs to be conserved. At the moment, I can just add the disks of VMs I retain some time later again to new machines, I do not need to conserve hardware configurations or so atm.

2

Note that some subdirs and files inside are owned by packages so you should not delete them carelessly, otherwise may create issues with ownership, permissions, ACLs, etc.

Also I experienced some SELinux denials in the past related to these locations when deploying BIOS and UEFI-based VMs:

/var/log/libvirt
/var/lib/swtpm-localca
/var/log/swtpm

Reproducing SELinux issues may be difficult as some of them are specific to QEMU’s system/session mode, or BIOS/UEFI firmware, or affected by race conditions, e.g.:

  • A VM deployment initially fails due to SELinux denial for accessing the log file, but starts to work on the second attempt with the same VM name.
  • An UEFI VM deployment fails due to SELinux denial for creating swtpm-localca, but it works correctly with an older version of SELinux policy, so mainly affects new installations.

Although you can rule out SELinux related issues with permissive mode, other system customization can also affect the setup, including systemd and sysctl configs.

To isolate the testing setup, I used nested virtualization to provision libvirt with kickstart and reliably trigger certain issues.

1 Like
3

Thanks for the suggestions :classic_smiley:

In the given cases I identified so far, the issues about that should be neglectable, but I have everything to roll back anyway. But indeed, I will only delete the contents of the folders and not the folders themselves, as they might be more risky than their contents. That’s quicker than reproducing their ownership/profiles, and with their ownership/profiles being set properly (I assume the top level has not changed over time), I assume the contents get the right ownership automatically once created by the very application itself. The only issue I experienced in such a case in the past is the root<->qemu ownership case, but the VM tools used to be able to solve that on themselves. At the worst, I can compare. But in any case, thanks for raising awareness :classic_smiley: Indeed a major point to consider for such an approach :open_mouth:

It’s a SELinux “confined user” environment (sysadm_u), permissive is not useful there. I indeed rely on SELinux being actively enforcing, both for production and testing. But at the moment, virtualization is broken with SELinux’s user confinement the way I have it configured (which compared to the current Fedora’s defaults, is a customized setup). That’s the major reason for the approach, as I need SELinux to be active, but also need VMs again (but don’t have the time to do a “clean” solution). Obviously, that virtualization is broken atm in SELinux user confinement also rules out nested virtualization, which indeed would be great to have :frowning:

Interesting is that I do not have that dir. I do not use swtpm at the moment, but I did in the past on this machine, and it is still installed. Maybe its not needed in all circumstances, I’m also not sure if the case I used it was user session or system, too long ago.

1 Like
4

Everything done as intended: I did in advance a systemctl stop for the services+sockets of libvirtd and cockpit.

After moving all mentioned contents to a temporary backup directory (so that I can easily/quickly recover them) and ensured the dir’s otherwise to be empty, I did a dnf reinstall qemu* cockpi* libvirt* virt-man*. Then, I restarted the services and tried again: new defaults have been created. virt-manager’s GUI remains broken, but that is the case for “all times” in confined accounts, so that was expected, I use virt-manager mostly for configurations, for which it is fine, but the primary goal: I can now use cockpit again :partying_face: I just had to setup a new virtual network, as that was gone too (indeed an expected outcome). I did that in virt-manager. No issues :classic_smiley:

I already set up a Fedora Kinoite instance in cockpit. The current default seems system mode. I am not convinced if everything is as default as in a new Fedora installation from scratch as some things feel a little eccentric, but that might be related to my system, which also otherwise is no longer equal to current installation defaults. Anyway, it works, that’s most important :classic_smiley:

Thanks again @vgaetera for your incentives :classic_smiley:

1 Like