SELinux preventing libvirtd to work with LVM devices

Ask Fedora
, ,
1

I’m trying to use LVM devices for Libvirt virtual machines on a freshly installed Fedora 34 system but SELinux just won’t let me:

$ virt-install --name debian0 --ram 2048 --vcpus 2 --disk path=/dev/vg0/debian0.disk0 --os-variant debian10 --cdrom debian-11.0.0-amd64-netinst.iso 

Starting install...
ERROR    internal error: process exited while connecting to monitor: 2021-09-22T00:18:17.008067Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/vg0/debian0.disk0","aio":"native","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: Could not open '/dev/vg0/debian0.disk0': Permission denied
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///session start debian0
otherwise, please restart your installation.

Of course, no mere user should have access to raw block devices, but I explicitly gave permissions to myself to access that disk:

$ ls -LlZ /dev/vg0/debian0.disk0
brw-rw----+ 1 root disk system_u:object_r:fixed_disk_device_t:s0 253, 2 Sep 22 02:11 /dev/vg0/debian0.disk0

$ getfacl /dev/vg0/debian0.disk0 
[...]
user::rw-
group::rw-
group:libvirt:rw-
mask::rw-
other::---

$ groups
christian wheel libvirt

Syslog has the following interesting lines about this:

Sep 22 02:19:54 horus libvirtd[56078]: unable to set security context 'unconfined_u:object_r:svirt_image_t:s0:c417,c506' on '/dev/vg0/debian0.disk0' (errno 1)
Sep 22 02:19:54 horus audit[57123]: AVC avc:  denied  { open } for  pid=57123 comm="qemu-system-x86" path="/dev/dm-2" dev="devtmpfs" ino=604 scontext=unconfined_u:unconfined_r:svirt_t:s0:c417,c506 tcontext=system_u:object_r:fixed_disk_device_t:s0 tclass=blk_file permissive=0
Sep 22 02:19:54 horus libvirtd[56078]: Unable to read from monitor: Connection reset by peer
Sep 22 02:19:54 horus libvirtd[56078]: internal error: qemu unexpectedly closed the monitor: 2021-09-22T00:19:54.319161Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/vg0/debian0.disk0","aio":"native","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: Could not open '/dev/vg0/debian0.disk0': Permission denied
Sep 22 02:19:54 horus libvirtd[56078]: unable to set security context 'system_u:object_r:fixed_disk_device_t:s0' on '/dev/dm-2' (errno 1)

So, short of disabling SELinux I tried to create a special policy:

# ausearch -w | grep denied | grep -vE 'abrt|dnsm|accounts|geoclue|mand|systemd|xdm' | audit2allow 

#============= svirt_t ==============
allow svirt_t fixed_disk_device_t:blk_file { ioctl lock open };

# ausearch -w | grep denied | grep -vE 'abrt|dnsm|accounts|geoclue|mand|systemd|xdm' | audit2allow -M local
# semodule -i local.pp

With that in place, the error changed somewhat:

$ virt-install --name debian0 --ram 2048 --vcpus 2 --disk path=/dev/vg0/debian0.disk0 --os-variant debian10 --cdrom ~/Downloads/debian-11.0.0-amd64-netinst.iso 
Starting install...
ERROR    internal error: qemu unexpectedly closed the monitor: 2021-09-22T00:27:59.437900Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/vg0/debian0.disk0","aio":"native","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: Could not stat file: Permission denied

So, it now tries to acces some kind of stat file but fails. Syslog doesn’t have more clues here, and the audit log doesn’t show new AVC denials:

Sep 22 02:29:29 horus libvirtd[61132]: unable to set security context 'unconfined_u:object_r:svirt_image_t:s0:c253,c289' on '/dev/vg0/debian0.disk0' (errno 1)
Sep 22 02:29:29 horus libvirtd[61132]: Unable to read from monitor: Connection reset by peer
Sep 22 02:29:29 horus libvirtd[61132]: internal error: qemu unexpectedly closed the monitor: 2021-09-22T00:29:29.084533Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/vg0/debian0.disk0"
,"aio":"native","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: Could not stat file: Permission denied
Sep 22 02:29:29 horus libvirtd[61132]: unable to set security context 'system_u:object_r:fixed_disk_device_t:s0' on '/dev/dm-2' (errno 1)

With SELinux in permissive mode the virt-install command succeeds and the virtual machine appears can be installed.

Does anybody have an idea how to tackle this? Searching the interwebs for “LVM” and “Libvirt” and “SELinux” did return a surprisingly low number of results :-\

3 Likes
2 
VG="vg0"
LV="debian0.disk0"
sudo tee /etc/udev/rules.d/99-libvirt-lvm.rules << EOF > /dev/null
SUBSYSTEM=="block", \
ACTION=="add|change", \
ENV{DM_VG_NAME}=="${VG}", \
ENV{DM_LV_NAME}=="${LV}", \
RUN+="/usr/bin/chcon -t svirt_image_t %N"
EOF
sudo systemctl restart systemd-udevd.service
sudo lvchange -a n /dev/${VG}/${LV}
sudo lvchange -a y /dev/${VG}/${LV}
1 Like
3

Thanks for the reponse, @vgaetera, but no dice. Even with svirt_image_t set, the error message is the same:

$ virt-install --name debian0 --ram 2048 --vcpus 2 --disk path=/dev/vg0/debian0.disk0 --os-variant debian10 --cdrom ~/Downloads/debian-11.0.0-amd64-netinst.iso

Starting install...
ERROR    internal error: qemu unexpectedly closed the monitor: 2021-09-27T22:08:43.511310Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/vg0/debian0.disk0","aio":"native","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: Could not open '/dev/vg0/debian0.disk0': Permission denied
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///session start debian0
otherwise, please restart your installation.

$ ls -LZ /dev/vg0/debian0.disk0 
system_u:object_r:svirt_image_t:s0 /dev/vg0/debian0.disk0

And syslog has:

Sep 28 00:08:43 horus libvirtd[7938]: unable to set security context 'unconfined_u:object_r:svirt_image_t:s0:c592,c964' on '/dev/vg0/debian0.disk0' (errno 1)
Sep 28 00:08:43 horus libvirtd[7938]: Unable to read from monitor: Connection reset by peer
Sep 28 00:08:43 horus libvirtd[7938]: internal error: qemu unexpectedly closed the monitor: 2021-09-27T22:08:43.511310Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/vg0/debian0.disk0","aio":"native","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: Could not open '/dev/vg0/debian0.disk0': Permission denied
Sep 28 00:08:43 horus libvirtd[7938]: unable to set security context 'system_u:object_r:fixed_disk_device_t:s0' on '/dev/dm-2' (errno 1)

So, apparently libvirtd expects the disk device to with a fixed_disk_device_t context. But even manually setting fixed_disk_device_t gives the same error :-\

4

I spoke too soon: while the SELinux context may have been correct now, the libvirt user could still not write to the disk device according to the standard permissions set. But with the correct ACLs set, it seems to work:

sudo setfacl -m g:libvirt:rw /dev/vg0/debian0.disk0

Now, virt-install still issues a warning, but the installation continues anyway:

Sep 28 00:32:07 horus virt-install[12122]: unable to set security context 'unconfined_u:object_r:svirt_image_t:s0:c613,c663' on '/dev/vg0/debian0.disk0' (errno 1)

Extending the .rules file accordingly:

$ cat /etc/udev/rules.d/99-libvirt-lvm.rules 
SUBSYSTEM=="block", ACTION=="add|change", ENV{DM_VG_NAME}=="vg0", \
	ENV{DM_LV_NAME}=="debian0.disk0", \
	RUN+="/usr/bin/chcon -t svirt_image_t %N", \
	RUN+="/usr/bin/setfacl -m g:libvirt:rw %N"

Thanks for the hint, @vgaetera!

1 Like