VM performance issues on FC34 with Ryzen 5950X

,

I’ve been trying to identify the source of the VM issues that I’ve been having, and I’m starting to believe it’s a hardware compatibility issue that I haven’t figured out how to address.

Recently built up a new system intended to be used as my daily driver:
Ryzen 9 5950x, 128G Ram, Nvidia Quadro M2000, freshly installed FC34

I did configure FC34 to use XFS instead of the default btrfs (primarily due to familiarity).

The first set of VM that I started to build out was a training VM that I have a static build process for. This training VM is to be used to provide tutorials for others, and has a well documented installation process (part of the training). This is a RHEL 7 VM, started on a RHEL 7.7 base, and upgraded to the latest version. The point of this is to say there should be no functional reason why this VM should not work/install correctly.

During the installation of the VM I for the first time ever saw a message along the lines of “This processor is not part of the Red Hat supported hardware list, please consult blah blah blah”. This of course was my first indication there was an issue. The system still worked reasonably well but seemed extremely slow. I increased the CPU/RAM on the VMs from 1CPU/4g to 2CPU/8G and finally to 4CPU/16G to try to resolve the issue.

I’ve been using virt-manager to configure/manage my VMs. I tried multiple CPU architectures for emulation purposes - qemu64, kvm64, host-passthrough, host-model. All options resulted in the same extreme sluggishness on any CPU intensive operation.

The configuration/deployment process for the application (Red Hat Certificate System) failed every time, no matter what architecture or CPU/RAM configuration I used.

I recently re-deployed the same configuration on an FC31 box using a 2CPU/4G ram setup - and the VMs and application deployed without any issues.

Assistance in isolating the configurations required to deploy RHEL 7 VMs with the given hardware configuration would be appreciated.

1 Like
lscpu; lsmem; free -m; swapon; zramctl
sudo virt-host-validate
virsh dumpxml VM_NAME

pastebin.com

lscpu; lsmem; free -m; swapon; zramctl

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   48 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              16
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      25
Model:                           33
Model name:                      AMD Ryzen 9 5950X 16-Core Processor
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         2200.000
CPU max MHz:                     5083.3979
CPU min MHz:                     2200.0000
BogoMIPS:                        6787.95
Virtualization:                  AMD-V
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        8 MiB
L3 cache:                        64 MiB
NUMA node0 CPU(s):               0-31
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and secc
                                 omp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitiza
                                 tion
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, IBRS_FW, STIBP
                                  always-on, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pa
                                 t pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt 
                                 pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid e
                                 xtd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_
                                 1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_lega
                                 cy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch os
                                 vw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perf
                                 ctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb st
                                 ibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a 
                                 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 x
                                 saves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero i
                                 rperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save
                                  tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfth
                                 reshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vae
                                 s vpclmulqdq rdpid overflow_recov succor smca fsrm
RANGE                                 SIZE  STATE REMOVABLE BLOCK
0x0000000000000000-0x000000107fffffff  66G online       yes  0-32

Memory block size:         2G
Total online memory:      66G
Total offline memory:      0B
               total        used        free      shared  buff/cache   available
Mem:           64206        2825       59173         195        2206       60533
Swap:          12287           0       12287
NAME       TYPE      SIZE USED PRIO
/dev/dm-1  partition   4G   0B   -2
/dev/zram0 partition   8G   0B  100
NAME       ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle         8G   4K   80B   12K      32 [SWAP]

sudo virt-host-validate

[root@midgard ~]# virt-host-validate
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : FAIL (Check that the 'kvm-intel' or 'kvm-amd' modules are loaded & the BIOS has enabled virtualization)
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for device assignment IOMMU support                         : PASS
  QEMU: Checking if IOMMU is enabled by kernel                               : PASS
  QEMU: Checking for secure guest support                                    : WARN (Unknown if this platform has Secure Guest support)
   LXC: Checking for Linux >= 2.6.26                                         : PASS
   LXC: Checking for namespace ipc                                           : PASS
   LXC: Checking for namespace mnt                                           : PASS
   LXC: Checking for namespace pid                                           : PASS
   LXC: Checking for namespace uts                                           : PASS
   LXC: Checking for namespace net                                           : PASS
   LXC: Checking for namespace user                                          : PASS
   LXC: Checking for cgroup 'cpu' controller support                         : PASS
   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
   LXC: Checking for cgroup 'cpuset' controller support                      : PASS
   LXC: Checking for cgroup 'memory' controller support                      : PASS
   LXC: Checking for cgroup 'devices' controller support                     : PASS
   LXC: Checking for cgroup 'freezer' controller support                     : FAIL (Enable 'freezer' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'blkio' controller support                       : PASS
   LXC: Checking if device /sys/fs/fuse/connections exists                   : PASS

virsh dumpxml pki-training

<domain type='qemu'>
  <name>pki-training</name>
  <uuid>deee1c18-d75c-46aa-b51b-94d9d600a2e9</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://redhat.com/rhel/7.7"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <vcpu placement='static'>8</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-5.2'>hvm</type>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state='off'/>
  </features>
  <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>kvm64</model>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rhel7.7.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='sda' bus='sata'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x16'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:9b:1b:82'/>
      <source bridge='virbr0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='2'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
    </graphics>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/urandom</backend>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </rng>
  </devices>
</domain>

Edits: formatting

1 Like
lsmod | grep -e kvm; rpm -q -a qemu\* libvirt\* | sort
2 Likes
  • Updated Bios to include SVM
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  • lsmod | grep -e kvm
kvm                  1007616  1 kvm_amd
irqbypass              16384  1 kvm
ccp                   110592  1 kvm_amd
  • rpm -q -a qemu\* libvirt\* | sort
libvirt-bash-completion-7.0.0-6.fc34.x86_64
libvirt-client-7.0.0-6.fc34.x86_64
libvirt-daemon-7.0.0-6.fc34.x86_64
libvirt-daemon-config-network-7.0.0-6.fc34.x86_64
libvirt-daemon-config-nwfilter-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-interface-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-libxl-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-lxc-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-network-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-nodedev-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-nwfilter-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-qemu-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-secret-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-core-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-disk-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-gluster-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-iscsi-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-iscsi-direct-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-logical-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-mpath-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-rbd-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-scsi-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-sheepdog-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-storage-zfs-7.0.0-6.fc34.x86_64
libvirt-daemon-driver-vbox-7.0.0-6.fc34.x86_64
libvirt-daemon-kvm-7.0.0-6.fc34.x86_64
libvirt-glib-4.0.0-1.fc34.x86_64
libvirt-libs-7.0.0-6.fc34.x86_64
qemu-audio-alsa-5.2.0-8.fc34.x86_64
qemu-audio-oss-5.2.0-8.fc34.x86_64
qemu-audio-pa-5.2.0-8.fc34.x86_64
qemu-audio-sdl-5.2.0-8.fc34.x86_64
qemu-audio-spice-5.2.0-8.fc34.x86_64
qemu-block-curl-5.2.0-8.fc34.x86_64
qemu-block-dmg-5.2.0-8.fc34.x86_64
qemu-block-gluster-5.2.0-8.fc34.x86_64
qemu-block-iscsi-5.2.0-8.fc34.x86_64
qemu-block-nfs-5.2.0-8.fc34.x86_64
qemu-block-rbd-5.2.0-8.fc34.x86_64
qemu-block-ssh-5.2.0-8.fc34.x86_64
qemu-char-baum-5.2.0-8.fc34.x86_64
qemu-char-spice-5.2.0-8.fc34.x86_64
qemu-common-5.2.0-8.fc34.x86_64
qemu-device-display-qxl-5.2.0-8.fc34.x86_64
qemu-device-display-virtio-gpu-5.2.0-8.fc34.x86_64
qemu-device-display-virtio-gpu-pci-5.2.0-8.fc34.x86_64
qemu-device-display-virtio-vga-5.2.0-8.fc34.x86_64
qemu-device-usb-redirect-5.2.0-8.fc34.x86_64
qemu-device-usb-smartcard-5.2.0-8.fc34.x86_64
qemu-img-5.2.0-8.fc34.x86_64
qemu-kvm-5.2.0-8.fc34.x86_64
qemu-system-x86-5.2.0-8.fc34.x86_64
qemu-system-x86-core-5.2.0-8.fc34.x86_64
qemu-ui-curses-5.2.0-8.fc34.x86_64
qemu-ui-egl-headless-5.2.0-8.fc34.x86_64
qemu-ui-gtk-5.2.0-8.fc34.x86_64
qemu-ui-opengl-5.2.0-8.fc34.x86_64
qemu-ui-sdl-5.2.0-8.fc34.x86_64
qemu-ui-spice-app-5.2.0-8.fc34.x86_64
qemu-ui-spice-core-5.2.0-8.fc34.x86_64

When using Clear CPU Configuration for CPU model, it will default to “qemu64”, but receives the following message on the RHEL 7 VM:

The above error is not found when using kvm64, or Copy host CPU Configuration (Which displays as EPYC).

Still having the same sluggish performance issues / inability to deploy standard build.

1 Like

Try importing the disk image as a new VM in GNOME Boxes.
Then you can specify the OS type as RHEL when configuring the VM.
It should generate the config using an optimized template for the specified OS.

gnome-boxes configured it for host-passthrough according to the XML, and for some reason decided to give it 32 CPU.

I did however notice some issues with the rpmdb which may have been the result of previous configuration issues (e.g. svm not being enabled properly). I’m working on a fresh rebuild of my test setup to see if the results are different now that that has been rectified.

1 Like

Okay, updating for conclusion.

The source of the problem was ultimately SVM not being turned on. I find this rather peculiar as the symptom was sluggishness, freezing, and more poor performance as opposed to an outright denial like I’ve experienced with intel-VT.

The secondary symptom was a result of the first. Looking back at the logs I had a failed RPM update on my deployment that causes duplicate packages to be present on the system. This resulted in application dependencies to be missing, which only manifested themselves during deployment.

1 Like