F39 upgrade broke VM

updated my VM Fedora from f38 to F39 Workstation and now boot times are 3 times more sometimes 17 minutes and after i got booted to Loginscreen no mouse inputs/cursors shown inside VM anymore mouse is there since i can see when it higlights some buttons, but still no visible cursor.

i know it is only VM and some small issues, but still issues

How does the output following command look like?

 systemd-analyze blame
25.475s dev-ttyS13.device
25.475s sys-devices-platform-serial8250-tty-ttyS13.device
25.474s sys-devices-platform-serial8250-tty-ttyS16.device
25.474s dev-ttyS16.device
25.474s dev-ttyS18.device
25.474s sys-devices-platform-serial8250-tty-ttyS18.device
25.472s sys-devices-platform-serial8250-tty-ttyS11.device
25.472s dev-ttyS11.device
25.472s sys-devices-platform-serial8250-tty-ttyS19.device
25.472s dev-ttyS19.device
25.472s dev-ttyS15.device
25.472s sys-devices-platform-serial8250-tty-ttyS15.device
25.469s sys-devices-platform-serial8250-tty-ttyS27.device
25.469s dev-ttyS27.device
25.469s sys-devices-platform-serial8250-tty-ttyS2.device
25.469s dev-ttyS2.device
25.468s dev-ttyS8.device
25.468s sys-devices-platform-serial8250-tty-ttyS8.device
25.468s sys-devices-platform-serial8250-tty-ttyS25.device
25.468s dev-ttyS25.device
25.464s dev-ttyS29.device
25.464s sys-devices-platform-serial8250-tty-ttyS29.device
25.463s dev-ttyS5.device
25.463s sys-devices-platform-serial8250-tty-ttyS5.device
25.462s dev-ttyS31.device
25.462s sys-devices-platform-serial8250-tty-ttyS31.device
25.462s sys-devices-platform-serial8250-tty-ttyS26.device
25.462s dev-ttyS26.device
25.461s dev-ttyS17.device
25.461s sys-devices-platform-serial8250-tty-ttyS17.device
25.461s sys-devices-platform-serial8250-tty-ttyS10.device
25.461s dev-ttyS10.device
25.461s sys-devices-platform-serial8250-tty-ttyS28.device
25.461s dev-ttyS28.device
25.461s dev-ttyS0.device
25.461s sys-devices-platform-serial8250-tty-ttyS0.device
25.460s dev-ttyS12.device
25.460s sys-devices-platform-serial8250-tty-ttyS12.device
25.460s sys-devices-platform-serial8250-tty-ttyS9.device
25.460s dev-ttyS9.device
25.459s dev-ttyS6.device
25.459s sys-devices-platform-serial8250-tty-ttyS6.device
25.458s dev-ttyS7.device
25.458s sys-devices-platform-serial8250-tty-ttyS7.device
25.458s dev-ttyS30.device
25.458s sys-devices-platform-serial8250-tty-ttyS30.device
25.458s dev-ttyS4.device
25.458s sys-devices-platform-serial8250-tty-ttyS4.device
25.458s sys-devices-platform-serial8250-tty-ttyS20.device
25.458s dev-ttyS20.device
25.457s sys-devices-platform-serial8250-tty-ttyS3.device
25.457s dev-ttyS3.device
25.457s sys-devices-platform-serial8250-tty-ttyS1.device
25.457s dev-ttyS1.device
25.456s dev-ttyS22.device
25.456s sys-devices-platform-serial8250-tty-ttyS22.device
25.455s dev-ttyS21.device
25.455s sys-devices-platform-serial8250-tty-ttyS21.device
25.452s sys-devices-platform-serial8250-tty-ttyS23.device
25.452s dev-ttyS23.device
25.452s sys-devices-platform-serial8250-tty-ttyS24.device
25.452s dev-ttyS24.device
25.446s sys-devices-platform-serial8250-tty-ttyS14.device
25.446s dev-ttyS14.device
25.431s sys-module-configfs.device
25.327s dev-disk-by\x2dlabel-fedora_localhost\x2dlive.device
25.327s dev-disk-by\x2dpartuuid-efd4facb\x2d9941\x2d4a67\x2d8526\x2d4409ef9af1f4.device
25.327s dev-disk-by\x2duuid-27cabaab\x2d64eb\x2d4c02\x2d8bbe\x2d6b40c731275e.device
25.327s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000\x2dpart3.device
25.327s dev-nvme0n1p3.device
25.327s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000_1\x2dpart3.dev>
25.327s sys-devices-pci0000:00-0000:00:16.0-0000:0b:00.0-nvme-nvme0-nvme0n1-nvme0n1p3.devi>
25.327s dev-disk-by\x2did-nvme\x2deui.f4666158b1ef73cb000c29653e90b58d\x2dpart3.device
25.327s dev-disk-by\x2ddiskseq-3\x2dpart3.device
25.327s dev-disk-by\x2dpath-pci\x2d0000:0b:00.0\x2dnvme\x2d1\x2dpart3.device
25.299s dev-nvme0n1p2.device
25.299s sys-devices-pci0000:00-0000:00:16.0-0000:0b:00.0-nvme-nvme0-nvme0n1-nvme0n1p2.devi>
25.299s dev-disk-by\x2dpath-pci\x2d0000:0b:00.0\x2dnvme\x2d1\x2dpart2.device
25.299s dev-disk-by\x2ddiskseq-3\x2dpart2.device
25.299s dev-disk-by\x2did-nvme\x2deui.f4666158b1ef73cb000c29653e90b58d\x2dpart2.device
25.299s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000\x2dpart2.device
25.299s dev-disk-by\x2dpartuuid-84567a1d\x2d08f1\x2d4418\x2d941c\x2dccf229eca701.device
25.299s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000_1\x2dpart2.dev>
25.299s dev-disk-by\x2duuid-5e8568a2\x2dee16\x2d4afb\x2db6f9\x2d36b895eba23b.device
25.297s sys-subsystem-net-devices-ens160.device
25.297s sys-devices-pci0000:00-0000:00:15.0-0000:03:00.0-net-ens160.device
25.291s dev-disk-by\x2did-nvme\x2deui.f4666158b1ef73cb000c29653e90b58d\x2dpart1.device
25.291s dev-nvme0n1p1.device
25.291s sys-devices-pci0000:00-0000:00:16.0-0000:0b:00.0-nvme-nvme0-nvme0n1-nvme0n1p1.devi>
25.291s dev-disk-by\x2ddiskseq-3\x2dpart1.device
25.291s dev-disk-by\x2dpartuuid-865ad022\x2d2681\x2d4523\x2da8fe\x2d1e97df960fa7.device
25.291s dev-disk-by\x2dpartlabel-EFI\x5cx20System\x5cx20Partition.device
25.291s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000\x2dpart1.device
25.290s dev-disk-by\x2duuid-F94D\x2dC2A6.device
25.290s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000_1\x2dpart1.dev>
25.290s dev-disk-by\x2dpath-pci\x2d0000:0b:00.0\x2dnvme\x2d1\x2dpart1.device
25.289s dev-nvme0n1.device
25.289s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000.device
25.289s dev-disk-by\x2did-nvme\x2deui.f4666158b1ef73cb000c29653e90b58d.device
25.289s dev-disk-by\x2ddiskseq-3.device
25.289s dev-disk-by\x2did-nvme\x2dVMware_Virtual_NVMe_Disk_VMware_NVME_0000_1.device
25.289s sys-devices-pci0000:00-0000:00:16.0-0000:0b:00.0-nvme-nvme0-nvme0n1.device
25.289s dev-disk-by\x2dpath-pci\x2d0000:0b:00.0\x2dnvme\x2d1.device
23.439s dracut-initqueue.service
 2.947s plymouth-quit-wait.service
 2.403s dev-cdrom.device
 2.403s dev-disk-by\x2dpath-pci\x2d0000:00:07.1\x2data\x2d2.device
 2.403s dev-sr0.device
 2.403s dev-disk-by\x2dlabel-Fedora\x2dWS\x2dLive\x2d38\x2d1\x2d6.device
 2.403s dev-disk-by\x2ddiskseq-2.device
 2.403s sys-devices-pci0000:00-0000:00:07.1-ata2-host1-target1:0:0-1:0:0:0-block-sr0.device
 2.403s dev-disk-by\x2dpath-pci\x2d0000:00:07.1\x2data\x2d2.0.device
 2.403s dev-disk-by\x2did-ata\x2dVMware_Virtual_IDE_CDROM_Drive_10000000000000000001.device
 2.403s dev-disk-by\x2duuid-2023\x2d04\x2d13\x2d22\x2d15\x2d10\x2d00.device
  959ms initrd-switch-root.service
  597ms firewalld.service
  415ms NetworkManager-wait-online.service
  356ms systemd-udev-trigger.service
  292ms systemd-journal-flush.service
  204ms packagekit.service
  199ms fwupd.service
  190ms user@1000.service
  182ms ModemManager.service
  179ms abrtd.service
  179ms polkit.service
  177ms udisks2.service
  171ms power-profiles-daemon.service
  165ms systemd-resolved.service
  159ms accounts-daemon.service
  150ms systemd-tmpfiles-setup.service
  140ms NetworkManager.service
  132ms systemd-remount-fs.service
  129ms systemd-journald.service
  129ms lvm2-monitor.service
  126ms systemd-oomd.service
  126ms upower.service
  124ms systemd-vconsole-setup.service
  121ms systemd-network-generator.service
  119ms avahi-daemon.service
  112ms systemd-modules-load.service
  109ms bluetooth.service
  107ms geoclue.service
  101ms var-lib-nfs-rpc_pipefs.mount
   96ms systemd-tmpfiles-setup-dev-early.service
   93ms chronyd.service
   93ms dracut-shutdown.service
   93ms initrd-cleanup.service
   92ms systemd-localed.service
   91ms modprobe@drm.service
   82ms systemd-userdbd.service
   82ms dracut-pre-pivot.service
   79ms dev-hugepages.mount
   77ms plymouth-switch-root.service
   77ms kmod-static-nodes.service
   77ms dev-mqueue.mount
   77ms dracut-cmdline.service
   74ms sys-kernel-debug.mount
   73ms systemd-sysctl.service
   72ms sys-kernel-tracing.mount
   69ms systemd-udevd.service
   69ms systemd-binfmt.service
   69ms rtkit-daemon.service
   67ms systemd-fsck@dev-disk-by\x2duuid-5e8568a2\x2dee16\x2d4afb\x2db6f9\x2d36b895eba23b.>
   61ms systemd-fsck@dev-disk-by\x2duuid-F94D\x2dC2A6.service
   61ms systemd-zram-setup@zram0.service
   61ms run-vmblock\x2dfuse.mount
   57ms sssd-kcm.service
   57ms dev-zram0.swap
   56ms auditd.service
   55ms systemd-random-seed.service
   54ms import-state.service
   52ms virtqemud.service
   51ms systemd-logind.service
   44ms systemd-tmpfiles-setup-dev.service
   43ms uresourced.service
   43ms switcheroo-control.service
   38ms colord.service
   36ms home.mount
   36ms cups.service
   34ms systemd-machined.service
   34ms plymouth-read-write.service
   34ms dracut-pre-udev.service
   33ms systemd-user-sessions.service
   32ms proc-sys-fs-binfmt_misc.mount
   31ms gssproxy.service
   29ms dbus-broker.service
   26ms systemd-sysusers.service
   23ms rpc-statd-notify.service
   23ms systemd-fsck-root.service
   21ms systemd-update-utmp.service
   21ms realmd.service
   19ms user-runtime-dir@1000.service
   17ms boot-efi.mount
   17ms initrd-udevadm-cleanup-db.service
   17ms plymouth-start.service
   16ms modprobe@loop.service
   15ms initrd-parse-etc.service
   14ms gdm.service
   13ms wpa_supplicant.service
   13ms systemd-rfkill.service
   11ms boot.mount
   10ms sys-fs-fuse-connections.mount
   10ms modprobe@dm_mod.service
    8ms modprobe@fuse.service
    6ms systemd-update-utmp-runlevel.service
    4ms tmp.mount
    4ms modprobe@configfs.service

Can we also see systemd-analyze chritical-chain? I think all those 25 sec services are depending on an earlier service and only starting up after.

What do we learn from systemctl status dracut-initqueue.service? (This shouldn’t take 23 secs)

EDIT: When you search for dracut-initqueue.service, there are quite a few topics where people are experiencing slow boots related to dracut-initqueue, see Search results for 'dracut-initqueue.service' - Fedora Discussion

This sort of issue is problematic to troubleshoot since it requires to set up a proprietary virtualization solution.
I run a Fedora VM in KVM/QEMU+libvirt and I have not noticed any regression after upgrading to the latest release.

The time the unit took to start is printed after the "+" character.

graphical.target @20.893s
└─multi-user.target @20.892s
  └─plymouth-quit-wait.service @11.886s +8.994s
    └─systemd-user-sessions.service @11.765s +76ms
      └─remote-fs.target @11.718s
        └─remote-fs-pre.target @10.333s
          └─nfs-client.target @10.333s
            └─gssproxy.service @10.256s +76ms
              └─network.target @10.193s
                └─wpa_supplicant.service @19.173s +49ms
                  └─basic.target @6.539s
                    └─dbus-broker.service @6.404s +127ms
                      └─dbus.socket @6.366s
                        └─sysinit.target @6.342s
                          └─systemd-resolved.service @5.705s +634ms
                            └─systemd-tmpfiles-setup.service @4.954s +691ms
                              └─import-state.service @4.483s +342ms
                                └─local-fs.target @4.458s
                                  └─boot-efi.mount @4.082s +370ms
                                    └─boot.mount @4.015s +29ms
                                      └─systemd-fsck@dev-disk-by\x2duuid-5e8568a2\x2dee16\x2d4afb\x2db6f9\x2d36b895eba>
                                        └─dev-disk-by\x2duuid-5e8568a2\x2dee16\x2d4afb\x2db6f9\x2d36b895eba23b.device >
lines 1-25/25 (END)

Good catch. The OP didn’t mention they use (proprietary) VMware rather than a Fedora supplied solution.

VMware is notoriously slow in supporting new kernels etc, which is why I gave up on them long ago. What you typically need is a new version of the guest tools - without them, hardware interfcaces are emulated and slow.
Where you typically get them is somewhere on the VMware forums from some VMware developper who cares more about Linux than his company does. That was my experience back then at least.

Well seems like properiaty and commercial softwares are huge red flags and then just see good luck for that, but I did update all open-vm-tools today that was even backed from Fedora even got updated open-vm-tools to other VM tho Fedora is highest/latest on that section, but we’ll if using commercial and properiaty drivers is such a huge red flag in here time to switch other distros

It’s not a red flag per se. But first, if you ask for help and use a tool from outside Fedora it’s helpful if you mention that. And second, due to the closed nature of VMware, there’s really not much we can do to help you. The best advice I can give you is to either use something supported, or - if you want to keep using VMware - to not jump onto a new Fedora release first day it comes out. This just doesn’t go well with how VMware keeps up with kernel development etc (if it still is like it used to be).

1 Like

it is okey i switched all VM’s to LTS Deb release that works