Stuck mid-upgrade again !@... this time to Fedora 44

My attempt to upgrade from F43 to F44 appears to have gotten stuck mid-upgrade. Similarly to how my F42 β†’ F43 upgrade got stuck: Stuck mid-upgrade to Fedora 43

The upgrade seems to have paused at about 45%. The system is responsive, in that I can switch to vttl2 and log in as root. However the upgrade does not seem to be progressing.

What should I do??

I am using the dnf system upgrade according to the documentation as I have done for about a decade: Making sure you're not a bot!

sudo dnf system-upgrade download --releasever=44 --allowerasing followed by sudo dnf offline reboot

Here are some screenshots of the suggested troubleshooting steps/commands from last time: Stuck mid-upgrade to Fedora 43 - #42 by francismontagnac





CPU is not at 100% i.e. no infinite loop. RAM is not low or hitting swapping. Hard drive activity is idle, with very occasional blips of the HDD activity light. I also checked that none of the storage volumes are 100% used.

This is issue is very unsettling. Any suggestions?

Progress!!!


From this, it can be seen that msttcore-fonts-installer is stuck because it is waiting for wget to fetch something but this is not possible because there is no network connection at this time with the offline updater.

I tried killing the wget process (kill 97569) but it said process not found. Running the pstree command again and again shows the wget process changes every time or is not present, so it seems to be timing out/failing and then respawning.

I did kill 12557 to kill the parent process, and the upgrade got unblocked!

There is also another report of this same issue: (Solved) Offline upgrade from 43 to 44 stuck (it was in infinite loop because of msttcore-fonts-installer)

EDIT: The upgrade concluded successfully! Before the next upgrade, I will consider removing msttcore-fonts-installer which comes from the onlyoffice repo.

Not sure if related to this issue, but after rebooting after the upgrade, the default kernel in the grub menu was for F43, and indeed uname -r confirmed the booted kernel was the F43 kernel.

The F44 entry in grub would not boot and gave an error of files not found. I tried using dracut to regenerate the F44 kernel and it failed:

sudo dracut -v /boot/initramfs-7.0.12-201.fc44.x86_64.img 7.0.12-201.fc44.x86_64

dracut[I]: Executing: /usr/bin/dracut -v /boot/initramfs-7.0.12-201.fc44.x86_64.img 7.0.12-201.fc44.x86_64
dracut[I]: *** Including module: nss-softokn ***
dracut[I]: *** Including module: bash ***
dracut[I]: *** Including module: systemd ***
dracut[I]: *** Including module: fips ***
dracut[I]: *** Including module: fips-crypto-policies ***
dracut[I]: *** Including module: systemd-ask-password ***
dracut[I]: *** Including module: systemd-battery-check ***
dracut[I]: *** Including module: systemd-cryptsetup ***
dracut[I]: *** Including module: systemd-initrd ***
dracut[I]: *** Including module: systemd-journald ***
dracut[I]: *** Including module: systemd-modules-load ***
dracut[I]: *** Including module: systemd-pcrphase ***
dracut[I]: *** Including module: systemd-sysctl ***
dracut[I]: *** Including module: systemd-tmpfiles ***
dracut[I]: *** Including module: systemd-udevd ***
dracut[I]: *** Including module: rngd ***
dracut[I]: *** Including module: i18n ***
dracut[I]: *** Including module: drm ***
dracut[I]: *** Including module: plymouth ***
dracut[I]: *** Including module: systemd-sysusers ***
dracut[I]: *** Including module: crypt ***
dracut[I]: *** Including module: dm ***
dracut[I]: *** Including module: fs-lib ***
dracut[I]: *** Including module: kernel-modules ***
dracut[I]: *** Including module: kernel-modules-extra ***
dracut[I]: *** Including module: lvm ***
dracut[I]: *** Including module: prefixdevname ***
dracut[I]: *** Including module: tpm2-tss ***
dracut[I]: *** Including module: rootfs-block ***
dracut[I]: *** Including module: terminfo ***
dracut[I]: *** Including module: udev-rules ***
dracut[I]: *** Including module: dracut-systemd ***
dracut[I]: *** Including module: initqueue ***
dracut[I]: *** Including module: usrmount ***
dracut[I]: *** Including module: base ***
dracut[I]: *** Including module: memstrack ***
dracut[I]: *** Including module: shell-interpreter ***
dracut[I]: *** Including module: shutdown ***
dracut[I]: *** Including module: openssl ***
dracut[I]: *** Including modules done ***
dracut[I]: *** Installing kernel module dependencies ***
dracut[I]: *** Installing kernel module dependencies done ***
dracut[I]: *** Resolving executable dependencies ***
dracut-install: ERROR: could not locate dependency libc.so.6 requested by '/var/tmp/dracut.dWhssFI/initramfs/usr/lib/libgcrypt.so.20.5.1'
dracut-install: ERROR: could not locate dependency ld-linux.so.2 requested by '/var/tmp/dracut.dWhssFI/initramfs/usr/lib/libgcrypt.so.20.5.1'
dracut-install: ERROR: could not locate dependency libc.so.6 requested by '/lib/libgpg-error.so.0.39.3'
dracut[F]: Resolving executable dependencies failed

rpm -qf /usr/lib/libgcrypt.so* and rpm -qf /usr/lib/libgpg-error.so.0* showed they were not owned by any package, so I moved/removed them:

mkdir /tmp/mystraylibs
sudo mv /usr/lib/libgcrypt.so* /tmp/mystraylibs/
sudo mv /usr/lib/libgpg-error.so* /tmp/mystraylibs/

I also increased space on my ~1GB /boot volume (was ~80% used, now 71% used with 264MB free) by limiting the kernels to the last two instead of the last three, as per this thread: Systems with <=1 GB /boot partition may see "need more space" error during system update I figured it’s better to have two kernels and the system recovery kernel than three kernels and no system recovery.

After all this, dracut succeeded, and sudo grub2-editenv list showed the latest F44 kernel as the saved entry in saved_entry=. Rebooting also selected and booted the F44 kernel. I think this upgrade adventure is safely over now.

When packages from an out-of-tree repo cause issues that may interfere with upgrades to a new version the issue is usually seen when performing one of the first 2 steps in this doc.

If not there, and you have identified the problem as you indicate, then it is usually best to disable the repo that package comes from and remove the problem package before performing the upgrade.

When a version upgrade fails as yours has seemed to do, the fix is usually fairly simple
sudo dnf distro-sync --releasever=44 --allowerasing --skip-broken --setopt protected_packages=

Here for 2 systems on 7.0.12-201.fc44.x86_64, I see:

% sudo dnf5 provides /usr/lib64/libgcrypt.so.20.7.2 
[sudo: authenticate] Password: 
Updating and loading repositories:
 RPM Fusion for Fedora 44 - Nonfree - Update 100% | 339.3 KiB/s | 317.2 KiB |  00m01s
 RPM Fusion for Fedora 44 - Free - Updates   100% | 487.4 KiB/s | 546.9 KiB |  00m01s
 google-chrome                               100% |  11.6 KiB/s |  12.5 KiB |  00m01s
 Fedora 44 - x86_64 - Updates                100% |   5.7 MiB/s |  22.1 MiB |  00m04s
Repositories loaded.
libgcrypt-1.12.2-1.fc44.x86_64 : A general-purpose cryptography library
Repo         : @System
Matched From : 
Filename     : /usr/lib64/libgcrypt.so.20.7.2

libgcrypt-1.12.2-1.fc44.x86_64 : A general-purpose cryptography library
Repo         : updates
Matched From : 
Filename     : /usr/lib64/libgcrypt.so.20.7.2

libgcrypt-1.12.2-1.fc44.x86_64 is installed on my system as well. Note that it provides files in /usr/lib64 and not /usr/lib/

For some reason I seem to have had remnants of older 32-bit versions of this library lingering, which only now caused issues.

After removing them:

$ ls /usr/lib/libgcrypt.so.*
ls: cannot access '/usr/lib/libgcrypt.so.*': No such file or directory

$ ls /usr/lib64/libgcrypt.so.*
/usr/lib64/libgcrypt.so.20  /usr/lib64/libgcrypt.so.20.7.2