Help! System stuck with two versions of packages after late failed upgrade

Ask Fedora
,
1

Hello,

I need to urgently upgrade a Fedora 33 system to Fedora 34. (Yes I’m using one version behind as I use some third party software which can be unstable on the current release.)

I’ve done many upgrades with dnf-system-upgrade before but this one failed when it was almost finished. I found later that this was because /var had filled.

I’ve now expanded /var another 2GiB, but my system is stuck. It thinks it is running Fedora 34 but still has thousands of duplicate packages from Fedora 33. dnf commands fail stating that they’d remove systemd.

I’ve tried running reposync (etc) and have gone through the troubleshooting steps but I can’t seem to fix the problem. Everything I try is stuck.

Attempting to upgrade to Fedora 34 again with dnf system-upgrade fails stating that protected packages (like dnf itself) would be removed!
I hit the same issue with distro-sync commands.

Could you please help me un-pick this problem?

Thanks very much!

2 Likes
2 
$ sudo dnf repoquery --duplicates | wc -l
Last metadata expiration check: 0:54:31 ago on Sat 06 Nov 2021 08:36:45.
3754
$ rpm -qa | wc -l
3991

sigh.

1 Like
3

At this stage I’m totally stuck.

I’m tempted to try the instructions here which suggest booting from a live CD and then commenting out all files in /etc/dnf/protected.d but would appreciate someone can validate this as it seems a bit drastic!

1 Like
4

If you end up with my problem this should work:

  • backed up the repo files /etc/yum.repos.d/
  • hard-coded the old release version in to the repo files in /etc/yum.repos.d/
  • ran sudo dnf distro-sync again and identified each package it complained about
  • ran dnf reinstall for all packages which were duplicated, effectively rolling them all back to the old release.
  • I had to forcibly remove one package which was duplicated, shim-x64 the UEFI bootloader by commenting it out in /etc/dnf/protected.d/shim.conf then reinstalling the older version from the old release (this PC boots from BIOS anyway so worst-case it should have still booted)
  • stopped the primary application from starting by disabling the systemd unit
  • I then ran distro-sync again, which now pulled my version down to the old release (33 in this case)
  • rebooted and it came back up
  • restored the original files from /etc/yum.repos.d/
  • ran through the upgrade process again (with more available space in /var)
  • The transaction test failed with a bunch of sssd related packages and some xorg packages. I am not using sssd on this system, it’s only got local auth, so I removed the sssd and sssd-common package.
  • run the upgrade process again by following the documentation

I actually forgot the step to restore the original repo files(!), so I accidentally “upgraded” back to the older release. This caused the system to be unbootable, breaking something under /boot as no kernel nor the recovery option booted.
I booted of a live image, set up the chroot as per the instructions and had a pain of a time trying to recover missing libraries and packages, most in /usr/lib64/ but I ended up copying them from the live image’s copy just to get by. Fortunately after some mucking around I was able to fully update all the packages to the old release (33) then go through the upgrade process in the chroot. The system rebooted and it’s now in the new release.
No doubt I have a lot of mess to clean.

1 Like
5

Lessons Learnt

  • If you need to recover, give yourself hours of time to do so - it’s non-trivial.
  • Ensure /var has at least 3 GiB free, even if the installer doesn’t fail due to low space
  • If possible, avoid complex “pet” systems which are so difficult to rebuild that you’re forced to spend hours recovering them
1 Like