Trouble updating from fc36 to fc38

Ask Fedora
, , ,
21

I have imported the new GPG key for virtualbox and removed the conflicting packages which I actually didn’t have when I upgraded as you can see in the log file I created just before doing the upgrade luckily! (sudo dnf list installed |& tee jazz6-dnf-installed.tx)
% grep dropbox Dropbox/jazz6-dnf-installed.txt
dropbox.noarch 1:2020.03.04-3.fc35 @rpmfusion-nonfree

This is very neat, I didn’t know about these tools!
Here are the links for
dnf list installed version 36 prior to update https://paste.centos.org/view/a4f791cb
dnf system-upgrade download --disablerepo=rpmfusion* --allowerasing --releasever=38 https://paste.centos.org/view/3a6ded25
dnf distro-sync https://paste.centos.org/view/e951d16a
dnf upgrade --refresh https://paste.centos.org/view/0643c8d5
As you can see these updates do not provide as many packages as the system-upgrade since the current rpm database is not up-to-date and since 21 Nov 2022 I have added packages (texlive and others)…
Do you think there would be a way to use automatically the log of the system-upgrade to reinstall the ones missing in the distro-sync or upgrade?
Many thanks for your help! I learn a lot!

22

You can start with system-upgrade and perform distro-sync after upgrading.
All DNF transactions will be logged and can be inspected later.
Also update the GPG key URL to fix the related error:

sudo dnf config-manager --save --setopt virtualbox.gpgkey=\
https://www.virtualbox.org/download/oracle_vbox_2016.asc
23

Thank you I have updated the GPG key URL to fix the error.

I am not sure to understand what you mean by start with system-upgrade. The log for the system-upgrade I provided was for my attempt two days ago which failed and was the reason I started this discussion.

I have now a system with what looks like a working version 38 rpms but without record in the rpm database apart from the three fedora-release rpms updated manually under your advice. The rpm database still contain an old state of the rpm database from 21 Nov 2022 which was not the state I started the upgrade two days ago and most of the rpms in the database are not installed any longer.
If I try to run the system-upgrade now it fails

% sudo dnf system-upgrade download --releasever=38
[sudo] password for patrick:
Before you continue ensure that your system is fully upgraded by running “dnf --refresh upgrade”. Do you want to continue [y/N]: y
Error: Need a --releasever greater than the current system version.

For completeness here is the log of dnf history
% dnf history
ID | Command line | Date and time | Action(s) | Altered

1585 | system-upgrade upgrade | 2023-05-04 16:22 | D, E, I, O, R, | 3522 EE
1584 | install ffmpeg-5.1.3-2.fc36. | 2023-05-04 16:08 | Install | 5
1583 | remove ffmpeg-5.1.3-2.fc36.x | 2023-05-04 15:32 | Removed | 5 <
1582 | remove libheif | 2023-05-04 15:19 | Removed | 1 >
1581 | install libheif.x86_64 | 2023-05-04 15:19 | Install | 1

As you can see the last transaction is the system-upgrade I did two days ago that resulted in this corrupted rpm database from 21 Nov 2022. I can get the log of the transaction with dnf history info 1585 and here is a link to the output https://paste.centos.org/view/e9004b51

I am not sure how to further proceed, any advice would be welcome.
Also it is still not clear what actually happened and failed during this upgrade 36->38.

24

Some further information that might be useful, here is the output of the command
% rpm -qa | grep ^kernel | sort
kernel-6.0.8-200.fc36.x86_64
kernel-core-6.0.8-200.fc36.x86_64
kernel-devel-5.19.16-200.fc36.x86_64
kernel-devel-6.0.8-200.fc36.x86_64
kernel-devel-matched-6.0.8-200.fc36.x86_64
kernel-headers-6.0.5-200.fc36.x86_64
kernel-modules-6.0.8-200.fc36.x86_64
kernel-modules-extra-6.0.8-200.fc36.x86_64
kernel-srpm-macros-1.0-14.fc36.noarch

while here is a list of the kernel files I had just before the system-upgrade two days ago
% grep ^kernel jazz6-dnf-installed.txt
kernel.x86_64 6.2.14-100.fc36 @updates
kernel-core.x86_64 6.2.14-100.fc36 @updates
kernel-devel.x86_64 6.2.14-100.fc36 @updates
kernel-devel-matched.x86_64 6.2.14-100.fc36 @updates
kernel-headers.x86_64 6.2.6-100.fc36 @updates
kernel-modules.x86_64 6.2.14-100.fc36 @updates
kernel-modules-core.x86_64 6.2.14-100.fc36 @updates
kernel-modules-extra.x86_64 6.2.14-100.fc36 @updates
kernel-srpm-macros.noarch 1.0-14.fc36 @fedora

Clearly the database is not correct and dates back 21 Nov 2022, note that all the kernels files are not here now and have been removed since this date.

25

Since system-upgrade fails, proceed with distro-sync.

Also check the output:

sestatus; ls -l -a -Z /var/lib/rpm{/,}; grep -e "\s/\s" /etc/mtab
26

Here is the output of the three commands
% sestatus ; ls -l -a -Z /var/lib/rpm{/,} ; grep -e “\s/\s” /etc/mtab
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: permissive
Mode from config file: permissive
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
/var/lib/rpm:
total 255968
drwxr-xr-x. 1 root root system_u:object_r:rpm_var_lib_t:s0 126 Oct 28 2022 ./
drwxr-xr-x. 1 root root system_u:object_r:var_lib_t:s0 1060 May 4 16:28 …/
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 0 May 4 16:31 .migratedb
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 262078464 Nov 21 13:08 rpmdb.sqlite
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 32768 May 6 11:38 rpmdb.sqlite-shm
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 0 Nov 21 13:08 rpmdb.sqlite-wal
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 0 Oct 28 2022 .rpm.lock

/var/lib/rpm/:
total 255968
drwxr-xr-x. 1 root root system_u:object_r:rpm_var_lib_t:s0 126 Oct 28 2022 ./
drwxr-xr-x. 1 root root system_u:object_r:var_lib_t:s0 1060 May 4 16:28 …/
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 0 May 4 16:31 .migratedb
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 262078464 Nov 21 13:08 rpmdb.sqlite
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 32768 May 6 11:38 rpmdb.sqlite-shm
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 0 Nov 21 13:08 rpmdb.sqlite-wal
-rw-r–r–. 1 root root system_u:object_r:rpm_var_lib_t:s0 0 Oct 28 2022 .rpm.lock
/dev/nvme1n1p3 / btrfs rw,seclabel,relatime,compress=zstd:1,ssd,discard=async,space_cache,subvolid=257,subvol=/root 0 0

I can wait for your feedback regarding the output of the commands before I initiate a dnf distro-sync

27

I just figured out there is a log command for dnf system-upgrade log but I am not sure how to use the information to access the log
% dnf system-upgrade log
The following boots appear to contain upgrade logs:
1 / 4c7186e329774db6a97cdde3797ba910: 2023-05-04 16:21:11 36→38

There might be some useful information on what went wrong?

28

Actually, /var/lib/rpm should be a symlink to /usr/lib/sysimage/rpm:

sudo mv /var/lib/rpm{,.bak}
sudo ln -f -n -r -s /usr/lib/sysimage/rpm /var/lib

This is how it looks for me on Fedora 38:

> ls -l /usr/lib/sysimage/rpm /var/lib/rpm
lrwxrwxrwx. 1 root root  26 May  6 20:38 /var/lib/rpm -> ../../usr/lib/sysimage/rpm

/usr/lib/sysimage/rpm:
total 111104
-rw-r--r--. 1 root root 113737728 May  6 18:36 rpmdb.sqlite
-rw-r--r--. 1 root root     32768 May  6 20:40 rpmdb.sqlite-shm
-rw-r--r--. 1 root root         0 May  6 18:36 rpmdb.sqlite-wal
29

Wow well spotted! On my system I have
% ls -lt /usr/lib/sysimage/rpm /var/lib/rpm
/usr/lib/sysimage/rpm:
total 124760
-rw-r–r–. 1 root root 32768 May 6 18:29 rpmdb.sqlite-shm
-rw-r–r–. 1 root root 0 May 6 15:02 rpmdb.sqlite-wal
-rw-r–r–. 1 root root 127721472 May 6 15:02 rpmdb.sqlite
-rw-r–r–. 1 root root 0 May 6 14:12 .rpm.lock

/var/lib/rpm:
total 255968
-rw-r–r–. 1 root root 32768 May 6 11:38 rpmdb.sqlite-shm
-rw-r–r–. 1 root root 0 May 4 16:31 .migratedb
-rw-r–r–. 1 root root 0 Nov 21 13:08 rpmdb.sqlite-wal
-rw-r–r–. 1 root root 262078464 Nov 21 13:08 rpmdb.sqlite
-rw-r–r–. 1 root root 0 Oct 28 2022 .rpm.lock

which actually shows that rpm --rebuilddb recreated the sqlite database in /usr/lib/sysimage/rpm, I was surprised that the date of the database in /var/lib/rpm was still 21 Nov even after the command but did not react. The newer database in /usr/lib/sysimage/rpm is half the size of the other one but I am not sure how to vuew the differences. Do you know how to check this?

I have now mv /var/lib/rpm to /var/lib/rpm.bak and created the soft link to /usr/lib/sysimage/rpm
ls -lt /usr/lib/sysimage/rpm /var/lib/rpm
lrwxrwxrwx. 1 root root 26 May 6 18:58 /var/lib/rpm → …/…/usr/lib/sysimage/rpm/

/usr/lib/sysimage/rpm:
total 124760
-rw-r–r–. 1 root root 32768 May 6 18:59 rpmdb.sqlite-shm
-rw-r–r–. 1 root root 0 May 6 15:02 rpmdb.sqlite-wal
-rw-r–r–. 1 root root 127721472 May 6 15:02 rpmdb.sqlite
-rw-r–r–. 1 root root 0 May 6 14:12 .rpm.lock

and rerun the command to find the kernel in the rpm database which seem to to be the same ones
% rpm -qa | grep ^kernel | sort
kernel-6.0.8-200.fc36.x86_64
kernel-core-6.0.8-200.fc36.x86_64
kernel-devel-5.19.16-200.fc36.x86_64
kernel-devel-6.0.8-200.fc36.x86_64
kernel-devel-matched-6.0.8-200.fc36.x86_64
kernel-headers-6.0.5-200.fc36.x86_64
kernel-modules-6.0.8-200.fc36.x86_64
kernel-modules-extra-6.0.8-200.fc36.x86_64
kernel-srpm-macros-1.0-14.fc36.noarch

Many thanks again for your expertise!
I am waiting for your feedback before doing anything :slight_smile:

30

It looks fine, you can proceed with distro-sync.
To minimize the risks, it is best to run in a text TTY.
Prepare a Fedora live media if something goes wrong.

31

I have prepared a Fedora live media and I am going to proceed in a text terminal.
I will come back to you when ran
Many thanks again!

1 Like
32

Sorry I was busy doing other things but I finally got time to run the distro-sync which ran successfully. Many thanks again!
I will have to do some cleaning and check missing packages but it seems to work fine. I have run a dnf remove --oldinstallonly.
Do you have any suggestions what I ought to check?
Many thanks again for your expertise!

33

Great, you can follow the wiki for post-upgrade instructions.

34

Thank you again for your expertise, that is great!

I followed the post-upgrade instructions, some of which I was aware but I have a few questions

Regarding the boot

I am not completely sure whether I shall remove the file /boot/efi/EFI/fedora/grub.cfg.rpmsave
The files looks very different size and in content
-rwx------. 1 root root 9125 Nov 18 17:30 /boot/efi/EFI/fedora/grub.cfg.rpmsave
-rwx------. 1 root root 143 Nov 21 13:03 /boot/efi/EFI/fedora/grub.cfg

Shall I try to update the grub bootloader on BIOS system as adviced? I am not sure at all.
The command given returns
% sudo mount | grep "/boot "
/dev/nvme1n1p2 on /boot type ext4 (rw,relatime,seclabel)

Can I run safely
sudo grub2-install /dev/sda
by replacing the device /dev/nvme1n1p

Currently I have in /etc/fstab
UUID=2f804dab-0863-46f9-ae02-766ea3ff5a7c / btrfs subvol
=root,compress=zstd:1 0 0
UUID=6fe4680f-f52e-4706-8411-7cf3681888aa /boot ext4 defaul
ts 1 2
UUID=AD63-A589 /boot/efi vfat umask=0077,shortname=win
nt 0 2
UUID=2f804dab-0863-46f9-ae02-766ea3ff5a7c /home btrfs subvol
=home,compress=zstd:1 0 0
UUID=2f804dab-0863-46f9-ae02-766ea3ff5a7c /var btrfs subvol
=var,compress=zstd:1 0 0
#UUID=2f804dab-0863-46f9-ae02-766ea3ff5a7c /opt btrfs subvo
l=opt,compress=zstd:1 0 0

Last question, can I regenerate a rescue kernel by running the two commands?
% rm -f /boot/vmlinuz-0-rescue-* /boot/initramfs-0-rescue-*.img
% /usr/lib/kernel/install.d/51-dracut-rescue.install add $(uname -r) “” /lib/modul
es/$(uname -r)/vmlinuz

Many thanks for your help again!
Patrick

35

Do not run that command if your system is installed with EFI.

Keep the current one that is newer and smaller.

I guess it should work.

36

Thank you again!
I have two last questions for this thread.

The first one is regarding cleaning. I have kernel 36 files in boots
ls -l /boot/36
-rw-r–r–. 1 root root 255899 May 1 02:12 /boot/config-6.2.14-100.fc36.x86_64
-rw-------. 1 root root 40415907 May 4 16:52 /boot/initramfs-6.2.14-100.fc36.x86_64.img
lrwxrwxrwx. 1 root root 46 May 3 10:01 /boot/symvers-6.2.14-100.fc36.x86_64.gz → /lib/modules/6.2.14-100.fc36.x86_64/symvers.gz
-rw-------. 1 root root 8435991 May 1 02:12 /boot/System.map-6.2.14-100.fc36.x86_64
-rwxr-xr-x. 1 root root 14228680 May 1 02:12 /boot/vmlinuz-6.2.14-100.fc36.x86_64*

which are not registered in the rpm and dnf database so I assume I can just delete them.
My question is do I need to regenerate the grub entries at all with the command
sudo dracut --regenerate-all --force
More generally in what situation would I need to use this command?

My last question is do you think the problem I had in the first place was due to the conflict between the two rpm databases?

Many thanks again