Dnf fails to rollback with "Cannot find rpm nevra" error if rpm does not follow naming standard

barryascott · August 22, 2023, 5:57pm

I hit a problem that is baffling me.

After an dnf update or dnf install I see an error doing a dnf history rollback.

What does the Cannot find rpm nevra error mean?

In the first case that I saw this in I could downgrade and reinstall the named package.
All the nevra can be found in enabled repos. But rollback gives up after the error.

This is the output from the rollback

Following this install

# dnf history rollback 34
Last metadata expiration check: 1:55:02 ago on Tue 22 Aug 2023 15:45:07 UTC.
Error: The following problems occurred while running a transaction:
  Cannot find rpm nevra "platform-python-3.6.8-47.0.1.el8_6.x86_64".
  Cannot find rpm nevra "python3-libs-3.6.8-47.0.1.el8_6.x86_64".

[root@prx29a ~]# dnf install python3-devel
Last metadata expiration check: 1:54:39 ago on Tue 22 Aug 2023 15:45:07 UTC.
Dependencies resolved.
==============================================================================================================
 Package                  Arch     Version                                   Repository                  Size
==============================================================================================================
Installing:
 python3-devel            x86_64   3.6.8-45.0.1.el8                          odsoracle_distro_builder    64 k
Installing dependencies:
 platform-python-devel    x86_64   3.6.8-45.0.1.el8                          odsoracle_AppStream        250 k
 python3-rpm-generators   noarch   5-7.el8                                   odsoracle_AppStream         25 k
 python36-devel           x86_64   3.6.8-38.module+el8.5.0+20329+5c5719bc    odsoracle_AppStream         16 k
Downgrading:
 platform-python          x86_64   3.6.8-45.0.1.el8                          odsoracle_BaseOS            85 k
 python3-libs             x86_64   3.6.8-45.0.1.el8                          odsoracle_BaseOS           7.8 M

Transaction Summary
==============================================================================================================
Install    4 Packages
Downgrade  2 Packages

Total download size: 8.3 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): platform-python-3.6.8-45.0.1.el8.x86_64.rpm                            458 kB/s |  85 kB     00:00
(2/6): python3-rpm-generators-5-7.el8.noarch.rpm                              760 kB/s |  25 kB     00:00
(3/6): platform-python-devel-3.6.8-45.0.1.el8.x86_64.rpm                      1.0 MB/s | 250 kB     00:00
(4/6): python36-devel-3.6.8-38.module+el8.5.0+20329+5c5719bc.x86_64.rpm       644 kB/s |  16 kB     00:00
(5/6): python3-devel-3.6.8-45.0.1.el8.x86_64.rpm                              1.3 MB/s |  64 kB     00:00
(6/6): python3-libs-3.6.8-45.0.1.el8.x86_64.rpm                                10 MB/s | 7.8 MB     00:00
--------------------------------------------------------------------------------------------------------------
Total                                                                          10 MB/s | 8.3 MB     00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                      1/1
  Downgrading      : python3-libs-3.6.8-45.0.1.el8.x86_64                                                 1/8
  Downgrading      : platform-python-3.6.8-45.0.1.el8.x86_64                                              2/8
  Running scriptlet: platform-python-3.6.8-45.0.1.el8.x86_64                                              2/8
  Installing       : python3-rpm-generators-5-7.el8.noarch                                                3/8
  Installing       : platform-python-devel-3.6.8-45.0.1.el8.x86_64                                        4/8
  Installing       : python36-devel-3.6.8-38.module+el8.5.0+20329+5c5719bc.x86_64                         5/8
  Running scriptlet: python36-devel-3.6.8-38.module+el8.5.0+20329+5c5719bc.x86_64                         5/8
  Installing       : python3-devel-3.6.8-45.0.1.el8.x86_64                                                6/8
  Cleanup          : platform-python-3.6.8-47.0.1.el8_6.x86_64                                            7/8
  Running scriptlet: platform-python-3.6.8-47.0.1.el8_6.x86_64                                            7/8
  Cleanup          : python3-libs-3.6.8-47.0.1.el8_6.x86_64                                               8/8
  Running scriptlet: python3-libs-3.6.8-47.0.1.el8_6.x86_64                                               8/8
  Verifying        : platform-python-3.6.8-45.0.1.el8.x86_64                                              1/8
  Verifying        : platform-python-3.6.8-47.0.1.el8_6.x86_64                                            2/8
  Verifying        : python3-libs-3.6.8-45.0.1.el8.x86_64                                                 3/8
  Verifying        : python3-libs-3.6.8-47.0.1.el8_6.x86_64                                               4/8
  Verifying        : platform-python-devel-3.6.8-45.0.1.el8.x86_64                                        5/8
  Verifying        : python3-rpm-generators-5-7.el8.noarch                                                6/8
  Verifying        : python36-devel-3.6.8-38.module+el8.5.0+20329+5c5719bc.x86_64                         7/8
  Verifying        : python3-devel-3.6.8-45.0.1.el8.x86_64                                                8/8

Downgraded:
  platform-python-3.6.8-45.0.1.el8.x86_64                 python3-libs-3.6.8-45.0.1.el8.x86_64
Installed:
  platform-python-devel-3.6.8-45.0.1.el8.x86_64  python3-devel-3.6.8-45.0.1.el8.x86_64
  python3-rpm-generators-5-7.el8.noarch          python36-devel-3.6.8-38.module+el8.5.0+20329+5c5719bc.x86_64

Complete!

ilikelinux · August 22, 2023, 6:11pm

Does this help:

https://www.reddit.com/r/Fedora/comments/mwrh6n/constant_system_failure_after_the_last_update/

barryascott · August 22, 2023, 6:18pm

Thanks for the pointer!

That gives one possible workaround, but I suspect I can safely use that work around.
I’ll test tomorrow.
I’m worried that the RPMs that I’m trying to roll back will not be replaced with the old ones.

I’m planning to read the dnf code to findout why it prints that error tomorrow if no one knows the answer.

ilikelinux · August 22, 2023, 6:28pm

This might give some more insights what nevra means.

computersavvy · August 23, 2023, 2:43am

It looks like every one of those listed files is an el8 file and all seem to be from an odsoracle_* repo. Apparently the system does not like rollbacks with that repo and the el8 files.

Is there a reason you are not using the python3 packages provided directly from fedora that would match everything installed and keep it updated. I see that you apparently are using python 3.6 and not the current python 3.11.

vekruse · August 23, 2023, 4:48am

If you have problem with Oracle Linux, you should ask the Oracle support.

barryascott · August 23, 2023, 6:57am

Thanks for the script, I think that is what i will have to implement.

Fyi i asked here about the issue as I guessed that there is more expertise with DNF here.

jakfrost · August 23, 2023, 11:26am

Hello @barryascott ,
Could you please mark @vgaetera script response as the solution so this is “solved”?

jakfrost · August 23, 2023, 1:12pm

So your script is only a partial solution to the problem then, is that correct to state? It occurs to me that GuixSD has this solved.

barryascott · August 23, 2023, 4:40pm

This will only work because the specific transaction i need to rollback did not install packages only updated them.

Somewhere in the rpm/dns databases one of the packages is claimed to be installed (dnf) and not installed (rpm —qf) at the same time. Also the install repo is @System, which seems to be at the bottom of the problem.

Thanks for all the help folks.

barryascott · August 23, 2023, 4:46pm

In the dnf code there is a comment that says to rollback the rpm must be in the same repo that it was installed from. This is not strictly true, but i think because one of the nevra is recorded as coming from @System its impossible to install.

FYI nevra is name, epoch, version, release, arch. E.g all the parts of a fully qualified package.

vekruse · August 23, 2023, 6:31pm

If you try to rollback your latest dnf update transaction, you will get the same results, namely the exact packages version of the upgraded packages prior to the upgrade is no longer available from the repository, and therefore the rollback cannot be done. NEVRA just means the exact package name and version.

barryascott · August 23, 2023, 7:46pm

In my specific case all versions, nevra, of the rpms are available.

That is because at work we snapshot ol8 plus our code into artifactory for each release of software. Then we have two snapshots mapped to repos.
The current snapshot and the next snapshot.
The upgrade goes from current to next.
The rollback goes from next to current.
Once we know the release is stable in production we set current to be next.

Somewhere in the way we kickstart and upgrade systems there is a gotcha that can break things sometimes, its very rare and very annoying.

In the specific case I am debugging i can reinstall the new rpm and i can downgrade the rpm successfully.
Its only history rollback that fails.

vgaetera · August 23, 2023, 10:41pm

(post deleted by author)

barryascott · August 24, 2023, 6:35am

From debugging the dnf code it is restricting where it looks for the RPMs when doing a rollback. I have given up tracing the code as it has gone into the C++ parts of dnf to do the query for the rpm, i do know it reports not found from the C++ query.

What I do not kbow is where the database is that it is searching.
I am sure it is not the /var/lib/dnf/history.sqlite db.

vgaetera · August 24, 2023, 6:45am

(post deleted by author)

barryascott · August 24, 2023, 9:53am

I am using a fake name for the RPM, its one private to my work place.

# dnf -q list day-job-package | cat
Installed Packages
day-job-package.noarch    1.3-111345.0-el8            @System

# dnf -q list day-job-package-1.3-111345.0-el8.noarch | cat
Installed Packages
day-job-package.noarc    1.3-111345.0-el8              @System

# rpm -q day-job-package-1.3-111345.0-el8.noarch
package day-job-package-1.3-111345.0-el8.noarch is not installed

# rpm -q day-job-package
day-job-package-1.3-111345.0-el8.noarch

As you can see there is a worring package is not installed from rpm.

vgaetera · August 24, 2023, 11:05am

This may be the result of a naming policy violation:

day-job-package - name
1.3 - version
111345.0 - ?
el8 - release
noarch - arch

The version and release tags do not permit dashes:

> sudo dnf -q repoquery --qf "%{VERSION} %{RELEASE}" | grep -c -e "-"
0

vekruse · August 24, 2023, 11:38am

Bingo! Good find.

It could be interpreted in this way as NVRA (without the epoch)

day-job-package-1.3 - name
111345.0 - version
el8 - release
noarch - arch

That is: exactly two dash characters after the name part.

barryascott · August 24, 2023, 1:15pm

Oh! You are right that is not a valid name.

I’ll take that up with the team responsible at $DAYJOB.

Thank to everyone for all your help with this problem.

Topic		Replies	Views
Cannot remove Wine, says there's a protected package 'systemd-udev' Ask Fedora f37 , pipewire , toolbx , wine , wireplumber , silverblue	1	847	January 16, 2023
Dnf update FAILED several times Ask Fedora	21	4566	May 15, 2023
Dnf rollback doesn't work because Fedora updates repo is only one package deep Ask Fedora mesa , dnf , workstation	5	89	July 28, 2024
DNF upgrade error Ask Fedora f31	11	2719	December 6, 2019
Manually reinstall/rebuild/repair DNF/YUM Ask Fedora dnf	3	9941	July 29, 2020

Dnf fails to rollback with "Cannot find rpm nevra" error if rpm does not follow naming standard

Related Topics