F41 Change Proposal: Switch to DNF 5 (System-Wide)

Switch to dnf5

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Wiki
Announced

:link: Summary

Change the default package manager from dnf to dnf5.

:link: Owner

:link: Detailed Description

This proposal will implement several topics, which are outlined below.

:link: Provider of the dnf command

This change proposes to switch the current provider of the /usr/bin/dnf symlink from dnf to dnf5. Currently, the symlink target is /usr/bin/dnf-3, provided by the dnf sub-package, python3-dnf. Upon implementation of this change, the symlink will point to /usr/bin/dnf5, provided by the dnf5 package.

:link: Prepare the upgrade path

The dnf5 package, serving as the new provider of the /usr/bin/dnf symlink, will obsolete the dnf package starting with Fedora 41. Upon the release of this dnf5 package, upgrading the system or installing dnf5 will replace the existing dnf package on the system. Additionally, the dnf5 package will provide a /usr/bin/yum symlink for backwards compatibility and the dnf-automatic command will be obsoleted.

:link: Feature parity with dnf

We aim to cover the majority of use cases available in the existing dnf package. However, there are some features that may not be implemented in time. Nevertheless, we plan to deliver them at a later stage.

:link: Plugins

The progress of implementing plugins to match the current set from the dnf-plugins-core package is tracked upstream. Among the missing plugins, we still plan to implement:

  • debuginfo-install plugin
  • reposync plugin

:link: Modularity

As support for modularity was retired in Fedora 39, dnf5 currently only implements a basic feature set for listing and enabling/disabling modules.

:link: Background service support

A new daemonized service, dnf5daemon, utilizing the D-Bus interface, is prepared for clients as a sub-package. This will serve as an alternative or replacement for the PackageKit layer. Integration of dnf5daemon support into the default Fedora user interface, GNOME Software, is currently in progress

:link: Documentation of API changes

The public interface has undergone significant changes to enhance the user experience and remove unused and obsolete code components. To facilitate user migration to the new CLI and API interfaces, a guide was prepared covering all differences compared to the interface provided by the existing dnf package, along with examples of typical use cases.

:link: Deployment tasks

During the deployment of the dnf5 package manager as the new default, several adjustments need to be made both to the infrastructure and the dnf5 package itself. Some of these adjustments are detailed below. To ensure synchronization and address all necessary changes, we’ve established an upstream tracking issue.

:link: Feedback

As this is the second iteration of such a proposal, we’ve gathered a lot of feedback from various sources during the first attempt to accept this change.

:link: FESCo inputs

A ticket discussing the reasons why the contingency mechanism was invoked for the first attempt of the proposal was opened by FESCo. It includes a list of items that are either incomplete or in progress.
Below is a list of issues from the ticket that are still unresolved or require clarification on their current status. Other items not mentioned below are considered completed.

:link: Switch in ELN

This should not block the proposal, as the current plan is to target RHEL 11. Integration can occur there after the proposal is implemented for Fedora 41.

:link: Aligning configuration with the current state in dnf

All overrides to match the current state of dnf configuration will be provided to the Fedora release project, see below.

:link: System upgrade and offline transactions

The implementation work has been completed and is already present upstream. We anticipate extensive testing during the summer, and we also plan to organize testing days for this purpose.

:link: Dropping the Snapper plugin

In dnf5, we’ve adopted a new approach for implementing functionality that was previously handled by the Snapper plugin in dnf.
We’re introducing the Actions plugin, which offers more capabilities than the Snapper plugin, including support for running external applications before or after transactions and interacting with the dnf5 configuration. Here is the documentation for the Actions plugin, which includes examples of how to emulate the behavior of the Snapper plugin.

:link: Messages from RPM scriptlets

An issue with output from RPM scriptlets is the potential length, coupled with the absence of a standardized policy for distinguishing between important and unimportant messages.
Currently, all messages are logged in the dnf5 log files, with differentiation based on their originating scriptlets, representing an improvement over dnf. Additionally, in case of transaction errors, the scriptlet output is included in the standard output. Furthermore, there is already a resolved ticket in the infrastructure to incorporate logs on the builders.

:link: Testing days

We’ve already held two dnf5 test days for Fedora 38 and 39. We’ve made efforts to document all reported issues in our upstream tracking system, and major issues should now be resolved. Some of these issues were related to user documentation, improving command-line outputs, and enhancing overall user experience. These topics are next on our priority list after completing the functionality for mandatory commands and plugins.

:link: Fedora QA scenarios

We’ve started a discussion thread on the requirements for accepting this proposal from a QA perspective. A list of relevant test cases and criteria has been mentioned, which we’ll review to ensure we’ve covered everything on our end.

:link: Fedora CI readiness

The dnf project is also deployed in the CI pipeline. We’ve initiated communications with this team to ensure that all dnf functionality used there is either already implemented in dnf5 or can be addressed through an alternative dnf5 method. We’ve already received some feedback from the first iteration of the proposal.

:link: Tracking ssue upstream

When implementing the first iteration of this proposal, we created an upstream ticket to track all bugs or lack of needed functionality. All items have been addressed, and only several known deployment issues remain, which need to be managed at the time of the next switch.

:link: Benefit to Fedora

The new dnf5 will significantly improve the user experience and performance. Detailed descriptions of individual areas are provided below.

:link: Reduced footprint

The dnf5 package is a fully-featured package manager that doesn’t require Python dependencies.
It also reduces the number of software management tools in Fedora by replacing both the dnf and microdnf packages.
The installation size of the dnf5 stack in an empty container is approximately 60% smaller than the dnf installation.
Currently, dnf, microdnf, and PackageKit use their own cache, leading to significant metadata redundancy. With dnf5 and dnf5daemon, which share metadata, this redundancy will be eliminated.

:link: Enhanced performance

Loading and downloading repository metadata now occur concurrently.
Package query operations, including processing numerous command-line arguments, have been significantly accelerated.

:link: Lowered maintenance costs

Many functional duplicates in dnf were eliminated during the development of the new dnf5 package manager. This was partly because the integration of the original PackageKit and dnf libraries into the original libdnf library was never completed.
Plugins are now included in the same package as the core functionality.

:link: Unified user experience

Consistent user experience is offered to users across servers, workstations, and containers, as dnf5 is the sole package manager deployed there. Existing dnf, yum, and microdnf commands will be linked to dnf5, while compatibility aliases for essential use cases will be provided to facilitate migration.
Configuration files will be shared among dnf5 components.
API users will encounter unified code style and naming conventions. Various scripting language interfaces are now provided from a single source using SWIG bindings (formerly CPython and SWIG).

:link: Scope

:link: Proposal owners

The remaining work on the proposal can be divided into several sections.

:link: Feature implementation

:link: System upgrade

This command is essential for upgrading the system to the next release. While the implementation is already completed, we plan to conduct extensive testing, including community participation, to minimize the risk of issues occurring in production.

:link: History command

The functionality related to manipulating transaction history has not yet been implemented. However, following the completion of the system upgrade functionality, it is currently our top priority. Due to the significant overlap between the functionality in the history command and the system upgrade functionality, we anticipate its readiness shortly thereafter.

:link: GNOME Software support

The integration of dnf5 support, particularly dnf5daemon, into GNOME Software is currently underway. Developers from both DNF5 and GNOME Software are closely connected and regularly synchronize the progress of their work.

:link: Documentation

While our current priority is achieving full coverage of user and API documentation, there may still be some undocumented parts of the code. Please don’t hesitate to report any such issues upstream, and we’ll endeavor to address them promptly.

:link: Early access for developmental branch users

The intention is to implement the proposed changes in the Rawhide developmental branch prior to the date specified in the Contingency Deadline section, targeting the transition period between March and April. It is anticipated that certain mandatory items for the regular release, as outlined in previous sections, may not be completed by this time.

:link: System upgrade

This functionality is unnecessary as Rawhide operates on a rolling release model.

:link: GNOME Software

Rawhide users will continue to utilize the current PackageKit backend connected to the existing libdnf interface. These libraries can coexist with the new dnf5 package on the same system. Although the setup is not ideal due to differences in package state metadata formats stored at separate locations, resulting in inefficient storage usage, this is generally imperceptible for typical GUI users. Furthermore, the underlying RPM DB remains the sole shared source of information about installed packages.

:link: Other developers

The following components are already prepared for transition to dnf5:

  • Ansible
  • Lorax
  • Mock
    Below is a list of dependencies that have not yet integrated with dnf5.

:link: Anaconda

Migration to the dnf5 API will not be implemented at this time. After discussion with the Anaconda team, it was determined that there isn’t sufficient capacity and time to complete the work on schedule. Therefore, the existing dnf4 Python bindings from the python3-dnf package will continue to be used for now.

:link: GNOME Software

This integration is already addressed in the section above, as it involves collaborative efforts.

:link: Release engineering

:link: Building with dnf5

The Fedora infrastructure has been utilizing dnf5 for building Fedora 40+ chroots since before the Fedora 40 mass rebuilds began. This implementation was based on the system-wide change outlined here.

:link: Apply downstream configuration overrides

Starting with dnf5, distro-specific overrides of the default configuration values are implemented using drop-in directories. Options with different values upstream compared to the current state of dnf in Fedora will be filed in a pull request against the Fedora release project.

:link: Update of kickstarts for image composes

The dnf team will prepare a pull request to include the packages for the new default dnf5 package manager in the Fedora Kickstarts project.

:link: Update of package groups definitions

The dnf team will prepare a pull request to include the packages for the new default dnf5 package manager in the Fedora Comps project.

:link: Update of KIWI image descriptions

The dnf team will prepare a pull request to include the packages for the new default dnf5 package manager in the Fedora KIWI descriptions project.

:link: Upgrade/compatibility impact

:link: Running the upgrade

The dnf5 package will be installed on the system to provide /usr/bin/dnf, replacing the existing dnf package as it becomes obsolete. Additionally, the dnf automatic tool will be replaced by the new dnf5 automatic plugin.
Below is an example of performing this upgrade transaction.

$ sudo dnf upgrade Last metadata expiration check: 0:01:09 ago on Wed 13 Mar 2024 05:48:25 AM EDT. Dependencies resolved. ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: dnf5-plugin-automatic x86_64 5.1.14-1.fc41 rawhide 120 k replacing dnf-automatic.noarch 4.19.0-1.fc40 Upgrading: dnf-data noarch 4.19.0-1.fc41 rawhide 40 k python3-dnf noarch 4.19.0-1.fc41 rawhide 551 k Installing group/module packages: dnf5 x86_64 5.1.14-1.fc41 rawhide 599 k replacing dnf.noarch 4.19.0-1.fc40 replacing yum.noarch 4.19.0-1.fc40 Installing dependencies: fmt x86_64 10.2.1-3.fc40 rawhide 125 k libdnf5 x86_64 5.1.14-1.fc41 rawhide 991 k libdnf5-cli x86_64 5.1.14-1.fc41 rawhide 229 k Installing weak dependencies: bash-completion noarch 1:2.11-15.fc41 rawhide 360 k Transaction Summary ================================================================================ Install 6 Packages Upgrade 2 Packages

:link: Binaries and symlinks

The existing dnf binaries will remain on the system and will be available at /usr/bin/dnf-3, which refers to the Python 3 version to which dnf was migrated in the past. Additionally, they will be accessible at /usr/bin/dnf4, which is a symlink to the dnf-3 script and denotes the major version of the dnf binary. This binary naming convention is also used for /usr/bin/dnf5, which will become the target of the /usr/bin/dnf symlink.
Below is the output of the tree command showing the expected file structure after the upgrade.

:link: Before upgrade

$ tree /usr/bin/ -P dnf* /usr/bin/ ├── dnf → dnf-3 ├── dnf-3 └── dnf4 → dnf-3

:link: After upgrade

$ tree /usr/bin/ -P dnf* /usr/bin/ ├── dnf → dnf5 ├── dnf-3 ├── dnf4 → dnf-3 └── dnf5

:link: Different system state

The transactional history in dnf and dnf5 is not shared, and they now use different formats. Transactions performed in dnf will not be visible in dnf5, and vice versa.
While the history database is not migrated to dnf5, when running a transaction in dnf5 for the first time, an attempt is made to convert and load the existing system state from dnf. This should preserve information about the reasons for installed packages and prevent them from being treated as user-installed, requiring manual removal from the system instead of being seen as dependencies of explicitly removed packages.

:link: Compatibility

While the majority of CLI use cases for managing packages should remain the same, the dnf5 API has undergone significant changes. To ease adoption, dnf5 will provide compatibility aliases for commands and options. Additionally, user output will differ, and we plan to offer machine-readable support for most commands. For more details, please refer to the section above.
Applications encountering difficulties with dnf5 adoption can continue using the existing dnf CLI and API provided by the python3-dnf and libdnf packages. These libraries can be used in parallel on the system, but modifying installed software is not recommended due to differences in system state, as mentioned in the previous section.

:link: How To Test

:link: Copr repository

A testing Copr repository has been set up with dnf5 already deployed as the default package manager. Instructions on how to proceed are provided alongside.

:link: Quay containers

We have also prepared Fedora container images with the dnf5 stack preinstalled on quay.io, making it easy to test in isolation with Podman.

:link: Side-tag for testing

Before pushing the new dnf5 into the Rawhide compose, we plan to announce the prepared side-tag containing this new package publicly. This will allow interested parties to test it against their software.

:link: Testing days

We are planning at least two iterations of testing days before dnf5 is delivered into Fedora 41. One will focus on testing the overall functionality of dnf5, with an emphasis on parts that were not tested during previous testing days. The other iteration will center around testing the system upgrade functionality.

:link: Communication channels

Community feedback, including bug reports, issues, or feature requests, is highly encouraged.
Our primary communication channel is upstream, where you can report issues, participate in discussions, or even propose pull requests. We are happy to review them.
Issues specific to a particular version of the dnf5 package can also be reported through the Bugzilla tracking system, which we also monitor.

:link: User Experience

:link: Faster query processing

The processing of package metadata is now significantly faster. Executing commands such as repoquery to list packages available in repositories is now twice as fast compared to dnf. Similarly, operations like listing dependencies or parsing numerous command-line arguments are notably expedited, potentially saving users seconds to tens of seconds in waiting time for the results.

:link: Consolidated and streamlined API

The API for managing packages, working with repositories, and solving package dependencies is now consolidated into a single component, providing a unified solution. The original dnf API underwent a review process, during which unused workflows and obsolete methods were removed, while improving usability for users.

:link: Enhanced command-line outputs

Transaction tables now offer more detailed information, verbose scriptlet outputs are redirected and organized by package name into log files, individual commands come with their own man pages, bash completion has been enhanced, and numerous other improvements have been made.

:link: Dependencies

:link: Owned by our team

:link: dnf-plugins-core

Installed plugins will persist on the system and remain functional using the dnf4 binary from the python3-dnf package.
With the exception of the system upgrade plugin, all essential plugins are now implemented in dnf5 and are provided by the dnf5-plugins package or dnf5 directly.

:link: dnf-plugins-extras

Installed plugins will persist on the system and remain functional using the dnf4 binary from the python3-dnf package.
Porting the functionality to dnf5 is currently of low priority, and its implementation depends more on community involvement. The functionality of the Snapper plugin is already covered by the dnf5 actions plugin.

:link: Requirements on dnf

This is the most critical category of packages as issues connected with them can potentially disrupt the system upgrade path.
Here, we can split components into two groups.
The first group utilizes dnf from the command line interface (CLI), and thus, they either need to adjust to the new syntax and behavior of dnf5 or switch to utilizing the existing dnf4 binary directly.
The second group consists of components providing plugins for the dnf command. These plugins will remain functional using the binary from python3-dnf, requiring packaging changes to depend on this package instead.
auter calamares copr-builder cpanspec dnfdragora etckeeper-dnf fedora-review fedora-upgrade kiwi-systemdeps-core libdnf-plugin-subscription-manager lpf mock osbuild perl-CPAN-Plugin-Sysdeps rbm rpmdistro-repoquery supermin system-config-language

A tracking issue was created in advance to inform maintainers about the planned switch to dnf5.

:link: Requirements on python3-dnf

Here, we can also categorize components into two groups.

The first group consists of components providing plugins for the dnf command. These plugins will remain functional using the binary from python3-dnf, requiring packaging changes to depend on this package instead.

The second group utilizes the dnf’s Python API, and they should not be directly affected by the change. However, testing is still necessary, and it is strongly recommended to consider porting to the dnf5 API.

anaconda-core copr-builder dnf-plugin-diff dnf-plugin-ovl dnfdaemon fedora-easy-karma fedora-review fedrq lorax mock-core-configs module-build-service modulemd-tools needrestart policycoreutils-devel pungi python3-dnf-plugin-cow python3-dnf-plugin-flunk_dependent_remove python3-dnf-plugin-perfmetrics python3-imgcreate python3-libreport retrace-server subscription-manager system-config-language

:link: Requirements on libdnf

These packages utilize the libdnf API and should not be directly impacted by the change. However, testing is still necessary.
Tools that modify system software, such as PackageKit, may exhibit different behavior when used alongside dnf5 to manage the same system, as previously described.
Additionally, it is strongly recommended to consider porting to the dnf5 API in this context.
PackageKit copr-builder libdnf-plugin-subscription-manager libdnf-plugin-swidtags libdnf-plugin-txnupd

:link: Requirements on python3-hawkey

These components utilize unsupported Hawkey Python bindings and should not be directly impacted by the change. However, thorough testing is necessary. Again, it is strongly recommended to consider porting to the dnf5 API in this context.
mock-core-configs modulemd-tools python3-rpmdeplint retrace-server

:link: Contingency Plan

:link: Contingency mechanism

To revert to the previous state with dnf as the default package manager on the system, the following steps would be necessary:

  • Packaging changes in dnf5 to remove the obsoletion of dnf and provider of the /usr/bin/dnf symlink.
  • Untagging the candidate dnf5 package from the compose.
  • Components that adapted to the dnf5 CLI must synchronize in this process and revert the changes to use dnf4 again. Proactive communication will be conducted similarly to how components were informed about the dnf5 migration.

:link: Contingency deadline

Branch Fedora Linux 41 from Rawhide

:link: Blocks release?

No

:link: Documentation

:link: Official documentation

:link: Upstream project

:link: Changes between dnf and dnf5

:link: D-Bus API documentation

:link: Contributing guide for developers

4 Likes

Removed f39

Given that a first security evaluation found security bugs (dnf5daemon-server: Local root Exploit and Local Denial-of-Service in dnf5 D-Bus Components | SUSE Security Team Blog), maybe it would be nice if another company or Red Hat security could be asked to take a another look at it before we make it the default.

3 Likes

Looking at Document changes in dnf5-plugin-automatic files between DNF 4 and 5 · Issue #1289 · rpm-software-management/dnf5 · GitHub, I conclude that the new plugin will not be a drop-in replacement for dnf-automatic. Could you be explicit about this in the change proposal, and include instructions on how to do the migration? Will there be a script that converts existing configuration on package upgrade, or do I have to follow some manual steps to replicate my existing dnf-automatic setup?

1 Like

I don’t understand this from Changes in DNF5 in comparison to DNF — dnf5 documentation

Distro-sync command

  • When any argument does not match any package or it is not installed, DNF5 fail. The behavior can be modified by the --skip-unavailable option.
  • Dropped distrosync and distribution-synchronization aliases

As it is documented, the current DNF functionality is:

As necessary upgrades, downgrades or keeps selected installed
packages to match the latest version available from any enabled
repository. If no package is given, all installed packages are
considered.

That seems a lot different from failing if an argument doesn’t match.

(Same goes for the Downgrade command.)

1 Like

I also really, really think that if we’re going to do this, it’s our chance to become consistent with everyone else in usage of “update” and “upgrade”. (Generally, “update” means to get latest fixes within current release version, while “upgrade” is used to mean “new major release” — probably “new Fedora Linux release” in this context.)

4 Likes

More regular end user here but I wholeheartedly agree with your reasoning on this. Sometimes I just… mistype the command I want to use out of pure muscle memory from previous package managers and I feel unifying the terminology would be hugely beneficial.

I love DNF, it’s by far and away my favourite package manager, so using this chance to make it even better by bringing it more in like with what most users might expect feels like a no brainer to me

1 Like

What about the system-upgrade plugin? The checklist in that issue shows it hasn’t been implemented yet, and that is the advertised way to do an upgrade in the docs offline (Upgrading Fedora Using DNF System Plugin :: Fedora Docs), so the fact it is missing right now leaves a hole compared to the current dnf.

(I know that I always do offline upgrades because a long time ago I had an online upgrade crash the terminal it was running in and leave things in a weird state - so I always do offline upgrades now).

2 Likes

This is not consistently held even in Linux package managers. APT uses “update” to “refresh metadata cache”, for example.

And system upgrades are not upgrades, they are package syncs (analogous to pacman -Syuc), version comparison is ignored to ensure that packages match the versions that are in the target distribution release repository (that’s why you’ll sometimes see downgrades and it’s fine).

1 Like

A post was merged into an existing topic: F42 Change Proposal: Fedora Plasma Workstation (System-Wide)

Hi Otto,

Actually, there shouldn’t be any functional issues when transitioning from dnf4 automatic to the dnf5 counterpart. The only thing is the relocation of configuration files.

All the differences will be documented in the dnf4 vs dnf5 changes doc. On this place, we will also provide instructions on how to make the switch.

Hi Matthew,

That seems a lot different from failing if an argument doesn’t match.

The snippet you’ve provided from the changes documentation highlights the current strict behavior of dnf5. By default, commands now terminate the operation if a non-matching argument is passed. However, you can still opt for the lenient behavior by using the --skip-unavailable option.

Hi Ian,

The system-upgrade has already been implemented, see also docs here.

Our GitHub issues labeled as epics can sometimes be misleading for external viewers. While the majority of the functionality is already implemented, there may be some minor aspects that are still outstanding, resulting in the issue remaining open. For further details, please navigate to the specific issue.

1 Like

Hi Timothée,

That’s a valid point. We’ve already reached out to the Red Hat Security team regarding this matter. However, it seems that it’s outside the scope of their standard workflow because the dnf5 package isn’t scheduled for inclusion in RHEL systems yet (the current target being RHEL 11).

1 Like

Yes, but… I don’t understand how this renders distro-sync and the others obsolete. It seems like apples and oranges. Or apples and teapots or something.

1 Like

Yes, but… I don’t understand how this renders distro-sync and the others obsolete.

Hmm, but we don’t mention anywhere that distro-sync or downgrade are now obsolete, do we? The documentation you’ve linked only highlights the differences in individual commands between dnf4 and dnf5. Or perhaps I’m misunderstanding. :slightly_smiling_face:

To be honest, I don’t entirely understand why dnf5 should support modules at all beyond maybe listing them. But I’d prefer not even that. Maybe if it’s a requirement in order to upgrade…

Oh! I read the “Dropped” in

  • Dropped distrosync and distribution-synchronization aliases

too strongly! I thought the whole command was being dropped for some reason!

2 Likes

I’m still confused over it. . . I read it twice and approaching my second cup :coffee:

1 Like

I think I’m going to compile a diff of both dnf3 and dnf5 somehow, because there are some extremely useful features I think I’m going to miss out on by moving up. Also supporting people on the forums and beyond. . . :exploding_head: