Are delta RPMs dead?

Hi, I live in a very poor country where bandwidth is not cheap. Recently I saw a proposal to remove delta RPMs in Fedora 39 because they “take too much space in mirrors”, “use too much CPU”, “never work”, “always fail”, “always give MD5 mismatches”, “are broken by design”, “Fedora doesn’t do delta RPMs correctly unlike openSUSE” and other issues. Today I updated to get KF5.109 and plasma 5.27.7, where I got

Delta RPMs reduced 346.1 MB of updates to 75.3 MB (78.2% saved)

A few weeks back, I got

Delta RPMs reduced 183.9 MB of updates to 13.6 MB (92.6% saved)

Quite often, like once every few weeks, delta RPMs save up to 50% of size. This is quite significant given I update once a week unless there are browser updates. They also save time, as internet is very slow here unless I’m using the local mirror which is not updated regularly and often has zchunk errors.

Now, why are delta RPMs being phased out? Given they are auto generated by koji and mirrors already calculate storage requirements including delta RPMs, how does their removal help Fedora?

2 Likes

I often see similar results with the delta rpms. Can you point to the proposal to do away with the deltas? I have not seen any official notification about that. A proposal is a suggestion that may or may not be followed.

2 Likes

Because moat people see either no size saving or less then a couple of percent saving. My experience is the same and I update once a week as well.

Given that the discussion on fedora dev mailing list was favouring dropping delta rpm generation. I don’t know when or if that will happen.

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/NLH6LCDDLEPOP74UOLNQG4NIIVBM7CTU/

Additional information :

Also, if most of your downloads takes less than a second for 10 or so rpms, you don’t really have a benefit for delta rpms.

It is important to realize that not all people have internet connects that are that fast, and for whom it is important to have delta rpms reducing the download size. Also some people need to pay real money for every byte they download or for every minute they are connected.

3 Likes

Here is some data about delta RPMs from my systems.

I have records of most of the updates I have done on 6 Fedora hosts since the start of the year.
Some are KDE deveopment systems one fileserver/imap-server, one web server.

The updates are run approx 1 week apart.
The number of updates I have records for is 114.

The total size installed is 42,454.7MB that downloaded as 39,049.7 MB which is a 8.7% saving.

74 (65%) of the updates saved less then 5%.
96 (84%) of the updates saved less then 15%

For the record I’m not for or against the delta RPMs.

Agreed. Those who are on metered connections would probably benefit from the drpm while those who are not metered would not really care.

I agree that delta RPMs can be useful for slow or metered connections, but I’m not so sure it is worth to keep enabled by default, as disabling it often makes upgrades faster.

I think it’s sad to make life for people on (expensive) metered or slow connections (yes, that is still a reality for millions) more difficult or leave them behind. It would be more inclusive to fix delta rpm and enable bandwidth savings for those who need it.

As I mentioned in discussion thread for the Change proposal (I am the author of the proposal), I doubt that Delta RPMs actually provide you any benefits on metered connections. Assuming you do not upgrade daily, you’re getting the worst of both worlds (constant cost of Delta RPMs in the repository metadata, but small benefits since you do not upgrade daily).

“Fixing” Delta RPMs would involve pretty large changes in how they are implemented (i.e. keeping Delta RPMs for longer and / or generating deltas between more versions) - nobody is willing to invest into making the technical changes required, and that’s not even accounting for the increased requirements for storage / compute.

PS: I hope you aren’t upgrading your system via GNOME Software, because it doesn’t support Delta RPMs at all (because the PackageKit DNF backend lacks support for them) :sweat_smile:

2 Likes