I was learning how delta updates work and AI is fumbling. For true delta updates where you get the binary diffs of each file that changed, you can only really support static deltas for a few versions. Those are static delta updates, which I think also the Google Chrome (on Windows) updater uses.
But AI says OSTree doesn’t need to store delta updates for every version to every other version, instead it compares the commits of the image of the two versions, and “downloads the changed files”. I steered it towards how it works and it admits that OSTree helps you not download the whole package, but it has to download the whole file that changed, not applying binary diffs inside them. But it also says that Fedora Atomic also provides static deltas which have binary diffs of every file that changed.
I wanted to verify what happens because the main reason I am wanting to switch to a distribution like Fedora Atomic is for delta updates. Delta updates is a basic mechanism and optimization for software updates and I need it on Linux. But the download sizes I still experienced was a bit underwhelming. I still have to download like 300 MB for a single commit jump , updating a few small system packages (not Firefox here). I feel like the total size of those packages themselves would be smaller then the data I downloaded. How much size overhead does this Git-like commits of an image have?
Even for Flatpak, I had to download like a 100 MB update for Chromium, which is half of its package size. On Android, I don’t need to download that much for Google Chrome updates, as for what I can see.
I should try out Snap. That has true binary diffs?
ostree does a bit more than just transfer the entire file if static deltas are used. There are some heuristics involved but it can and will use binary diffs for files where that is a good choice (e.g. the diff is smaller than the entire file) which might just not be that common of a scenario especially when stuff like media assets are in play; or programs get recompiled against other libraries.
Now if Flatpak repositories are generating static deltas between commits: I don’t know, since some remotes use the OCI-transport which wraps a commit inside an OCI container I assume those at least won’t use static deltas. Flathub theoretically could since they use an ostree remote.
If OSTree uses static deltas, then they can only support a handful of commit jumps. If I update from a old commit, I won’t get the binary diffs? My Fedora installer image was slightly old (kernel 6.19 when now 7 is being used) and so after install I had to download over a GB. But that doesn’t seem to be any different from the 300 MB I have to download for only a couple of system packages. Why so much?
AI says Snap uses true binary diffs. Do you have information on that? Should I start using Snaps? I will give it a try.
ostreecan use static deltas; they are generated by the repository owners between two commits. It is unlikely that they are generated between all commits; if a repository does generate them then it’s more likely for them to exist between sequential commits.
As I said I don’t know if they are being used on the Flatpak remotes you are using.
snapd, also has heuristics for enabling binary diffing and also only creates binary diffs between sequential revisions thus as far as I remember updating over multiple revisions incurs a full download (but this could be wrong or have changed).
I don’t know what you mean by “true binary diffs”. ostree uses bsdiff if I recall.
Overall the direction is instead to move to bootable container images that are split into layers. To help with deltas, we will likely slow down the release cadence for stable users.
I am running dnf in a toolbox. This was almost the exact same update as I can remember that I did on my Fedora Atomic host, which had to download 300 MB. Here it literally has to download 8MiB. What is going on?
edit - Also for static updates (if supported as you say), does it get binary diffs for the whole image, or for individual files? Because the image can be manipulated, as I have uninstalled Firefox, so image level binary diffs wouldn’t work.
But what is your take on the update sizes between rpm-ostree and dnf that I mentioned? I just received another update of 270 MB download and 323 MB written, for these packages (all are listed in the image)?
Are these packages supposed to take 323 MB? Does me applying an override to the base image have to do anything? With delta updates, I expected up to a few tens of MB of updates. All updates, do update a few basic packages have been around 300 MB of download. We are not even updating like an app. I just want any distro do provide the download sizes like Android.
Whether or not you have made changes to the system via package layering does not impact what is downloaded during update as rpm-ostree first downloads what is needed to update the base image and then does what is needed for package layering.
