Flatpak using much more storage space than installed packages

I noticed that flatpak was using a lot of storage space (11 GB in /var/lib/flatpak). So I ran flatpak list --columns=ref,size to see what packages were using the most space. I got the following:

$ flatpak list --columns=ref,size
Ref                                                                                 Installed size
io.dbeaver.DBeaverCommunity/x86_64/stable                                           300.8 MB
io.dbeaver.DBeaverCommunity.Client.mariadb/x86_64/stable                             51.8 MB
net.sourceforge.qtpfsgui.LuminanceHDR/x86_64/stable                                  39.5 MB
org.freedesktop.Platform/x86_64/21.08                                               557.0 MB
org.freedesktop.Platform.GL.default/x86_64/21.08                                    375.5 MB
org.freedesktop.Platform.GL.nvidia-495-44/x86_64/1.4                                984.1 kB
org.freedesktop.Platform.html5-codecs/x86_64/18.08                                    8.8 MB
org.freedesktop.Platform.openh264/x86_64/2.0                                        778.2 kB
org.gnome.Platform/x86_64/41                                                        765.6 MB
org.kde.KStyle.Adwaita/x86_64/5.12                                                   17.8 MB
org.kde.Platform/x86_64/5.12                                                          1.2 GB
org.kde.PlatformTheme.QGnomePlatform/x86_64/5.12                                     14.3 MB
org.kde.PlatformTheme.QtSNI/x86_64/5.12                                               3.6 MB
org.kde.WaylandDecoration.QGnomePlatform-decoration/x86_64/5.12                      14.0 MB
org.signal.Signal/x86_64/stable                                                     369.0 MB

The installed sizes of all the packages add up to about 3.7 GB…way less than the 11 GB total used by flatpak. I’d expect there to be some overhead, but it appears flatpak is using almost 3 times as much space as the total of all the installed packages! Does this indicate a bug in flatpak? Is it possible that there are temporary files generated by flatpak that should have been deleted but weren’t?

Hi @jhaiduce Flatpak packages do carry more dependencies. read Flatpak - Fedora Project Wiki.

2 Likes

This is expected. Apart from a few common runtimes, these flatpaks bundle all their dependencies—so even if two flatpaks use library A, they’ll each carry a copy of it (unless it’s included in the few available runtimes). I’m quite sure this is the case for Flatpaks from Flathub but I haven’t checked to see how Flatpaks from the Fedora repositories are generated (they’re generated from our rpms but I don’t know the pipeline well enough)

This is different to how software that is installed using rpm/dnf works—there, as far as possible, we only have only one copy of each dependency as a system level package that is used by all other packages.

4 Likes

You can try app images and snaps snaps are smaller than flatpak but it take slightly more time to open in my device but in ssd it is not a issue. Else rpm i always there if you want the smallest package.

1 Like

Do these not bundle their dependencies?

That explains why flatpaks use more space than rpms… but I would expect the space used by those bundled dependencies to be reflected in the installed size of the flatpaks. Why isn’t it?

1 Like

Hello @jhaiduce ,
Flatpaks are containerized apps, and indeed they do bundle a lot of their dependencies, and if the flatpak maintainer isn’t caring there will invariably be more than you would expect. However the internals of Flatpak are explained very well at https://docs.flatpak.org/en/latest/under-the-hood.html. If you read through their doc’s you will see how they work and how the runtimes are used etc… I think if you do a bit of digging into this you will find that there is not as much bloat as suspected and that the missing storage you cannot account for becomes accounted for.

4 Likes

Thanks @jakfrost, I see in the page you linked that flatpak uses hard links to version files, which could result in some files being double-counted by tools like du and baobab. I’m not sure how to verify that this actually accounts for the extra space though.

1 Like

Yes they do bundle there dependency with there apps and run in a container same as flatpak but with there packaging way the ultimate app is smaller than the flatpak now i have downloaded them from both for various apps and come to the conclusion… that snaps are smaller in size than flatpak.!
ex:






but some time reverse can be possible.

Are you on btrfs? Try:

    sudo btrfs filesystem du -s /var/lib/flatpak
3 Likes

No, I’m using ext4.

After reading On application sizes and bloat in flatpak – Alexander Larsson and /var/lib/flatpak/repo/objects Folder is taking up almost 12 GBs of space · Issue #3894 · flatpak/flatpak · GitHub I learned that:

  • The du command does detect hardlinks and avoids double counting them if one inventories all inter-linked folders with a single du command
  • /var/lib/flatpak/app is supposed to only contain hardlinks to /var/lib/flatpak/repo/objects

So I run du on all the subdirectories of /var/lib/flatpak, listing repo first:

$ sudo du -hs repo app runtime appstream exports oci overrides
3.2G	repo
2.2M	app
482M	runtime
6.7G	appstream
360K	exports
2.3M	oci
8.0K	overrides

What this seems to show is:

  • app does contain some things that aren’t just hardlinks to repo/objects, but they don’t take a lot of space (only 2.2 MB)
  • appstream takes a lot of space (6.7 GB) that can’t be explained away as hardlinks to repo/objects or anywhere else

That seems peculiar. I think that’s supposed to be just metadata? (It’s what’s shown in GNOME Software.) What’s the disk usage for subdirs within that?

1 Like

Here’s the contents of the appstream subfolders:

$ sudo du -hs repo appstream/*
[sudo] password for jhaiduce: 
3.2G	repo
2.9M	appstream/fedora
2.7M	appstream/fedora-testing
6.6G	appstream/flathub

It looks like most of that is in appstream/flathub/x86_64:

$ sudo du -hs repo appstream/flathub/x86_64
3.2G	repo
6.6G	appstream/flathub/x86_64

appstream/flathub/x86_64 contains lots and lots of stuff, mostly hidden (.) subdirectories (736 in total).

Individually they don’t take a ton of space, but apparently they add up. Here are the first few after sorting largest first:

$ sudo du -hs repo appstream/flathub/x86_64/.[^.]*|sort -rh|head
3.2G	repo
19M	appstream/flathub/x86_64/.bfcf1134648654c3e04d9dce3e9060e53dc8a164809db4e448de70cfa56efd60-0VTHC1
19M	appstream/flathub/x86_64/.1507904d235c9fb08a2a9298cfbfa03988bb547ff7c2c239825889fa0c3727e8-6YFPC1
18M	appstream/flathub/x86_64/.ec1b858e97eb09107b9660ed635db7f8b738bed10ffb4e5c8e8b731bbcef7dff-AFGJA1
18M	appstream/flathub/x86_64/.eba42bc391663af8b972accacbf382895e091ccd3c64a3ef857d2eaca1e852b1-66OLA1
18M	appstream/flathub/x86_64/.eb27a1619b76191fabf9908eb2f5811c3bad62c3e1c56343fc31430a60a2826e-1NHCA1
18M	appstream/flathub/x86_64/.e7ce007780ca5f7eb33438c595e74ec37e611c6c9b36906544e2678365c0138a-6SX090
18M	appstream/flathub/x86_64/.e4ee47b9b502c874f29834bc9540ee4028c3d85673a0f8d4eec8e6ba29208bf7-2LAUA1
18M	appstream/flathub/x86_64/.e3448f48b4a03fbe10f3abb41cbb464032ae4bf0dcbb26e042695c308008f543-7TM190
18M	appstream/flathub/x86_64/.db07388fa705bcd023a4fba2daf63dbd48a9c8218d996d16d9ccb8c7e9c7b88c-UDNMA1
2 Likes

No idea what happened but something is really wrong with /var/lib/flatpak/appstream. If it was my system I would delete (rm -rf) that directory. Appstream is going to be rebuilt by Gnome Software.

A couple questions about this before I do it:

  • Will my flatpak apps still work in the meantime (i.e., after I delete the contents of /var/lib/flatpak/appstream but before Gnome Software rebuilds it)?
  • Is there a way to rebuild the appstream directory from the command line instead of waiting for Gnome Software to do it?

That’s… a lot. I think you should be able to remove the contents of appstream/*.

    flatpak update --appstream

will refresh from the command line

3 Likes

Thanks @mattdm and @augenauf, I renamed appstream to appstream.old just to be safe in case something goes wrong. Flatpak apps run without error. But I get errors trying to update the appstream folder:

$ flatpak update --appstream
Updating appstream data for remote fedora
Updating appstream data for remote flathub
Error updating: Error deploying appstream: Opening content object d232a00e9b8f42e532d6505ae230413a53c9d2bdde7466f4bf46355dd3d496e1: fgetxattr(user.ostreemeta): No data available

Hmmm — does flatpak repair --system help?

2 Likes

I get the same error if I run flatpak repair --system before I run flatpak update --appstream.