Flatpaks and media codecs

There already are at least two topics about adding support for H264 encoded video. In both cases the solution boils down to installing packages from rpmfusion. That’s good and fine as long as you use (layered) rpms. What will be the prefered way to solve this problem once Silverblue ships Flatpaks?

I have looked at several Flatpaks from Flathub and the situation appears to be a little chaotic. Some Flatpaks include codecs, Totem appears to have an extension (org.gnome.Totem.Codecs) and others use runtime extensions like org.freedesktop.Platform.ffmpeg or org.freedesktop.Platform.html5-codecs.

Is the availability of an easy way to add additional codecs an issue Fedora is concerned with, even if Fedora can’t distribute these codecs? A 3rd-party-extension to the runtime provided by Fedora would probably be the most convenient solution, if that is possible, or will users have to replace Flatpaks provided by Fedora (Firefox, Totem…) with Flatpaks provided by Flathub or other alternative repositories if additional codecs are required?

3 Likes

IMO, if those particular codecs are to follow the logic of flatpak design they would likely be made into extensions of the runtime (possibly freedesktop.org). Something like the rust or java sdk extensions. That way all flatpak apps requiring them could find them.

1 Like

I’m not familiar with how the extensions work that you’re talking about, but when you say that “all flatpak apps requiring them could find them”, does that mean that the flatpak app would have to declare upfront that it needs extensions X and Y?

When I think about media codecs being on rpmfusion, I assume it’s because they’re potentially non-free (and therefore not always desired or acceptable by everyone). In @heliopath’s example, would Totem need to say that it requires all of the known codecs, or could the user decide to install a codec (or not) only when they need it?

There are two ways to use extensions for flatpak applications currently. One is to include them in the manifest of the applications flatpak build file. The other way is to install them separately as an extension to whatever platform they belong. The second option affords the opportunity to use them with other flatpak bundles. A specific example I ran into was trying to get the various JDK’s I use to work with IntelliJ IDEA Flatpak and Netbeans Flatpaks. I was having to go through efforts to circumvent the rpm-ostree host system, which seemed counter-intuitive to me in the purist sense of Silverblue and what makes it desirable to use. I tried and successfully built the IDEA flatpak with my own modifications to the build manifest file to include the extensions I wanted, but Netbeans didn’t benefit from it. When I just installed the extensions for the JDK’s the result for IDEA was the same and Netbeans could also use those extensions when I pointed it there. So to answer the last question, the extensions are available for me now as needed by whatever app I decide to point at them, irrespective if they actually need to use them.
The deal with flathub not being setup on Silverblue out of the box, is the same reason as the rpm-fusion free and non-free repo’s not being included out of the box on Workstation, licensing of them is not inline with Fedora’s founding principles and therefore Fedora prohibit’s it. So just like every other time with Fedora, you as a user have to decide how “free” your system is truly.

2 Likes

Thanks for your reply @jakfrost! I understand better now, but I’ll likely have to try some things myself to fully get it. :slight_smile:

1 Like

If you want to check out what there is at flathub, providing you have it setup with flatpak, type the command flatpak remotes to get a listing of your configured remotes. Then use flatpak remote-ls flathub to see what is available on flathub, it will be more than you would see with Gnome Software. Once you have identified the package you are curios about you can use flatpak info <pkgname> to see details about it, or just install it if desired. If your curious about what you have installed type flatpak list, or running type flatpak ps.
[Edit]:Here is a link to the wiki about extensions https://github.com/flatpak/flatpak/wiki/Extensions some good info there.

1 Like

As a side note, the way normal extensions and SDK extensions (what you described) is a bit different.

The “normal” extensions (which is what the codecs are) are declared in the manifest (in this case, the Platform manifest IIRC) with their mount points manually given. This is also how e.g. Hexchat plugins work.

SDK extensions are automatically mounted to /usr/lib/sdk when you’re using the full SDK as your platform, or they can be used during the build process via the sdk-extensions option.

That is true, and during the build process as an application extension they get installed in the $HOME/.var/app/some.application.dir and aren’t readily accessible across the filesystem I thought.

There is now an extension for Totem codec’s on flathub.

1 Like

To sum this up and dumb it down for people like me: The Fedora runtime is compatible with and can be extended by freedesktop runtime extensions from Flathub but the Fedora runtime maintainers have to define extension points to mount these extensions?

In theory, yes. In practice, I’m actually not sure if the fd.o extensions for e.g. ffmpeg would have slightly different library requirements, since they were of course designed for the fd.o runtime, not the Fedora one.

If you want to see any extension points declare in the metadata for the runtime, you can do flatpak info -m my.runtime.

The Fedora runtime is based on the freedesktop runtime, like gnome’s or kde’s, so it will include at least what the freedesktop runtime has for extension points. Important to note is what the application you are wanting to use an extension with uses for a runtime, then you can look at the available extensions. Some are specific to the 18.08 freedesktop runtime while others appear to be provided for both 1.6 and 18.08.

If the Fedora runtime is based on the FreeDesktop runtime, is it possible to run a Fedora-packaged flatpak with the org.freedesktop.Platform.ffmpeg-full extension? I’m not seeing an easy way to do that on my Silverblue laptop.

It’s not based on the freedesktop runtime, though.

Upstream Firefox is now looking at official Flatpak support, so it’s probably best to wait for that as it’s likely going to use the Freedesktop SDKs.

1 Like

[citation needed]

1 Like

In Firefox-Flatpak from the Fedora registry I cannot play all videos (YouTube’s VP9-encoded videos work, but Facebook’s for example don’t). org.freedesktop.{ffmpeg,openh264} are installed. Any ideas?

It’s probably not possible. These extensions are for the freedesktop-runtime while Firefox is using the fedora-runtime. This really is a major annoyance.

Ah bummer, that renders the Flatpak-version pretty unusable…