Developing applications using Flatpak-packaged editors/IDEs

Hi!

I’m really excited about Fedora Silverblue and how I could convert my “classical” Fedora workstation to it.

Currently, I’m exploring using Flatpak versions of certain applications instead of their classical RPM counter-parts. And I must say it works well for many applications.

One area where I’m having trouble figuring out how things should work in the new Silverblue landscape is with Flatpak-packaged editor/IDEs in relation to application’s software stacks.

For example, a developer is working on a Python application and wants to use Visual Studio Code (OSS version) from Flathub as his development environment.

The org.freedesktop.Sdk/x86_64/1.6 runtime on which the current app/com.visualstudio.code.oss/x86_64/stable is based on (at the time of writing) provides Python 3.5.2 and Python 2.7.12.
This is serious limitation since two new Python versions were released in the mean time, 3.6 and 3.7, and are available in Fedora “classic” Workstation.
Furthermore, one can’t simply configure VS Code to use the host-installed Python executable with a corresponding Python virtual environment since such attempts fail with:

/home/tadej/.local/share/virtualenvs/temp/bin/python: error while loading shared libraries: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory

More generally, when someone develops an application he usually wants to have complete control of the whole software stack, from the version of the language it uses to all the dependent libraries and their versions.

How does Fedora Silverblue plan to tackle this use case?

10 Likes

Thanks for posting this! I helped create Silverblue, and I’d probably say this problem is my biggest pain point too. It’s where the flatpak vs podman/docker vs host divisions comes strongly to the fore.

In some sense it’s a generic flatpak issue - I’m not sure if there is one upstream or not.

But it’s also a Silverblue issue in that on a classic system one could more easily take the “out” of using flatpak just for simple/fixed apps, and install IDEs+tooling on the host. You can of course do that on Silverblue too…but it’s fighting the system a bit.

The other alternative is to install the IDE inside a pet podman container and give it access to X/Wayland etc.; would require reinventing some of the flatpak⇔desktop integration.

Yet another path would be for flatpak to grow some mechanism to “derive” from an app (much like OCI layering) and add e.g. other Python versions or Rust/whatever to it. Flatpak today does have Extensions but I am not sure about using them for this case.
Probably the best thing here would be for an IDE maintainer to comment.

3 Likes

I do a lot of golang. I found this by another golang writer trying flatpak: Go -mess with flatpak+vscode - DEV Community

Seems to work pretty well so far. Though not ideal having to keep everything in home folder, but it works.

@walters, thanks for sharing your thoughts. Since this is quite a new area, one could think that things are just not properly documented.

I did a bit of searching in Issues · flatpak/flatpak · GitHub and the only related thing I’ve found is develop using flatpak · Issue #1412 · flatpak/flatpak · GitHub.

Therein, Alexander Larsson mentions how Pitivi uses Flatpak as a development environment, but that is a bit different use-case since there you have an upstream project providing a 700+ lines Python script implementing the Flatpak-based development environement along with a Bash script providing aliases and shortcuts for this environment.

I think you’ve nailed it here. For example, I’ve just stopped using
VSCode’s Flatpak and switched back to using the host-installed RPM version.

Yes, I could probably do the same on Silverblue via rpm-ostree layering but as you’ve said, it’s fighting the system a bit.

Yes, one could probably do that, but it feels pretty hackish and very manual from the outset.

This approach has the most potential, in my opinion. But as I’m reading the current Extensions description, Flatpak Extentions appear to be something static, i.e. a Flatpak extension is similar to an RPM subpackage, it is available to extend the file-system of a Flatpak to provide translations or debug information that is not part of the main Flatpak.

On the other hand, a developer would need a user-defined read-write OCI layer on top of a Flatpak file-system where he could install his application’s binaries and libraries. E.g. install the desired version of Python and pip-install the desired versions of application’s libraries.

Agreed. Do you know any we could @ mention to look at this topic?

@geekgonecrazy, thank’s for the pointer.

I’m afraid this solution is very Go-specific since Go applications are usually distributed as single statically linked binaries and you can get away by putting them in your home folder and bind-mounting it in the Flatpak sandbox.
But this won’t work with other languages and applications that aren’t statically linked. Not to mention that I don’t want to manually fetch these binaries and store them in my home folder.

So, this is a commonly asked question, and there is no full answer to it, but there are several options that, by themselves or as a combination, moves toward solving this.

The first option is the one chosen by gnome-builder. gnome-builder tightly integrates with flatpak and doesn’t actually build things in the context of the gnome-builder sandbox and its runtime, instead if recursively creates other sandboxes for builds. This means that you get to pick any runtime for your builds which may be completely different than the one that runs gnome-builder itself. This is quite powerful, but requires a lot of intimate support for this model in the IDE.

The second option is to extend the app/runtime via flatpak extensions. Flatpak already supports this in the Sdk, with the org.freedesktop.Sdk.Extension extension. Flathub currently hosts sdk extensions for mono, openjdk 8+9, rust and gcc 7 + 8. These are compilers that are not supported natively by the Sdk, but you can still use them.

Effectively what this mean is that if you install say org.freedesktop.Sdk.Extension.rust-stable and run something against the freedesktop sdk (or derivates) a directory will appear in /usr/lib/sdk/rust-stable/ that has the compiler and tools for rust. All you have to do is add it to PATH and then you can build some rust app.

It has been proposed to add a similar extension point for IDE-like tools so that you can easily add the tools you need to them. This would be done in a way that allows these extensions to be shared between different IDEs.

The third way is to use the dbus service that comes with flatpak to break out of the sandbox. This needs the app to have explicit permissions for this, but once you have this you can use the flatpak-spawn --host command in the sandbox to start commands on the host (or call the dbus directly which is what gnome-builder does when it spawns you a terminal on the host inside its window).

I think different problems call for different solution, but a combination of the above is pretty compelling.

3 Likes

The first option is the one chosen by gnome-builder. gnome-builder tightly integrates with flatpak and doesn’t actually build things in the context of the gnome-builder sandbox and its runtime, instead if recursively creates other sandboxes for builds. This means that you get to pick any runtime for your builds which may be completely different than the one that runs gnome-builder itself. This is quite powerful, but requires a lot of intimate support for this model in the IDE.

There’s a variant of this - which is for the IDE to do builds in a container. If the end thing you are creating is a graphical application, then doing the builds as Flatpak builds makes a lot of sense. If the end thing you are creating a server app, then a container build is going to work better - and also allows using prepackaged Fedora dependencies. I don’t know of any examples of setting up a Flaptak’ed IDE to target container builds, but many IDE’s will have some sort of container support that can potentially be adapted for this purpose.

2 Likes

Right, that’s the case for me - I am using Silverblue primarily to develop server-side apps for Kubernetes (and base operating system components), not desktop apps. So the flatpak IDE story basically crosses all 3 systems (flatpak, dev/pet containers with podman, and local oc cluster up testing).

I’m a newbie at this but I had briefly looked at what Theia IDE is doing and I like some of their ideas Theia - Cloud and Desktop IDE Platform for composabilty

This tooling issue is real and I guess a lot of us need to readjust to this container mindset to be productive.

Can you reach out to Christian Hergert, the author of Gnome-Builder?

Maybe you could somehow redirect Python has a wrapper for flatpak-spawn --host python3 or similar? This is how GNOME Builder’s and Tilix’s terminal work.

BTW as for Atom I’ve created some wrapper scripts for things I use and that needs to be spawned on the host system.

Actually, I guess, you can use the exact same ones for VS Code/OSS Code (see difference here), too. Or any other IDE in a flatpak, for that matter.

It is based on the suggestion by @refi64 to use flatpak-spawn. Explanation on how to setup it is there in the repo.

1 Like

From reading the extensions description, this seems like exactly what it’s meant for. Means extensions for Atom, VSCode, etc. can be created per language (Python, Go, PHP, Ruby, etc.) which can contain all the tools needed for full language/IDE support (library, linter, etc.). I’m assuming a Flatpak extensions is easily distributed on Flathub and conveniently displayed in the GNOME Software app as add-ons to the main application. That makes it user friendly to extend your favorite editor with support for the required languages so the Atom/VSCode in-application extensions work too. Even makes it easy to scope extensions for major language versions (Python 2/3, PHP 5/7, etc.).

Anything with a wrapper requiring users to alter the language commands inside their editor to flatpak-spawn, or building all kinds of podman containers, or using fedora-toolbox is simply not user friendly. If we wanted to tinker with everything, we would probably all be using Gentoo. I know the solution is upstream in Flatpak/Flathub, but i.m.h.o. Fedora needs that to be a “it just works” solution.

1 Like

I just switched to running emacs from inside toolbox and I am way happier than I was with previously having it layered on the host, and trying to hook up e.g. M-x compile to run in my previous dev container. Also briefly tried GNOME Builder inside toolbox, seemed to work OK but I have too many years invested in Emacs at the moment.

This pattern of running GUI apps inside toolbox definitely has disadvantages too, e.g. GNOME Shell/Software doesn’t know about them, but on the flip side most editors and IDEs will Just Work this way to integrate with your build tools.

1 Like

toolbox create and toolbox enter are pretty user friendly. Also being able to use dnf as you would in workstation is pretty user friendly. The toolbox container idea was meant for developers, and can be tweaked to suit then trashed and rebuilt from scratch a different way. But it isn’t exclusively for dev’s, and it does provide a very neat rootless container to play around with some ideas in. If I could spin up a container for my windows app’s I need for my business I’d be overwhelmed by joy for finally being able to ditch the last vestige of a horrid OS. Instead of having to run it in a VM, ughh.

What I’m currently doing with VS Code is taking advantage of workspaces and the shell-args config option to automatically enter the toolbox for a project when I open the terminal while I have the project’s workspace open. I don’t get to use the Rust extension with the RLS this way (at least I haven’t gotten it to work yet) but I can adjust to not having it by getting in the habit of running Cargo Check often. IMO the containers are worth the workflow adjustments.

This is a cool idea, can you show an example of the config? I’d like to try that

Sure! I’m at work right now but I’ll post something tonight.

{
        "folders": [
                {
                        "path": "<project folder>"
                }
        ],
        "settings": {
                "terminal.integrated.shellArgs.linux": ["-c", "flatpak-spawn --host toolbox enter -c <container name>"]
        }
}

So this is what my .code-workspace file looks like for one of my projects. This was generated automatically by VS Code. If you don’t want to use workspaces there’s also folder settings you can set in VS Code’s integrated settings editor. You can get to the settings editor with the gear in the bottom left corner. From there you switch to Workspace Settings and you’ll find the option under Features → Terminal → Integrated > Shell Args: Linux. The cool thing is that since these .code-workspace files are just json these could be created automatically with a script. I plan on writing a script that will automatically generate a container and a code-workspace automatically, and maybe also enter the container and install certain packages according to arguments given.

2 Likes

Thanks for sharing! I’d be interested in seeing how the script turns out. I haven’t really tried extending VScode that much. I suppose I should take another shot at it

Well the script I was thinking about writing either would be just a bash script or a small program, probably written in Rust. The code-workspace file is just a JSON file describing the path of the project and its workspace-specific settings, so you wouldn’t necessarily need VS Code to make it. As a matter of fact, that particular option just has you open the “settings.json” file (which is your workspace file if you are in workspace settings tab) and edit it manually anyway.