F42 Change Proposal: Integrate FEX in Fedora Linux (Self-Contained)

Integrate FEX in Fedora Linux

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Wiki
Announced

:link: Summary

FEX is a fast emulator that allows one to run x86 and x86-64 binaries on an AArch64 Linux host. FEX requires a number of supporting components, including a RootFS image, and integration with krun to support 16k page-size hosts. The purpose of this Change is to integrate FEX itself and its supporting components into Fedora Linux, to provide a delightful out-of-box experience for users that want to run x86 and x86-64 binaries on their aarch64 systems. This also includes integration into the AArch64 Fedora KDE spin as a non-blocking component of the spin.

:link: Owner

:link: Detailed Description

When running Fedora Linux on an AArch64 host, one can normally only run AArch64 binaries. This can be a problem if the user needs to run preexisting software that was built for x86 or x86-64, which is still the predominant architecture. If the software is opensource it can sometimes be recompiled (or, even better, packaged in Fedora), but this isn’t always an option and could require significant work on the user’s part. For proprietary software, there is no viable option short of persuading the author to release a native AArch64 build.

FEX allows the user to sidestep this issue entirely by making it possible to run x86 and x86-64 binares on AArch64 Linux hosts, as if they were native. FEX achieves this via emulation, and it integrates with the binfmts infrastructure to provide a seamless experience. To ensure wide compatibility, FEX is also able to leverage a distribution-shipped root filesystem tree (RootFS) to provide core system libraries that the emulated binaries might need.

FEX only supports AArch64 host systems running a 4k page-size kernel. This is the default in Fedora Linux, but it presents a problem for Fedora Asahi Remix, as Apple Silicon Macs use a 16k page-size. To address this, we will leverage krun to run FEX inside a microVM with a 4k page-size kernel, thus providing a compatible environment with minimal overhead.

:link: Feedback

(pending the initial Change discussion)

:link: Benefit to Fedora

Users will be able to run x86 and x86-64 binaries out of the box when running Fedora Linux on their Aarch64 systems. This particularly improves the usability of Fedora KDE for day-to-day usage on AArch64 systems.

:link: Scope

:link: Upgrade/compatibility impact

No expected upgrade or compatibility impact, this is a purely additive Change.

:link: How To Test

Once the relevant components have been packaged and the comps group has been created, this Change will be testable with:

  1. dnf install @fex-x86-emulation
  2. …
  3. profit!

:link: User Experience

Users will be able to run x86 and x86-64 binaries out of the box when running Fedora Linux on their Aarch64 systems.

:link: Dependencies

This is pretty much self contained.

:link: Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) Don’t ship this
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? No

:link: Documentation

The Change owners will write documentation on how to use FEX, both for general Fedora systems and in the context of Fedora Asahi Remix.

:link: Release Notes

Last edited by @amoloney 2024-09-12T15:54:03Z

Last edited by @amoloney 2024-09-12T15:54:03Z

3 Likes

How do you feel about the proposal as written?

  • Strongly in favor
  • In favor, with reservations
  • Neutral
  • Opposed, but could be convinced
  • Strongly opposed
0 voters

If you are in favor but have reservations, or are opposed but something could change your mind, please explain in a reply.

We want everyone to be heard, but many posts repeating the same thing actually makes that harder. If you have something new to say, please say it. If, instead, you find someone has already covered what you’d like to express, please simply give that post a :heart: instead of reiterating. You can even do this by email, by replying with the heart emoji or just “+1”. This will make long topics easier to follow.

Please note that this is an advisory “straw poll” meant to gauge sentiment. It isn’t a vote or a scientific survey. See About the Change Proposals category for more about the Change Process and moderation policy.

How will this interact with qemu-user-static-x86 which already owns the binfmt setup for running x86 on other arches?

What happens if both are installed on the system?

Does this Change seek to make FEX the default implementation for x86 binfmt on all non-x86 arches? Just on aarch64? Or will qemu remain the default and FEX just becomes an alternative?

Do you want FEX installed by default on some aarch64 live media?

1 Like

This is a good question and something we will test and report back on.

FEX only support aarch64 hosts, so this will be a noop for other arches. This Change does not seek to make FEX the default or preinstall it in stock Fedora deliverables. We do plan to preinstall FEX in Fedora Asahi Remix, and the KDE SIG is looking into options to better integrate FEX into their deliverables.

1 Like

It is getting preinstalled on Fedora KDE AArch64 specifically to validate the integration in stock Fedora.

1 Like

This sounds like a great idea for Asahi users especially.
Is some kind of integration with Flatpak also planned, seeing that more and more popular (often x86 only) apps are available as a Flatpak, or is that out of scope for now?

1 Like

…snip…

FEX only supports AArch64 host systems running a 4k page-size kernel. This is the default in Fedora Linux, but it presents a problem for Fedora Asahi Remix, as Apple Silicon Macs use a 16k page-size. To address this, we will leverage krun to run FEX inside a microVM with a 4k page-size kernel, thus providing a compatible environment with minimal overhead.

Is the krun wrapper going to be used everywhere? Or just on Asahi?

:link: Feedback

(pending the initial Change discussion)

:link: Benefit to Fedora

Users will be able to run x86 and x86-64 binaries out of the box when running Fedora Linux on their Aarch64 systems. This particularly improves the usability of Fedora KDE for day-to-day usage on AArch64 systems.

So this will support x86 (32bit) as well?
Does that depend on a 32bit rootfs being constructed? Or the single
rootfs will have both 32 and 64bit?

I’m not sure we want to support this as I hope we can stop making 32bit
stuff someday soon.

Thanks for working on this…

1 Like

We haven’t tested Flatpak specifically yet, but I don’t see a reason why it shouldn’t work there as well. I’ll add this to the list of things to check, thanks for bringing it up.

2 Likes

The plan is to use it on any systems that don’t run 4k page-size kernels. So yes, primarily Asahi (and that’s what we’ll test against), but it should also work for e.g. aarch64 systems running 64k page-size kernels.

Yes, it’ll support x86 as well, and we’ll have a single rootfs that will include both 32bit and 64bit packages. You can see the current list at PR#83: Add FEX-RootFS definition - fedora-kiwi-descriptions - Pagure.io (with the caveat that this is likely larger than it needs to be). If and when Fedora decides to drop 32bit builds altogether we can revisit this.

Hmm, just a guess, but wouldnt this need to be executed from within the Flatpak container?

If yes, then you need a runtime extension, and somehow auto-install that on aarch64. If not, then you need a second method, I guess, apart from binfmt, to detect amd64 Flatpaks and run them accordingly.

It’s probably best to ask Flatpak people this, they should know this.

I had a look at a discussion about hardened_malloc for flatpaks, which may be in a similar category.

But not having this wouldnt stop it from being a positive addition, it would just be less useful.

1 Like

I suppose. But it seems weird to push a new feature/thing with support
only to remove it (hopefully soon) after… on the other hand, I suspect
there’s fewer and fewer people who care about that support.

Steam is still 32 Bit and that is like half the reason people want FEX on their ARM machines.
Maybe in the future Steam will be fully 64 Bit and Wine and Proton use the new WoW64 mode by default that doesn’t need 32 Bit Linux libs; at that point you’d only be giving up some older native Linux games and obscure old proprietary Linux software.
But for the foreseeable future x86 32 Bit support is absolutely a thing people will want.

1 Like

This change proposal has now been submitted to FESCo with ticket #3273 for voting.

To find out more, please visit our Changes Policy documentation.

This change has been accepted by FESCo for Fedora Linux 42. A full list of approved changes to date can be found on the Change Set Page.

To find out more about how our changes policy works, please visit our docs site.

I’m assuming this is copypasta, but just to clarify: this Change is for F42, not F41 :slight_smile:

1 Like

Correct :sweat_smile: Fixed now!

2 Likes

disclamer: I’m the main author of box64, so of course my answer is biased.

Yet, I don’t see why FEX should be installed “by default” on aarch64 version of fedora. It will kill the competion, and will add a 1 GB (or more?) download size for the needed rootfs.

Also, to the argulment that box64 does not support 32bits it’s now in developpement, and can already runs some program & game (admitedly not steam yet, but it’s of course in the TODO)
Also also, box64 is compatible with 16k pagesize, and can run many games and program out-of-the box on 16k pagesize.

Finaly, box64 is also compatible with RiSC-V architecture (and even Loongarch).

For alternate architecture like Aarch64 or RiSC-V, the x86 translation layer is a critical piece of software, as important as the browser for example. I must say I am chocked and disapointed to see that in future fedora, a solution will be imposed to end-user!

This isn’t implemented yet, but the plan is to install a binfmt shim that would download and install FEX and the rootfs on first use, instead of baking them into the install image. This would also allow users to pick a different implementation if they so desire. box64 is already packaged in Fedora and it’s definitely something that’s available for our users, as is QEMU. That said, we believe that at the moment FEX does provide a better out-of-box experience for users on aarch64, so that’s what why the Change is focused on it (also see our feedback section). I definitely agree that on other architectures such as RISC-V box64 is a better option.

1 Like

So you tested both FEX and Box64 to get to this conclusion. To the point on making it a default choice?

I don’t understand how the binfmt shim is working? Will it ask for the end-user to choose something at the first x86_64 encouter? or will it download FEX and all it’s depandencies directly (just asking the user if that’s fine to download it)?

Side note: that box64 package is severly out of date. More than 1.5 year old.