F42 Change Proposal: Switch to EROFS for Live Media (self-contained)

Switch to EROFS for Live Media

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Wiki
Announced

:link: Summary

Switch the read-only filesystem image format from SquashFS to EROFS for Fedora live media.

:link: Owner

:link: Detailed Description

In recent years, there has been increasing adoption of a new, more advanced read-only filesystem for a variety of use-cases called the Enhanced Read-Only FileSystem (EROFS). Support for EROFS as the backing read-only filesystem for live environments was introduced in Dracut v103. This change switches over the following live media to use EROFS instead of SquashFS:

  • all kiwi-produced live media. Currently:
    • KDE Desktop
    • KDE Mobile
    • LXQt
    • MiracleWM
    • COSMIC
    • Xfce
    • Budgie
  • Fedora CoreOS Live media
    • i.e. fedora-coreos-42.2025XXXX.91.0-live-iso.x86_64.iso

Note: When future editions/spins get moved to kiwi they will inherit this change.

:link: Feedback

:link: Benefit to Fedora

EROFS is considerably more actively developed than SquashFS, and offers more modern file system features that can be utilized in the future.

:link: Scope

  • Proposal owners:

  • Other developers:

  • Release engineering: #12520

  • Policies and guidelines: N/A (not needed for this Change)

  • Trademark approval: N/A (not needed for this Change)

  • Alignment with the Fedora Strategy: N/A (not needed for this Change)

:link: Upgrade/compatibility impact

There should be no visible impact. This only affects live media used to make fresh installations.

:link: How To Test

Once the change is applied, users can grab nightly Fedora live images that fall in scope (such as the KDE and LXQt images) to test it. Simply booting the images in a VM and installing the environment is a sufficient test.

For Fedora CoreOS, nightly rawhide installation media can be used to perform an install.

:link: User Experience

There should be no visible impact to users. Live installations continue to work as expected, and live environments may be slightly faster.

:link: Dependencies

N/A. Everything has been in place and supported since Fedora Linux 41.

:link: Contingency Plan

  • Contingency mechanism: Revert back to SquashFS.
  • Contingency deadline: Final Freeze.
  • Blocks release? Yes.

:link: Documentation

N/A (not a System Wide Change)

:link: Release Notes

Fedora Linux live environments now use the Enhanced Read-Only FileSystem (EROFS), a modern, feature-rich read-only filesystem.

Last edited by @amoloney 2025-01-15T22:55:46Z

Last edited by @amoloney 2025-01-15T22:55:46Z

1 Like

How do you feel about the proposal as written?

  • Strongly in favor
  • In favor, with reservations
  • Neutral
  • Opposed, but could be convinced
  • Strongly opposed
0 voters

If you are in favor but have reservations, or are opposed but something could change your mind, please explain in a reply.

We want everyone to be heard, but many posts repeating the same thing actually makes that harder. If you have something new to say, please say it. If, instead, you find someone has already covered what you’d like to express, please simply give that post a :heart: instead of reiterating. You can even do this by email, by replying with the heart emoji or just “+1”. This will make long topics easier to follow.

Please note that this is an advisory “straw poll” meant to gauge sentiment. It isn’t a vote or a scientific survey. See About the Change Proposals category for more about the Change Process and moderation policy.

Is this just a swap-in of a similar-but-newer tech for another? Would this allow us to finally get rid of the the media-check step?

Just swaps over to an equivalent technology, it doesn’t really let us get rid of the mediacheck step, as that’s a whole image integrity check thing.

1 Like

If every block has integrity, so does the whole image, right? :slight_smile:

But really, I should generalize my question. The proposal says:

more modern file system features that can be utilized in the future.

What might some of those be, and what might they be used for?

1 Like

It is unclear to me whether I should have commented here, rather than on the devel@lists.fedoraproject.org mailing list. In case my comments don’t count there, I’ll cut and paste it here, because I do want my views to be known.

Neal Gompa wrote:

It’s mostly that it behaves more like a real filesystem. It uses
extents, supports inline compression, DAX, and other features that you
expect from advanced filesystems.

That’s just word salad. Squashfs is a real filesystem, with real filesystems techniques: inodes, extents, inline compression (I think you mean inline decompression here). In fact it has the features that EROFS has, and it has had them for years.

There’s one thing the EROFS developers never talk about, and that’s metadata compression. EROFS doesn’t do it, but Squashfs does, and it has done since 2002.

In a supposedly technical proposal, I would have expected to see at the least:

  1. Motivations and reasons for why it is necessary, including:
    1.1 The current pain points with Squashfs. Is there a problem with compression size? Speed? Reliability?
    1.2 The techniques that EROFS has that solves these problems.
    1.3 Real world tests and benchmarking results that backup/illustrate the problems and the solution.

  2. A risk analysis of the possible consequences:
    2.1 In real-world situations is EROFS less or more stable than Squashfs with corrupted images.
    2.2 Have you tested? What were the results? Undetected file corruption?, kernel OOPses?, lockup?

You say you have worked with the EROFS developers to improve compression. I see no dialogue or discussion with me, or on Squashfs websites/mailinglists where you reached out to voice your issues/concerns with Squashfs and have them addressed. This strikes me as putting the cart before the horse, and suggests the motivation is not driven by technical problems with Squashfs, but because EROFS is seen as newer or there are politics involved, with Squashfs being sucked into the Flatpak/Snap standoff. I have absolutely nothing to do with Canonical or Snaps. If you don’t want people to run Snaps on Fedora, don’t do it via the backdoor of deprecating and then disabling Squashfs (I notice Squashfs is deprecated on RHEL 10 and is to be removed for no discernible technical reason).

There’s a lot more detail about it on the EROFS website:
🔎 Overview — EROFS filesystem project

If you believe everything that the EROFS developers claim then I’ve got a bridge to sell you (I’ve got a bridge to sell you: The conman who sold the Tower of London, London Bridge and a Piccadilly mansion to gullible American tourists - MyLondon).

The tests are all done in way to flater EROFS and put Squashfs in the worst light. For example see Deduplication among versions — EROFS filesystem project. They deliberately disable Squashfs metadata compression because EROFS doesn’t support it, and use 128K blocks despite Squashfs supporting 1Mbyte blocks (look for the line [1] SquashFS uses –b 131072 by default, -noI will disable its metadata compression.). By doing this they deliberately reduce the compression and speed of Squashfs in their tests. This IMHO is biased and unethical. But that is how it is.

So far this proposal looks like something from the EROFS marketing department and is remarkably light in motivations and technical detail.

Phillip


Dr. Phillip Lougher (Squashfs author and maintainer)

3 Likes

The proposers say they have been working with the EROFS developers to improve compression and speed. In fact the relevant discussion is here https://github.com/coreos/fedora-cor…er/issues/1852

Latest results are here https://github.com/coreos/fedora-cor…ent-2578574295

No wonder they don’t mention any of this in this proposal. After weeks of effort by the EROFS developers, Squashfs still has better compression, and is much faster at generating the filesystem, and extracting them.

So they’ve bent over backwards to switch to a filesystem which is worse in all respects. So why are they proposing doing it. This is politics dressed up as something else. Shame on you.

I already responded to this on the devel thread on the mailing list, but I guess I will respond here too.

As I am one of the snap stack maintainers in Fedora and EPEL, I do not care about this fight. I do know that the OCI world is adopting EROFS, and Flatpak is getting sucked into that. I couldn’t care less about the Flatpak vs Snap thing, as I prefer to use neither. I use some Flatpaks and snaps as I am required to, but otherwise prefer regular packaging systems like RPM.

The initial driver for exploring the switch was because RHEL 10 is dropping SquashFS support. My guess about that is that since the OCI world and Red Hat’s bootc are using EROFS (through composefs), they want to consolidate things around it. I had also been previously concerned about the long dry spell around squashfs-tools maintenance, but that situation has improved in recent years.

No one is proposing to drop SquashFS in Fedora, certainly not me.

I am aware of the bias. My tests were done with much more equivalent settings. Kiwi built images have used 1MB blocks for quite some time for squashfs, and I force the same block size for EROFS in our kiwi definitions. Initially, EROFS was >50% larger than SquashFS and getting it to comparable size would blow out the image build time by 5x. The EROFS developers noticed my comparisons and worked to improve both compression and time to creation to make it possible to match SquashFS.

As one of the upstream developers for the kiwi image build tool, I have no intention of removing SquashFS support. It’s perfectly fine and usable. My only issue with it is that using unsquashfs to install live environments causes the environment to freeze, which is not great. This doesn’t affect Anaconda, but it does affect Calamares (which Fedora itself does not use).

And EROFS does have a couple of downsides right now for the CoreOS folks (notably the lack of single file extract), but they want to switch to it anyway for technology consolidation reasons (as I mentioned earlier).

This change proposal has now been submitted to FESCo with ticket #3356 for voting.

To find out more, please visit our Changes Policy documentation.