Hello all,
a couple of months ago I submitted a proposal for EXT4 mount option/policy on LKML:
this is a version/upgrade from the basic idea formetly named “rotalloc”:
screenshots, patch, additional information on the related github repo:
I’m here trying to find out if there is a potential userbase for such option?
thanks
Mario Lohajner
Can you describer what the use case is for your proposed option?
Initially, this project was motivated by reports from users who observed that some SSD/NVMe devices do not always handle highly localized allocation patterns particularly well. Early on, I associated this mostly with wear leveling, and I admit that wear leveling was part of the original motivation. However, during development it became clear that the allocation policy has potential benefits beyond that.
The intended use cases are:
-Filesystems where performance degradation appears to correlate with allocation concentration near specific LBA ranges.
-Workloads with heavy overwrite cycles, such as scratch disks, cache volumes, temporary build directories, VM images, and similar high-churn storage.
-Users who believe the filesystem should actively help avoid allocation hotspotting instead of relying entirely on device firmware to manage it.
The core idea is relatively simple:
-Rotate the starting block group for new allocations across per-CPU allocation zones.
-Use stream allocation consistently, encouraging files to be placed sequentially while distributing new allocations more evenly across block groups.
-Implement the policy as a separate allocation path, avoiding additional branches and complexity in the existing allocator.
-Leave the on-disk format unchanged.
-Allow the rotating allocator to evolve independently while keeping the existing allocator stable.
-Reduce repeated allocation activity within the same filesystem regions and naturally discourage allocation hotspotting.
Ted repeatedly asked me to identify the workload that would justify such an option. My goal was to demonstrate classes of workloads where this allocation policy may be beneficial, rather than to present a specific enterprise deployment that already depends on it.
To be completely transparent, I do not currently have a large production user or company requesting this mount option. The project originated from observed behavior, user reports, experimentation, and an attempt to determine whether an alternative allocation strategy could provide measurable benefits for certain workload patterns.
While the original motivation involved allocation distribution, the test results suggest there are additional benefits. In particular, some high-concurrency tests showed significantly improved tail latencies compared to the standard allocator, which is why I believe the idea deserves further investigation beyond the original wear-leveling discussion.
Results are available here:
https://github.com/mlohajner/RRALLOC/tree/main/5)%20high%20concurency%20tests
The repository also contains implementation details, design notes, and benchmark descriptions. The included LKML discussion is provided as additional context, documenting the technical arguments, questions, and concerns raised during the upstream review process.
But do these users know that the firmware is bad?
Sounds like you are in arms race with ssd firmware.
If the firmware does perfect wear levelling are these changes unnecessary?
Well, I’m not saying that device firmware is bad, nor am I presenting rralloc as a workaround for firmware limitations or a wear-leveling fix. In fact, I think it is useful to step back from the wear-leveling discussion for a moment.
What rralloc provides is an alternative allocation policy. The goal is to avoid repeatedly concentrating new allocations in the same filesystem regions by rotating allocation starting points and distributing allocation activity across block groups. In practice, this reduces allocation hotspotting and naturally avoids repeatedly allocating the same areas over and over again.
The policy also maintains sequential allocation patterns, so the intent is not simply to spread data around the device, but to do so while preserving good file layout characteristics and avoiding unnecessary fragmentation.
Real world test, allocation maps and direct comparison to default alocator behavior is here:
https://github.com/mlohajner/RRALLOC/tree/main/3)%20proof%20of%20concept
https://github.com/mlohajner/RRALLOC/tree/main/7)%20real%20world%20test
If you look at the test results and summaries, the observed effects appear to go beyond the allocation geometry itself. Some workloads, particularly highly parallel allocation workloads, showed improved tail latencies and throughput characteristics. My current hypothesis is that these improvements are related to allocation distribution and reduced contention, but that deserves further investigation and independent confirmation.
https://github.com/mlohajner/RRALLOC/tree/main/4)%20sythetic%20tests
https://github.com/mlohajner/RRALLOC/tree/main/5)%20high%20concurency%20tests
Another important point is that rralloc is implemented as a completely separate allocation path. The existing allocator is left unchanged. The heuristics used by rralloc are intentionally different from those used by the default allocator, so keeping them isolated avoids adding complexity or overhead to the normal allocation path.
As a result, filesystems that do not enable the mount option are unaffected in any way. A user can choose to enable rralloc only on a specific ext4 filesystem where they believe the allocation policy may be beneficial, while all other filesystems continue to use the standard allocator unchanged.
Not on a SSD it will not work as described as the logical mapping is not the same as the physical storage mapping.
That mapping is part of wear leveling and and algorithm is the firmware as I understand it.
Wear leveling == hot spot avoidance.
Okay so easy to avoid of added.
When I refer to reducing allocation hotspotting, I’m referring to hotspotting within ext4’s allocation decisions (e.g. multiple writers converging on the same block groups and allocator state). Any effect on the physical media access pattern is indirect and depends on the underlying device and its FTL implementation. At the filesystem layer, the control we have is how we distribute allocations across the filesystem address space and block groups.
The proposal operates entirely at the filesystem allocation layer. Whether the underlying device is an HDD, SSD, NVMe device, RAID controller, DM target, loop device, or something else, ext4 still has to decide how logical block allocations are distributed.
The SSD’s FTL and wear-leveling logic may remap those LBAs internally, but that does not eliminate contention within the filesystem allocator itself. Since the FTL implementation is device-specific and unavailable to the filesystem, rralloc does not rely on any assumptions about its behavior.
The goal of rralloc is to distribute allocation activity across ext4 block groups, reducing allocator-side concentration and contention. The observed tail-latency and throughput improvements come from that behavior.
The claim is that rralloc changes ext4 allocation behavior in a way that can reduce allocator contention and improve performance for certain workloads, particularly those involving high-concurrency allocations.
Impact on wear leveling is an indirect side effect of the changed logical allocation pattern. The primary goal is improved allocation behavior and reduced allocator contention through more distributed allocation geometry. Additional benefits come from removing reliance on the hash array of global goals to keep indvidual file sequentiality/locality, but I would prefer not to go too deep into details here.
Allocation maps demonstrating the behavioral difference:
https://github.com/mlohajner/RRALLOC/tree/main/7)%20real%20world%20test/2)%20rralloc%20allocator
Original motivation and visualizations:
https://github.com/mlohajner/RRALLOC/tree/main/1)%20initial%20motivation
At a user-visible level, this alternative allocation policy can make NVMe/SSD storage behave in a way that better exposes its parallelism and low-latency characteristics. It is designed with SSD/NVMe-class devices in mind rather than HDD-era assumptions, which is why it is optional and can be enabled or disabled at mount time depending on workload and system characteristics.
The change is to prevent multiple writes from creating fragmented file?
Do you have benchmarks showing the improvements?
No offense, but most of your questions in this thread have an answer in GitHub - mlohajner/RRALLOC: Support for the round-robin allocation policy as a new mount option. Policy rotates the starting block group for new allocations. · GitHub, which was posted in the initial post. Did you go through those folders?
Sometimes it’s better to stay quiet and let people who understand ext4 and file system internals comment here.
I have some knowledge of these matters.
The OP could have pointed me to the background reading.
it’s actually there…
I just checked the links.
What is not addressed is a comparison to the default allocation algorithm or why this change is needed. Maybe I read too fast, but I’m not seeing answers to my questions.
Quick clarification on fragmentation and the comparison to the default allocator:
Intra-file locality (sequentiality) is preserved, stream allocation is used consistently, so blocks within a single file are still placed contiguously.
It changes where the starting block group is chosen for new allocations. Across the test runs, fragmentation with rralloc is equal to or lower than the default allocator, never worse.
The effect being targeted is allocator-side:
under concurrent write workloads, the default ext4 allocator tends to have multiple writers converge on the same block groups / allocator state, which creates contention.
rralloc rotates the starting block group per allocation (via per-CPU allocation zones), spreading that allocation activity more evenly across the filesystem’s block groups. That’s where the tail-latency differences in the high-concurrency benchmarks come from.
Allocation maps and direct side-by-side comparisons against the default allocator are here:
https://github.com/mlohajner/RRALLOC/tree/main/7)%20real%20world%20test
At this point I’m not proposing this as a replacement for the default allocator.
Currently, It’s an optional, mount-time policy, doesn’t touch the on-disk format, and is meant to be characterized further on wider userbase with modern SSD/NVMe storage.
Disabled by default, can be enabled or disabled per filesystem/mount without affecting anything else.