Update: This proposal was rejected. Instead, all hardware compatibility lists for aarch64 were dropped. See details in this post.
Hi everyone, this is a proposed change for Fedora Release Criteria. It is related to a wide set of changes that Fedora Quality team need to perform this cycle, which are summarized here, so please feel free to read that for more background information and general overview, thanks.
Proposal: Reduce the release-blocking ARM hardware definition to the Raspberry Pi 4 device family, and apply this also to Fedora IoT, assuming it remains an edition. Currently the official definition in the Basic release criteria points to ARM/Supported_Platforms, which has not been updated since 2020. Confusingly, ARM#Supported_Hardware_and_Devices also exists. There is also a “Supported IoT devices” entry which links to Reference Platforms, so we have three lists of supported ARM hardware. The main purpose of this proposal is to reduce the burden of testing, but it would also clarify that confusion and remove the burden of reconciling and updating all these lists. Under the proposal there would be one blocking hardware family for all of Fedora - Raspberry Pi 4. We would envisage adding Pi 5 whenever sufficient support for it is in place. As part of the proposal, the Hardware Tests table at Template:General_IoT_test_matrix would be reduced to only RPi 4.
Justification: as explained in the summary article above, the Red Hat Fedora quality team is now smaller in size. Manual ARM hardware testing is a considerable burden on the team (and anyone else who tries to do it), due to the flaky behaviour and slow performance of SBCs in general. The ‘supported hardware’ definitions for ARM and the IoT test platform list don’t seem to be updated or refined very often, and we do not have all the hardware available in any case.
In practice it seems like the vast majority of real-world use of Fedora on ARM is with Raspberry Pi family devices. Making Raspberry Pi 4 the only release-blocking hardware will go a long way to clarifying the situation, reducing the difficulty of blocker bug review, and reducing the testing burden to a more manageable level.
I would not be comfortable with this given the lack of affordable systems of this type. As community SIGs are going to be asked to do their own ARM testing, SystemReady SR systems are going to be out of reach of nearly all of them.
Is there a common use case for virtualized ARM/IoT? I consider the major use case for ARM to be the “tinkerer/hobbyist” space, so playing with a Raspberry Pi for various uses, and the major use for IoT to control some small appliance, i.e. all of them bare-metal focused. Am I wrong?
At the same time, I think VMs are currently release-blocking even for ARM, because this basic criterion doesn’t have any architecture restrictions, so it should apply to all release-blocking images. @adamwill Am I reading that correctly?
Yeah, you are. As we do have substantial openQA testing on aarch64, it’s not really a concern - it’s pretty safe that we always work in virt, at least in virt as openQA uses it (qemu with virtio devices).
Right now, it shows me the 8G (ram) board costs ~$227, and the 32G variant is ~$327, plus shipping, and all the bits (nvme, case, psu, better GPU?, keyboard, mouse, etc) one needs to purchase if they don’t already own.
The solidrun honeycomb LX2, and AsRock Ampere Altra motherboard + 64 core CPU are also still available. Both of those are SystemReady certified as well, and while both of them are a few years old, the ampere in particular, is proper server class hardware, yet seems like a reasonable value considering how much new 64 core CPU’s sell for.
Both of them are less expensive compared to consumer class high end laptops or desktops, which presumably are ‘in reach’.
You see it from the OSS philosophy point of view, while we’re looking at it more from the practicality point of view. OSS philosophy is important for Fedora, and I have no objections to “we only want to support OSS-friendly aarch64 hardware” stance, if that’s what Fedora wants (let’s compare it with our x86_64 approach as well, though). However, this would bring even less practicality to our products, because the amount of users running those boards will be even lower (please correct me if I’m wrong).
We suggested RPi4 because we believe it’s the most used aarch64 board for Fedora (sadly, we have no data to back this up). If some other board or types of boards is more widespread, please help us figure it out, and we have no issues to adjust the proposal. (I was really hoping we would get a lot of feedback from the ARM SIG… I made sure to email the arm list. But I don’t even know who’s included in it.)
If we target a board which is not widespread, what purpose does it have to block on it? It makes no sense to me to block all Editions just because there’s an OSS-friendly board that few people use, and it has some issues. At that point, we could even go to blocking on ARM in virtual environment only, and it would have the same outside practicality, and more internal practicality (easier validation).
So if we need to narrow down the scope of supported boards (and the Quality team believes we need it), it makes sense to us to pick the board that most Fedora users will benefit from (through validation).
RPi4 is probably the most widespread one in the tinkerer market. @pboy would you have a guess what would be the most widespread board in the Server market?
Unfortunately, we have no data either. I think we should add this question to our next year user survey.
RPi4 seems to be most widely used model. But for server it is the least usable. It is incredible slow, no mass storage integrated, clumsy board design, etc.
The Pine64 RockPro64 board has earned a reputation for significantly better performance, reliability, and good driver support. And it is quite widespread, at least in my experience.
And despite the need to reduce testing efforts, we should not disregard the long-term perspective and the political and philosophical perspective of OSS. We will never be satisfied with RPi. It is too proprietary for that. If we are going to bow to the numbers, we should at least actively offer and promote one OSS friendly and more powerful alternative. The Pine64 would be a solid option. Radxa Pi5b as well (for servers, GUI still doesn’t work).
FESCo discussed this today, and decided in principle to drop the specific lists of supported ARM hardware and instead go with the same case-by-case approach we use for x86_64. I’ve posted Release criteria proposal: drop ARM and IoT hardware compatibility lists with the proposed criteria changes for that. I will also talk to the kde team and @jlinton about how we want to approach this on the validation matrix pages - still have specific expected tested platforms there (but update them), or just have a generic ‘platform testing’ matrix or suchlike, and have reporters list the platform they tested on as a result comment?