✅ Proposal: Limit release-blocking status of BIOS systems to just certain scenarios

Project Discussion
, , , , , , ,
1

:white_check_mark: Update: This proposal has now been put into action.

Hi everyone, this is a proposed change for Fedora Release Criteria. It is related to a wide set of changes that Fedora Quality team need to perform this cycle, which are summarized here, so please feel free to read that for more background information and general overview, thanks.

Proposal: Reduce BIOS-based systems release blocking status from covering all scenarios (on parity with UEFI) to just limited scenarios. The following would stay release-blocking in BIOS mode:

  • Installations of release-blocking desktops, Server and Everything images which use the default automatic partitioning layout to a single empty SATA or NVMe drive.
  • Installation of Server images which use the Blivet-GUI partitioner to set up software RAID 1 onto two local empty disks following the steps in Server documentation.
  • Cloud image boot in Amazon EC2 (no change).
  • CoreOS image boot in its blocking environments (no change).
  • System upgrades (no change).
  • OS and application functionality (no change).
  • Anaconda rescue mode (no change).
  • Both bare metal systems and virtual machines are covered in the cases above.

The following use cases would no longer be release-blocking in BIOS mode (but would be kept blocking in UEFI mode):

  • Any partitioning layouts not specified above.
  • Any storage device types not specified above.
  • The fallback video driver (available as the “basic graphics mode” from the install media).

Justification: Currently most of our installation tests (both manual and automated), especially related to partitioning, are duplicated for BIOS and UEFI. New features, like Anaconda’s webUI, also need to be tested in both modes, because the partitioning scheme fundamentally differs (biosboot vs ESP partitions) and configurations like multi-disk setups or RAIDs exercise different code paths. We need to save resources somewhere, and simplifying BIOS systems verification would help us a lot. The Quality team feels that BIOS-only systems (incapable of booting over UEFI) are now a rarity, surely more than a decade old, and in the past years we even see a trend of hardware (especially laptops) no longer even supporting CSM and depending on native UEFI only (which means it’s getting harder for us to even test BIOS). Under this proposal, we would still keep the BIOS mode blocking, but only in a very narrow, simple-to-test use case, to make sure BIOS support hasn’t been inadvertently broken. We would no longer block on all the different installation combinations, though.
Note: Cloud images were kept in the blocking path, because booting through Xen on EC2 is included in our current release criteria and Xen has BIOS boot as a requirement. CoreOS images kept the same status in order to be in sync with Cloud. Server images on SW RAID 1 were kept in the blocking path, because many data centers still support only BIOS boot and SW RAID1 is the prevalent use case.

4 Likes
2

While people might be installing the new releases on Fedora on old BIOS-only hardware, it seems unlikely that they would create a new complicated server install with fancy hardware on such an old system. If such systems are being used, then mostly likely they are upgraded, not installed afresh. So limiting the release-blocking status seems reasonable.

3

I maintain a set of - and still meet new (as in freshly obtained, not freshly produced) - BIOS-only laptops and PCs. And lot of them are alive and well.

As I understand it, this proposal only means shrinking of the release-blocking test coverage for such systems, nothing else. Which sounds reasonable :+1:

2 Likes
4

+1 from me, I agree with this proposal.

5

I oppose this proposal because it would prevent Nobara, bazzite OS, and March7th Vista from being installed in an MBR traditional BIOS environment.

1 Like
6

A post was merged into an existing topic: Fedora Quality scope reduction announcement and summary

7

This proposal is a change to what Fedora considers blocking for a release. This doesn’t mean they’re dropping support for the scenarios they’re dropping from blocker status here, it just means that they will be treated as standard bugs and not ultra-critical release blocking ones. It’s essentially just a change in what Fedora will be testing as far as I know. I don’t see how this proposal would possibly prevent those distros from being installed on an MBR BIOS-based PC.

In fact, it doesn’t even sound like they’re dropping the case you’re concerned about: Installing release-blocking desktops (i,e. Workstation and KDE) will still be on the test list, and it will still be release blocking if they fail. I would think that Workstation / KDE can be installed, a derivative distro surely can as well (or it’d be their issue, not Fedora’s)

7 Likes
8

+1, that is still pretty significant test coverage for an increasingly legacy boot method.

9

Thanks @rosaaeterna for a great explanation. Yes, we would still be blocking on a default install even on BIOS (but not on advanced partitioning setups).

Also, it’s a bit funny to use gaming-oriented downstream distros [1] as an argument here, because if there’s a PC segment that surely has the highest percentage of modern (meaning less than 10-15 years old) UEFI-capable hardware, it’s gaming :slight_smile:

[1] Except for March7th Vista, I have no idea what that is, and Google doesn’t know either.

Truth be told, while all of these proposals are written as proposals, the reality is that we simply don’t have the capacity to continue doing them (especially now that our team is half the size). If new features get added regularly (and they are), some features also need to get removed at a similar rate, and Fedora has been pretty bad at implementing this. That’s why for removal we’re trying to pick ideally old features that hopefully won’t affect much of our current audience.

10

March7th Vista operating system is my develop operating system

11

March7th Vista is based on Fedora Linux.

12

I think support for BIOs must be scaled back. Corner cases should be redefined and not tested for

13

Server Edition Working Group discusses this proposal on the Aug 30 meeting.

“Server WG agrees with the proposal on the condition that a simple Software Raid installation in accordance with Server documentation is added with blocking status. (@pboy:fedora.im, 17:48:39)”

The reason is: According to our user poll, providing Web Service is by far the most frequent use by our users (in their mix of services). Operating a publicly accessible server on premise is generally not practical, as Internet access is too limited. Instead, one basis is to rent a root server (bare metal) in a data center, which several providers offer at a reasonable price (compared to various cloud offerings). The data center management and administration system of many providers (still) requires BiosBoot, so that newer servers are also set up for this. So this is not about old technology that has more or less become obsolete, but about current usage. At the same time, the installation of two disks running in an SW Raid 1 is the rule.

This configuration has been tested (manually) by Server WG for years and is specified in detail in the documentation. It therefore does not put any strain on the QA team. The critical difference to the other installation procedures is in the handling of the BiosBOOT partition, which must be installed in parallel on the disks involved. This was optimized by the Anaconda team several years ago and simply needs to be kept working as is.

A reference to a later fix is not applicable in this case, as the installation medium is not being updated. A corresponding server installation would therefore be impossible for an entire release cycle. That is not acceptable, for Server.

14

Thanks, Peter.

Can you please add a link to that documentation, so that I can see precisely what “simple Software Raid installation” means? What layout, how to create it, etc.

This is interesting. To my knowledge, the GTK UI of Anaconda doesn’t offer having RAID on top of the whole disk, just on top of a single partition. So I thought the use case of having two full disks (including biosboot) on RAID1 is impossible to configure in GTK. This is changed in WebUI, where it is now possible. Can you clarify/share exact installation steps/guide?

I’d love to agree with this, but I can’t :slight_smile: I spent full two weeks (or maybe more) in Fedora 42 testing just SWRAID in Anaconda WebUI, because it was heavily broken. That currently doesn’t affect Server, which uses GTK UI, but most likely will in the future. I appreciate your manual testing (can we please make it show up in our release validation matrices?), but we spend a lot of time on it as well, debugging failures and doing our tests. Especially when the GTK UI has two custom storage editors (and a third one for WebUI). And it’s those custom editors that we wanted to avoid for BIOS in the future.

Once we clarify the exact use cases that Server deeply cares about (for BIOS), perhaps a solution would be to create test cases exactly matching these use cases, and then Server people could help us validate those during release. Honestly, we would most probably use automation to run those, so it’s not much work once they’re written, but the continuing work is in debugging failures, which is exactly where we would appreciate Server WG’s help. Does that sound reasonable?

15

The upcoming published documentation is currently on staging: Exkurs: Configuring a software RAID upon interactive installation. The official currently published version only covers custom mode, which is insufficient. Some additional information you find in the Installation Introduction and in the Fedora Server interactive local installation. Some more information about Fedora on rented server hardware you find here (comments / corrections much appreciated).

Currently, the doc covers the general case of SW raid. Release blocking should only apply in the case of two HD Raid 1 SW raids.

We use SW raid of partitions, not the whole disk. We haven’t investigated which method is better under which circumstances. Only one was available. Off the top of my head, I would say that if work needs to be reduced, then we can do without a raid of entire disks.

What I meant is, UI-wise, the handling of partition creation etc. is the same. Anaconda has just to allow and enforce the installation of biosboot on all included disk, as it does for the boot partition. That was added when we made GPT the default.

Thank you for this. But currently, it is a special situation. It is in development of a complete new functionality. Of course, there is a huge amount of testing required. But when the development is done, there will be much less.

Yeah, we had a discussion with Adam to adjust the server specific page to make it easier to find the locations of the manual tests. Currently, it costs us nearly as much of time to find it as to perform the tests.

I don’t think that’s a good idea. For Server, we need it, indispensable. Probably not both of the custom editors, but at least the advanced one.

Automation is much preferable, of course. On the other hand, we have to do the manual testing if the installation anyway to check the validity and accuracy of our documentation. And of course we are happy to help with testing. So far, no version has been available for us to test the server-related functions – or at least we haven’t been aware of one. And testing hardware is currently my domain. I am the only one who has the necessary “superfluous” hardware available.

16

This is extremely helpful, thanks. It’s very detailed, I understand the required workflow much better now.

I would like to link to it from our test cases (and maybe even the release criterion), once the staging version (including the blivet-gui section) is officially published. Any ETA for that?

Yeah, I understand it now after reading the documentation.

Ok. If I want to specify that “Server installation on two disks in SW RAID 1 is also blocking”, I have basically three options how to define the blocking workflow:
a) say that all advanced editors are blocking (that would mean Custom and Blivet-GUI at present)
b) pick one of the editors, so either Custom or Blivet-GUI, and say that that one is blocking and must always work
c) say that at least one editor must work at all times. It doesn’t matter which one, but at least one.

Obviously, a) requires the most QA time, so b) or c) is preferable for us. Peter, do you have any opinion on these options?

The are wiki redirects that you can bookmark (just make sure to bookmark the redirect before it redirects) that point to the latest results page. For Server, that would be:

You can also subscribe or look at archives of test-announce which contain links to the latest results.

And there are test statistics which display how long it was since a particular test case was last tested.

We will definitely need to make this more approachable and discoverable if we want more participation from individual teams. We will try to improve this.

Sorry, I don’t understand what you mean by “version”, can you explain? Thanks.

Some of the tests require bare metal, yes. But many of them (including SW RAID testing) can easily be done with just a virtual machine.

17

I just updated the main repo so the extended article should be visable in about 10 mins past the next hour.

The Server WG opted for option b) and the Blivet-GUI. Just in case the advanced GUI doesn’t work they can use Blivet and achieve anything they could with the Advanced GUI.

!agreed We agree to limit the blocking bug property for BiosBoot to the default installation and the installation of a 2 disk SW raid1 installation useing the advanced blivet GUI

I think we found an excellent solution to keep important properties of Server and to reduce the workload of the QA team.

And thanks for the links! I’ll put them into our tracking issue as a template to make it easier for us to enter the test results.

The VM provides a reduced and standardized runtime environment. On rfeal hardware, things too often fail.

By the way, to get rid of BiosBoot we should modify the default LibVirt VM to UEFI and GPT, right?

18

Hey @kparal - thanks for looping us in. We discussed this in our community meeting today.

AGREED: We will respond in the discussion forum thread on this topic.  Generally we don't
see an issue with Fedora QA dropping manual testing of CoreOS+BIOS because we have
CI that covers this use case and a once a Fedora cycle test day that covers manual testing for it.

Hopefully if we do find issues in BIOS boot we can get them fixed, but we certainly don’t expect Fedora QA to manually test CoreOS BIOS booting.

1 Like
19

During the F43 test week, I will perform the Fedora CoreOS test case for a Bare Metal installation on an old BIOS-only machine.

1 Like
20

Thanks! I’ll reword the proposal shortly.

I believe we currently block only on the default way a VM is created in tools like virt-manager and gnome-boxes, which is BIOS, and not block on customizations like switching it to UEFI. So it’s not same case as with bare metal, where we so far have blocked on both. Therefore switching VMs to UEFI by default is not our current priority, we just flow with the upstream decision.

Thanks for update. Just to make sure there’s no misunderstanding, it was about dropping the release-blocking status (i.e. relevant bugs will stop the whole release of next Fedora, until fixed) of BIOS deployments of CoreOS. We’re trying to limit the scope of BIOS blocking path because of potential issues that can arise and we need to deal with, it’s not just about saving time on manual testing. (The CoreOS team has a very strong test process, and we’re very happy with it, we don’t need to perform much CoreOS testing on our end).

We envisioned a very limited scenarios where BIOS path is release-blocking, and as you can see above, teams like Cloud and Server then identified limited areas where BIOS boot is still critical for them (e.g. Xen for Cloud), so we’re accommodating the proposal. If CoreOS team haven’t found cases where BIOS boot is a critical functionality for you, that’s great, and we can move forward with this proposal.