Quality Assurance for Silverblue

Yesterday I rebased to Silverblue 36; really smooth process, would recommend (excepting a bug in GNOME Software which completely breaks it - can’t win them all :eyes:). I actually did this on Tuesday, and something was wrong so I just did a rpm-ostree rollback and returned to a working desktop - all while on a work call :grin: I’m really bought in to Silverblue at this point, as a concept.

However, none of the Flatpak’d versions of pre-installed packages were updated ready for release. This means that I got half-G42, half-G41. I raised this on the issue tracker, where it turns out that the person who does this is on long-term leave and it hasn’t been picked up by anyone else.

This has made me realise that although I love using Silverblue, up until this point I was assuming it gets the same Quality Assurance that Workstation does. I essentially have half a F36 upgrade here :sweat_smile: My expectation was that this would be picked up on and, if not blocked F36 (which makes sense) then blocked Silverblue 36. But this wasn’t the case.

Silverblue cannot be expected to reach parity with Workstation on everything (yet), but where it does have parity it’s important that this is maintained. If people have a poor experience with Silverblue “the product” then it will switch them off Silverblue “the concept”, which is bad for Fedora and bad for the Linux Desktop ecosystem.

So, a question: how does QA for Silverblue currently happen, is it possible to get these sorts of issues to be considered blockers (for Silverblue if not for Fedora), and how can we get to that place?

I’d also like to help with this :slight_smile: but I’ve never really been able to find who/where the Team Silverblue are :sweat_smile: I’m on the Silverblue Matrix channel and I’ve been reading through this Discourse but both seem to be largely user forums.

3 Likes

Sorry you hit that. To answer your questions, we can’t partially block a release. It’s all or nothing (we’d like to make that possible in the future, but there’s a lot of release engineering work to make that happen). So we define release-blocking deliverables. When the Silverblue team thinks they’re ready, they can propose to add Silverblue to that list.

In the meantime, since Silverblue isn’t a release-blocking deliverable, the QA team doesn’t put a lot of direct effort into it. There are occasional test days, although I’m not sure one was performed this release cycle. If you’d like to join the QA team, I’m sure they’d love to have some help in running tests on Silverblue pre-release. Bugs you find won’t block the release (unless they also affect Workstation or other blocking deliverables), but they’re more likely to get fixed the earlier we find them.

1 Like

I totally agree to what Ben said. Also, way back in the day, we used to have these Silverblue Test Day. I run most of these test days (from QA side of things) and I would love to have Silverblue added to the list of test days this release. In the past, I have tried to get in touch with some of Silverblue folks and haven’t heard much back from them. There are a bunch of more things, as QA team we want to run to increase testing footprint through various archs and DE.

3 Likes

I’ve never participated in a test day but looking it over it seems pretty reasonable. I use silver blue as my daily driver and would love to help.

1 Like

What do you think would lead to best bang-for-the-buck? Is there a job description type thing that we could link people to? Don’t have time to do this myself but would love to help find someone who might be interested in getting involved.

1 Like

Separately from the topic of QA before release, can we make the known issues more visible to users checking out SB? In recent hours I’ve seen a couple of folks on the matrix chat and the forums trying SB36 and not being aware of these know issues - which is a real shame for them.

On the announcement for SB36 ( from https://silverblue.fedoraproject.org/ ), there is a section:

As always, before installing or upgrading your system to Silverblue 36, make sure to check the latest known issues.

That “issues” link goes to a special “Common Issues” category in ask.fedoraproject.org, which lists general Fedora 36 issues and doesn’t currently include these Silverblue specific issues.

Can the release announcement be amended to contain links to the Silverblue issue tracker? Or should we start proposing SB specific issues be included in the Common Issues category?

2 Likes

I’ve gone ahead and proposed a Common Issue for the GNOME software update problem: GNOME Software Updates tab: “Operating System Updates Unavailable: your operating system is no longer supported.” - Ask Fedora

3 Likes

The release validation and updates testing sections of the Join QA link I posted are the closest. Updating at Beta and using that is a good way. People who are interested in getting started should contact the QA team.

2 Likes

Thanks for the responses everyone. I am currently working through the “Joining the Package Maintainers” steps (although they seem to only apply to RPMs) and I’m looking at joining the QA team as well to contribute to testing.

Maybe a stupid question: who are “the Silverblue team”? Is there a working group, or is it part of another working group, or something?

Yes, we can do that. Website is here: GitHub - fedora-silverblue/silverblue-site: website for Fedora Silverblue

2 Likes

@pauldoo
I’ve made an issue and pr to add the silverblue issue tracker link to the release announcement. Maybe it need some work before being actually merged but just to let you know I’ve picked it up.