The Future of Bugzilla in Fedora

I posted this on the Fedora Development mailing list, in the thread RHEL moving to issues.redhat.com only long term, and am re-posting here for broader discussion.

Here’s what I’m thinking: we should plan to migrate to something else in the next three years. There are two reasons for this, and only one of them is that Red Hat is moving away. The second is that Bugzilla isn’t a great tool for what we need anyway.

On the first part: yes, there’s a long, slow sunset, but I think realistically we’re going to see business-side attention drop significantly, and we’ll have correspondingly worse and worse service. Right now, there are basically only two people keeping it all going, which is heroic. I don’t think it’s too much pulling-back-the-corporate-curtain to guess that if one or both get tired of that and decide to go start a yak farm, Red Hat won’t prioritize hiring backfill dedicated in the same way. We could ask CPE to keep it going, but… there’s so much more I’d like to ask CPE. (And non-CPE volunteers? There’s so much that’s more interesting!)

So to the second part: Bugzilla isn’t what we really need anyway.

I’m not saying Bugzilla has been terrible — it’s served us well, in fact. And I have personal fondness for it that, which I do not for, say, the wiki. :classic_smiley: Buzilla is it’s deeply integrated in a lot of our processes, and we’ve got a lot of automations and so on. It’s important. But… there’s a lot that could be better. I think specifically:

  1. It doesn’t serve as a single place to track everything. We’ve got stuff scattered around Bugzilla; issue trackers in Pagure GitLab, and GitHub; pull requests in all of those places; and more. Not to mention upstreams (and inconsistent approaches to tracking upstream issues). I’m not arguing that we need ALL things in one place, but it’s important that Bugzilla isn’t that now anyway.

  2. Bugzilla a terrible experience for end users. Usually it’s the Wrong Place. When a user has a problem, that may or may not be caused by a software defect. This is a whole topic of its own, but in short, it’s really better for most users to report problems at Ask Fedora, and then possibly after triage a bug should be filed and tracked somewhere.

    Many of our users are advanced and recognize real defects and file good reports, but this leads to even more frustration, because our response is inconsistent. Maybe that report should actually be upstream. Maybe it’s in a dependency package that’s really only loosely maintained. For whatever reason, a lot of perfectly good reports end up closed EOL, which is never a good outcome.

  3. We’re inconsistent with PRs vs issues, which is confusing and makes more duplication. I have seen cases where a packager complained that someone filed a PR that they never noticed, saying “this should be a bug so I’ll see it”, while others close bugs with “please send a PR”. We could make stronger statements about policies, but as long as these two things exist in separate places, that tension will keep coming back.

Plus plenty of minor things: Our workflow still is shoehorned around a bunch of RH-centric stuff (lots of fields, flags, and statuses that we don’t really use or want). It’s not great for the review workflow, or for some of the other things we’ve twisted it to. A component-centric approach makes it hard to track larger issues. The SCM workflow is … not good.

And I’m sure there’s more. But I’m not really here to complain about Bugzilla. It’s just actually not filling our needs. I therefore think that Jira / issues.redhat.com wouldn’t either — although it’s got a ton of features on top, it’s fundamentally the same kind of thing.

We need define exactly what we do need, and figure out how to get that, in a sustainable way going forward. And fortunately, we DO have a few years, so for once we could do this before it’s a crisis.

I think we should create a project to figure this out. In fact, I think it should be a Fedora Objective. But, it also shouldn’t be a completely blue-sky initiative — we should avoid trying to develop a new gigantic piece of software that we own. (See past results on that, sigh.)

We do have some pieces already in place that should be part of the foundation (or at least other metaphorical bricks in the construction):

  1. Ask Fedora is the place for users to report and discuss problems. This is our first-line of support, and it’s where we can do triage. Then, software defects may or may not be tracked relating to those conversations.

  2. Package-specific issues should be next to the PRs. Let’s enable issue tracking on src.fedoraproject.org (with whatever gitforge we end up with that on). I think this makes sense for initial package review too, although that would need some workflow changes.

  3. Bodhi and the CI results system and all of that. This is update-centric, but does also have a lot of issue-tracker-like characteristcs in the “karma” system, and it is inherently close to our release process. Maybe some of the process that currently goes through Bugzilla could move here.

  4. The Fedora account system and Fedora message bus, plus the packager dashboard, the (under significant update!) Fedora notifications system, and so on. We have a nice underlying infrastructure for tying things together, and we can do more integrations. (Look at how Copr uses Discourse, for example. Or the Blocker Bugs app and its connections with Pagure and, um, Bugzilla.)

Obviously that’s not nearly sufficient. But we should look at all of our needs around tracking issues, plans, projects, blockers, and defects; consider different packaging cases with various relationships with upstreams; and imagine ideal workflows for users, packagers, project managers, spin and edition teams, QA, and all the other stakeholders. The last time we did this was in 2013, so… it’s a good 10-years-later exercise! (Infrastructure Fedora bug tracker - Fedora Project Wiki)

Then, once we have a good shared understanding of what we want, start fitting pieces like the above into that picture, define the gaps, and then find exactly what we need to fill them.

10 Likes

As I said on the devel thread, part of my plan for this year is to begin the process of seeing what we need in a bug tracker so we can make informed decisions. I’m going to put together a survey so we can analyze the data instead of trying to glean comments from mailing list/Discussion threads. Expect to see that after the F36 GA.

5 Likes

Could we not just use the gitlab issues setup?

4 Likes

Maybe (probably?). First we need to have a solid understanding of what our requirements are.

4 Likes

FWIW, GNOME did the same “recently”, and Gitlab offers pro features for free, for open source projects.

I’m an adept user and an occasional contributor. I feel much more satisfied with GNOME flow now.

You might ask them about their experience on the matter. Learning from others’ success or failures is always better than learning from owns’.

4 Likes

To add more, you might ask Debian about their experience as well, since Debian is a distribution. They switched to GitLab a couple of years ago thanks to GitLab switching to a DCO, so they have a lot of experience with GitLab.

6 Likes

What happened to this discussion in the end?

1 Like

It is still ongoing, but has not been active lately. This was a topic at the recent Fedora Council Hackfest (this happened last week) and there are some communications to be sent to the community due from that discussion, but one of the main points is that bugzilla is not a platform the project should and will choose to maintain long term, so we do still need to figure out a bug tracking solution in the very near future.

As the projects operations architect & change wrangler, this will be something I will drive, in close collaboration with the main users of bugzilla currently. I’m going to take Ben’s suggestion as mentioned originally and survey our community.

I think having this information will help us choose the right set up, and coupling that with the outcomes of our git forge evaluation, it might make it more straightforward to see what might suit the requirements of our bug tracking needs too.

Working towards a solution for our bug tracking needs is squarely on my radar for the next few months, so expect to see this topic come up a little more frequently this year :slight_smile:

6 Likes

Kinda reviving this thread as there was a recent discussion.

I think it is pretty crazy how many issue trackers the small Fedora projects have, and for sure confusing.

  • packages: RH Bugzilla
  • Atomic Desktops (aka. OSTree SIG?): Gitlab.com (an entire repo just for the issuetracker), also has a list of all dedicated issue trackers
  • Silverblue: Github (again an entire repo just for the issues)
  • Kinoite: Pagure in one place with the other KDE issues
  • rpm-ostree: Github but under the CoreOS name
  • Sway Atomic: Gitlab.com
  • Budgie Atomic: Pagure
  • missing: a selfhosted Gitlab, the fedora pagure instance or a dedicated pagure one

This is very bad UX but I think it comes from personal preference of some devs for certain platforms. But it is extremely confusing.

Edit: thanks @siosm for the list of the dedicated issue trackers. This is already helping a lot

About Bugzilla, from someone who has filed tons of bugs, but never received, triaged or manages them:

  • it is very hard to find duplicate issues easily. I mainly use bookmarks to skip some “first steps” which also involves duplicate search (oops)
  • the UI is very messy and barely usable on the phone
  • it is hard to find the list of “bugs I have reported”
  • for sure there is a “CHANGEME: I dont know what component” category needed. Also a “not listed” subcategory, if a package entry is missing for example
  • you receive emails about your own comments

I dont think using the issue systems of the alternatives makes sense, as as far as I understood you would need tons of empty repos for each product.

I also prefer how much info a user can give, the tags, importance, etc. This is also possible with Github yaml templates but less transparent and unified (when clicking box y tag x is set automatically)

3 Likes

You missed one of the (perhaps) most important
trackers moving forward, which is the RH issue
tracker (Jira based, as I recall). Given that
Fedora ends up being highly linked to RH,
there seems to be no (reasonable) future that
for packages that end up being in both RH
and Fedora where the jiri tracking system
will not be a place Fedora contributors (and
sometimes users) will end up having to
traverse at least some of the time for the full
picture of the packages and issues.

Please make it easier to report bugs :party:

1 Like

7 Likes

If it makes it easier, I’m for it !

@boredsquirrel, not if you configure Log in to Red Hat Bugzilla correctly - I don’t:

This is true for all Bugzilla instances, although the granularity available is dependent upon the plugins/customisations present, and more frequently, the version in use.

…It’s not Trac.


Despite how much I love using GitLab, I once would have insisted that this wouldn’t be sufficient due to how utterly important issue correlation is.

However, with the recent implementation of doc/user/project/issues/related_issues.md · 1139c65323bfa1638278d6976fb504b9555ef997 · GitLab.org / GitLab · GitLab, even transferring the current Bugzilla database without data loss would be surprisingly feasible.

Though, the lack of Have a visual issue dependency graph (#273597) · Issues · GitLab.org / GitLab · GitLab is problematic. I remember how damn useful it was in Phabricator (and Bugzilla does actually provide this functionality, but it’s as hidden as most functionality is in Bugzilla).

GitHub is evidently insufficient for myriad reasons (who wants vendor lock-in to a proprietary provider?) but especially because it doesn’t provide correlation in an accessible manner yet, per Add relation between issues · community · Discussion #4928 · GitHub.

Indeed. I don’t think zealous to predict that using a paid-for and proprietary platform would be problematic in the long-term.

1 Like