Fedora Badges: long term maintenance discussion🙇

[siddharthvipul1 (Vipul Siddharth (He/Him))]

Hey Folks,

Fedora Badges is one of the most loved projects in our community but it’s in a bad state right now. There are multiple problems with it ranging from outdated compatibility with our authentication system and messaging bus, to it using a not-so-friendly framework for community members (Pyramid). All of these combined make it a tricky application to approach and fix.

The Community Platform Engineering team have been discussing in recent weeks what would be the best solution to fix badges, whether a rewrite of the application would be best or whether a refactoring is the way to go. The problem though is that unfortunately Fedora Badges doesn’t fit exactly into the CPE team mission, but we do understand its importance in the Fedora community.

We felt that the best course of action to make Badges better for everyone is to try to make space for a project ‘team’ of sorts that is made up of folks from the CPE team and Fedora community volunteers who would work on developing a solution for the Badges service together, from ideation to delivery. But first we need to assemble the Avengers Team and some terms and a proposal of who will take on what work needs to be agreed on.

While the Badges application is not within the CPE teams remit to maintain, the CPE team would like to offer to:

• Collaborate with the community maintainers to investigate the best course of action to make the Badges service more stable and maintainable
• Pair with the community maintainers to assist them in rewiting the application
• Assist them while integrating the newer service/version with the rest of the Fedora projects apps.
• Provide power and ping to the service
• Help with writing good documentation for troubleshooting and a standard operating proceedure for the service to make sure its easier for people to understand how it works and contribute to in the future

On the community maintainer(s) side, we would like them to:

• Contribute to the writing of the new app
• Keeping up to date with Fedora’s application integration
• Updating/resolving CVEs and dependencies change
• Responding to issues/PRs opened by community members.

So before our team can move ahead, we would like to reach out to the Fedora community to ask for a maintainer(s) for the Badges application who will be able to work (part time is fine) with the CPE team in developing a better Bagdes for all!
If you are interested in working with us to improve the Badges application and become its maintainer, please respond here before 2022-09-14T18:30:00Z.

[andilinux (Andi Artz)]

Hey @siddharthvipul1

Great news!

I am interested to help u out as one of some maintainers Badges application.

v/r

Andi/andilinux

[steiner (Daimar Stein)]

Great news indeed, love to see Badges finally getting some love again, it might not look a lot like it, but I always found it to be a great way to motivate new and long time users alike to contribute to the community and participate in its events!

I wish I had the technical skills to help with that.

[mattia (Mattia Verga)]

Since Bodhi also is based on Pyramid framework, maybe the new maintainer and the CPE team can find a way to share some code between the two apps? The Bodhi interface to the authentication system has just been be reworked by abombard; the messaging bus interface was already been migrated to fedora-messaging long time ago.

[mattdm (Matthew Miller)]

See also:

Fedora-Council/tickets ticket #400: Fedora Badges project need some help

[erolkeskin]

Hey @siddharthvipul1

While i was in fedora-infra meeting today, nirik suggested me to take a look at this topic. I would love to work on this project as one of the maintainers I’m actively contributing to the projects in fedora-infra team such as Anitya. And in order to understand the infrastructure behind Fedora, i joined the fi-apprentice group. I can actively work on Badges project.

[gui1ty (Penguinpee)]

I have been following the recent discussion from the back rows. I would very much like to help were I can. For example by going through open tickets. Sadly, the mailing list appears to be dead in the water. Only four posts this year so far.

Join Fedora Badges community :: Fedora Docs ← Sadly nothing there, yet. Time permitting I will dig deeper trying to understand the setup of the system and where I might be able to help out.

Above is what I posted in #fedora-badges about 8 hours ago in response to @riecatnor posting a link to this discussion and another one started by @bcotton.

Sadly, and I apologize for using this word a third time, I received no response. I think less resolute people would already have given up.

I don’t want to come across as all negative, but there are lessons to be learned here and simple actions to be taken.

Direct people to the right channels

The topic on the IRC channel has probably not been updated in some time. The link to Pagure is a good starting point, but the direct link to open a new ticket could be left out. Next is a link to a Telegram channel, which greets you with the following message:

Hi there, you have found the Fedora Badges Telegram Group, now retired. We are using IRC and Element/Matrix, please join us on one of those platforms!

So, basically, back to where you came from and where it was quiet to start with.

The last link brings you to Join Fedora Badges community :: Fedora Docs, which does have some helpful information, but it also prominently directs you back to the very quiet mailing list and the (a little less) quiet IRC channel.

At the bottom is the empty As a system administrator section, which is the section I pointed to in my IRC message and closes the circle for me.

Lesson: Make sure information is up to date in as many places as possible.

To end on a positive note: My offer posted on IRC (see the beginning of the post) still stands. I would very much like to help out. I have a background in system administration and some knowledge of scripting languages like Python. I would like to join @erolkeskin in giving Fedora Badges the love it deserves, making sure requests are responded to and badges added and updated in a timely fashion.

[mattdm (Matthew Miller)]

@penguinpee

Sadly, the mailing list appears to be dead in the water. Only four posts this year so far.

I think there’s a general problem with small-team mailing lists where they can easily turn into ghost towns like that. Since most of the discussion (that isn’t real-time in Matrix/IRC) seems to be happening here, we could consider closing the mailing list in favor of adding a badges tag to this site.

That doesn’t automatically make people show up, of course, but it would direct people to where the current activity is, and if there is a slowdown in the future, there are more mechanisms for people to notice it incidentally even if they aren’t directly following the tag.

[gui1ty (Penguinpee)]

@mattdm

I think there’s a general problem with small-team mailing lists where they can easily turn into ghost towns like that. Since most of the discussion (that isn’t real-time in Matrix/IRC) seems to be happening here, we could consider closing the mailing list in favor of adding a badges tag to this site.

I was thinking along the same lines and put it down as an item on the growing list of things to consider for bringing badges into shape again. I see some parallels here with respect to real time communication and choice of client. Since badges seem to attract more new skool users, for lack of a better term, we should probably focus communication on new skool channels.

My previous post may have come across as mostly negative. Therefore, I’m glad to report that things have gained some momentum. I discussed badges with @erolkeskin over the weekend and we both spent time getting to know badges better; the technical part, how it’s set up and where to start and whom to poke.

Meanwhile I had a quick chat with @riecatnor on IRC(for me)/Matrix(for her) and she’s helping me out getting things started again as well.

[erolkeskin]

I’m happy to inform you all that I’ve started to making a list regarding what should be done to get fedbadges project in a good shape again get fedbadges in a good shape again · Issue #90 · fedora-infra/fedbadges · GitHub .Thanks to Kevin, I have access rights to underlying platform, and been examining logs in order to gain insight about the current problems. Hoping to get start solving those issues in a matter of two days.

I would love to get everyone’s feedback regarding the issues revolving around badges project.

[mattdm (Matthew Miller)]

I know that @ryanlerch did some extensive investigating. Due to summer chaos — or, actually, winter chaos for him since he is Down Under — I didn’t sync up with him on this as I’d hoped to. But I’d like to make sure we don’t lose what he’s found.

[t0xic0der (আকাশদীপ ধর)]

Hi @mattdm, you should be able to find his progress on the investigations here Fedora Badges — ARC notes documentation (fedora-arc.readthedocs.io)

[gui1ty (Penguinpee)]

It seems we are making progress.

@mattdm I meant to ask you about the script you wrote for Magical Experimental Fedora Badges Topic! - Fedora Discussion Is that script available somewhere?

Kind of a redundant questions, I know, you being the leader of a FOSS project. So, I guess, I’m actually asking: Where can I find it?

[mattdm (Matthew Miller)]

@penguinpee

@mattdm I meant to ask you about the script you wrote for Magical Experimental Fedora Badges Topic! Is that script available somewhere?

Yes, of course. It’s at Tree - badgebot - Pagure.io. See badgemirror.py.

There’s also a webserver configuration on one of my personal systems which gets the webhook and executes the script. That’s pretty hacky and very specific to my ancient setup so I didn’t put it in the repo.

[erolkeskin]

Great, this document pretty much confirms what is written down on GitHub issue. As i can see from the logs of fedbadges, there are couple of issues more too, i’m planning to give an explanation about them once i’m sure. For example,
while it’s initializing fedmsg-hub,it tries to get messages between the last time it ran and now. This causes the systemd to restart service because of the holdoff time gets over. Unfortunately it goes to infinite loop after that, tries to get messages again, restarts again…

Good news is, current development state shows us that fedbadges is already ported to Python3 , for a short time we can use datanommer’s latest models and then we can replace datanommer with datagrepper client. Updating to latest datanommer models, makes that piece of code not needed anymore, so it will reduce the complexity: Tree - fedora-infra/ansible - Pagure.io

Also we are going to need to log the time it takes for awarding a badge. I do know that Tree - fedora-badges - Pagure.io kind of badges causes issues since it tries to get all records at once. As far as i can see from logs, this causes the memory to ran out of space. But i’m guessing that this is not the only badge that performs bad, so logging those times will be great opportunity to clearly see which badges performs poorly.

[erolkeskin]

I guess, i made an early call when i told that “already ported to Python3”. Unfortunately that’s not the case. The vast majority of source code is migrated to Python3 but there are some areas that needs polishment. Unfortunately due to the sudden stop of development, things seems like not in a working state on “develop” branch.

Current code in development branch tries to create those “Badges” with in “badgr-server” GitHub - fedora-infra/badgr-server: Open Badge issuing and management with Django. But it seems like this project is abandoned 2 years earlier, and never made it’s way to prod or staging environment.

So if we do want to have fedbadges in a working state as soon as possible, we should probably need to revert back to use “tahrir-api” instead of “badgrclient”. But if we will go with the “complete rewrite” option none of the things i said above matters.

[erolkeskin]

Well one more thing i want to add here is that, the current implementation is somewhat fragile on a point. Let’s see that with an example: fedbadges/consumers.py at develop · fedora-infra/fedbadges · GitHub
In this line, it states that we need to wait for a specified of time before processing the message to prevent race conditions with datanommer. But this is not a reliable solution since “sleeping” is not a long-term solution for race-conditions.

In the fedbadges, instead of listening every topic and every message from bus, we can listen just a one message, “datanommer.event_processed”.

This type of event would fire after datanommer ends processing any event/message. Since we do know that datanommer captures all possible messages, there will be a corresponding “datanommer.event_processed” for every possible event/message on the system.

This way we can easily prevent race conditions between fedbadges and datanommer.

[erolkeskin]

Another idea to have more consistency is if we are going to have separate repository for badge yaml definitions, it’s for the best to create a testing pipeline for that repository. This way we can be absolutely sure that created/edited yaml definitions works flawlessly with the current version of fedbadges.
Same goes for the fedbadges repository too. We can create a pipeline stage that checks the latest changes in the fedbadges project doesn’t introduce a breaking change with current yaml definitions we have.
As far as i know this kind of cross-project changes can be tested with Zuul CI, and the good thing is we are already using it on another projects like Anitya