Concluding Fedora Badges ARC investigation

Project Discussion
, ,
#1

Hey everyone and wish you folks a very happy and prosperous new year!

As we head towards the end of the extensive ARC[1] investigation for Fedora Badges, the scope for the development and maintenance of the system has become clearer than ever before. Not only did we largely explore the system in its current state, but we also delved down the path of reworking it with two probable approaches - one, being the revitalization of the existing codebase[2] and the other, being the complete rewrite of the project from the ground up[3].

Observations

Allow me to go further in detail about our observations and the conclusions that we deducted from them.

  1. Fedora Badges is not a project, but rather a system constituting multiple projects like

    • Messages Consumer (or fedbadges[4]), which listens in to the (now, deprecated) fedmsg[5] bus to know about the events that lead to awarding contributors
    • Web Interface (or tahrir[6]), which allows users to view their badges and that of others, the leaderboards of the top awardees, award badges to other users etc. from a web browser.
    • Database library (or tahrir-api[7]), which allows interaction with the database model at ease by abstracting the database classes, for tasks like creating and awarding badges.
    • Collection of artworks and rules (or fedora-badges[8]), which houses all the related artworks and awarding conditions, tracks issues and discussions and has documentation.
    • Badge awarding scripts (or awarding cronjobs[9]), that are triggered not by the messages, but rather run periodically to check if the conditions for awarding have been met.

  2. A lot of these project repositories contain undeployed or incomplete feature implementations, do not have a functional testing environment and code compliance checks, use comparatively lesser-known frameworks and libraries, still rely on the (now EOL’d) Python 2, do not have a working project configuration and are inadequately documented.

  3. Efforts put into the codebase in its current state would be largely spent on cleaning/maintaining the aforementioned items and lesser on working on new features and designs, and with the lack of documentation that helps people do so, this becomes less incentivizing of an opportunity for the contributors to partake in maintaining the system.

More about the current implementation can be found here[10]. I highly recommend reading this through if you are interested to know how the existing system works.

Approaches recommended

  1. Revitalization of the existing codebase - This approach aims to utilize as much part of the existing codebase as possible and update them to fall in line with the more recent functional requirements and deployment environments. This would take significant effort as the documentation about the system in its current state and the dependencies it makes use of is scarcely available. I recommend reading the following writeups if you are interested to get involved in developing Fedora Badges.

    • ARC proposal - Revitalize Fedora Badges[11]

  2. Complete rewrite from the ground up - This approach aims to inherit the ways of the existing system and add meaningful changes to the workflow to meet the more recent functional requirements. This gives contributors a fair opportunity to partake in making choices, developing technologies and writing documentation for a system that they would maintain. I recommend reading the following writeups if you are interested to get involved in developing Fedora Badges.

    • ARC proposal - Rewrite Fedora Badges From The Ground Up[12]
    • Entities involved[13]
    • Interactions possible[14]
    • Technologies suggested[15]

Conclusion

Having explored both the options, we have come to the conclusion that it makes a lot more sense to inherit and expand upon the methodologies employed in the current codebase but implement the same from ground up in a new functional rewrite.

Proposed roadmap

  • Step 1 - Understand the project remit of the system to decide on the scope
  • Step 2 - Get started with creating/inheriting the database schema for users
  • Step 3 - Dump a snapshot of the existing database according to the new schema
  • Step 4 - Brainstorm and document the interactions expected with the system
  • Step 5 - Connect with the Fedora Design team to begin with the mockup designs
  • Step 6 - Develop and document the Messages Consumer alongside Accolades API
  • Step 7 - Test both against the dumped database and the existing Collection
  • Step 8 - Develop and document the Accolades CLI and test its functionality
  • Step 9 - Build Liberation frontend alongside the Fedora Websites & Apps Team
  • Step 10 - Document and Test the frontend against the above mentioned entities
  • Step 11 - Deploy and document the system on STAGING with the Fedora Infra team
  • Step 12 - Release Accolades CLI as a Python package on PyPI and as RPM
  • Step 13 - Announce public testing invites for the staging deployment
  • Step 14 - Announce deprecation status for the existing Fedora Badges system
  • Step 15 - Migrate database changes continually to keep up with the changes
  • Step 16 - Switch the environment to production and… PROFIT?

Of course, there are and will be more things to account for than what meets the eye but from a Mount Everest high perspective - these are the steps we would want to suggest in the roadmap.

Thanks

The in-depth analysis of the current state of the system could not have been possible without the foundational investigations done by @ryanlerch. I would also like to thank @zlopez, @lenkaseg and @patrikp from the Community Platform Engineering team[16], and @sayanchowdhury, @gui1ty, @bogomil and @ank22 who participated in the ARC investigation and provided their valuable insights on what this system can be developed into. I cannot appreciate @jflory7 and @eocarrol enough for kicking off the joint community effort for revamping the system.

Of course, last but not the least, thank you folks for reading this wall of text. Please feel free to reach out to me if you have any questions regarding this and I would be glad to help you.

Let us build Fedora Badges again. Together.

  1. https://fedora-arc.readthedocs.io/en/latest/index.html ↩︎

  2. https://fedora-arc.readthedocs.io/en/latest/badges/proposal_revitalize.html ↩︎

  3. https://fedora-arc.readthedocs.io/en/latest/badges/proposal_rewrite.html ↩︎

  4. ↩︎

  5. ↩︎

  6. ↩︎

  7. ↩︎

  8. https://pagure.io/fedora-badges ↩︎

  9. https://pagure.io/fedora-infra/ansible/blob/main/f/roles/badges/backend/files/cron ↩︎

  10. https://fedora-arc.readthedocs.io/en/latest/badges/current_implementation.html ↩︎

  11. https://fedora-arc.readthedocs.io/en/latest/badges/proposal_revitalize.html ↩︎

  12. https://fedora-arc.readthedocs.io/en/latest/badges/proposal_rewrite.html ↩︎

  13. https://fedora-arc.readthedocs.io/en/latest/badges/prop_rewrite_entities.html ↩︎

  14. https://fedora-arc.readthedocs.io/en/latest/badges/prop_rewrite_interactions.html ↩︎

  15. https://fedora-arc.readthedocs.io/en/latest/badges/prop_rewrite_technologies.html ↩︎

  16. ↩︎

8 Likes
Regular meeting cadence for Fedora Badges?
Structuring the Fedora Badges sub-teams
So we investigated Discourse to confirm if it is worthy of being the Fedora Badges frontend
#2

So then! It seems like a total rewrite is the most viable option. That is a huge undertaking… but I think Badges is one of the most treasured and loved applications in the Fedora Community. Even hearing in last month’s social call how so many people started their Fedora journeys with Badges was validating.

I didn’t see any specific calls-to-action that you might want from this conclusion. But perhaps a close review of the proposed technologies is helpful? I am seeing a mix of Python Flask and VueJS as the headline technologies the new stack is built on?

:tada: :100: :tada:

1 Like
#3

Great investigations, and thank you for organizing all this knowledge! :vulcan_salute: :100:

Would it use SPA Vue for all of it, or would there be static pages? If there are, it might be better to use something more geared towards static websites, like 11ty or Astro for those parts.

The component library is going to be Vuetify. The framework would be Nuxt?

#4

Please refer to the conclusions section[1] and the proposed roadmap section[2] of the document. This should get us an understanding of what needs to be done and when.

Yes. With the use of VueJS and NuxtJS by the Fedora Websites and Apps Team, even the Community Platform Engineering team has gotten started with using progressive frameworks like the same in projects like FMN[3]. I would like to derive as much part of the codebase as possible from the rewrite, as it winds up by Q2 2023 and base a couple of our internal entities (like Liberation frontend and Accolades API) atop it to give us a quick start.

  1. https://fedora-arc.readthedocs.io/en/latest/badges/index.html#conclusions ↩︎

  2. https://fedora-arc.readthedocs.io/en/latest/badges/index.html#proposed-roadmap ↩︎

  3. ↩︎

#5

SPA Vue only. Negative on the static pages side of things as there would be a lot of interactivity[1] on the Liberation entity (i.e. web frontend) that would necessitate the page items to be reactive to the changes made to/from the Accolades API entity (i.e. backend service).

Vuetify happens to be a suggestion but if required, we can always pivot to a friendlier and/or more extensible component library. Negative on the Nuxt as being a static site would limit the functionality of the Liberation entity to a great extent and would not fall in line with the interactions planned with the Accolades API entity.

  1. https://fedora-arc.readthedocs.io/en/latest/badges/prop_rewrite_interactions.html#la ↩︎

#6

Sorry, I should have been more specific. Is there a specific part of the proposal/conclusion you would like feedback on? Is it the conclusions and proposed roadmap sections? I’m trying to understand how we can best provide feedback and insight into this conclusion, since there is a lot to see.

Although, it seems like folks are starting to dig in and add commentary about the proposal anyways, so I guess that works too. I was just wondering if there was something specific that you wanted folks to pay close attention to, or if there was anything you weren’t fully decided or sure of yet.

#7

Not a problem. It really helps if we get feedback and thoughts on the entire study but it would help us the best if we get those for at least the conclusions and proposed roadmap sections, as they are derivatives/deductions of the entire study and can be (more or less) representative of it.

Of course, for folks who are interested in helping us out with the initiative[1], it would be great if they can provide feedback on the proposed entities[2], their interactions[3] and suggested technologies[4]. These would form the foundational workflow of the system so we need to get these right at all costs.

  1. Hehe, I should use this word sparingly LOL ↩︎

  2. https://fedora-arc.readthedocs.io/en/latest/badges/prop_rewrite_entities.html ↩︎

  3. https://fedora-arc.readthedocs.io/en/latest/badges/prop_rewrite_interactions.html ↩︎

  4. https://fedora-arc.readthedocs.io/en/latest/badges/prop_rewrite_technologies.html ↩︎

#8

Thank you @t0xic0der and everyone else involved in working on and concluding the Badges ARC investigation :raised_hands: :raised_hands:

I am very encouraged to see exactly what needs to be done to rebuild badges in documented form. I think I would have some valuable input for the “brainstorm and documentation about the interactions expected within the system” step, and I’d be happy to help coordinate work with the Fedora Design team. I am not a developer or UI/UX designer, but I am very familiar with the use cases for badges and am a long time member of the Design Team.

I want to note that we(badges design) have a new template developed for badge designs that we have yet to implement. Before we do the work of moving all the artwork into the new template and creating files, it would be good to get an understanding of the requirements/recommendations for the new system (e.g. file size, padding, etc). In relation to the roadmap, I am curious when those requirements/recommendations would become available.

#9

Thank you for volunteering, @riecatnor!

We can totally use the context that you have gathered around the Fedora Badges system while working with it all these years.

Coming to the badges design template implementation part, I think that we still have a long way before we get there. That, of course, does not necessarily mean that folks cannot work on it alongside the foundational UI/UX design and engineering work - in fact, they are encouraged to do so. That would only help us to be able to swiftly plug in the work and test the compatibility of it all when we reach somewhere around Step 7 and Step 15 of the proposed roadmap.

I cannot emphasize enough how important the documentation part is - to a point that I seem to have explicitly stated it being one of the outcomes of multiple roadmap stages, as you have rightly pointed out. I like to think that the stronger and more comprehensive documentation we have, the more we empower our contributors to partake in the process of building the system with us. We really appreciate your assistance in strengthening our documentation.

#10

Nuxt is used for SPA’s, it only turns into static if the option is turned on. It’s used more for it’s file based routing and conventions/configs. Vuetify can be used alongside it. A more flexible alternative would be TailwindCSS.

#11

I don’t know if this is something that I just misheard or totally imagined, but I don’t see Discourse listed in in the list of proposed technologies. Discourse has a badges feature that has been neat to engage with on other forums I check out.

Are there plans to integrate Discourse into the new badge system? Is it too complicated or not worthwhile enough to add it into the mix?

#12

While the Nuxt’s SPA mode can be helpful in auto-generating the routes, structuring the project in a standard manner, provide with a baseline scaffolding, auto SEO improvements etc. with a batteries-included approach, it would also limit the flexibility of doing them all the way we see fit.

As I mentioned before, we are still open to the choice of component libraries. I, for one, am still a bit sceptical about Vuetify because we plan on using exclusively Vue 3 with Vite here + 100% composition API and the last I heard, Vuetify had the Vue 3 support still in its infancy.

Let us find a component library built atop Tailwind CSS rather than using it as-is to keep ourselves from making the frontend a big challenge. Fedora Badges is already a complex system having around 6 internal and 4 external entities so it would be unfair to assign all the skill points to just one thing :stuck_out_tongue_winking_eye:

#13

One thing that we wish to achieve with the remake of Fedora Badges is to provide some semblance of a system that other FOSS communities can get inspired from and extend from. The fact that having our systems bound to third-party service that is not managed by our infrastructure and engineering community just puts me off. I mean, if the folks actively working on developing the system have limited access to what happens there (and how), we cannot empower new contributors to participate even if we have all the intentions to do so. These were my ideological reasons.

Coming to my more technical reasons, the badges implementation in Discourse, leaves a lot to be desired. There is no proper implementation of leaderboards and I am not sure if folks can manually add new badges into the system and award them as they see fit. Surely, we can make efforts to add those things but it would be a lot of bending Discourse to fit our purpose (difficult) than to make something which natively has our purpose (easier). Although, if we follow up on the experiment that @mattdm did[1], we totally can add viewing of badges here as a “nice-to-have” goal.

Note: I would be unavailable for the discussions during the weekends. See you folks on Monday.

  1. ↩︎

So we investigated Discourse to confirm if it is worthy of being the Fedora Badges frontend
#14

Discourse is open source software, and has a pretty flexible plug-in system. I can see this objection were it proprietary software, but that’s not the case. Why not leverage existing open source infrastructure where we can?

We’ve been bitten by trying to go it alone on community infrastructure projects before — I love the idea, but if that’s the plan let’s make sure we have a sustainable approach.

That’s not quite true. Maybe not “proper”… but not far off.

  1. The user list serves as a leaderboard over different periods — here, for example, for “Likes Received”: Fedora Discussion received. It doesn’t do badges, but could be extended to.

  2. There is a new thing, “Discourse Gamification”, which is specifically to make leaderboards. Discourse Gamification - plugin - Discourse Meta. This also does not currently work with badges, but it’s on the upstream roadmap.

You can:

That’s what I’m doing here — https://discussion.fedoraproject.org/badges.

1 Like
#15

That said, Discourse certainly isn’t a complete solution. Most of the component projects you’ve listed above would still need some separate existence…

  1. It could provide a leaderboard and end-user interface for seeing your own badges and those of others.

  2. It could also serve as the authoritative database of awarded badges. [1]

  3. With some more work, it could also provide a workflow for awarding arbitrary badges. [2]

On the other hand, we’d still need something separate for:

  1. The message bus consumer and rules engine
  2. Time-based scripts
  3. QR codes and magic links

And, like the who-has-the-badges database, it could hold the authoritative images and descriptions, but I don’t think that’s really best.

There is also the badge-path thing I’d like to have, and that might be harder to fit in. I can see some ways to make it work, but a dedicated UI would be easier.

  1. I’m not sure it should, though. ↩︎

  2. The easiest way I can see to do this: a special category where people could post in per-badge topics, which would create message-bus notifications which would feed back into the system… ↩︎

#16

Thanks for the helpful context, @mattdm!

We most certainly can have some kind of integration between the Discourse platform and the actual Accolades API entity, so folks can see the badges they have acquired (and those of others) from within the Discourse interface. The other entities, as you have rightly pointed out, would have to be separate projects serving their purpose that this interface can feed off from.

The badge path, for one, is something that might get introduced in the future in the Discourse Gamification plugin but it’s uncertain right now if it would. As a result, I think having a read-only integration (like we do now) would be a good start for this “nice-to-have” feature which we can, of course, expand to adding new badges, awarding contributors and much more such actions.

1 Like
#17

Yeah, I definitely want to keep syncing badges to this site even if it doesn’t end up being the primary end-user UI.

1 Like
#18

One advantage of using it would be that our stack would be the same as the Websites and Apps team, and we could take advantage of the components’ library being built and keep the look and feel uniform across the projects.

1 Like
#19

@t0xic0der et al. thanks a lot for the very detailed analysis. I have only read the information/discussion in this thread so far, but I will go through the documents mentioned in the footnotes as well.

The conclusion to rebuild rather than refactor confirms opinions that have been voiced before. In particular I talked to @erolkeskin quite a bit about revamping badges before. Unfortunately he didn’t make it to our kick off meeting in December, but I’d really like to hear him chime in as well. I will give him another shout out by e-mail as well.

Since I’m not a developer, I will stay out of the discussion on APIs and frameworks. I would like to volunteer for the implementation and documentation part, having experienced first hand how daunting the lack of proper documentation can be.

I might be able to provide more feedback once I’ve read through all the documents. I’m still coming out of year-end hibernation. :yawning_face:

3 Likes
#20

Let’s make you into one during the project. :stuck_out_tongue_winking_eye:

With you on that. We could use all the help we can get.

Thanks for volunteering and I am looking forward to your extended feedback.

1 Like