Implementation details for Fedora Discussion badges

Continuing the discussion from Badges for Ask Fedora (and Fedora Discussion overall):

This is for @gui1ty, primarily, I think. :classic_smiley:

I’ll submit badge requests as usual for those. And I can[1] make the Data Explorer queries that return a list of people who should have the badge. The question is: how should we bridge this into the badges system?

Once I have created the appropriate queries, they can be run via an API call:

and I can create an API key which will allow access to those queries (and not anything else).[2]

The question is: where should that run? Should that be part of the Badges infrastructure? Or, should it be a separate little app that runs periodically and puts messages on the Fedora Message Bus for Badges to consume?

Or something else?

I’m imagining that whatever it is would run daily, so badges would happen on a bit of a delay rather than instantly. We could also have something based on webhooks related to particular events (like posting), but that seems like a lot of complication for a first pass…


  1. possibly with some help from someone who is better at SQL than me ↩︎

  2. We wouldn’t even need to treat such a key as secret, since the queries will be restricted and won’t return anything sensitive ↩︎

1 Like

We should track this discussion in the epic for the Fedora Messaging integration phase. I think it best fits in our timeline for the new system there:

I hesitate to plug this into the current legacy system because Badges is still running the fedmsg library, not the newer fedora-messaging library. I suspect automated badges would need to be scripted carefully and may not always work reliably (that is, without fixing the underlying issue with the messaging consumer in the current legacy system).

2 Likes

We’ll need a service that runs periodically (and/or is triggered by webhooks). With the new architecture, that service could either:

  1. emit messages on the bus (new, old, or both), to be processed by the Collector entity, or
  2. be a new entity that is parallel to and separate from the Collector, calling Accolades directly.

The latter seems like it’s probably the cleanest, really. The Discourse API call part is trivial (all the query complexity is on the Data Explorer configuration side on the site here), assuming

  1. it runs periodically rather than by webhook trigger, and
  2. it is stateless — every query gives the full list of people who should have the badge.

Adding webhook triggers would be a lot of additional complexity, but might be worth it eventually for the “instant gratification” factor. I’d keep it out of scope for the first pass, though.

I don’t think we’d want to make it keep state, but we could perhaps make queries that would only return people who have qualified for the badge in the past 12 hours (or whatever frequency). Then, we’d probably also want separate full queries which run weekly or something to make sure nothing is missed. Again, I don’t think we’d want that in the first iteration.

And, we could also have a “zeroth iteration”, where the queries are available but not automated…

1 Like

:+1:
I want to put as little time and energy into the old, frail system. We may yet have to for connecting Badges to the renewed FMN, so people get notified about all the badges they are hording.

But other than additional badges, that can be implemented in the current ecosystem, I want to steer clear of touching the code at all.

However, putting this into some epic to be implemented in Badges 2.0 at a later stage, is certainly within scope. Does Discourse emit any messages at all, that we could use to award badges with using what we have in place now? Then I’d be willing to take a look at implementing that, so people get at least something for the time being.

1 Like

Makes sense. At that stage, which architecture makes most sense to you? Bridge through the message bus, or a “Discourse Query Collector”?

It does — see https://apps.fedoraproject.org/datagrepper/v2/search?category=discourse. However, the history does not go back to the beginning of Fedora Discussion (or Ask), and there is some data loss from a few months ago when there was a DNS problem (of course it was DNS). There’s no easy way to ask Discourse to send all missed messages.

Makes sense. At that stage, which architecture makes most sense to you? Bridge through the message bus, or a “Discourse Query Collector”?

The bridge sounds like it would make life easier, since we interface
with the message bus already. Provided the bridge provides the same
level of fine grained information. But I would like a developers opinion
on that as well [1].

We could use the query collector to initially award badges to users
already active on Discourse. As a manual, one off, effort, that is.

It does — seehttps://apps.fedoraproject.org/datagrepper/v2/search?category=discourse. However, the history does not go back to the beginning of Fedora Discussion (or Ask), and there is some data loss from a few months ago when there was a DNS problem (of course it was DNS). There’s no easy way to ask Discourse to send all missed messages.

DNS is hard! Ask the people at ISC. :stuck_out_tongue_winking_eye:

From a quick glance it looks like we could use those for awarding
simple badges. Badges that do not require knowledge of a user’s past
activity. Anything under the Single-event badges header in your
post
.


  1. I’m looking at you @t0xic0der. :eyes: ↩︎

1 Like

Hey there,

This does look like an interesting discussion that I plan on responding to, once I am done with the Fedora Linux 38 Release Party.

Barely getting some time/space to breath here :wink:

1 Like

Webhooks? I think Discourse has those.

We can make use of the project that @dkirwan had been working on for a while now, discourse2fedoramessaging in order to be able to generate messages on the bus whenever a certain event takes place. That in turn can be taken up by the Fedora Badges Messages Consumer entity from the Datanommer and we can get an event that would lead to a badge being awarded.

Unless, of course, if the problem is not as easy as I take it as.

1 Like

As much as I like the idea, the bridge project might just add one more entity to an already complex system for the folks involved to develop and maintain. Discourse2FedoraMessaging does sound like the correct answer to this and we can use the events propagated by the Discourse application on the Fedora Messaging bus to award badges impromptu.

I asked them[1]. They said it is easy[2].


  1. No, I did not ↩︎

  2. No, they did not ↩︎

Maybe the main take away for now is, that we need to take note of different ideas and make sure the new Badges is developed with them in mind. Or more general: Badges needs to be modular, extensible, so it can provide interfaces for different systems.

The actual implementation can follow once we have something tangible.

So, in order to keep track of all those ideas, we should probably (ask people to) open tickets in GitLab and link them to the appropriate milestones.

Two problems with webhooks is: 1. there aren’t hooks related to everything we want to award, and 2. Discourse has no way to replay hooks for history.

OK, that indeed looks like a problem. If there are not enough webhooks to cover events that we want to award badges for then a lot of these kinds of contributions would fly under the radar. For non-technical ways of contribution, I wanted to leverage Fedora Badges (albeit as a last resort) to track contributions and the absence of ample amounts of webhooks for every event does push us back on that endeavour.

If we have the Discourse2FedoraMessaging system in place - we technically would be sending messages on the bus whenever any event takes place and the Datanommer project would keep on archiving these messages in real-time. This essentially means that this information can be later accessed with the use of Datagrepper and hence, historical events can be probed into by the messages consumer entity easily.

Yes, but it took a while for that to get in place, and then there were some problems for a while where it wasn’t reliable. It should be all good going forward… but I don’t want to miss what we already have.