Strategies to collect metrics on participation / engagement on Discourse vs. mailing lists

Hi all, I wanted to start a conversation on ways we can measure participation and engagement on Discourse with whatever tools or metrics are available to us.

Elsewhere, I made an (unfortunately) biased evaluation of engagement based on metrics immediately available to me (number of threads, number of posters, number of replies). These metrics are helpful, but on their own, they can be biased (e.g. if a specific topic explodes in popularity one month, but all other months are quiet). I wanted to see if other people had ideas on strategies we could use to collect metrics here.

It would be cool if we can use fedmsg to play with Discourse metrics too. I figure @bex @mattdm @sanja might have some thoughts here.

2 Likes

I was discussing with a SEO analyst who gave me a quick free review when comparing users, devel, and council-discuss MLs to Discourse as a first test. Obviously, she also said it’s like comparing apples and oranges and one would have to dig deeper and do some actual comparisons and measuring from now on rather than digging up numbers from the past.

The findings from her preliminary analysis (she’s not in the open source community and her job is SEO and marketing):

  • MLs are not SEO-friendly but have high potential in terms of links
  • the forum ranks on more keywords in search engines, more potential to rank well

It doesn’t really tell us much, other than confirming that for search engine visibility, we’d need to improve the ML SEO if we’re not using Discourse.

1 Like

Waiting for more input from others here as I’ve already given my opinion in favour of Discourse for various reasons previously.

Here’s one more purely personal reason that is an advantage of Discourse for me: I really despise the fact that ML participation means your e-mail is widely available to the public. And if people are now telling me to use a throwaway for that, I’ll counter with 2 things:

  1. throwaways will be taken less seriously,
  2. it’s either creating an account on Discourse where you have more control over what happens to what you post versus a ML where your email is publicly distributed to numerous other places where you have absolutely no control, but both need an account creation and regular checking. For those people telling me to create an alias for that, I can say the same: use the mail notifications on Discourse and get the posts delivered to your inbox. Both are inconvenient for someone who’s a heavy user of one versus the other.

It all comes down to personal preference. The rest is what we’ll have to measure in terms of engagement and discussion value (e.g. purely automated posts to an ML are less valuable in the metrics than an actual discussion, empty threads with no answer on Discourse are bad, etc.).

2 Likes

@sanja @jwf
+1 that MLs vs Discourse is apples:oranges.

I had to address this question at work a few years ago and I’m happy to go into details.
IMHO, Discourse is a better solution for this use case (ie. a forum for fedora contributors) because:

  • Reputation Scoring helps drive the best answers to the top
  • Badges are easier to manage and extend to accommodate new objectives.
  • Cross-referencing of topics and solutions is easier and less centralized.
  • History is more accessible to people joining a discussion.

Perhaps a more direct comparison would be to compare this discourse site vs the collection of Fedora boards on Reddit, StackOverflow, Quora, Medium, Twitter, etc.

In fact, if this discourse app could somehow aggregate and curate that traffic, it would be a big contribution.

Just sayin’

2 Likes

If I start new discourse topic will an email get send to the mailing list?

No, the mailing list is not connected to Discourse. You can get mail notifications individually but it’s not hooked in with the mailing list itself.

I’m working on a plan to get mailing list mode on discourse to be usable in any way, shape or form. There are a lot of annoying things that we can’t really work around with mailing list mode, but it’ll be better than nothing for teams like CommOps that are stuck with Discourse alone.

1 Like

Hmmm, IIRC there is a way to have a bot identify empty threads… They could be esacalated to someone who could get a badge for resolving them with dispositions such as:

  • “Sorry, it looks like no one has an answer for this…”
  • "Sorry, this might get a better response if you were more specific, gave more context , mentioned what resources you already had, etc.
  • “This is the wrong forum for this topic, try XYZ instead?”
1 Like