Trying an AI/ML sentiment analysis system

Site Help & Feedback
1

After coming down hard on ChatGPT in this topic, I’m going the other direction here and have enabled an experimental Discourse feature called “DIsorder”. This plugin uses an AI/ML classifier to automatically flag topics which might be problematic.

Note that flagging for moderator attention is all that it is configured to do. It can also warn users if their post is classified this way, or even block posts entirely. I don’t anticipate using these latter features (at least, not with thresholds anywhere that might have false positives).

As such, this feature should be invisible to most users — but I thought y’all might like to know. (And you can read more at the link above.)

@moderators, as noted in the docs, there are thresholds we can tweak as need be. Please let me know if you’re finding something over-aggressive (or broken, or whatever).

Also: there is a multi-language model, but for now I’ve left it at the default. In the future, I’d like to enable that in the “Ask in Other Languages” category — or, ideally, per language.

1 Like
2

I’m in favor of this specific application, but I feel like this is something we shouldn’t do silently. Even if it’s buried in the terms of service somewhere or something, people should have a way to know that it’s happening.

3

I agree, just to do things out in the open.

Just for the sake of discussion :smiling_face: is this system so much different compared to an antispam system? At the end of the day, even classical antispam systems use some sort of AI/ML classifier to automatically flag topics which might be problematic. (And we know, their work often fail).

1 Like
4

First, a mea culpa. I saw this by way of the @moderators mention and missed that this is actually a public post. So my “this is something we shouldn’t do silently” objection was invalid from the beginning.

Yes and no. Practically speaking, it’s pretty similar. But it feels different. I think this is because spam filtering is necessary if you want to put something on the Internet. But the sentiment analysis feels less existentially-necessary and more judgmental. I think it’s valuable as a way to clue the moderators into a topic that might be getting too heated, but I can see people (particularly those involved in a free software project) feeling like it’s invasive. But we’re not hiding it, so let’s see how well it works.

1 Like