I have been looking at the user interface for the “notifications” application (aka FMN) located at Fedora Notifications and just wondering if anyone has some raw feedback on how they use this application.
Firstly, if you are not aware of FMN, it is a web application that was designed to be the central place to configure email (and IRC) notifications. the interface allows you to configure an enable filters that read the messages send to the fedora message bus (basically every action on fedora infra sends a message – like building a package, starting a meeting, ending a meeting, etc, etc) You can then filter and receive email notifications for only what you want to get.
The main idea was that FMN is the only thing that sends you email notificaitons, so users have a central place to turn the emails on and of, and configure them.
So, long story long, i’m after some raw feedback – here are some questions to inspire you:
FMN? never heard of it.
What fedora applications do you get email notifications from (either directly or by FMN)
Do you use the IRC notifications in FMN
Do you use the email notifications in FMN
Have you spent time creating and configuring your own filters in FMN?
I knew about it and tried using it a long time ago (3-4 years?) for Fedora Badges. But I remember it seemed buggy because it spammed me with notifications long past.
At the time I used FMN, this was my primary use case. I already get emails for most of the things I follow in Fedora, but sometimes it is nice to get things in front of me that might take longer for me to catch up to with email.
Although today, it seems like Matrix support is more relevant than IRC. I’m not sure if all the various fedora-messaging IRC bots we had before were hooked up to the same infrastructure, but I always found the team-specific IRC bots with fedora-messaging filters helpful. It would be nice to have such bots for Matrix too.
I never used email notifications.
At the time I used FMN, it was mostly to notify me any time I received a Fedora Badge. I may have had a few filters for other services, but I can’t remember.
Getting notifications from Fedora Discussion into IRC/Matrix private messages seems useful to me, especially for people who may not keep up as actively with the discussion site.
Fedora Discussion is actually an interesting case here too (for email notifications at least) – the emails from discussion are always going to be better than FMN can provide, so what i understood of the primary goal of FMN (a single place to manage fedora emails) has somewhat moved with the introduction of quality 3rd party products like discourse.
Agreed – and I would not have the interest to duplicate Fedora Discussion emails with FMN emails for this site. Not sure if Discourse has a clean API for this, but just my recent notifications (as I have configured them from Discourse) into Matrix/IRC would be nice. But I definitely think this is a “nice to have.”
Yes. I have it notify me of build success/failures in COPR and Koji.
Same as above (possibly with some slightly different tuning)
Not beyond setting up basic alerts.
I like the idea of FMN, particularly the idea of it being the one place to configure email notifications. That said, in practice it doesn’t really work. We get email notifications from Bugzilla, Pagure, Discussion, Bodhi, etc.
I’m not sure that I’m using FMN in a very effective way (in other words, I’m not getting the value I could from it). The existing implementation is…daunting.
+1 to Justin’s comment about Matrix support. For me personally, it’s mostly irrelevant because my IRC and Matrix messages end up in the same place.
The performance is my biggest complaint right now. Some times I don’t get build notifications for a day or two later, which is…not helpful.
I did in the past, but I eventually came to the conclusion that doing so was a terribly bad idea so reverted to the defaults and instead use my email filters to discard anything I don’t want.
So now you’re probably wondering why I found it so problematic.
The answer to that is really quite simple - as soon as you make any sort of customisation you are immediately cut off from any changes that are made to the default filters which means that over time, as notifications change and changes are made to the default filters to reflect that, the notifications you are getting get worse and worse. You receive noise that the default filters would suppress and worse don’t get new messages that might be important.
Even once you realise you are missing things it’s virtually impossible to work out what never mind to try and merge the changes in using a cumbersome web UI and you typically wind up having to revert to defaults and then reapply your own customisations on top of the new defaults.
Having said all that it’s not clear there is a good alternative - if the user filters were an extra layer on top of the defaults they would only be able to further restrict the selection and wouldn’t easily be able to widen it. Maybe two sets of user filters? One which is ORed with the defaults and then a second which is ANDed with the output of that operation?
As I hinted above I also found the web UI to quite a cumbersome way to manipulate something as complex as the notification filters - a simple text filtering language with filters kept in a git repo where you could view history and rebase over new defaults would probably be more usable in many ways.
I’ve spent a lot of time trying to configure FMN in the past. I really liked the idea, but eventually discovered major issues which nobody was fixing. I gave up on FMN almost completely. I still have it enabled (by email), but I wiped all my setup and only enabled the default filters set, which had the least rough edges. From time to time, I still consider disabling FMN completely, especially when I receive an empty notification or a json dump, but the volume is low and I just don’t care anymore…
FMN is an example of the 80/20 state. The remaining 20% to make it actually well usable would, I believe, take a lot more effort than it took to build the current state. I filed quite a few issues on infra github in the early days, they are probably still rotting in there.
Now, to be a bit more specific, going from my memory:
It is quite common to receive either an empty notification or a json dump, both of which are completely unhelpful. Even if you enable full details and trigger links, you still have no idea why you received that message and how to stop it (I don’t think you can), which is very infuriating. The more you stray from the beaten path of the default filters, the more you see this.
FMN architecture seems to be something like: source events → user filters → formatters → notifications. Those formatters need to be provided by the source project (we provided some for Taskotron in the past). However, even when the formatter is missing, and therefore the source even is not converted into a user-readable description of what happened, the message is still sent to the user (either blank or a json dump). This is a completely broken design. The user is getting spam he can’t do anything about. If the message is not formatted, it should not be sent.
It’s very hard to debug why I receive a certain message. For example, it seems obvious that FMN should indicate which of my filters sent this message. But it doesn’t. I feel powerless to tune the message setup. I have to toggle magic knobs and then I either receive a lot of noise or almost nothing, and I have no idea why.
Related to that, the “example messages that match this filter” when editing a filter almost never show anything. Again, working with a black box. This all means that configuring things takes a lot of time (you have to wait until some actual messages arrive and decide whether it’s what you want) and effort.
The UI for editing filters is difficult and confusing. It took me some brain-wrapping to understand how rules apply in a filter, and why there are so many ignored rules in default filters, and whether that means they are just ignored or whether they must be false in order to trigger the message… and I already forgot most of that.
Because all rules are ANDed together in a filter, sometimes you need to create a lot of similar filters just to be able to express rule1 and rule2 and (rule3 or rule4 or rule5).
Filters can be exported, but can’t be imported. So they can’t be easily copied, you can’t easily experiment with different approaches (and restore the original back if it doesn’t work).
Your whole setup can’t be backed up, you have to do it individually filter by filter. (And then, you still have to restore option). So if you ever want to try if a clean slate would solve your problems… have fun restoring everything afterwards.
The default filters management is… non-existent. It seems that if you ever do a slight tweak to a pre-defined filter, your copy will forever override the default one, even if the default one is updated in the future. And the one way to go back to the default version (or even just check whether your version is out of date) is to completely wipe your profile. Combine that with export&import problems above.
I’m sure I forgot a lot of important things, but I think I included at least some. I honestly expected FMN to be in a state of the clinical death, to be removed eventually. If they are plans to actively maintain it and improve it, that would be of course great. I think some parts would need to have some major redesign, though. If I could have a magical wish, I’d make it smaller, simpler, more controlled instead of free-form. Instead of letting the user configure whichever actions and rules he wishes, and then send him empty unhelpful emails, I’d rather see a narrow set of possible actions/rules, which actually work and look great. The pre-defined filters lifecycle would need to be completely redesigned. The web UI would need to get easier to understand and the export/import or save/load system would need to be better.
Sorry if I sound overly critical, it’s just because I burned a lot of time on FMN and never achieved a satisfactory result You asked for raw feedback, though
1: I think I know what you’re talking about. The thing that sends me email anywhere from hours to days after an event happened.
2: They don’t say where they come from. Those that contain comprehensible information tell me about events in Git, Koji, Bodhi and badges as I recall. Sometimes there are completely useless messages from who-knows-where.
3: No.
4: Yes, mostly those that are on by default.
5: Once or twice, long ago. It was far from easy. I have no idea how much the web interface has changed since then. At the time, it did not achieve the goal of being a central place to turn emails on and off. I would like to turn off some useless messages, but I have no idea how to do that while keeping the useful ones.
6: Obviously some bottleneck needs to be removed so the messaging can keep up with mass rebuilds.
The timestamp in the notification should be used to set the Date field in the email, so when the notification arrives three days late, it at least shows the time when the event happened. Anyone who wants to know where along the path it was delayed will read the Recieved fields.
Nah, I’ve heard of it and am using it for quite some time.
What fedora applications do you get email notifications from (either directly or by FMN)
Koji, Koschei, bodhi, Bugzilla and badges
Do you use the IRC notifications in FMN
Yes, but they come with such an incredible delay, that they are next to useless.
Do you use the email notifications in FMN
Yes.
Have you spent time creating and configuring your own filters in FMN?
No, the last time I glanced over them, they went over my head (although that has been a long time ago)
Honest thoughts / ideas?
I find the notifications generally very useful and would miss not having them, but currently their usefullnes on irc is really limited as they come delayed (sometimes by a whole day).
I was not aware of FMN until I saw this. I just set my e’mail address. Not using IRC. Currently I get Fedora e’mail from the QA Mail lists and Fedora Magazine. The set up verification could be faster. I would guess I’ve been waiting about 15 minutes now for the e’mail. I would like to get notifications currently I search around here to find out what’s going on.
When waiting for a fixed bug to be released, it is useful to get one time notification when a new version of a software is detected, when an attempt was made to automatically package it, when somebody sends a patch to fix/bump the package, when it is finally built and scheduled for update, moved to testing and released. But setting this is too cumbersome.
The whole software delivery pipeline can be made into one actionable dashboard. At least FMN as the key component for the infrastructure is there. But the UX for this specific story is not.
You asked for raw feedback, though 