F40 Change Request: Privacy-preserving Telemetry for Fedora Workstation (System-Wide)

Project Discussion Change Proposals
, ,
652

Fedora does mostly guide itself, but I find it amusing that you would suggest that Redhat is harvesting free contributions. Redhat employees and infrastructure make massive contributions to the Fedora community so it is not unreasonable for them to partake of those contributions downstream, hell tons of people simply harvest without contributing…what have I done lately for Fedora other than use the product.

I have huge respect for Red Hat contributions to Fedora and other upstream projects. What I do not like is they should not shove their profit based agenda into Fedora which they call “community” distro. Not that I call this proposal a secret profit based agenda. But the stated goal clearly tells whether “Red Hat is achieving it’s goals with Fedora”. This is the disturbing point. I’m all for opt-in telemetry if it gives it’s users (not the manufacturer) advantage. In fact I contribute to Mozilla Firefox’s telemetry in all my PCs. What matters to me is intention behind the data collection. I do not trust Red Hat because I already saw their anti-competitive behavior and how they restrict their customers from exercising their GPL rights. I don’t want Fedora to become another Red Hat product serving Red Hat instead of users.

Red Hat should respect Fedora community’s opinions and leave Fedora to operate independently serving it’s users.

People are posting their voices against this proposal not because they want don’t want to help improve Fedora but they don’t want to see Fedora as just another spyware along the line of Windows, MacOS, VSCode, Audacity etc.

6 Likes
653

I agree with this part 100%, but I do want Redhat to get out of Fedora what they need…chicken->egg egg->chicken. It is mostly a symbiotic relationship that has worked so far. Let’s see what the actual proposal ends up looking like.

1 Like
654

I agree this could be worded better in the proposal, but I want to point out a crucial distinction — you’ve paraphrased this as “Red Hat is achieving [its] goals with Fedora”, but the actual wording (quoted above) is about a specific goal: “whether Red Hat is achieving its goal to make Fedora Workstation the premier developer platform for cloud software development”.

The overarching goal of Red Hat’s investment in Fedora is to make sure this is a successful, healthy community project that explores innovation while also providing a good upstream for RHEL. That’s what they pay me to help with — and the first part is important, not just the last. (Fortunately, the last is also nicely in line with the Fedora Mission.) Beyond that, I don’t think it’s right to say that anything is a “Red Hat goal”, exactly. Rather, different people at Red Hat (both organizationally and personally) want to accomplish lots of different things in Fedora, or to help Fedora accomplish various others. None of those are around productization or commercialization.[1]

It’s dangerous for me to speculate about the RHEL business side of things, because that’s really not my area, but on this specific goal…

Red Hat makes and sells a “RHEL Workstation” product, but it is a very enterprise-oriented thing. Its customer base is not really like the Fedora user base at all. Red Hat formerly had a “Desktop” product, but a few years ago, dropped that. It’s really hard to make any money at all from a general-purpose desktop OS, and Red Hat has decided not to try. Red Hat IT has even switched from RHEL for the official corporate desktop offering to a Fedora Linux build.

But, out in the world, that leaves an ecosystem vacuum.

As a community project, we in Fedora do have a very strong interest in providing a general-purpose end-user Linux desktop OS. A little more than a decade ago, when Fedora was kind of at its most depressing point[2], there was some talk (inside and outside of Red Hat) of steering Fedora away from the desktop, focusing on a cloud/server OS. We could have taken that approach, and ended up being something like Alpine or Wolfi. We decided, instead, to embrace the “Fedora.next” plan[3] and encourage Workstation, Server, Cloud — and a plethora of other spins and deliverables.

And that’s worked out! We (Fedora) have a great desktop offering.

Many people at Red Hat would like Fedora to succeed in filling strategic desktop ecosystem space without being a product or productized[4]. And in particular, to succeed in areas which are obviously key to the Red Hat business (like, “cloud software development”[5]). That doesn’t mean that’s the only area that RH wants Fedora to succeed (again, see the overall goal!), but it’s one that’s pretty easy to sell to management and the business folks.

Of course our amazing desktop relies on the wider community, both in Fedora and upstream — that’s how it should be. But, there’s also a lot of Red Hat investment. It is reasonable to me that the Desktop team backs the goal of positioning Fedora Workstation for success in a way they can use to justify increased investment in their team, Fedora overall, and the Linux desktop.

  1. Well, at least not serious ones. For all of the recent feeling of big-corporate, RH does have a culture where people are encouraged to have different ideas, and so I’m sure someone has these. ↩︎

  2. see this talk! ↩︎

  3. see Fedora Strategy 2028: Focus area review (Editions, Spins, and Interests) ↩︎

  4. because there’s not seen to be a future in that ↩︎

  5. You can see free and open source investment in this in other areas, like container tooling (and in particular Podman Desktop ↩︎

7 Likes
655

Thanks for your detailed reply Matthew. As long as this proposal is actually for the community and data is collected with explicit consent which I can inspect, I’m more than happy to share my data with my beloved project.

2 Likes
656

BTW, it’s taken me a while, but I actually see a boring QA-related problem with this.

We’d want to make sure the ‘yes, telemetrize me!’ option actually, y’know, works. But we probably don’t want the automated test system to always opt in to telemetry, because then you’ll get a bunch of bogus data from the test system running constantly.

So…the best option I can think of is to add a new test (sigh) which just opts in to telemetry, checks that that worked, then shuts down and does nothing else. All other tests would opt out of telemetry. Filtering out any data the test sends would be a problem for the developer.

This might actually be somewhat awkward to implement (sigh) because the way things are currently, we do an ‘install and boot’ test, which completes g-i-s, then uploads a disk image for all the other tests to use. So I don’t see a nice way to implement the design above while keeping just a single ‘install’ test. We’d probably have to have the new ‘telemetry’ test run through the entire install process just so it can get to g-i-s and test the telemetry, which is a bit of a drag but I don’t immediately see a way around it…

657

Since you want to test both the “telemetry enabled” and “telemetry disabled” paths through Initial Setup, the model you describe is probably what you need. Unless you can do this:

  • complete installer, upload this image A.
  • boot image A, complete initial setup, opting out of telemetry, upload B
  • boot A again, complete initial setup opting in, do whatever test, discard
  • run rest of tests against B

I can see three ways to do this of the top of my head.

  • By default (actually I think it’s hardcoded) batches of events are only submitted once an hour. I presume the test will not take an hour. So unless you force an upload, nothing will typically get uploaded.
  • There is a concept of environments – prod, dev, test – where you tell the metrics daemon which environment to upload to. (At Endless we run prod & dev all the time and test only as needed.)
    • There is actually special logic in the metrics daemon to switch to the dev environment if it detects that the booted system is using a nightly or staging build of Endless OS. This could be extended; though personally I am not fond of it.
  • You could use the “site” mechanism to tag test systems as being test systems. Then those analysing the data can exclude channels matching the site configured in the openQA suite.

When you talk about checking that opting in works, are you thinking to actually test that uploads happen? And similarly for opting out. (Personally I would simply check that the relevant dbus property has the expected value.)

How do you exclude openQA from countme?

658

We could do it the way you describe (I thought about that too after posting my last post) but I’m not sure I like it. The image uploads aren’t free in terms of time or required space on the server, I try to avoid having too many of them. Probably the standalone test is the least worst choice.

edit: oh, hey, you know what, we can solve this with snapshots. Should’ve thought of that earlier. Take a snapshot before the g-i-s step, do throwaway test routes (like enabling telemetry), reset to snapshot after each, then do the ‘real’ route (disabling telemetry) and upload the final image…that should work.

There is at least one test that takes quite a while - apps_startstop, which checks we can at least launch and then quit every app on the menus by default. (For obvious reasons, this test would also likely produce some bad bogus data). That takes ~30 minutes when it passes. But if it starts failing - because GNOME fiddled with overview icons again, or fonts, or something - it can take much longer, because there will be a lot more timeouts. I think it can run over an hour in that case. So that’s one example of a case we’d have to watch out for.

The environment concept sounds nice, but OTOH, I like openQA to test things as close as possible to exactly how users will experience them. Making it use a non-prod environment somehow might be a small enough change to be OK, though.

How does the ‘site’ mechanism work, exactly?

I wouldn’t be that interested in testing that the metrics system actually works, TBH. I guess the most important things to test would be:

  1. Does the opt-out button actually opt-out (crucial)
  2. Does clicking either button cause any kind of critical problem (crucial)
  3. Does clicking the opt-in button actually turn the right things on (meh)

I wouldn’t bother testing whether data upload actually works, I don’t think.

I don’t do anything special to exclude openQA from countme. I don’t know if anyone else does. I would hope, I guess, that the system is set up to exclude all data from inside Fedora infra (where openQA runs); that should be possible, AIUI.

1 Like
659

One thing I’m not understanding here. The terms “Fedora” and “GNOME” are getting tossed about as if they were interchangeable. But there are other Spins of Fedora being produced at the same time as the GNOME spin, despite what seems an attempt to ignore them.

If I am using Cinnamon Desktop, how will this affect that install? Will the telemetry option present itself there? Or perhaps it will it get installed in the background with no UI to configure it? Will there be pieces of it along with all the other chunks of GNOME that get installed even when we don’t want them?

660

In theory, it won’t unless the program is expanded in the future.

661

You drop a keyfile into somewhere /etc/metrics and then, when a batch of metrics is uploaded, the key/values in this file form a part of the “channel” that each event is associated with. (The other things that define the channel are, on Endless OS, the original installation image, and whether the system is live, dualboot, or neither.)

On my main Endless OS laptop I have some values like “Will’s house” set in here, which means that I can identify on the (dev) server which events came from my own system. Otherwise I wouldn’t be able to tell them apart from events from other devices that happen to have been installed from the same OS image.

The intended real-world use is: you have a bunch of computer labs in different schools in various cities, and you want to be able to slice the data by school and/or city. So the administrator manually tags them. (The keyfile has some misleading name involving “location”. “Location” tends to sound like automatic geolocation, which this is not, hence the (incomplete) rename.)

662

My understanding is that the intent is for the telemetry to only be used for the Workstation Edition, because Endless wrote the software to be wired into GNOME. It is currently not generally useful, and I’m not sure if anyone upstream is interested in making it generally useful.

That said, the way it gets included on the Workstation Edition can be structured so that it doesn’t get preloaded on non-Workstation variants and would not get installed even if you manually installed some GNOME components.

663

If possible, I think that’s the way to go. It keeps things clean. We could make the distinction that Workstation is the one that has this telemetry thing but every other variant does not. Otherwise it would bring up questions of why this thing is installed on a system without a GUI or consent check, and is it running. From a PR perspective, if it’s not going to come with the consent screen or even be running, it should not be in any other Fedora variant.

However, it’s my understanding that this is only for Fedora Workstation to begin with. Just mentioning this as it would be good to have it cleared up.

664

@catanzaro Could you elaborate how your proposal differs from the gnome-info-collect tool and why the telemetry needs to be shipped with the distribution, instead of being a downloadable package? Additionally, since this seems to be targeting GNOME, why is this only being proposed for Fedora (Workstation)?

I think most users can acknowledge the benefit of valid and representative data, however it should be the users explicit choice whether the user wants to share this data, and just as important, whether or not the user wants any telemetry software (whether enabled or not) installed on the user’s OS.

As quoted I believe that a downloadable telemetry package would be a great compromise, as it would keep the image and any forks clean of telemetry, while still allowing users to easily opt-into telemetry. By separating the telemetry from the OS into a package user trust should also be significantly improved. Additionally it would make the removal of the telemetry very user-friendly by simply uninstalling the package, instead of needing to find any appropriate toggles in settings. (Re-)using the gnome-info-collect tool would also save a lot of work for the developers and allow for data to be collected and compared cross-platform.

So far, the only reason provided for shipping the telemetry on an OS level instead of a package that I could find, was that the number of responses is not high enough and mainly consists of a subset of dedicated users leading to non-representative data. This however is not a valid reason for integrating telemetry on the OS level.

Instead, GNOME should focus on better communicating such efforts and most importantly making the process of contributing as easy and open as possible. I would therefore propose the following ideas:

  1. Reinstate and use gnome-info-collect as the cross-platform telemetry tool for GNOME development. This post already provides a great starting point.
  2. To better address “regular users” (in the sense of less involved in the development of GNOME/Fedora), highlight the gnome-info-collect package within the GNOME software utility. This should repeatedly offer users (who use the GNOME software utility) the opportunity to opt-into telemetry by simply installing the package.
  3. Add the link to the already mentioned post or a new dedicated webpage in the GNOME tour. The inclusion of the reference to the package within GNOME tour should yield similar attention levels as with the current proposal, as users will be exposed to the option to opt-in at the same interval, assuming a continued bi-annual release cycle for both Fedora and GNOME.
  4. On the GNOME contribute page start the page with something along the lines of:
    “One of the easiest, yet very effective ways of contributing, is to install our telemetry package, which collects hardware and usage data from your system. This data will anonymously combined with the data of other users to form aggregate usage metrics. These collected metrics can aid developers to better identify areas of improvement, focus their time and cater to the needs of the user base.”
1 Like
665

I believe this to be true.

But this is not the reason AFAIK. There is nothing intrinsically GNOME-specific about eos-metrics-event-recorder / eos-metrics / eos-metrics-instrumentation. (Unless you count being implemented with GLib.)

I wouldn’t say that we at Endless are actively interested in other desktops, but if someone did want to use it elsewhere, go for it. Of course if you wanted to have settings to enable and disable it on other desktops, you’d need to implement those. And if you want some other desktop’s components to record metrics, you’d need to implement that.

666

It was not designed to be used outside Endless, but adapting it to Fedora won’t be hard and nothing would stop KDE from doing the same.

gnome-info-collect is a survey tool. It’s a completely different purpose.

667

The only metrics of interest for the entire distribution would be essentially equivalent to the Steam Hardware Survey. I have wanted something like that built into all Fedora variants for a long time, because the majority of our problems are hardware-related. Knowing the hardware would allow us to better understand how to prioritize hardware enablement and eliminate a lot of guesswork about what people are using Fedora with.

Essentially, I want Smolt again.

For KDE Plasma software, there’s already an opt-in KDE telemetry inquiry as part of the first-run welcome wizard in KDE Plasma. I don’t need two of those.

4 Likes
668

why not having two checkboxes next to each other to opt-in?

669

Hi all,

It’s evident that there is a lot of revision to be made and feedback to be captured before this proposal is submitted to FESCo. The proposal owner has asked for more time to digest the discussion and is not ready to submit the proposal in its original proposed state. While this thread (and the many others it has spawned) has proven how powerful open discussion is, it has also been contentious at times and it may not be constructive nor in anyone’s best interests to keep this discussion going. A lot of valid points have been made across the board and unless there is something that is blatantly absent, either for or against or somewhere in between, that has not already been voiced by someone, it might be best to allow the proposal owner time to work on the proposal and we can discuss their revised version when they are ready to re-submit one.

We will post a new thread when the proposal owner feels that the proposal is ready to be re-proposed for feedback, and thank you to everyone for engaging thus far. It has clearly been a tough topic in tough times, so thank you all for showing up and showing how much you care about the Fedora project.

7 Likes
670

To clarify, should the community think of this as the original proposal being withdrawn, and then a new proposal will be submitted at a later date that takes into account the feedback provided?

Is there a timeline on when the new revised proposal will be submitted, whether its weeks or months away?

671

I would say certainly consider it paused. Edits are being made to the original and the owner needs more time to incorporate all the feedback. When they feel they are ready to re-submit their proposal, as ‘interim wrangler’(I don’t know what else to call myself in this situation) they will let me know, and I will create a new discussion thread so the community has the opportunity to review the revised proposal and provide feedback again. No changes go to FESCo without open discussion in our community, and given how strongly people felt about this proposal, it is only appropriate to have space to provide feedback on the next iteration.

With regard to timelines, I have nothing concrete to offer here unfortunately as it is an F40 change, so the proposal owner technically has a few months to work on their proposal before resubmitting if they needed that time.
My personal suggestion is to enjoy Flock in a few weeks :slight_smile: and we will create a brand new discussion post when the proposal is ready for feedback to ensure it wont be lost as a reply in this thread.

I hope this helps!

2 Likes