F40 Change Proposal: Passim Peer-to-Peer Metadata (Self Contained)

Wiki Link

Announce Link

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

:link: Summary[edit]

Passim is a local caching server that broadcasts specific shared metadata to other clients on your local network to reduce the amount of duplicate data downloaded from the internet.

:link: Owner[edit]

:link: Current status[edit]

  • Targeted release: Fedora Linux 40
  • Last updated: 2023-08-29
  • [ devel thread]
  • FESCo issue:
  • Tracker bug:
  • Release notes tracker:

:link: Detailed Description[edit]

Much of the software running on your computer that connects to other systems over the Internet needs to periodically download metadata or information needed to perform other requests.

Everybody downloads the same file from a CDN, and although a CDN is not super-expensive, it’s certainly not free. Everybody on your current network (perhaps thousands of users) has to download the same 1MB blob of metadata from a CDN over a perhaps-expensive internet link.

What if we could download the file from the CDN on one machine, and the next machine on the local network that needs it instead downloads it from the first machine? We could put a limit on the number of times it can be shared, and the maximum age so that we don’t store yesterdays metadata forever, and so that we don’t turn a ThinkPad X220 into a machine distributing 1Gb/s to every other machine in the office. We could cut the CDN traffic by at least one order of magnitude, but possibly much more. This is better for the person paying the cloud bill, the person paying for the internet connection, and the planet as a whole.

:link: Feedback[edit]

IPFS is an existing project that’s existed for many years and allows sharing with other users not on your local network. It’s not packaged in any distributions and not trivial to install correctly. It’s main drawback is that it requires a internet to IPFS “gateway” which costs a large amount of money for the LVFS, and that it’s not EAR/ITAR compliant.

I’ve asked for feedback already on fedora-devel and have already started making changes and suggestions from that discussion – for instance splitting out a -libs subpackage. See Fedora Development — Adding Passim as a Fedora 40 feature? for the discussion.

:link: Benefit to Fedora[edit]

Fedora will consume less bandwidth when checking for firmware updates. Per user there is only a 2MB/day saving, but for millions of Fedora users this adds up to a huge amount of saved data (and money) globally.

:link: Scope[edit]

The code is already written, tested and ready to go. The passim package is already included in Fedora (although not installed by default) and fwupd needs an upstream release which includes this functionality – which is scheduled for 2 weeks time.

:link: Upgrade/compatibility impact[edit]

Old versions of fwupd will be updated and start sharing metadata with other local users with no changes required.

:link: How To Test[edit]

Install two physical machines or VMs with Fedora 40 that are both on the local network. Run fwupdmgr refresh on the first, and observe that passim dump or https://localhost:27500/ lists the published metadata file . Then run fwupdmgr refresh --verbose on the second machine and see that the file is downloaded from https://localhost:27500 rather than cdn.fwupd.org. Avahi needs to be enabled and running, as does passimd although both are autostarted as required.

:link: User Experience[edit]

Each user will use 2MB less bandwidth per day when there are other users on the local network with the same metadata file. There is no user-visible difference to any operation.

:link: Dependencies[edit]

None; fwupd will recommend passim to be installed by default and autolaunch it as required.

:link: Contingency Plan[edit]

Change Recommends: passim to Suggests: passim in the fwupd.spec file so that it’s not autoinstalled by default. In this case fwupd will fall back to downloading from the CDN every day.

:link: Documentation[edit]

:link: Release Notes[edit]

Fedora now uses a peer-to-peer service called Passim to reduce the amount of bandwidth used when downloading metadata.

From the Passim README:

Here we’ve assuming your local network (aka LAN) is a nice and friendly place, without evil people trying to overwhelm your system or feed you fake files. Although we request files by their hash (and thus can detect tampering) it still uses resources to send a file over the network.

We’ll assume that any network with working mDNS (as implemented in Avahi) is good enough to get metadata from other peers. If Avahi is not running, or mDNS is turned off on the firewall then no files will be shared.

Trusting networks based on mDNS support is not a good idea, there are good reasons to have mDNS enabled on untrusted networks (like using a printer on a semi-public network).

Also from the Passim README:

The file is sent when requested from a URL like https://192.168.1.1:27500/filename.xml.gz?sha256=hash – any file requested without the checksum will not be supplied

Since a lot of files have unique filenames and are public, obtaining the hash is not very hard. An attacker could request known filename+hash pairs to guess information about a device. This can be used for fingerprinting and might even be enough to uniquely identify a host across networks. The 2mb data cap doesn’t really help mitigate this attack.

An attacker could request known filename+hash pairs to guess information about a device

Have you got a specific example? I don’t see how the presence of metadata that every client is going to be downloading can provide any additional details about a host.

This can be used for fingerprinting and might even be enough to uniquely identify a host across networks.

How?

Have you got a specific example? I don’t see how the presence of metadata that every client is going to be downloading can provide any additional details about a host.

Clients only download firmware that they use, so the firmware exposes information about hardware (including peripherals), and the last time updates were downloaded.

How?

You use available firmware (which identifies hardware) to identify users, because it is unlikely that a significant portion of users on the same network have exactly the same hardware. Since fwupd can also update some peripherals, this makes even less likely that there will be hosts with the same firmware files cached. Even if this isn’t enough for identifying individual users, it can still be used to collect data for advertisers.

Fedora Workstation ships with all of the unprivileged ports open by default.

Having a web server running by default on an open port seems like a risk that outweighs the relatively minor benefit for most users.

Having a web server running by default on an open port seems like a risk that outweighs the relatively minor benefit for most users.

If the server is run as its own user and is properly sandboxed, this can be done relatively safely. The API is pretty small too, so there will be less security bugs. Its not a full web server with obscure bugs features.

Clients only download firmware that they use, so the firmware exposes information about hardware

Ohh this proposal is only about the metadata – the shared XML catalog that’s downloaded by all users regardless of installed hardware. You can use passim to share firmware too, but it’s not going to be enabled by default for Workstation as I agree there would be a privacy issue.

Sorry, I thought it was firmware-specific metadata. Helping reducing LVFS bandwidth usage is a good thing.

1 Like

Apologies for the late replies, but I missed it when this was announced.

Is passim explicitly limited to only sharing LVFS metadata in some way, or is passim a general-purpose tool that can be manually configured to share other data?

You describe this as a Self-Contained Change, but it occurs to me that if we want to support it for non-Workstation use-cases, there also needs to be communication and coordination done with groups like the Cloud and Server Working Groups to set up default firewall rules. On non-Workstation installs, the default firewall configuration is much stricter (in the case of Server, basically only sshd and Cockpit are open by default).

There were earlier comments in the threads that also make assumptions about how it’s sandboxed. I’d like to see that described in the Change Proposal, because FESCo will need this information to make a decision on allowing passim to be enabled by default (when installed).

1 Like

Is passim explicitly limited to only sharing LVFS metadata in some way

It would be set up to share LVFS metadata by default

or is passim a general-purpose tool that can be manually configured to share other data?

You certainly can publish anything (sudo passim publish on the CLI) but I’m not expecting anything to do that for F40.

if we want to support it for non-Workstation use-cases

I’d be fine with just workstation for F40.

make assumptions about how it’s sandboxed.

I think https://github.com/hughsie/passim/blob/main/data/passim.service.in is pretty good – although we’d need selinux policy changes I suppose.

Hmm, so it’s a brand-new server, written in C, enabled by default, running as root, listening on a public port.

I appreciate that quite a lot of work was put into making locking down the service, but why is it running as root‽

I don’t have any particular expectation that the code is not implemented correctly, but enabling both the server and client by default seems to create quite a bit of risk. The client runs as root too and will do periodic lookups on the network. This means that both the server and the client can be attacked easily by anyone on the same local network. Even assuming that there are no memory handling bugs, there could even be bugs like DoS through algorithmic complexity or large memory allocations.


If we want to save downloads, why don’t we do this with rpm packages? Any Fedora machine downloads order of magnitude more stuff for normal upgrades. While lvfs is a valid case, it’s also quite so niche that it’s strange to enable local sharing just for one thing used by this one service.

1 Like

but why is it running as root

I don’t think it is: systemd.exec

The client runs as root too

No, the fwupdmgr client definitely runs as non-root. The fwupd daemon runs as root, but that only accepts files over a file descriptor and can’t ever touch the network.

there could even be bugs like DoS through algorithmic complexity

Well there could certainly be bugs, but I don’t think we have to prove that software has no bugs before including it in Fedora – otherwise we’d never ship anything.

If we want to save downloads, why don’t we do this with rpm packages?

The metadata changes every day, and is security sensitive with timing, in that you don’t want to wait for the user to upgrade the OS just so that you can tell them that there’s a critical severity 10/10 firmware update available.

it’s also quite so niche that it’s strange to enable local sharing just for one thing used by this one service

I’ve deliberately scoped this very small for F40, but dnf metadata would certainly be in scope for future releases.

As it’s a new project, it would have been great if it had been written in a memory safe language so that we don’t keep adding critical software written in C to our distributions.

But given the limited scope and real effort made to harden the service using systemd features, this seems reasonable.

i’m globally in favor of the approach. We should be able to share hash-identified content on the local network to speed up updates / reduce server load / bandwidth costs. This will be a boost for people with limited, expensive or capped data plans.

I filed a (very lightly tested ;)) pull request to run as a dedicated user: Use a dedicated user to run the service by keszybz · Pull Request #18 · hughsie/passim · GitHub. It’s probably better to discuss details of security policy there.

The metadata changes every day, and is security sensitive with timing, in that you don’t want to wait for the user to upgrade the OS just so that you can tell them that there’s a critical severity 10/10 firmware update available.

All of those things apply to normal packages and package metadata too… In a great majority of cases, a swift update of software will be much more relevant than a swift update of firmware. I don’t want to say that the update of firmware is not important: it is, but the other updates are at least as important.


Have you done any fuzzing of the code that parses the input from the network connection?

1 Like

Much appreciated, thanks. I’ll do some testing and discuss upstream on that ticket.

Have you done any fuzzing of the code that parses the input from the network connection

Not yet; passim_server_handler_cb is really just a very small wrapper on top of SoupServer (which has had fuzzing work done). I certainly could add a fuzzing hander and split out the processing, but I’m not sure how valuable it would be given the additional processing [not done already by SoupServer] is really minimal.

1 Like

Closing change proposals for F40. The ship has sailed. Of course we can still discuss if needed, but please open new topics for that.