Why was podman chosen for mutable container environment on silverblue?

Hello, I’ve been looking at different Linux containerization technologies for a while now. Despite that, I still have a lot of questions about how it works precisely. So my question is: why was podman chosen as the implementation of a mutable distribution in silverblue? If I understand correctly, podman is supposed to be a replacement for docker. But docker isn’t very well suited for a mutable container - it uses immutable images, and container is just a mutable overlay over said image. You can’t really upgrade a container, you’ll just have to create a new one from newer image. Shouldn’t something like systemd-nspawn, LXC, etc. be used instead? As far as I know, they create mutable containers by default. Some of them even provide unprivileged containers too. Also podman is still in development and lacks features compared to the more mature options available.

I don’t think it was chosen for mutable environments because Silverblue was using it before Toolbox was created.

It works with rootless containers (unlike systemd-nspawn for now), uses the widespread OCI image format (unlike systemd-nspawn and LXC), and doesn’t use a daemon (unlike Docker). It has a friendly interface (unlike LXC) that is also compatible with Docker (unlike systemd-nspawn) and has a SELinux policy that Fedora uses. Developers are also involved in Fedora. Overall, that makes it clearly the best choice.

As far as I can tell, toolbox is the official way of creating a mutable environment on silverblue. I can see how podman can be a good docker replacement, but I don’t really understand why it was chosen as toolbox’s backend. Updating podman’s container to a new version requires reinstalling all previously installed packages, which is somewhat annoying. Also you can’t, if I understand correctly, easily layer new container on some other already existing one. And I don’t think these problems can be fixed with updates since they’re fundamental podman mechanics. Also, correct me if I’m wrong, aren’t all containerization solutions use the same (or slightly modified) selinux policy?

My point is: I don’t think that podman is the best possible solution for solving the mutable environment problem we have on silverblue. Maybe I’m misunderstanding something, but that’s how I feel about toolbox right now.

The original toolbox was a wrapper script for systemd-nspawn. I honestly don’t remember all the details, but after due diligence on this, podman was chosen as a better option. I realize that not remembering the details isn’t super helpful to this discussion, but Mark outlined quiet a few. If you’re missing something in toolbox please let us know and open a bug. If you prefer using a tool like nspawn, please continue to use it. We include systemd-nspawn by default in silverblue and it’s an incredibly valuable tool.