Background
Fedora Atomics are great. I have Fedora CoreOS installed on my personal VPS server. I deployed my apps using containers, configured Zincati, and completely forgot about it. I have Fedora Silverblue installed on my work PC and laptop. It’s solid, and I haven’t worried about updates breaking the system since then.
However, it needs a non-root package manager to work well for daily use. Flatpaks are great for desktop apps that require little interaction with the system, but they are not suitable for CLI tools or code editors. rpm-ostree requires restarts after installation, and local package layering can interfere with upgrades when using bootc. A package manager is needed to install user-level programs without fighting the immutable root.
Previously, I used Linuxbrew and chezmoi. It worked. But recently, support for Nix rolled out in Fedora 44, so I decided to give it a try.
Installation
Installing Nix on Fedora Atomic requires a bit of a hack: Nix needs to write to the /nix directory, which Fedora Atomic doesn’t allow by default, and Nix won’t accept a symlinked /nix. A workaround is to use a bind mount to mount /var/nix to /nix.
[Unit]
Description=Bind mount /var/nix to /nix
[Mount]
What=/var/nix
Where=/nix
Type=none
Options=bind
[Install]
WantedBy=local-fs.target
Experience
Nix Language
The Nix language itself is always seen as an obstacle for newcomers, but it is in fact quite straightforward and intuitive if you’ve learned a functional programming language before. There are only a few syntaxes:
- attrset/object/dictionary and array, like JSON/YAML
- function declaration: a few different forms, but you’ll know it when you meet a colon (
:) let .. in ..to declare scoped variables- a special kind of object called a
derivation, which is a recipe for how to build a directory inside the Nix store. Many concepts, like packages anddevShells, are in fact derivations with well-known output structures that Nix knows how to link into the right places. - Nix is lazy by default, so a lot of control flow is implemented with functions.
And thanks to LLMs, you can learn the language, and write configurations at the same time.
Home Manager
The experience is generally great. You can almost manage everything declaratively in your home, like programs, dotfiles, fonts, etc., and synchronize them between your devices through git.
There’s a useful option that makes Home Manager-installed packages better integrated with the system: [1]
targets.genericLinux = true
Issues & Workarounds
Graphics Driver
Nix apps are patched to load graphics drivers in /run/opengl-driver, which doesn’t exist in OSes other than NixOS. Home Manager recently provided an option to create the link for you: targets.genericLinux.gpu.
After enabling the module, Home Manager will give you a command to run after running home-manager switch. The command and relevant systemd service are as follows, for reference. [2]
#!/nix/store/i27rhb3nr65rkrwz36bchkwmav6ggsmn-bash-5.3p9/bin/bash
set -e
# Install the systemd service file and ensure that the store path won't be
# garbage-collected as long as it's installed.
unit_path=/etc/systemd/system/non-nixos-gpu.service
ln -sf /nix/store/7khlgj3b25j95pnnk9wj5sa9734m6s9x-non-nixos-gpu/lib/systemd/system/non-nixos-gpu.service "$unit_path"
ln -sf "$unit_path" "/nix/var/nix"/gcroots/non-nixos-gpu.service
systemctl daemon-reload
systemctl enable non-nixos-gpu.service
systemctl restart non-nixos-gpu.service
[Unit]
Description=GPU driver setup for Nix on non-NixOS Linux systems
[Install]
WantedBy=multi-user.target
[Service]
Type=oneshot
ExecStart=ln -nsf /nix/store/26lyy9zm656frjh6m0fl988b1fml9blx-non-nixos-gpu /run/opengl-driver
RemainAfterExit=yes
It works, but it gave me the inspiration that, since Nix is just a language that can produce derivations, which can be used to generate real directories in the store, we can just create a derivation in the flake output. With a paired systemd service that links the store path to the real system location, we can implement any modification to the system.
For example, buildEnv is a helper function from nixpkgs, which merges multiple derivation directories, folder by folder to produce a new derivation.
{
prefix = pkgs.buildEnv {
name = "system-nix-prefix";
paths = [ inputs.home-manager.packages.${pkgs.stdenv.hostPlatform.system}.default ];
};
graphicsDrivers = pkgs.buildEnv {
name = "system-nix-graphics-driver";
paths = with pkgs; [
mesa
libvdpau-va-gl
intel-media-driver
];
};
}
Then run commands like this. You can wrap them inside a bash script called something like fedora-nix-rebuild, just like NixOS:
sudo nix build --out-link /var/nix-system/prefix .#prefix
sudo nix build --out-link /var/nix-system/graphics-drivers .#graphicsDrivers
Add a systemd service to link /run/opengl-driver to /var/nix-system/graphics-drivers. And create an environment config that adds /var/nix-system/prefix/bin to $PATH and /var/nix-system/prefix/share to $XDG_DATA_DIRS. [3]
[Unit]
Description=Link Nix system graphics drivers
After=local-fs.target
ConditionPathExists=/var/nix-system/graphics-drivers
[Service]
Type=oneshot
ExecStart=/usr/bin/ln -nsfT /var/nix-system/graphics-drivers /run/opengl-driver
RemainAfterExit=true
[Install]
WantedBy=multi-user.target
PATH=${PATH}:/var/nix-system/prefix/bin
XDG_DATA_DIRS=${XDG_DATA_DIRS:-/usr/local/share:/usr/share}:/var/nix-system/prefix/share
Now you have another global package manager and graphics drivers working properly. And the mental model is just like NixOS if you have multiple users.
-
If you have just a single admin user using Nix, put the config in your home directory.
-
If you have other admin users that use Nix:
- Put the config into a shared folder like
/etc, but other users will need root privileges to edit their home config. - Let other users put config inside their own home, but there is no promise that the graphics driver in the root will work with their programs, since they are not sharing flake locks.
- Put the config into a shared folder like
Fonts
Home Manager has a fonts.fontconfig.enable option. When set to true, it will generate the correct fontconfig in $HOME/.config/fontconfig/conf.d/. However, it’s not allowed by default for Flatpak apps to access it. And the actual font files are still located inside the Nix store, so you need to allow Flatpak to access the two locations by running:
flatpak override --user \
--filesystem=/nix/store:ro \
--filesystem=xdg-config/fontconfig:ro
A more declarative way is to add the following lines to your Home Manager config (generated by ChatGPT):
home.activation.flatpakFontAccess = lib.hm.dag.entryAfter [ "writeBoundary" ] ''
if command -v flatpak > /dev/null 2>&1; then
run flatpak override --user \
--filesystem=/nix/store:ro \
--filesystem=xdg-config/fontconfig:ro
else
echo "Skipping Flatpak font access override because flatpak is not on PATH"
fi
'';
Distrobox
You need to make the Nix store and GPU drivers visible inside the Distrobox.
distrobox create --name fedora-44 \
--volume /nix:/nix:ro \
--volume /run/opengl-driver:/run/opengl-driver:ro
P.S. Toolbx, which comes with Fedora, doesn’t support mounting volumes.
Update Size
The update size is significantly larger in bootc than in ostree. There are tools like chunkah and the zstd:chunked option in Podman to optimize the layering. However, because of the nature of OCI images, a bottom layer change will invalidate all layers on top of it, so there seems to be no real solution. Please let me know if you know of a solution!
UID/GID Drift
The UID/GID of each user/group is assigned during the image build. However, /etc/passwd and /etc/group won’t follow image updates if you’ve created new users or modified groups locally, which will almost certainly happen after installation. So be careful if you have RPM packages that create users or groups, such as wireshark or nix.
If the group/user ↔ GID/UID mappings are later updated in the image, your local file ownership and group membership may get messed up. See also: Add support for chowning across upgrades · Issue #1263 · bootc-dev/bootc · GitHub
Conclusion
Now I have a pretty solid setup. The OS is distributed by my custom bootc image, automatically updated by GitHub Actions and Renovate bot. It is fully customizable during the build phase by editing the Containerfile, and unbreakable after deployment.
Nix allows me to put large dependencies in my home directory when possible. It also acts as a dotfiles and font manager. What’s more, it’s fully declarative and makes it easy to sync configurations between my devices.
There are various ways to create a development environment now.
- Distrobox: better isolation.
nix develop: more reproducible and lightweight.- Dev Containers: one container per project.
It’s been a long time since the last time I tinkered with my computer and workflows. This experiment took about two weeks in my free time, and I’m quite satisfied with the outcome. Thank you, Fedora team, for your hard work making this possible!
However, there’s a known issue that this option will introduce duplicate entries in your
$PATHand$XDG_DATA_DIRS. ↩︎At the time of writing, the service has been replaced by a systemd-tmpfiles config in upstream. ↩︎
This service can also be replaced by a systemd-tmpfiles configuration. ↩︎
