Backup and Restore Silverblue root system state

Hi all !
I am using Fedora Silverblue v37+

Since v36 this change we can take advantage of Btrfs features like snapshots and rollbacks:

By moving /var into its own subvolume, it can be independently snapshot and rolled back from the “root” subvolume, which contains /etc and /usr.

The ability to snapshot “var” and use Btrfs send/receive to replicate only “var” permits for an efficient way of backing up the variable system data, similar to snapshotting “home” and replicating it as a way of backing up user data.

A clean install will restore the “root”, therefore it doesn’t strictly need to be backed up. Meanwhile “var” and “home” can be restored using snapshot replication via send/receive, or a favorite backup utility.

And since v37 with this other change /sysroot is read-only so that:

On rpm-ostree based systems, the real root (the root directory of the root partition on the disk) is mounted under the /sysroot path. By default it contains the state of the system (the content of var and etc) as well as the system versions themselves (each versioned copy of /usr) in the ostree repository (/ostree/repo).

Do those two changes enable us to restore a running and coherent system state ?
And what would be the best way to achieve this;

  • creating separate snapshots of subvolumes /var /root ?
  • or should we backup /sysroot instead ?

Since /etc will be in the /root subvolume and /var in it’s own subvolume, do we need to create and manage the snapshots of each subvolume simultaneously ?

I do not quite understand how to reap the benefits of a separate /var subvolume and readonly /sysroot for the purpose of backing up and restoring a Silverblue system state to a defined point in time.

Thanks !

While I don’t use either silverblue or kinoite I do have some thoughts on your question.

Both /etc and /var seem to be sections that are changeable.
Both /var and /etc contain files that are changed/updated as different programs and packages are installed or configured and as time passes.

Your idea seems great – But!
A Later reinstall in an attempt to recover, or just having to do a snapshot restore could easily result in those parts being out of sync if not done as a single snapshot of the whole system. This may or may not cause a problem depending upon the use case involved.

Thanks @computersavvy for your feedback

Indeed it seems to me, for recovery purpose, to do a single snapshot of all the btrfs volume at once; thus having in sync the same state for /root and /var.

The intended benefits of a separate /var subvolume as stated by the first change I have cited above, still evade me …

Years past, when disk drives were small, the dynamic nature of /var with logs, mail files, etc. often caused a system crash if left as part of the root file system. The workaround at that time to prevent the system crashes was to make /var its own partition/drive with an independent file system such that having /var run out of space did not cause a system crash.

Today, with having an abundance of drive space on most systems it seems totally unnecessary to split /var into a separate file system. In fact, if the /var subvolume is on the same btrfs volume as the root and home subvolumes then all have the same exact space limitations – the size of the total btrfs volume. Thus even having /var as a separate subvolume seems a bit outdated.

Using df on a btrfs system shows each subvolume with the same used and available space while du shows the actual space used by files within the particular directory tree.

It’s been as a Fedora Change accepted not even a year ago ! Last updated: 2022-02-09

Do you mean that even in a subvolume a full /var will crash a Fedora Silverblue system ?

If /var is part of the same btrfs volume as /, and the volume fills up (regardless of cause), then yes it may cause crashes and difficulty in recovery. There have been several threads recently where users reported problems and it was determined that the btrfs volume had at some point been filled up and caused the failure. I cannot point to one at present but constantly am reading here and have noted them.

Note this from my newly installed VM, and look at the data for /home and /.

# df
Filesystem     1K-blocks    Used Available Use% Mounted on
devtmpfs            4096       0      4096   0% /dev
tmpfs            1987928       0   1987928   0% /dev/shm
tmpfs             795172    1468    793704   1% /run
/dev/vda3       50763776 4200340  45894140   9% /
tmpfs            1987932      84   1987848   1% /tmp
/dev/vda3       50763776 4200340  45894140   9% /home
/dev/vda2         996780  237480    690488  26% /boot
/dev/vda1         613160   17780    595380   3% /boot/efi
tmpfs             397584     116    397468   1% /run/user/1000

[root@localhost-live user]# lsblk -f
NAME   FSTYPE FSVER LABEL                 UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
zram0                                                                                         [SWAP]
├─vda1 vfat   FAT32                       4726-38DE                             581.4M     3% /boot/efi
├─vda2 ext4   1.0                         0752c112-f782-4cf7-a6af-25a95c086288  674.3M    24% /boot
└─vda3 btrfs        fedora_localhost-live c358b97d-7d36-404b-8181-3d004b228193   43.8G     8% /home

In silverblue, having /var in a separate subvolume has advantages. I understand what you are saying about the legacy but there are modern reasons for separating them.

Since / is immutable, having /var(and /home) separate means you can take snapshots of those subvols. If they were together, you would need to take a snapshot of the entire partition which is pointless in this case.

I understand that and totally concur about /home, but it still seems that the potential for getting out of sync with /var having a separate snapshot from the / snapshot is risky.

I don’t really know how the file system is structured with both flatpaks and rpms that can be installed. As long as everything that can be changed is in sync then it seems OK, but I still have trouble wrapping my mind around the differences and how the ‘immutable’ system can be updated.

I have not yet decided to jump into that pool.

Is there a reason /var needs to be in sync with /? Isn’t it safe for them to diverge assuming it is done offline?