Disaster recovery - what is necessary and sufficient to backup?

Ask Fedora
1

I’d like to be able to restore my Fedora 41 Workstation setup in the event a disaster strikes the laptop, e.g. fire. Note I didn’t say “easily”, but that’d certainly be nice. I’m quite familiar with various backup strategies and am using borg (via Pika) for my ~, to a NAS that is in turn backed up across two physically disparate sites.

A borg backup of ~ is fine for my data but doesn’t capture the system config; this is something I’d normally make a non-issue via IaC (e.g. cloud-init + Ansible), but I don’t particularly want to have to manage my personal “daily driver” that rigorously.

I’m looking for a definition of the necessary and sufficient (i.e. bare minimum) information that is required to replicate an operating Fedora 41 Workstation instance to a newly installed instance. I’ve reviewed many of the posts here tagged as backup and there are many good ideas on system imaging, etc, I can’t really see the “necessary and sufficient” question addressed.

I expect I’d have a crude form of an answer if I got a second laptop and did a new install on it, then did a figurative rsync --dry-run --one-file-system a:/ b:/; but I trust there’s a better way (I also don’t have a second laptop available ;-).

For example:

  • things I installed or removed:

    • put dnf history list > ~/dnf-history in crontab, in turn backed up as part of my home dir

  • things I tweaked by editing them:

    • extract the files listed by sudo rpm --all --verify --nomtime --nomode --nouser --nogroup and add them to my regular borg backups (ignoring file attributes as for some reason there are many); separate borg backup as presumably will need to run as root
    • add /usr/local/ to the above, for completeness

  • things I configured via GNOME Settings or related apps:

    • I believe most of this is in GNOME’s ~/.* files, and thus captured in my ~ backup, but I presume system stuff such as network config is not; I expect there are a list of /etc/ paths I would need to explicitly backup, e.g. /etc/{NetworkManager,firewalld}
    • I suspect it’s a fool’s errand to try and be selective about /etc/, and simplest to just back the whole thing up and sort our the diffs later if needed…

Your insights on what I’ve missed in the above would be appreciated.

FTR, I have a bunch of other requirements/desires:

  • I don’t want a system image: rather I want to back up only the file-level delta between what comes with a Fedora 41 out-of-the-box and my machine’s current state

    • e.g. network configs, installed apps, GNOME Settings, anything in /etc/ (or ideally anywhere in /) that I tweaked and then promptly forgot about
    • the motivation for only backing up things that have changed is that a) it’s an implicit record of stuff I’ve fiddled with, and b) it’s far less data

  • set-and-forget: any approach that requires me to remember to do something is doomed to fail

  • I don’t need a particularly quick recovery time, replacing the physical hardware will take days to weeks so a few hours reinstalling/etc to get to a point where I can restore my previous state isn’t going to be a problem

FTR, I recognise there are some things that just can’t be backed up, e.g. firmware state (also likely not relevant as any replacement hardware may be a different model).

1 Like
2

There’s also the option to go with a mix:

  • System image (disk image or rather partition images of /, /boot and /boot/efi) after every system upgrade (so no more often than every 6 months), using a live ISO and GNOME Disks for example.
  • Regular data backup with an incremental backup system (e.g. Deja-Dup or Pika) of ~ and /etc/
  • Back up the list of user-installed packages.

When restore needed, the disk/partition images would be the base of a recovery (UUIDs will be preserved)[1]. Afterwards restoring the data, bringing system up to date and reinstalling additional packages.

For my particular setup, I rather reinstall the system from scratch, and only restore my home folder. There are less and less tweaks needed outside of the home folder nowadays[2].

  1. Or just swapping the damaged SSD/NVMe with the backup SSD/NVMe, in case only the disk is damaged, but laptop available and otherwise functional. ↩︎

  2. Those few system settings/tweaks needed can be made base on my notes. ↩︎

2 Likes
3

I’m not sure an answer exists, considering the subjective nature of what would be considered necessary and sufficient.

Tools to help make your own solution include: borg (of course), borgmatic (for ease of automation), and Relax-and-Recover/ReaR (bash tool for bare metal recovery including disk layout).

4

@bendl,

My goal is to only backup my data.

This is made possible by fully automating the provisioning and administration of systems and all the code needed to do this I consider part of my data. Disaster recovery is the provisioning of a new system and restoring my data.

There is a lot of discipline involved. Plenty of technical choices to make. For instance, I like to use a seperate storage device for everything that needs to be backed up and leave the boot/os device alone otherwise. How does that affect vm images, containers and images, databases etc all has to be considered and appropriately dealt with.

I have seen the practice of backing up everything but I’ve never seen those backups successfully used for full recovery. I have seen attempts to only backup the data but without full provisioning and all administration captured in code recovery was a nightmare.

5

Ta for the replies.

This is my experience too; though inevitably I forget about “that one thing” I set up as a system service for whatever reason.

As an example of what I’m aiming for, Apple’s Time Machine aims to “back up files on your Mac that weren’t part of the macOS installation”; though I’ve added one extension: “back up files on your workstation that weren’t part of OS installation or as installed via dnf”.

@steppybug: totally; this is how any reproducible system needs to be, to be reproducible. The hiccup is the “a lot of discipline involved” - for me, it’s far too much overhead for my personal machine (insert the proverb about the carpenter’s house being badly maintained).

2 Likes