Btrfs-patrol: snapshots before every dnf transaction, and one-command rollback on Fedora's default btrfs layout

Project Discussion
1

Hi all,

I’ve been working on btrfs-patrol, a small snapshot and rollback tool made for Fedora’s default install: btrfs with the root and home subvolumes, /boot on its own partition, and optionally LUKS. It’s now in COPR for Fedora 43 and newer (x86_64 and aarch64).

What it does

  • Takes a read-only snapshot of / before each dnf5 transaction, through libdnf5-plugin-actions. The description says what changed, such as upgrade kernel-core, mesa-dri-drivers +12 more; install tree.
  • Takes a daily snapshot with a systemd timer, and prunes old ones (50 by default; snapshots you mark as kept are never pruned).
  • Rolls back with one command: sudo btrfs-patrol rollback 42, then reboot. The current state isn’t lost; it’s kept as a snapshot you can roll forward to.

What it checks before a rollback

/boot isn’t part of the root subvolume on Fedora, so a rollback can leave you with a kernel the restored system has no modules for. btrfs-patrol refuses to roll back unless the snapshot has modules for the running kernel and that kernel has a boot entry, either a systemd-boot/GRUB BLS entry or a unified kernel image. It also refuses layouts that mount the root by subvolid=, which can’t follow a rollback, and undoes its steps if one fails.

It also handles SELinux: the snapshot store gets its labels, and it’s excluded from full relabels, so a relabel can’t mislabel a snapshot you might roll back to.

Install

sudo dnf copr enable jccl1706/btrfs-patrol
sudo dnf install btrfs-patrol
sudo btrfs-patrol setup --dry-run
sudo btrfs-patrol setup

setup shows what it will change (a snapshots subvolume mounted at /.snapshots, an fstab line with a backup of the old file, the timer) and asks before doing it. man btrfs-patrol has the full reference.

Why another tool?

Snapper and Timeshift are both good, but on Fedora’s default layout a rollback isn’t one step: Timeshift’s btrfs mode expects Ubuntu-style @ subvolumes, and Snapper’s rollback needs extra setup to boot the restored root. btrfs-patrol only targets Fedora’s layout, and tries to do that one thing carefully. It’s inspired by timepatrol (GitHub - abdeoliveira/timepatrol: BTRFS snapshot manager and rollback tool · GitHub) by Alan Barros de Oliveira, reimplemented in Python with no dependencies beyond btrfs-progs and util-linux.

Status

It’s young (0.4.1). I’ve tested setup, dnf and timer snapshots, and real rollbacks and reboots on a ThinkPad T480 (LUKS, LVM, systemd-boot, SELinux enforcing), on a stock Fedora Workstation install with GRUB, and on Fedora 44 VMs including one booting a unified kernel image — and a Fedora 44 to 45 upgrade rolled back to 44. Since 0.3.0 it can also snapshot other subvolumes: /home, or /var/log once it is one, each rolled back on its own. Snapshots aren’t backups, so please keep backups.

Source, docs and issues: GitHub - jccl1706/btrfs-patrol: Btrfs snapshot manager and rollback tool for Fedora, inspired by timepatrol · GitHub

Feedback is very welcome, especially from setups I haven’t tried: GRUB instead of systemd-boot, other disk layouts, Silverblue-style systems (not supported yet), or anything that makes setup refuse.

4 Likes
Question about my topic being moved to The Water Cooler
5

Team Workflows, where would this post belong to? I somewhat agree with @computersavvy that it somehow doesn’t really fit in Ask Fedora but it’s gonna get lost in the The Water Cooler dumpster.

On the other hand, user is requesting feedback, so maybe Ask Fedora is the best place.

btrfs

6

This tool is a really good idea.

Are you open to collaborators?

Regarding where the post should live, I would think Project Discussion, with the engineering tag would be appropriate, as @bookwar suggested.

7

Thanks! Yes, very much open to collaborators — issues and pull requests are welcome at GitHub - jccl1706/btrfs-patrol: Btrfs snapshot manager and rollback tool for Fedora, inspired by timepatrol · GitHub, and I’d be glad of testing on setups I don’t have: GRUB with LUKS, other disk layouts, and anything that makes setup refuse.

Since the first post, 0.3.0 adds snapshots of other subvolumes: /home, or /var/log once it’s a subvolume, each rolled back on its own, so undoing a bad update never touches your documents. It’s in COPR for Fedora 43+ on x86_64 and aarch64.

And thanks for the pointer on the category — Project Discussion with the engineering tag sounds right. I’ll ask the moderators to move it rather than repost.

8

I am really interested to test this tool. However now when testing the newest beta, i would prefer to focus on that. As I also help a lot in the Ask section, it is quite busy now till after the final release of F45.

I would like to know if this tool is able to undo a dnf upgrade to a never Fedora version?
Otherwise we do have dnf history list and dnf history info Nr and then we can also undo a with dnf it selves:

 dnf history --help
Usage:
  dnf5 [GLOBAL OPTIONS] history <COMMAND> ...
                                
Query Commands:                 
  list                          List transactions
  info                          Print details about transactions
                                
Software Management Commands:   
  undo                          Revert all actions from the specified transaction
  redo                          Repeat all actions from the specified transaction
  rollback                      Undo all transactions performed after the specified transaction
  store                         [experimental] Store transaction to a file

My opinion about the Project Discussion engineering you can se here:
Question about my topic being moved to The Water Cooler - #7 by ilikelinux

11

Thanks to the suggestions above I agree that this topic better belongs to the Project Discussion → engineering and/or websites-and-apps locations. It has been relocated.

2 Likes
12

This is an OK place for this request for feedback, but if you really want to promote the new open source software, that’s what Fedora Magazine is normally used for.

1 Like
14

Good question — rather than guess I tested it, on a Fedora 44 VM upgraded to 45.

Short answer: yes, with one caveat about the kernel.

  • dnf system-upgrade download creates no snapshot, which is correct — nothing is installed on the running system yet.
  • The offline transaction does trigger the pre-transaction hook. btrfs-patrol took a snapshot before the upgrade ran, described as “upgrade 7zip, abattis-cantarell-fonts +836 more; install fips-provider, kernel, kernel-core +33 more; downgrade libldb, libopenmpt, librepo +6 more; remove nwg-panel”. So you get an automatic pre-upgrade snapshot without doing anything.
  • Rolling back to it restored F44 completely: /etc/fedora-release back to 44, rpm -E %fedora = 44, packages the upgrade installed (fips-provider) gone, packages it removed (nwg-panel) restored, SELinux still enforcing, no failed units. The F45 state is kept as a new snapshot, so you can roll forward again.

The caveat is /boot. It isn’t part of the root subvolume, so a rollback restores /usr/lib/modules but not the kernels. After a release upgrade the ESP still offers the new kernel, which the restored root has no modules for. btrfs-patrol warns about exactly this:

warning: snapshot 14 has no modules for 7.2.5-300.fc45.x86_64;
         choose 7.2.4-200.fc44.x86_64 in the boot menu when you reboot

It permits the rollback because the kernel you are currently running is present in the snapshot, but you have to pick the older kernel at the boot menu. In my VM the older entry happened to be the default so it came back unattended — I would not rely on that being true everywhere.

On dnf history undo: it is a good fit for ordinary transactions, working within the RPM database. A subvolume rollback is a different thing — it restores the whole filesystem state, including files dnf does not own, in one step.

That said, this is one VM test rather than long experience with release upgrades; the tool was built for day-to-day dnf transactions. I will add the kernel caveat to the man page.

1 Like
15

If you want to restore the kernel(s) to the /boot partition after a rollback, I think you just need to call kernel-install (with the correct parameters, or maybe just “add-all”).

16

It relies on the previous package version existing in the repos.

For Fedora’s own packages this usually works ok, as long as you have the updates-archive repo configured, and you specify --enablerepo=updates-archive when doing the undo.

For third-party repos (including RPM Fusion) it can be problematic, because you can’t rely on being able to get the previous package version.

That’s not a problem when using btrfs snapshots, because the previous version is still on your disk in the snapshot, and doesn’t have to be downloaded from anywhere.

1 Like
17

@glb — good call, I tested it on the same VM, which was still sitting in the rolled-back state.

kernel-install add-all on its own does not clear it. It ran fine and reinstalled the F44 kernel, but the F45 UKI stayed in the ESP. The reason is structural: kernel-install list enumerates kernels from /usr/lib/modules, and the rollback had already taken the F45 modules away, so add-all has no way to see the stale entry. It also rebuilds the images of every kernel it does find — on this VM that was a dracut run of several minutes.

What works is the targeted removal:

sudo kernel-install remove 7.2.5-300.fc45.x86_64

That took the UKI out of the ESP, and btrfs-patrol stopped warning entirely — before, the first line of rollback -n was warning: snapshot 14 has no modules for 7.2.5-300.fc45.x86_64; afterwards the output starts straight at the snapshot table. I rebooted to confirm rather than assume: boots in 4.2 s on the F44 kernel, no failed units, SELinux still enforcing.

So the two directions need different commands:

  • a kernel present in the snapshot but missing from /boot → kernel-install add-all, your suggestion, which is right for that case
  • a kernel present in /boot that the restored root has no modules for (the release-upgrade case) → kernel-install remove VERSION

That is now in the man page, and it suggests a feature: rollback already detects this exact condition in order to print the warning, so it could offer to remove those entries itself rather than leaving it as boot-menu discipline.

@pg-tips — that matches what I just measured, and the example turned out to be my own package.

On the same VM, transaction 17 upgraded btrfs-patrol 0.1.1 → 0.3.0. Trying to undo it:

$ sudo dnf history undo 17
Failed to resolve the transaction:
Cannot perform Install action, no match for: btrfs-patrol-0.1.1-1.fc44.noarch.

My COPR still offers 0.2.0, 0.2.1, 0.2.2 and 0.3.0, but 0.1.1 has aged out, so there is nothing to install and the undo cannot resolve. The snapshot taken before that upgrade still holds 0.1.1 on disk and rolls back without downloading anything.

It is not only third-party repos, either. On that VM dnf list --showduplicates bash offers exactly one version, because updates-archive is not configured by default — so an undo of a core package can fail the same way unless you have thought about it in advance.

That is a better statement of the difference than the one I made: dnf history undo reconstructs a past state from packages that must still be fetchable, while a snapshot is the past state. Thanks — I will work this into the README.

18

I am not sure if these guides would be of any utility here but it is an alternative snapshot solution using snapper and btrfs-assistant with optional encryption. However, the guides require a custom layout without a separate /boot partition.

19

0.4.1 is out, and it is a hardening release rather than a feature one. After the questions in this thread sent me looking, I reviewed every module and fixed what that turned up. The tests went from 162 to 212, one for each fix.

The findings worth naming, because most of them were silent:

  • A snapshot taken while /.snapshots was not mounted landed in the root subvolume instead. It reported :: created snapshot 1 and exited 0, but the entry became invisible the moment the real store was mounted again — never listed, never pruned, the space not reclaimable — and it reused an ID that already existed. The daily timer would have done that every day while reporting success. check and rollback both refused in that state; only snapshot did not.
  • rollback ran its checks, printed the plan, then waited at the confirmation prompt — an unbounded wait — and acted on those checks afterwards. In between, the timer or a dnf transaction can prune the target, or anything can create a nested subvolume. It now re-checks under the lock and refuses without touching anything.
  • A second Ctrl-C while a failed rollback was unwinding could abandon the unwind before the step that puts the root subvolume back under its name. That leaves no root subvolume, which is a boot into the dracut emergency shell.
  • Three ways the dnf hook could hit your transaction: an unexpected JSON shape raised out of the hook, a decode error from a corrupt config file was not caught by its guard, and replies were only time-bounded for their first byte, so a half-written one could block forever. The actions plugin treats a failing pre_transaction command as a plugin error, so each of those could have aborted a dnf transaction. Nothing this tool gets wrong is worth that.
  • Snapshot descriptions were printed to the terminal unsanitised. They come from dnf’s package lists as well as from you, and an escape sequence in one was obeyed rather than shown.

The kernel caveat from further up the thread is in the shipped manual page now, along with a prune-kernels command that removes boot entries whose modules a rollback took away.

sudo dnf copr enable jccl1706/btrfs-patrol
sudo dnf upgrade btrfs-patrol

@notarobot — thanks, those guides are worth linking. The distinction is the layout: they set up a system without a separate /boot, which avoids this whole class of problem, but you have to install that way from the start. btrfs-patrol is aimed at the stock Fedora layout, where /boot is separate and outside the root subvolume — which is precisely why it checks that the kernel you will boot has modules in the snapshot before it lets a rollback through. If someone is happy to reinstall with a custom layout, snapper and btrfs-assistant are a well-trodden path; this is for people who already have the default one.

Still young software on the part of the system you most need to work, so keep backups that are not snapshots.

20

0.5.0 adds the thing this tool was missing: snapshots can now appear in the boot menu, so a system too broken to log in to — after rm -rf /etc, or an update that leaves you at an emergency shell — can be booted from a snapshot and rolled back from there, with no live USB.

That was the gap between “undo a bad update” and “recover a broken system”, and it is the part I most wanted to close.

It is off by default, because the boot menu belongs to the whole system and filling it is not something to do to someone unasked:

[boot]
entries = 3

The newest snapshots then get an entry each, written when a snapshot is taken and removed when one is deleted or pruned. They are Boot Loader Specification Type #1 entries. They also work where kernel-install uses layout=uki and there are no Type #1 entries at all, because the kernel and initrd that go into the unified image are still on the boot partition.

Booting one gives you the snapshot read-only — nothing you do is kept and the snapshot cannot be altered — with /home writable as usual, and btrfs-patrol rollback <ID> restores the root subvolume, which is not running at that point, so nothing is using it.

What testing it actually taught me, since this is the interesting part and none of it came from the test suite:

  • systemd.volatile=overlay seemed the obvious way to give a read-only snapshot a writable root. It hangs: PID 1 loads SELinux policy, relabels /dev, and stops. Fedora’s initrd does not build the overlay it asks for. Plain ro boots fine — the first attempt reached a full graphical session with zero failed units.
  • A unified kernel image bakes systemd.machine_id= into its command line, which makes systemd treat the machine-id as transient and commit it to disk at boot. On a read-only root that fails and the system comes up degraded. Dropped.
  • initrd= has to be dropped too. A Type #1 entry names the initrd in its own field; the argument is added by the loader at boot, and keeping both tells it about the initrd twice. I only saw this on real hardware, because the VM’s command line had no initrd= at all.
  • resume= is dropped: restoring a hibernation image of the normal system on top of a snapshot is not something to leave possible.

Everything needed to reach the disk — rd.luks.uuid, rd.lvm.lv, root=UUID — is carried over from the running command line rather than rebuilt, which is what makes it work on an encrypted machine.

Tested on systemd-boot, on two layouts: a VM using unified kernel images and no encryption, and a ThinkPad T480 with LUKS, LVM and ordinary Type #1 entries. On both it booted read-only with the desktop up, and on the T480 I rolled the system back from inside the snapshot boot.

Where I could use help: I have not tested this with GRUB. Fedora’s GRUB reads the same Type #1 files through blscfg, so in principle one entry serves both loaders, but “in principle” is not “tested”, and the README says so. Since Fedora Workstation ships GRUB, most people here are better placed to try it than I am. If you turn entries on and the snapshots show up — or do not — in your GRUB menu, I would like to know.

sudo dnf copr enable jccl1706/btrfs-patrol
sudo dnf upgrade btrfs-patrol

Still young software on the part of the system you most need to work. Keep backups that are not snapshots.

1 Like
21

I am also aware of the /boot which is ext4, and causes problems to work with Btrfs snapshots.
However since F44:

Keep this in mind to test in a VM. You could even test with the new Azure. It is now based on Fedora. You see, your tool could get famous very soon as long as you keep it simple and without changing the default layout!