Can't delete snapshot of subvolume

Hey,

I’m using Fedora 41 Silverblue for a month. Just liked the idea of immutable OS, but I got a problem from generating a snapshot which I cant delete anymore.

Here is what I have already tried and some logs: Markdown | vQg3L | Rocket Powered Pastebin

What can I do to remove this snapshot?

Thanks in advance

EDIT, posting logs

Volumes

patrick@fedora:~$ sudo btrfs subvolume list /
ID 256 gen 19536 top level 5 path @home
ID 257 gen 19536 top level 5 path @
ID 258 gen 19530 top level 5 path @data
ID 273 gen 15494 top level 258 path @data/admin/backups/volumes/home_20250130_0037
ID 274 gen 19360 top level 273 path @data/admin/backups/volumes/home_20250130_0037/home
ID 276 gen 19462 top level 258 path @data/admin/backups/volumes/home_20250204_0123

they are not mounted

>>> patrick@fedora:~$ mount | grep '/var/data/admin/backups/volumes/home_20250130_0037'
>>> patrick@fedora:~$ mount | grep '/var/data/admin/backups/volumes/home_20250130_0037/home'

but i can’t delete them…

>>> patrick@fedora:~$ sudo btrfs subvolume delete /var/data/admin/backups/volumes/home_20250130_0037/home/
Delete subvolume 274 (no-commit): '/var/data/admin/backups/volumes/home_20250130_0037/home'
ERROR: Could not destroy subvolume/snapshot: Read-only file system
>>> patrick@fedora:~$ sudo btrfs subvolume delete /var/data/admin/backups/volumes/home_20250130_0037
Delete subvolume 273 (no-commit): '/var/data/admin/backups/volumes/home_20250130_0037'
ERROR: Could not destroy subvolume/snapshot: Directory not empty

there is also another folder inside, which seems not to be a subvolume, but I can’t remove ith either

>>> patrick@fedora:~$ ls -l /var/data/admin/backups/volumes/home_20250130_0037/
total 0
drwxr-xr-x. 1 root    root     14 Jan  3 19:13 home
drwx------. 1 patrick patrick 502 Jan 29 23:24 patrick
 

>>> patrick@fedora:~$ sudo btrfs subvolume show /var/data/admin/backups/volumes/home_20250130_0037/patrick/
ERROR: Not a Btrfs subvolume: Invalid argument

>>> patrick@fedora:~$ sudo rm -rf /var/data/admin/backups/volumes/home_20250130_0037/patrick/
rm: cannot remove '/var/data/admin/backups/volumes/home_20250130_0037/patrick/.bash_logout': Read-only file system
rm: cannot remove '/var/data/admin/backups/volumes/home_20250130_0037/patrick/.bashrc': Read-only file system
rm: cannot remove '/var/data/admin/backups/volumes/home_20250130_0037/patrick/.mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}': Read-only file system
...

this is, what mount shows

patrick@fedora:~$ mount
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /sysroot type btrfs (ro,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on / type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /etc type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /usr type btrfs (ro,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /sysroot/ostree/deploy/fedora/var type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=4096k,nr_inodes=3545801,mode=755,inode64)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,size=5704352k,nr_inodes=819200,mode=755,inode64)
tmpfs on /run/credentials/systemd-cryptsetup@luks\x2d10474a4a\x2d88a9\x2d4afe\x2d957a\x2dc1a9598bf99a.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,nosuid,noexec,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13851)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,seclabel,pagesize=2M)
tmpfs on /run/credentials/systemd-journald.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-network-generator.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-udev-load-credentials.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-tmpfiles-setup-dev-early.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-sysctl.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-tmpfiles-setup-dev.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-vconsole-setup.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /var type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel,nr_inodes=1048576,inode64)
/dev/nvme0n1p2 on /boot type ext4 (rw,relatime,seclabel)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /var/home type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=256,subvol=/@home)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /var/data type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=258,subvol=/@data)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
tmpfs on /run/credentials/systemd-tmpfiles-setup.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-resolved.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=2852172k,nr_inodes=713043,mode=700,uid=1000,gid=1000,inode64)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
jetbrains-toolbox on /tmp/.mount_jetbraNlvAkH type fuse.jetbrains-toolbox (ro,nosuid,nodev,relatime,user_id=1000,group_id=1000)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)

And the filesystem is not ro

/dev/mapper/luks-... on /var/data type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=258,subvol=/@data)
1 Like

As pastebins can be deleted anytime, it’s best to post the content directly in a post here as that makes it easier to find in the future.

Try btrfs sub list -ta / and post that result, the table format is a bit easier to read and then also distinguishes the absolute/relative path for subvolumes.

The @ naming convention is non-standard in Fedora so I’m guessing this is a snapper or Timeshift convention. The idea of those utilities is to defer to their configuration and commands for managing the snapshots. If you start deleting snapshots outside of those regimes, it has a pretty good chance of confusing them at best, and at worst resulting in data loss.

$ sudo rm -rf

You might checkout some of the btrfs man pages, there’s quite a lot of documentation. While you can recursively rm subvolumes and read-write snapshots, this is suboptimal. For one it’s really slow as every file needs to use unlinkat() per file and directory, whereas btrfs subvolume delete will use an IOCTL that efficiently removes the file b-tree backing that snapshot.

Snapshots can be read-write or read-only, these appear to be read-only snapshots so they become immutable even to root.

man 5 btrfs
man btrfs
man btrfs subvolume

Not that btrfs subvolume has an option --subvolid and you can use the subvolume ID instead of a full path to subvolume (or subvolume snapshot). The subvolume ID, a.k.a. subvolid, is found in the first column of the btrfs sub list -ta / listing. This is often easier for nested subvolumes which can be anywhere.

What tool did you use to snapshot? Just manual btrfs?

It makes no sense, you have created the snapshot target subvolumes dont you? And they are in /var

How could this happen?

Hey, thanks for looking into this issue

patrick@fedora:~$ sudo btrfs sub list -ta /
[sudo] password for patrick: 
ID	gen	top level	path	
--	---	---------	----	
256	23308	5		<FS_TREE>/@home
257	23307	5		<FS_TREE>/@
258	23183	5		<FS_TREE>/@data
273	15494	258		<FS_TREE>/@data/admin/backups/volumes/home_20250130_0037
274	19360	273		<FS_TREE>/@data/admin/backups/volumes/home_20250130_0037/home

I think I added the @ to the volumes when I created them, I’ve read this is how Fedora handles them.

I did not use snapper and timeshift. I did not see the benefit over btrfs as I would still need this to send the snapshots to other devices (at least this is what I’ve found).

I’ve tried btrbk in the beginning, but than wrote my own script for handling old backups. This is because I haven’t created subvolumes for /etc and /var in the beginning, but also wanted to have a (partial) backup for them to avoid issues from messing something up.

It makes no sense, you have created the snapshot target subvolumes dont you? And they are in /var

Please help me to understand what makes no sense? I created home with the installer and data after installation. home handles apps etc, while I link all my Downloads, Desktop, AndSoOn to /data - they are mostly redownloadable or synced with some cloud service

OK I think I know what’s going on.

You need to delete subvol ID 274 before you can delete 273, because 274 is in 273. We don’t have recursive deletion of subvolume/snapshots in the user facing tools. (libbtrfsutil has an API for it though)

What do you get for:

btrfs prop get /var/data/admin/backups/volumes/home_20250130_0037

I’m guessing it’s read-only for some reason, and that’s why you can’t delete the nested home subvolume. And the nested home subvolume is why you can’t delete the subvolume containing it. Catch 22.

So you need to make home_20250130_0037 rw.

btrfs prop set /var/data/admin/backups/volumes/home_20250130_0037 ro false

And now you can delete the nested home subvol even though its read-only. And now you can delete subvol 0037.