Can't delete snapshot of subvolume

Ask Fedora
, ,
1

Hey,

I’m using Fedora 41 Silverblue for a month. Just liked the idea of immutable OS, but I got a problem from generating a snapshot which I cant delete anymore.

Here is what I have already tried and some logs: Markdown | vQg3L | Rocket Powered Pastebin

What can I do to remove this snapshot?

Thanks in advance

EDIT, posting logs

Volumes

patrick@fedora:~$ sudo btrfs subvolume list /
ID 256 gen 19536 top level 5 path @home
ID 257 gen 19536 top level 5 path @
ID 258 gen 19530 top level 5 path @data
ID 273 gen 15494 top level 258 path @data/admin/backups/volumes/home_20250130_0037
ID 274 gen 19360 top level 273 path @data/admin/backups/volumes/home_20250130_0037/home
ID 276 gen 19462 top level 258 path @data/admin/backups/volumes/home_20250204_0123

they are not mounted

>>> patrick@fedora:~$ mount | grep '/var/data/admin/backups/volumes/home_20250130_0037'
>>> patrick@fedora:~$ mount | grep '/var/data/admin/backups/volumes/home_20250130_0037/home'

but i can’t delete them…

>>> patrick@fedora:~$ sudo btrfs subvolume delete /var/data/admin/backups/volumes/home_20250130_0037/home/
Delete subvolume 274 (no-commit): '/var/data/admin/backups/volumes/home_20250130_0037/home'
ERROR: Could not destroy subvolume/snapshot: Read-only file system
>>> patrick@fedora:~$ sudo btrfs subvolume delete /var/data/admin/backups/volumes/home_20250130_0037
Delete subvolume 273 (no-commit): '/var/data/admin/backups/volumes/home_20250130_0037'
ERROR: Could not destroy subvolume/snapshot: Directory not empty

there is also another folder inside, which seems not to be a subvolume, but I can’t remove ith either

>>> patrick@fedora:~$ ls -l /var/data/admin/backups/volumes/home_20250130_0037/
total 0
drwxr-xr-x. 1 root    root     14 Jan  3 19:13 home
drwx------. 1 patrick patrick 502 Jan 29 23:24 patrick
 

>>> patrick@fedora:~$ sudo btrfs subvolume show /var/data/admin/backups/volumes/home_20250130_0037/patrick/
ERROR: Not a Btrfs subvolume: Invalid argument

>>> patrick@fedora:~$ sudo rm -rf /var/data/admin/backups/volumes/home_20250130_0037/patrick/
rm: cannot remove '/var/data/admin/backups/volumes/home_20250130_0037/patrick/.bash_logout': Read-only file system
rm: cannot remove '/var/data/admin/backups/volumes/home_20250130_0037/patrick/.bashrc': Read-only file system
rm: cannot remove '/var/data/admin/backups/volumes/home_20250130_0037/patrick/.mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}': Read-only file system
...

this is, what mount shows

patrick@fedora:~$ mount
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /sysroot type btrfs (ro,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on / type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /etc type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /usr type btrfs (ro,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /sysroot/ostree/deploy/fedora/var type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=4096k,nr_inodes=3545801,mode=755,inode64)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,size=5704352k,nr_inodes=819200,mode=755,inode64)
tmpfs on /run/credentials/systemd-cryptsetup@luks\x2d10474a4a\x2d88a9\x2d4afe\x2d957a\x2dc1a9598bf99a.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,nosuid,noexec,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13851)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,seclabel,pagesize=2M)
tmpfs on /run/credentials/systemd-journald.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-network-generator.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-udev-load-credentials.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-tmpfiles-setup-dev-early.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-sysctl.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-tmpfiles-setup-dev.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-vconsole-setup.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /var type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel,nr_inodes=1048576,inode64)
/dev/nvme0n1p2 on /boot type ext4 (rw,relatime,seclabel)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /var/home type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=256,subvol=/@home)
/dev/mapper/luks-10474a4a-88a9-4afe-957a-c1a9598bf99a on /var/data type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=258,subvol=/@data)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
tmpfs on /run/credentials/systemd-tmpfiles-setup.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
tmpfs on /run/credentials/systemd-resolved.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=2852172k,nr_inodes=713043,mode=700,uid=1000,gid=1000,inode64)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
jetbrains-toolbox on /tmp/.mount_jetbraNlvAkH type fuse.jetbrains-toolbox (ro,nosuid,nodev,relatime,user_id=1000,group_id=1000)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)

And the filesystem is not ro

/dev/mapper/luks-... on /var/data type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=258,subvol=/@data)
1 Like
3

As pastebins can be deleted anytime, it’s best to post the content directly in a post here as that makes it easier to find in the future.

5

Try btrfs sub list -ta / and post that result, the table format is a bit easier to read and then also distinguishes the absolute/relative path for subvolumes.

The @ naming convention is non-standard in Fedora so I’m guessing this is a snapper or Timeshift convention. The idea of those utilities is to defer to their configuration and commands for managing the snapshots. If you start deleting snapshots outside of those regimes, it has a pretty good chance of confusing them at best, and at worst resulting in data loss.

$ sudo rm -rf

You might checkout some of the btrfs man pages, there’s quite a lot of documentation. While you can recursively rm subvolumes and read-write snapshots, this is suboptimal. For one it’s really slow as every file needs to use unlinkat() per file and directory, whereas btrfs subvolume delete will use an IOCTL that efficiently removes the file b-tree backing that snapshot.

Snapshots can be read-write or read-only, these appear to be read-only snapshots so they become immutable even to root.

man 5 btrfs
man btrfs
man btrfs subvolume

Not that btrfs subvolume has an option --subvolid and you can use the subvolume ID instead of a full path to subvolume (or subvolume snapshot). The subvolume ID, a.k.a. subvolid, is found in the first column of the btrfs sub list -ta / listing. This is often easier for nested subvolumes which can be anywhere.

7

What tool did you use to snapshot? Just manual btrfs?

It makes no sense, you have created the snapshot target subvolumes dont you? And they are in /var

How could this happen?

8

Hey, thanks for looking into this issue

patrick@fedora:~$ sudo btrfs sub list -ta /
[sudo] password for patrick: 
ID	gen	top level	path	
--	---	---------	----	
256	23308	5		<FS_TREE>/@home
257	23307	5		<FS_TREE>/@
258	23183	5		<FS_TREE>/@data
273	15494	258		<FS_TREE>/@data/admin/backups/volumes/home_20250130_0037
274	19360	273		<FS_TREE>/@data/admin/backups/volumes/home_20250130_0037/home

I think I added the @ to the volumes when I created them, I’ve read this is how Fedora handles them.

I did not use snapper and timeshift. I did not see the benefit over btrfs as I would still need this to send the snapshots to other devices (at least this is what I’ve found).

I’ve tried btrbk in the beginning, but than wrote my own script for handling old backups. This is because I haven’t created subvolumes for /etc and /var in the beginning, but also wanted to have a (partial) backup for them to avoid issues from messing something up.

It makes no sense, you have created the snapshot target subvolumes dont you? And they are in /var

Please help me to understand what makes no sense? I created home with the installer and data after installation. home handles apps etc, while I link all my Downloads, Desktop, AndSoOn to /data - they are mostly redownloadable or synced with some cloud service

9

OK I think I know what’s going on.

You need to delete subvol ID 274 before you can delete 273, because 274 is in 273.

What do you get for:

btrfs prop get /var/data/admin/backups/volumes/home_20250130_0037

I’m guessing it’s read-only for some reason, and that’s why you can’t delete the nested home subvolume. And the nested home subvolume is why you can’t delete the subvolume containing it. Catch 22, even for recursive subvolume delete flag -R.

So you need to make home_20250130_0037 rw.

btrfs prop set /var/data/admin/backups/volumes/home_20250130_0037 ro false

And now you can delete the nested home subvol even though its read-only. And now you can delete subvol 0037.

10

Thanks, this was the solution I needed!

11

It has been added to btrfs-progs 6.12.

  • subvolume delete: add new option to do recursive subvolume deletion (for regular user delete only accessible subvolumes)

sudo btrfs subvolume delete -R /subvolume/subvolume2

R|–recursive

delete subvolumes beneath each subvolume recursively

This requires either CAP_SYS_ADMIN or the filesystem must be mounted with user_subvol_rm_allowed mount option. In the unprivileged case, subvolumes which cannot be accessed are skipped. The deletion is not atomic.

1 Like
12

Thanks! Good to know!

I did just test it in the case of a subvolume containing both rw and ro subvolumes, then setting the containing subvolume to ro, and then trying to delete it with and without -R and it fails with the same message:

ERROR: Could not destroy subvolume/snapshot: Read-only file system

Which makes sense because changing anything inside the subvolume requires write privileges and ro flag is set on the containing subvolume prevents it. The containing subvolume can’t be deleted because recursive subvolume delete is not atomic - it requires deleting nested subvolumes first. Again catch-22.

13

If I understand correctly, your tests are on the non-privileged case, right?
Because I did some tests, and I can delete the nested RO snapshots inside another subvolume by deleting the TOP subvolume.

emanu@fedora ~> sudo btrfs subvolume create /home/emanu/volumeTOP

Create subvolume '/home/emanu/volumeTOP'

emanu@fedora ~> sudo btrfs subvolume snapshot -r /home/emanu/volumeTOP /home/emanu/volumeTOP/VolumeRO_Nested

Create readonly snapshot of '/home/emanu/volumeTOP' in '/home/emanu/volumeTOP/VolumeRO_Nested'

emanu@fedora ~> sudo btrfs subvolume snapshot -r /home/emanu/volumeTOP /home/emanu/volumeTOP/VolumeRO_Nested2

Create readonly snapshot of '/home/emanu/volumeTOP' in '/home/emanu/volumeTOP/VolumeRO_Nested2'

emanu@fedora ~> sudo btrfs subvolume snapshot /home/emanu/volumeTOP /home/emanu/volumeTOP/VolumeRW_Nested2

Create snapshot of '/home/emanu/volumeTOP' in '/home/emanu/volumeTOP/VolumeRW_Nested2'

emanu@fedora ~> sudo btrfs property get /home/emanu/volumeTOP/VolumeRO_Nested/

ro=true

emanu@fedora ~> sudo btrfs property get /home/emanu/volumeTOP/VolumeRO_Nested2

ro=true

emanu@fedora ~> sudo btrfs property get /home/emanu/volumeTOP/VolumeRW_Nested2

ro=false

emanu@fedora ~> sudo btrfs subvolume list -ot /home/emanu/volumeTOP/ 

ID      gen     top level       path
--      ---     ---------       ----
3214    450659  3213            @home/emanu/volumeTOP/VolumeRO_Nested
3215    450660  3213            @home/emanu/volumeTOP/VolumeRO_Nested2
3216    450661  3213            @home/emanu/volumeTOP/VolumeRW_Nested2


emanu@fedora ~> sudo btrfs subvolume delete -R /home/emanu/volumeTOP

Delete subvolume 3213 (no-commit): '/home/emanu/volumeTOP'

emanu@fedora ~ [1]> sudo btrfs subvolume list -ot /home/emanu/

ID      gen     top level       path*
--      ---     ---------       ----*
1990    450664  666             @home/emanu/.var*
2543    406562  666             @home/emanu/Progretti*]
14

Previously, it was impossible to delete nested subvolumes when trying to delete the parent subvolume.

emanu@fedora ~ [1]> sudo btrfs subvolume list -ot /home/emanu/

ID      gen     top level       path
--      ---     ---------       ----
1990    450694  666             @home/emanu/.var
2543    406562  666             @home/emanu/Progetti
3220    450693  666             @home/emanu/volumeTOP

emanu@fedora ~> sudo btrfs subvolume list -ot /home/emanu/volumeTOP/

ID      gen     top level       path
--      ---     ---------       ----
3221    450692  3220            @home/emanu/volumeTOP/VolumeRO_Nested
3222    450693  3220            @home/emanu/volumeTOP/VolumeRO_Nested2
emanu@fedora ~> sudo btrfs subvolume list -ot /home/emanu/volumeTOP/

emanu@fedora ~> sudo btrfs subvolume delete /home/emanu/volumeTOP

Delete subvolume 3220 (no-commit): '/home/emanu/volumeTOP'
ERROR: Could not destroy subvolume/snapshot: Directory not empty

EDIT:

Ok, I managed to reproduce it, and maybe I understand the use case: initially, RW snapshots nested inside other snapshots, then converted to RO, resulting in multiple nested RO snapshots.

I think it makes sense for it to fail in this case.

emanu@fedora ~> sudo btrfs subvolume snapshot -r /home/emanu/volumeTOP/VolumeRO_Nested /home/emanu/volumeTOP/VolumeRO_Nested/
Create readonly snapshot of '/home/emanu/volumeTOP/VolumeRO_Nested' in '/home/emanu/volumeTOP/VolumeRO_Nested//VolumeRO_Nested'
emanu@fedora ~> sudo btrfs property set /home/emanu/volumeTOP/VolumeRO_Nested ro true
emanu@fedora ~> sudo btrfs property get /home/emanu/volumeTOP/VolumeRO_Nested/
ro=true
emanu@fedora ~> sudo btrfs subvolume delete -R /home/emanu/volumeTOP
Delete subvolume 3220 (no-commit): '/home/emanu/volumeTOP'
ERROR: Could not destroy subvolume/snapshot: Read-only file system
15

Privileged.

Is the top subvolume you’re deleting also set to ro? Use btrfs property set to make that top subvolume also ro.

That’s the problem OP had, and when I reproduce that, I’m not allowed to delete that top subvolume. Again, since that containing subvolume is ro, and since any modification inside is disallowed including subvolume removal, the attempt fails.