Displaying btrfs detail for admins

[steppybug]

Getting an understanding of btrfs details and leveraging them for all the things I used to do with lvm/xfs proceeds.

To find the information I am looking for I have come up with these based on clues I gleaned from your replies and further documentation searches. It is not as straight forward as I expected which may be due to me going about it the wrong way.

---

Any clues on better solutions?

---

What other questions would be useful to know the answers to?

---

##Discover btrfs filesystems, mounted or not, on a system:

lsblk -no UUID,PATH,FSTYPE,MOUNTPOINTS | grep btrfs

##Devices in a btrfs filesystem:

ls -1d /sys/fs/btrfs/<UUID>/devices/*

##Checksum algorithm:

#Would be great if this worked against a UUID
btrfs inspect-internal dump-super <DEVICE> | grep csum

grep . /sys/fs/btrfs/<UUID>/checksum

##Compression algorithm:
#for kickstarting the compress=no is needed for both fsoptions and mkfsoptions unless anaconda modified
#  part btrfs.111 --fstype="btrfs" --grow --size=1 --fsoptions="noatime,compress=no" --ondisk=/dev/disk/by-id/virtio-os
#  btrfs none --label=os --mkfsoptions="--data=single --metadata=single --mixed --compress=no" btrfs.111
#  btrfs / --subvol --name=r LABEL=os
#  After booting into kickstarted system the compress=* that anaconda adds can be removed from /etc/fstab
#As far as I can tell btrfs property cannot change filesystem compression, only per-{file,directory} compression on filesystems with compression enabled

#if compression enabled, algorithm will be indicated in feature name
ls -1 /sys/fs/btrfs/<UUID>/features/* | grep compress

##data/metadata mode:

#Would be great if this worked against a UUID
btrfs device usage <path to mount point of a mounted root or any subvolume>

#if exists, mixed otherwise isolated
ls -1 /sys/fs/btrfs/<UUID>/features/* | grep mixed_groups

##Profiles
#DUP profiles are not interesting to me in use cases this is applied to (ie: quality nvme and rigorous backups)
#data=single is default but keeping it in ks.cfg improves clarity

#Would be great if this worked against a UUID
btrfs device usage <path to mount point of a mounted root or any subvolume>

#The information is here but not in a great form
find /sys/fs/btrfs/<UUID>/allocation

---

The situation is MUCH better for xfs, ext4 and even vfat

xfs_info <mntpt|device>
xfs_db -r -c info <device>

tune2fs -l <device>

minfo -i <device>

[emanuc (Emanuele)]

@steppybug

Getting an understanding of btrfs details and leveraging them for all the things I used to do with lvm/xfs proceeds.

I’ll try to clarify a few points (as far as I know); for the rest, I’d mention @chrismurphy

@steppybug

So when the filesystem was made does the root tree-of-trees end up with compress=no? How do I query for this? For mounting the subvol the compress=no option does get propagated to /etc/fstab and compression can be turned on by setting it to compress=zstd:1.

You can’t set mount options for individual subvolumes.
If you need different options, such as compression, use the user-space utility btrfs property.
Also, ssd and space_cache=v2 are defaults in recent kernels.

example:
btrfs property set subvolX compression zstd:10

@steppybug

A fresh install storage information with traditional commands is informative but more detail is desired

For accurate space usage reporting, don’t use df; use btrfs filesystem df instead.

@steppybug

And with the btrfs specific commands I see the lack of compression in the Data ratio: row.

To check compression details, you can use compsize.

All the available commands and features are well-documented in the Btrfs documentation.

Hope this helps

[chrismurphy]

@steppybug

So when the filesystem was made does the root tree-of-trees end up with compress=no?

The installer treats compression as a function of each mount point, and therefore each Btrfs subvolume, and therefore the option makes its way into the /etc/fstab

However, mount option compression on Btrfs is file system wide. It’s not a per subvolume option. You can check the resulting fstab to see if compression is enabled anywhere.

Note that compression only applies to data extents. Metadata is not compressed.

Mount option compress=no is the same as omitting it everywhere for that file system. By default, compression is not used.

Some mount options are really VFS options, and those can be per subvolume because they’re really mount point options, and a subvolume mount is a bind mount behind the scenes.

@steppybug

And with the btrfs specific commands I see the lack of compression in the Data ratio: row.

The ratio report in usage is how many copies of block group types there are. Metadata is the file system itself. Data is the contents of files. Ratio 1 means one copy, i.e. if checksum verification fails, there is no self-healing possible because there won’t be a good copy.

@steppybug

What is the System,single: line describing?

system block group type holds the chunk b-tree. This b-tree is responsible for containing the logical ← → physical address mapping. Most addresses in btrfs are logical addresses, and point to one logical block, metadata or data. But it is the chunk tree that determines what physical device(s) and location(s) the actual block is located on. So most of the time btrfs is treating everything as having single blocks, but then at a lower level there’s the breakdown as to how many copies and were they are.

For all practical purposes, system bg’s are treated as metadata.

@steppybug

How do I query btrfs and get information on what has not been added back into the free space during that intervening time?

Both metadata and data on Btrfs are subject to delayed allocation and it’s pretty lengthy, 30 seconds. It’s possible for writes to get split up across commits, so it’s possible a write takes up to 60 seconds. A delete on Btrfs consumes space because of COW. Deleting a file means first writing metadata updates into unused space to reflect the delete, thus consuming more space initially. Btrfs needs lower the reference count of all shared extents for deleted files. And only extents with zero ref counts actually have their space freed.

File delete is a process that does take a bit of time.

[chrismurphy]

@steppybug

In the Linux realm LVM and xfs became so solid I shied away from getting into btrfs especially since none of my employers would seriously consider putting it into use. I really do consider figuring this stuff out fun and have high expectations for many hours of btrfs fun.

Upstream btrfs development list is chock full of more info. There are sysadmins that sometimes post there, more are on #btrfs libera.chat IRC. Zygo has a lot of experience with Btrfs in production environments. We’re also using Btrfs in Fedora infrastructure.

linux-btrfs.vger.kernel.org archive mirror

Of course so does Meta. They have millions of instances in production. They’ve told us Btrfs falls over at about the same rate as XFS. Both are susceptible to device problems including firmware bugs. Btrfs does thave more concentration of critical metadata in a small number of blocks, and the complexity of COW means no fixed location for the metadata means no assumptions can be made during repair like overwriting file systems. So it is different.

But this isn’t unfamiliar to you, having ZFS experience.

I mitigate risk with frequent cheap snapshots and replication of them with send/received. (See btrbk project, it’s also packaged for Fedora, for automating it).

Fedora Btrfs contributors and users are also on https://matrix.to/#/#btrfs:fedoraproject.org

Also, check, out the fpaste command and project for code examples of using btrfs. Some of the typical things I ask for are in there. It only looks at /, but you can see the commands and adapt for non root storage.

GitHub - sanjayankur31/fpaste: A command-line front-end for the Fedora Pastebin...

[chrismurphy]

@steppybug

xfs_info <mntpt|device>
xfs_db -r -c info <device>

tune2fs -l <device>

minfo -i <device>

Functional equivalent on btrfs is:

btrfs inspect-internal dump-super <device>

The block group profile is a function of the chunk tree, not the super block, so that information is found in btrfs filesystem usage <mountpoint> instead.

[chrismurphy]

Keep in mind Fedora is a community driven project. Anyone can contribute!

Get Involved

---

The upstream kernel and btrfs-progs projects are also community driven. The latest upstream documentation is at:

Welcome to BTRFS documentation! — BTRFS documentation

The upstream btrfs mailing list is where that documentation is discussed and where patches for improvements and fixes are sent. List subscribe info. And list archive.

---

It is also possible to ask each question as separate topics here in the discussion forum, and tag it with btrfs tag so interested parties are notified. I haven’t tried it, but Discourse (the software this forum is based on) supporting requesting discussion topics be turned into wiki articles.