BTRFS - I surrender! Good features BUT Complex, wastes time, not KISS, does not play well

Project Discussion
1

Hi
I have used BTRFS for the first time since March 2023 on my internal archives directory.

Using multiple drives of varying sizes created a luks1 encrypted ‘single’ data and dup meta volume. While I have used btrfs balanced, timeshift and my other develop tool scripts for managing borg backups rely a lot on standard linux file management OS tools. This is where the problem arises. The added complexity re all the btrfs fi or btrfs device commands and so destroys any notion of KISS simplicity.

While a newbie re BTRFS my experience has been lots of wasted time, huge learning curve, lots of wear on my SSDs storing 800 GB of data etc etc.

Am pulling the plug after 1.5 months messing around and I decided to write this to warn others that if you do not have the available time to go through a substantial learning curve, do not touch this and stick with ext2/3/4, xfs, ntfs, vfats which are easy to create, reliable, work well with all the linux OS filesystem management tools and interact well with all the common apps.

In the end BTRFS is still a work in progress and if you have the time to go through the learning curve fine. But for cases such as mine where I was forced out of centos to fedora trying to run a work computer trying to develop other things more important to my time, then for the btrfs filesystem, my recommendation to those, DON’T GO THERE!

My recommendations to the BTRFS developer. Keep It Simple Stupid - KISS! Better fs features does not compensate for complexity of use. Backwards compatibility so it is seamless with ALL apps and linux OS tools is equally important. This is where BTRFS fails and why I regard it as a work of progress for the more adventurous. On this basis I also think it should not be the default filesystem which still should be ext4 particularly for the new and curious.

1 Like
2

While I agree that btrfs is not for everyone, in my experience, one of it’s weaknesses is how far they have gone to make it compatible and be able to work like all other filesystems.

What applications have you found that don’t work with btrfs?

3

Programs that use the fanotify api don’t work on BTRFS subvolumes.

Examples:

  1. fsnotifywait
  2. fatrace

fsnotifywait (recent addition to inotify-tools)

git clone git@github.com:inotify-tools/inotify-tools.git
cd inotify-tools
./autogen.sh
./configure --prefix=/usr/local/stow/inotify-tools --enable-fanotify
make
sudo make install
cd /usr/local/stow
sudo stow -S inotify-tools
sudo fsnotifywait -r -F .
# Setting up watches.  Beware: since -r was given, this may take a while!
# Couldn't watch .: Invalid cross-device link

fatrace
fantoify fsid mark failing for btrfs subvolumes: Invalid cross-device link #3

1 Like
4

1/ timeshift. Works fine when initially run but when going to settings tab from that moment it thinks my btrfs disk is full and fails. Yet btrfs reports 65% full. But when you start without going into tabs it works fine and sees the right amount of remaining space. Clearly there an incompatibility there.
2/ Intermittently polkit error asks for one of my three disks forming a single volume on boot up for permission to mount disk with sudo pwd request. Am running luks1, serpent, 512 on three partitions, with each formated to btrfs (using sha512 crc), then added them using brtfs, balance to -dsingle -mdup and in fstab add mount option for 1 added using device=/dev/sda3, device=/dev/sdb1, device=/dev/sdb2 and compress i think was set to zstd:9. Due to different sizes forming the ‘btrfs’ vloume unable to use raid so relied on crc sha512 to check errors. sdb is an older ssd. Yes am newbie, may have made errors but it took over a month of btrfs article reading and playing to write one line in /etc/fstab and three lines /etc/crypttab to make it all sort of work. Clearly not perfect as a btrfs newbie so more work to do but that’s my point. Would have been way quicker with LVM on three partiions formated to ext4 without research all the nitty detail to balanace, run dsingle to maximise space and the list goes on. My mistake to use BTRFS but I was curious.
3/ Have my own scripts running borg backup much of which is disk management relying on uuid. But due to luks and each btrfs has a uuid the script just crashed and burned.
4/ Unlike timeshift vorta does seem happy with btrfs.
All this has been to much time wastage and I need to move on to other tasks. So going back to LVM, ext4 and encryption. I’ll let you know if I get same issues. But at least I can get this going in hours versus days without lots of reading re BTRFS and head scratching. It’s just not straight forward like LVM and mkfs. Further gparted is not that great with btrfs, particularly combined with encryption and csum options.

So don’t read my critique as pure negativity of BTRFS. I see it’s promise but in the end hard test data shows the ext4 outperforms btrfs, I’ve never had reliability problems like I am having with btrfs re space size or standard linux command managing disk & space etc etc. It’s an issue of benefit versus complexity and time. That’s where I think, except for simple use cases, BTRFS fails and is not ready for prime time. Filesystem are fundamental to an OS. That’s why I guess microsoft have not messed with theirs. If it aint broke don’t fix it is my mantra. Linux OS and ext4/xfs are not broken, so exactly what is btrfs fixing then?

If you talk to IT professionals, they like linux but they will also admit that while Linux is cheaper to acquire, to run it it’s more work. BTRFS from what I see only adds to that burden unless you run complex raid cluster maybe. But then I’d consider ZFS if I wanted that reliability.

My first use experience with BTRFS suggest a setup and management GUI is needed. Managing btrfs particularly for newbies via Command Line is too complex to get right. A GUI will enable consistency and correct configuration for particular selected feature options of which there are many. With a gui then it may succeed in prime time for use by non BTRFS experts.

2 Likes
5

I use BTRFS for the system partition. snapper works well for snapshots and helped me rollback multiple time without issues. Despite that, I trust ext4 more and have daily backups to an LUKS encrypted partition with ext4 on it.

In my experience BTRFS is fine for use as a system partition, I like the light snapshots and transparent filesystem compression. For mission critical uses, I would stick with more stable filesystems.

6

In what way are you using Timeshift on BTRFS? I have been using it for years without any problem and I know many users satisfied with Timeshift + BTRFS for its simplicity.
Unfortunately, Timeshift in BTRFS mode only supports the subvolumes “@” for rootfs and “@home” for the user’s home, if it does not work in other configuration it is a limit of Timeshift and not BTRFS.
Also try Btrfs Assistant.
I do not understand the complexity in your configuration, BTRFS is very flexible on multiple devices, because you can add or remove the storage devices on the fly and change the profiles.
A user who installs Fedora with the predefined partitioning has no complexity with BTRFS compared to ext4 or XFS.
If you want to understand why Fedora has changed the filesystem on BTRFS, read the modification proposal: https://fedoraproject.org/wiki/Changes/BtrfsByDefault

7

I am coming round as a BTRFS newbie to your thinking, ext4 trustworthy vs btrfs. In my 20+ yrs Linux use ext3/4 have never ever failed nor where they painful to use or learn needing need me to contact a forum for support or reading endless man pages and web. mkfs.ext4 /dev/sdxy and done never to think about it again. But btrfs format with multitude of ‘useful’ options does not fall in the KISS format and forget category.

Came across BTRFS assistant yesterday, visually helps but somehow it made my Vorta backup crash overnight? Been running OK until I installed assistant. Something to do with my admin. permissions suddenly changing overnight. Still investigating but since my backup sits on btrfs archive volume for that purpose this raises more questions about suitability.

This does not raise my confidence level.

Flexibility promise on add/remove multiple devices and compression I chose btrfs for a try. But so far also been happy with LVM in past. The compression and sha512 checksum sold me. Looked forward to accurate backups using single data mode across 3 ssd partitions of different left over sizes and single meta dup mode.

I considered OpenZFS for my backups but in the end my review suggested complexity & learning time and a professional notch above what I needed. I certainly was not investing in multiple device storage for which ZFS is best used. Didn’t realise that complexity and time was also an issue to btrfs in which case I would have gone ZFS due to older better proven.

I’ve never had so much trouble with any linux instal. Currently running F36 getting ready for upgrade to F38. That’s why change and ‘improved’ backups.

So I agree with your statement “For mission critical uses, I would stick with more stable filesystems”. My backups are mission critical as they store a lot of important financial data but also a lot of time and effort. Your qualification adds to my motivation for opening this discussion as warning that BTRFS is not quiet there yet for some.

8

Everything is ext4 and my media directories are XFS for large file performance.

My back up uses a new SSD and two partitions on an old ssd but not heavily used in past.
Each partiton encrypted with luks1, serpent, 512 and then created a btrfs volume formated with sha512 then using ‘add’ of all 3 partitions, followed by balance -dconvert=single -mconvert=dup (consered raid across three partitions for metadata).

All looked and worked OK per examples on net. Then transfered from old retiring backup drive my timeshift and the borg archives to this new btrfs and then my live changed for the worse. My btrfs volume is basic partition, no subvolumes. have not got my head around subvolumes versus partitions, again this per my point adds complexity I’ve whinged about for those who says it easy and work reliably. Maybe those have not used ext4 much in the past as a reliability reference!

Timeshift stores on btrfs volume but sources from the ext4 volumes. In it’s gui it runs OK and stores OK but when in gui tab I switch to settings it thinks the disk if full but my other df or btrfs fi show or usage say it’s 65 % full. Anyway from then timeshift does not work. If I close timeshift and restart into main front gui is reports accurate fs size and remaining space and works fine.

I am inclined to agree with A Ismail above, btrfs is for basic simple use cases but beyond, stick with tried and proven ext4 and let others debug btrfs promises & hype. I do not have time for this rubbish heap that I am now in. That was my motivation for writing this as a warning to others like me who not want to deal with a complex fs and who want to spend time on other more important things re their IT systems.

Before deciding to use btrfs I actually read https://fedoraproject.org/wiki/Changes/BtrfsByDefault

“For laptop and workstation installs of Fedora, we want to provide file system features to users in a transparent fashion. We want to add new features, while reducing the amount of expertise needed to deal with situations like running out of disk space. Btrfs is well adapted to this role by design philosophy, let’s make it the default.”

This is what sold me! But BTRFS adds it’s own complexity which makes up for the expertise needed to deal with running out of disk space.
1/ I MONITOR my disk space and have only by accident run out of it eg ‘cp bigfile /mnt/somemount’ and there was not mount by accident and filled up rootfs. But fix it I did in under 15 minutes but not the days re btrfs mudfest I am in now.

Clearly complexity versus simplicity is in the eye of the beholder. I am old school with 30 or more IT experience. If it works don’t fix it. I listened to that wiki and I am paying for it now cause I BROKE what was working.

So other people considering using BTRFS, be warned beyond the simple partitions formats. The fancy features beyond the simple partition format, ON WHICH BTRFS is sold are exactly what may get you into trouble and time wastage and no BTRFS is not fully compatible with Linux file management commands as there are from research example on the net and even this thread where that is not true.

9

It is highly unlikely that Btrfs Assistant would cause Vorta to crash. They are unrelated.

I use both and It isn’t even close. Btrfs acts like a pretty normal Linux filesystem with some advanced features on top. zfs is orders of magnitude more complicated. It requires careful planning and tuning. It is my preferred filesystem but you need to be willing to invest the time in learning and preparing to get the best results.

On the other hand, btrfs mostly “just works”. From reading your posts, it feels like you are making it more complicated than it needs to be.

10

I suspect your right, that btrfs mostly works as a normal linux fs but with some extra features on top.

It’s those extra features that interested me. My mistake probably dived straight into the extra features. But then where’s the warning re the extra feature use? Besides from my FS research before going down this path it is those features that interested me to BTRFS.

2 of my 3 partitions are on an older SSD (5+ yrs old) though not heavily used. So decided to sprinkle it with sha256 checksum. Mounted it with compression per fstab & crypttab
fstab
/dev/mapper/luks-dd129b86-469d-4042-8b90-00d1acb2a1bf /srv/lpssd/archives btrfs defaults,device=/dev/mapper/luks-dd129b86-469d-
4042-8b90-00d1acb2a1bf,device=/dev/mapper/luks-1e32597b-d73f-48de-a899-af443d96b150,device=/dev/mapper/luks-9b746590-b83b-4e35-
bb7c-1c07d5130a4f,compress=zstd:9,discard=async,x-systemd.device-timeout=0 0 2

crypttab

RK 19/4/2023: btrfs, archive ← archive2 <-archive3 refer /etc/fstab

luks-dd129b86-469d-4042-8b90-00d1acb2a1bf UUID=dd129b86-469d-4042-8b90-00d1acb2a1bf /root/Documents/encrypted-disks/archive-key
luks,nofail,discard
luks-9b746590-b83b-4e35-bb7c-1c07d5130a4f UUID=9b746590-b83b-4e35-bb7c-1c07d5130a4f /root/Documents/encrypted-disks/archive2-ke
y luks,nofail,discard
luks-1e32597b-d73f-48de-a899-af443d96b150 UUID=1e32597b-d73f-48de-a899-af443d96b150 /root/Documents/encrypted-disks/archive3-ke
y luks,nofail,discard

mkfs.btrfs with checksum sha256 with more importance on accuracy versus performance given 2 older partitions.

Encryption as my wife and I about to travel around Australia and may leave PC/Server behind with others. Wanted data and my other passwords secure when at rest. So yes layer of complexity for a reason and ext4/LVM have no problems with this as rootfs is like that. No issues. But does not have sha256 checksum which sold me re BTRFS and two older partitions. Have terrabytes of business data so felt standard crc not good enough.

So I agree with you. Maybe was more complex but for reasons and assumed from research that brtfs provide features that were useful particularly the Fedora wiki sold me. How wrong I was and how miss-leading the wiki was re hype of new features.

So agree with your conclusion NOW from bitter experience.

KISS is the warning to others, use btrfs for simple partitions.

But then exactly what is the point of BTRFS if the extra feature use makes it complicated? What? Use is BTRFS when running out of disk space with other FS requires higher skill? Really?

11

Need to add for amusement being an old fart using BTRF for first time, this is what it looks like for you, but for me, it’s me in the cement!

12

Moved timeshift from encrypted multi-disk btrfs volume mentioned above to encrypted ext4 partiton. Timeshift and gui now work properly. Read disk space propertly in all tabs. Did the move in under half hour versus leaving a being stuck in btrfs re above cement driveway.

Vorta/borg on the btrfs archive volume seems so far to be working fine on it’s own. Wait and see if too dump btrfs for the vorta archive completely or leave it as is.

But the fact that moving timeshift repository off BTRFS highlights my point and firming views that BTRFS is not quiet compatible with ‘some’ existing but much loved and important apps for desktops.

14

Defaults fine as long as I can chose when installing, alternatives FS going into the future. Further given that huge dominance of linux in the enterprise server area and phones, that’s a big call for distributions to dictate what filesystems they use to the actual users. There’s a reason that RHEL still is using ext4.

Server farms can not afford the risk of incompatibilities such as I have observed. In the end the market will determine the default FS. Not the distribution or it will be relegated to oblivion.

15

I don’t think Fedora or others will remove other options, so rest assured in this regard.

Maybe because RHEL is meant for the enterprise so it can’t afford the least issues. Fedora is more desktop oriented (I think) so it adopts new technologies faster.

16

btrfs is slow too

1 Like
17

Good Video. I always find DJ Ware’s videos very informative.

18

Thanks. I opened a discussion about setting the I/O scheduler on a per-file system type basis.

19

Well, from the DJ Ware video it seems that if you’re concerned about performance, you shouldn’t be using BtrFS. XFS seems to be a better choice unless you absolutely need a feature that is specific to BtrFS.

20

I wanted to dual boot and this stopped me. I fixed the problem by installing Fedora 38 with a cinnamon desktop. Now I an dual booting no problem.

I made a YouTube video of how I did it.

21

2 posts were split to a new topic: How do I set my avatar picture?