Split Fedora 33 BTRFS in two disks

mackissack · November 30, 2020, 4:38am

I have a brand new Lenovo IdeaPad 3 17" laptop.
I upgraded the original 128 GB NVMe M2 storage unit cloning it with dd into a 500 GB one.
Now I have about 345 GB in no assigned space

And I thought, how about to install the Fedora 33 system folders there?

The other storage unit is currently a 1 TB SATA HDD which I will clone into a 1 TB SSD.

So, I would have to create two (2) BTRFS volumes.
1st volume in the NVMe unit containing /boot, /, /var, /usr, /tmp
2nd volume in the SSD containing /opt and /home.
SDD unit will also hold all the Windows document folders and user data

Any comment, suggestion, recommendation or warning?

chrismurphy · November 30, 2020, 8:33pm

I suggest two separate Btrfs file systems, one on each drive. Exactly as you described doing it.

If you choose both drives, and do automatic partitioning, it will create one Btrfs file system across both drives. While this will work, it might result in some peculiar performance issues, e.g. some things will seem super fast when the read/write from the NVMe drive and then other things will be slower when read/write from the SSD. The SSD is probably not dramatically slower than than NVMe the vast majority of the time, so this effect might be subtle.

It is possible to do this with Custom partitioning or Advanced-Custom partitioning. Custom partition hides the details of what devices each mount point is located on. So as you add mount points, you’ll need to look on the right side interface, and click on either Devices>Modify (see figure 16) or for Btrfs mountpoints use Volume>Modify (see figure 21).

The Volume dialog will let you specify what drive the file system is on, and its name. So you’ll eventually end up with two volumes (two names), one on each drive.

Another possibly confusing thing is size. I suggest setting the size of the volume in the Volume dialog, along with the volume name and the device the volume is located on. And then when you create mountpoints on the left side UI, leave the desired capacity field blank. Mountpoints on Btrfs become subvolumes that are mounted at the specified mount point, and subvolumes don’t have size. The way this works in the simple case is, the first time you create a mount point that will go on Btrfs (for example /) the desired capacity field is honored if you enter a value; the second time you create a mount point that goes on the same Btrfs volume, the desired capacity field value is added to the same Btrfs volume. So it kinda acts like it’s setting capacity per subvolume but it’s really just adding it to the Btrfs volume, which acts as a pool. Subvolumes share that pool, just like a directory.

mackissack · December 2, 2020, 3:35am

Thank you, @chrismurphy.
I’ll put myself to work.
Have to clone the HDD into the SSD and then start the Fedora 33 installation from scratch.
Fedora 33 was already on the HDD using only one BTRFS volume.

chrismurphy · December 2, 2020, 6:24am

Ummm yeah, so be a bit cautious with cloning file systems. It will mean duplicate UUIDs and it will be ambiguous to the kernel which file system you want it to mount if the two drives are kept visible to the kernel following cloning. I’m not a super big fan of cloning except for data recovery purposes where you clone a drive, and put one of the copies in a safe place on a shelf. Duplicate fs UUIDs are bound to cause confusion either the kernel or human or both.

(a) just reinstall instead of cloning, and then rsync user data over to the new home or whatever.
(b) a better “clone” is something like

cd /
tar -acf bootefi.tar boot/

Now you have a copy of all the files in /boot and /boot/efi in bootefi.tar located in the root directory, which means it’s on the Btrfs volume. Assume vda1 for ESP, vda2 for boot, vda3 for btrfs. And vdb is the new drive, with vdb1, vdb2, vdb3 partitions created already, but no formatting.

btrfs replace start /dev/vda3 /dev/vdb3 /

This is a live migrate vda3 to vdb3 for the Btrfs file system mounted at /. And in another shell you can do other things:

mkdosfs -n EFI /dev/vdb1
mkfs.ext4  /dev/vdb2
umount /boot/efi
umount /boot
## update /etc/fstab with the new UUIDs for vdb[12]
mount  /boot
mkdir /boot/efi
mount /boot/efi
tar -xf bootefi.tar

In this case everything has its own unique fs UUID and you don’t need to worry about confusion. You will have to update the /etc/fstab and grub.cfg and efibootmgr to reflect the new fs UUIDs. So it’s not anywhere near as straightforward as just reinstalling. But no downtime, and quite fast because only in-use blocks are copied.

NOTE: Once this replace process completes, the original device (the one being replaced) has its magic wiped. This is not exactly a “clone” when the original goes bye bye. You can restore the magic but then Btrfs can’t tell the difference between the otherwise two exact copies - and that’s asking for confusion. What you probably want for cloning Btrfs is either: Btrfs send/receive to make “clones” of snapshots onto another device. You can use Btrfs on the receive side to get an exact copy of that snapshot. Or you can use any file system with the btrfs send -f option.

For those who want to replicate an entire single Btrfs volume, all subvolumes and snapshots, then you want the seed device method. This is also quite fast. The “sprout” (the copy) gets its own fs UUID so there’s no confusion which is which.

https://btrfs.wiki.kernel.org/index.php/Seed-device

mackissack · December 2, 2020, 11:49pm

I reviewed your instructions, @chrismurphy.
My idea on cloning the HDD into the SSD is to keep the Windows partition.
I wouldn’t want to mess with Windows.
On the other side, the HDD won’t be used on this laptop.

chrismurphy · December 3, 2020, 6:40am

Ahh OK gotcha, makes sense. It’s a bit hidden on Windows 10, but there is an integrated backup (might be called Windows Backup, local search will find it). Off hand I don’t know if it will backup everything or just user files.

For the OS itself, Microsoft offers their own images now, which are fully up to date and you don’t have to mess with the original manufacturer’s restore image, which tends to have a lot of 3rd party junk on it.
https://www.microsoft.com/en-us/software-download/windows10

I haven’t tried this lately but my recollection is if you go to that URL from Windows, it’ll provide a USB media creation tool.

ozeszty · December 3, 2020, 10:05am

Since those are only data partitions, no re-installation shoud be needed. Just in case there’s WoeUSB, which creates bootable USB from Windows ISO’s (available on linked site when visiting it from linux). For newer win 10 ISO’s you have to choose NTFS file system for it (in WoeUSB, that’s because system image grew above 4 GB supported by FAT). Installation from it may require to temporarily disable Secure Boot.

GUID of original partitions can be changed with gdisk, but it won’t be necessary if old disks will be formatted (entire disks, with partition tables).

chrismurphy · December 3, 2020, 7:11pm

I suggest using the Microsoft tool to create the USB stick, or in a bind just cp -a the contents of the ISO to a FAT32 formatted USB stick. That works for UEFI or BIOS, and UEFI Secure Boot doesn’t need to be disabled. I’d choose to not trust anything that proposes disabling UEFI Secure Boot as a work around.

I don’t see how NTFS can work because the firmware can’t read NTFS. It needs FAT to read the Windows bootloader, then it can read NTFS. But FAT32 has a file size limit of 4G, the file system limit is 2T.

mackissack · December 9, 2020, 10:21pm

Well, well, well…
update on the task.

Something unexpected happened.
Anaconda halts when starting the installation process.
Says it cannot check the amount of storage.

I don’t know if it is due to the existence of 5 partitions in the NVMe drive.
There is a Windows 10 Reserved Partition.
So, there are 5 total:

EFI partition
Reserved (16 MB)
Windows System
Recovery (1000 MB)
Fedora system

These are both drives with the Fedora filesystem defined:

Here is the NVMe drive after deleting the BTRFS:

What is weird to me is Anaconda makes no problem on the partition definition,
GParted simply wouldn’t allow to set a partition if it is any problem.

chrismurphy · December 10, 2020, 4:13am

Advanced-Custom takes some fiddling to figure out how it works, and you also have to be familiar with the Fedora Btrfs layout. So yea, it’s a bit confusing.

The problem is you’ve assigned the btrfs volumes themselves to mountpoints. Don’t do that. Instead, click on the Btrfs volume icon on the left side, and then click + to add a new subvolume, and assign that to a mountpoint. The subvolume name can be whatever you want but the automatic partitioning default is home for /home and root for /

Also missing is a /boot mount point. The installer will allow you to put it on Btrfs, but the default is to put it on ext4.

Last, I notice the Advanced-Custom UI doesn’t permit setting labels for file systems. At least Btrfs you can do it after installation: btrfs filesystem label / yaybutter

mackissack · December 11, 2020, 1:35am

Finally, task completed!
Followed @chrismurphy indications.
Have to say I had to find out that UEFI systems accept more than 4 partitions.
Here is how the GB NVMe unit was partitioned.

And here the 1 TB SSD

System is running smoothly.
And yes, the attained speed improvement is noticeable.

ozeszty · December 11, 2020, 9:26am

You should mark as a solution the post, where chrismurphy wrote what, how and why has to be set, this one just confirms he was right.

The file size limit was causing the problem. I’ll try cp -a the next time, current iso contains install.esd smaller than 4GiB (so WoeUSB doesn’t need to work around that too, now). Secure boot had to be disabled only during installation, nonetheless, not the best practice. Thanks.

mackissack · December 11, 2020, 5:15pm

Done, @ozeszty.
Need to make clear I didn’t know UEFI based systems allow partitions beyond de 4 limit.
I found this sailing around.
From that point on I followed @chrismurphy suggestion and created the ext4 partition fo /boot.
Then corrected all regarding mounting points.

As an old friend of mine once said, there are more dumb the askers.
One basic way to learn is ask.

ozeszty · December 12, 2020, 5:38pm

Yeah, I remember the heroes at school who weren’t afraid to ask when many more didn’t understand something, yet not many would speak up. The Internet made asking much easier, but still one has to be curious to learn.

MBR was limited to four primary partitions, but allowed more inside an extended partition. GPT fixed that and some other shortcomings, converting to it doesn’t require formatting.

mackissack · December 12, 2020, 7:52pm

So, am I then a hero, @ozeszty?
Well, I studied computer sciences in the mid '80s.
But many things have evolved since.
Forums help me a lot, thanks to persons more acquainted in the subjects than me, like you or @chrismurphy and others.

And studying is paramount.
It is imperative to search.

In the end, I learned how to do this splitting.
Thank you all.

chrismurphy · January 1, 2021, 9:38pm

(a) replace requires target device to be equal to or larger than the source (b) resize is not done for you.

If the source is bigger than the target, you can shrink the source first so that you can use the replace command (preferred over the device add then device remove method). You only need to shrink the file system, you don’t have to shrink the partition it’s on.

If the source is smaller than the target, following a successful replace, resize by:

btrfs filesystem resize max /

system · January 29, 2021, 9:38pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.