Hi, I have an Fedora installation with the following partition layout:
/boot (ext4)
/boot/efi (fat32)
/ (luks2 + ext4)
This works very well… until the point of a reinstall.
What I did already in preparation for the new install:
format /boot
format /boot/efi
delete everything on / except /home to delete the system but keep my userdata (cd /<mountpoint_of_system_partition>/ && rm -rf $(ls | grep -v home))
So now I have nothing on my disk anymore but my userdata. No OS, nothing that will get into the way of a new installation.
The next step would be to mount my three partitions in the installer and go ahead… right?
But unfortunately I get this:
(“You must create a new file system on the root device.”)
In the blivet mode it is the same.
But there is no reason to do this. My disk is already clean of any OS, no formatting needed.
How to get around this artificial limitation of the installer?
(I know I could copy over the data to another disk, reinstall, and copy everything back. But this costs hours of time and money for another disk, and I don’t like wasting my time and money on artificial limitations. So the goal here is to reinstall while keeping the data in place.)
For the short term:
A workaround would be sufficient.
Not 100% sure but I saw some python in combination with the installer. So if the installer is made in python, could we just edit out this check before installation? If so, which file to edit?
Or otherwise is there an alternative installer which doesn’t have this limitation?
For the long term:
I hope the installer will allow installing on already formatted partitions. Please make this warning skippable, like it was done for weak passwords for example.
What is missing from your post is the answer to the question, why do you need/want to re-install.
What you could do for the future is using separate partitions for / and /home. Then you can just reformat the root partition and keep /home. A backup of your data is still recommended.
Long answer:
Depending on the machine.
My current only install of Fedora is luckily not on my main machine yet, so I don’t care too much if anything goes wrong. On that, the reinstall serves the purpose of cleaning up the system, getting rid of old packages and files I forgot about and which are not needed anymore.
At the same time this gives me a good opportunity to vet Fedora for becomming my new main OS and recommending it to others.
For example on my main machine, where currently another distro is running, I would have to install Fedora in the same way: on an already existing partition. So I have to know before installation if this is possible. (good that I did the test)
Also I maintain some other machines for friends, family and others. Some of them are not the best at using linux. Over the years I had unbootable or bugged systems for various reasons. Instead of spending hours on debugging and searching for what went wrong, often it is faster to just install a fresh OS to get back up and running.
So for that case, I want to know that this is possible with the OS I recommend to others (in this case Fedora) and where I have to help out when something goes wrong.
I appreciate your advice. However I like having everything on one partition because this allows for sharing the space for system and user data.
Other options I have considered sadly always come with a downside. For example using btrfs + separate subvolumes for root and home would also allow for shared space, but in my tests btrfs comes with a significant performance downgrade compared to ext4, which is the reason I stay with ext4.
Not important enough for the machine I currently test Fedora on to justify the cost of a new backup disk for that.
However for important data of my main machine I do have a backup
This still has the problem that, with the current limitation of the Fedora installer, I would temporary only have one copy of my important data while installing Fedora on my main machine.
Meaning I still have to spend money on another backup disk plus the time to copy everything over.
And the only reason for that unnecessary trouble is an artificial limitation in software. Ugh
Which is why…
… I still hope someone has an idea for a workaround. Or the devs of the installer remove the limitation.
If you really need, you can try to install Fedora from scratch:
Using BTRFS will avoid “wasting” of dispace, with the subvol feature. It might be time for you to do a backup of data to external drive, and re-install using the BTRFS with encryption.
(or, do the more elaborate process of downsize existing partition, install Fedora to new diskspace, move data over, and extend new partition to take up all diskspace)
Fedora defaults to separate partitions in any new installation. It will, by default, want to create three subvolumes (uses BTRFS as default now, not etx4) at these mountpoints /, /var, and /home.
First if you are reinstalling, which you actually shouldn’t need to do if it was working fine, just do sudo dnf --refresh-upgrade. Then do a sudo dnf system-upgrade --releasever=37 to upgrade it to F37 if you wish to run the beta.
But in any case, your data should be properly stored somewhere on another device or your other system, probably using rsync.
That actually means you are willing to sacrifice, for lack of a better term, whatever is on the disk now. For the experimental system you described, you can actually choose a custom layout. I believe Anaconda, the installer asks you to choose between different basic option like BTRFS or LVM setup.
You could then assign the existing /boot and /boot/efi partitions to their designated mount points. The / partition will have to be reformatted in that scenario. There’s no way around that. I may have misread your initial post. There’s a way of re-using partitions. I did that myself not too long ago als coming from a different distro on my laptop. I’m not sure if the separate /boot and /boot/efi is supported. I’ll check.
Using either BTRFS (default) or LVM layout, you would still have all the space available for both. BTRFS, in that respect, is more flexible with its subvolumes. In LVM you would use one volume group assigned to the partition and then assign space in logical volumes to / and /home. Start small and expand as needed. Doable, but I’d go with BTRFS. Saves you a lot of hassle and cries for help from friends and relatives.
I think that is no longer true. Maybe re-run the tests in Fedora. There has been a lot of optimization for BTRFS.
I hope a clarified that the limitation is limited to having to format the partition assigned to /. Having said that not all partition layouts are supported and Anaconda sometimes requires some tickling () to get it to do what you want.
I believe @bleuzen mentioned quite valid reasons for doing a clean install. Even though upgrading is supported and works quite well, even I’m considering doing a clean install next time having gone through so many upgrades over the years.
I still have anaconda-yum-plugins-1.0-10.fc20.noarch installed on my machine.
I knew I missed something important. Anaconda has some hard requirements. One of them is that it will format the partition assigned for /. Normally that’s not an issue, since /home is in a separate partition, subvolume or logical volume, which can simply be mounted during/after installation.
Why that limitation is there has all sorts of reasons, most of which elude me, but they are there mostly foor good reasons. If you’d really want learn more about the reasoning, you would need to talk to some of the Anaconda maintainers. I doubt many, if any, hang out here.
Bottom line: you will have to get through the pain of re-installing once, separating / and /home. Thereafter you can enjoy the benefits of having these separated, not only for easier re-installation. There’s good reasons why it is common practice in Linux to separate stuff into different partitons/volumes and why most distros will default to that seperation for at least / and /home - and since UEFI als /boot/efi.
Thanks, this is something I will look into as a possible option.
Using EXT4 with everything on one partition does also avoid wasting dispace, only that it offers better performance, which is the main reason I prefer it.
(Also saves me a reformat (which is the point of this thread), since everything here is already ext4.)
I have a backup. But during reinstall the data on my main disk would be erased, leaving only one copy (on my backup disk) temporarily. And I like having at least two copies around.
This would be no problem if I could just leave the data in place on my main disk during reinstall.
This is what I did about a year ago when I switched back from btrfs to ext4
Unfortunately now that the disk is filled above 50% this would be an extremely long process of resizing/moving partitions multiple times.
Also, in general, this is a waste of time and should not be needed.
That moving/resizing partitions and copying data takes hours and wears down the disk, while reinstalling on the existing partition would be done in 10 minutes with the same end result… if only the installer would not have this weird artificial limitation.
So yeah, in the worst case, this would work. But hope for a better solution.
For example:
Since the installer seems to be made in python and with some luck is not in any form compiled, maybe we could do something like:
nano /path/to/installer/some/codefile.py
and comment out the code that produces the limitation and checks if / will be formatted.
And I want everything on one partition to share the space. Ext4, to be exact.
No worries, most important data is on a second disk. I actually use rsync already, nice tool, thanks for the tip
For now on this one specific machine, this would be possible.
But since this whole operation I’m trying to do also serves as a test for ‘the real thing’ later (on my main machine) this would not be the way to go forward.
To clarify: I want to reinstall, on one existing ext4 partition (where my userdata already lives on and the new system should join it), without formatting.
…which are just other layers slowing down and not needed… if there wasn’t this artificial limitation of the installer.
In general, there is a way around it. Every other distro I used does support installing on an already existing partition but Fedora.
I don’t want to reformat my disk for every reinstall. If there is really absolutely no way, I may have to use another distribution. Which is sad, because I like Fedora otherwise
Yes, between the two, definitely btrfs.
Sure, why not
Tests done with:
OS: Fedora 37 (Beta)
Kernel: 5.19
SSD: Samsung 980 Pro
Both test partitions (ext4 and btrfs) are freshly created (upon luks2) and do have enough space left (to avoid the filesystem getting full slowdown, if this is still the case nowadays)
The results…
EXT4:
BTRFS:
Also ran the tests a second time, numbers didn’t change much.
Hm, I think it got a little bit better since last time iirc. But still not convincing, ext4 is miles ahead still.
Also I work a lot with databases and virtual machines, which are one of btrfs pain points. I don’t want to have to think about every file and manually disable COW on individual ones. Btrfs feels like it gets into my way more often than not (I already used it for some months before switching back).
So while btrfs can be a nice filesystem, ext4 is still the better deal in my case, so I will stay with that.
I would be quiet happy if we could, even with some tickling, get it to do the trick
Just don’t know where the .py file of the installer are and what to edit to remove the limitation.
The only thing I could think of is that unexperienced users are warned of possible issues if they install over another system (if they forget to delete the old system before).
However the current way the installer handles it is to completely block the installation in case someone even tries, even when they know what they are doing. This is not a smart move. There should be only a warning in the installer, but it should not prevent advanced users to do what works perfectly fine.
They did this for example when choosing a password and you enter a weak one. The installer will warn you that this is a bad idea, but you can still continue the installation. It would make sense to do the same for the ‘reinstall on existing partition’ warning.
Makes sense.
Another bottom line: I will have to go through the pain on every reinstall (if I can’t trick the installer), since I don’t plan on separating / and /home for the shared space.
This indeed is a point for btrfs. However currently I care more about performance. Don’t get me wrong, I do like btrfs. Just in terms of performance optimization it is not quite there yet for my taste. Might check it out again some kernel versions in the future tho
(Also, for completeness, keep in mind the benchmark I posted above does not show the full story. Filesystems could perform different on different workloads.)
Well, there might still be parameters that could be tuned for BTRFS. But if performance is really what you are after, you should not enable disk encryption on the partition that you need performance from. That will always add a performance penalty, whether it’s used with BTRFS or EXT4. And that’s another good reason to separate your /home from your / and possibly the partitions / volumes that you want to store your databases / VMs on.
This was referring to a special case I have encountered, where a certain option needed to be selected, otherwise Anaconda would not allow mounting an existing BTRFS volume in the partitioning step. That scenario does not apply in your situation, I’m afraid.
I later realized that using only one partition was essential to your case. That’s why I posted an addendum.
Since you insist on having everything in a single partition, and Im not going to re-iterate the pros and cons, yes, every re-install will be as painful as this one.
Ultimately, it’s up to you to consider your options and maybe re-evaluate your choices. Kickstart maybe a way out for you.
I recently stumbled into this issue when trying out fedora 37. Here is what you can do to workaround this artificial limitation:
The installer sits at /usr/lib64/python3.11/site-packages/pyanaconda. If you encounter an annoying warning, just cd into that directory, use grep -r <string>, and then edit it out.
In this case, the following parts are the problem:
Additionally in my case, the calculation of remaining free disk space was broken. It said it needs “-1.81TB additional free space” (note the negative sign) - So I had to comment that out too:
Hi, status update:
I spent some time trying the Fedora from scratch route. Unfortunately I didn’t find an up to date and fully working guide out there. Best I got was a system that would show grub (and the installed kernel):
Tried gdm and sddm but none would start up after this (with their service enabled).
I had no idea what was missing, so I gave up for a while.
Not really. CPUs do have hardware acceleration for crypto operations these days, which is very fast. In tests on my machine, encryption only has a negligible impact on performance.
Btrfs on the other hand causes a much bigger hit on performance.
(Also encryption has valuable benefits, like in case I have to send in a drive for warranty, I don’t want to send them unencrypted data.)
Yeah, since it is the only way to have both: shared space and good performance
Thank you a lot! This is a really nice solution.
Can confirm it also works in my case.
First of all, I’m glad to hear you accomplished what you set out to.
Since hardly anything happens all by itself, may I suggest one (or both) of you consider submitting an RFE for Anaconda. As a starting point: Anaconda/Contribute - Fedora Project Wiki. Most likely it will be by way of a bug report (Log in to Red Hat Bugzilla), but do read the info on the wiki page first.
Maybe the “good reasons” for not allowing that behavior are no longer relevant and the installer can be changed. Feel free to cc me in the bug report (gui1ty).
) to get it to do what you want.
