I’ve decided to start the slow(super slow given the size of my wallet…) transition from hdds to ssds, at least for a good chunk of my data.
Now I’ve found out about this Logical Volume Management thingy and watched a video about how to combine multiple drives togheter into a single partition, and while those commands look a bit scary and the whole “if just one drive fails all of your data is gone” bit is giving me just a mega tiny itty-bitty little pause, I was wondering does anyone know if it’s even gonna work in my case since one of the ssds will slot directly into my motherboard into it’s secondary ssd slot(using the main one for my boot drive) while the other future ssds will be on something like this:
While I have the smaller more important/more irreplaceable stuff backed up not only on different devices but also in different locations I’ve never been financially able to create backups for all of my data…
Now if this whole switching to ssds thing pans out I suppose this time I could try to just keep the data on the hdds as well instead of wiping and selling them off like I usually do when I upgrade something.
That however does bring up another question like you mentioned(and while I am sure I could just google it since I’m here might as well ask and if nobody knows/answers then I’ll ask google) which is exactly how long can those hdds stay unplugged(in a drawer somewhere) before they start losing data?
Talking about regular 3.5 inch internal hdds(from Seagate if it matters), not external/removable ones. How often would I need to plug them in and for how long should I keep them plugged each time, couple of minutes, couple of hours, or what?
Cool, as long as LVM doesn’t care how they are connected that’s fine with me even if there should be some performance loss since they will still be way faster then hdds. And considering they will be used mostly for seeding stuff and that I already upload almost 1TB of data per day with hdds, I think the people should be pretty happy with the upgrade regardless, no?
Yeap, after I got a bit more comfortable with Linux I remember at a certain point I actually did google and read up on the differences between ext4 and btrfs, and I concluded that I simply don’t need the things that btrfs can do and that ext4 is a better fit in my case.
I suppose I could read up on it again(cause of course I forgot everything the second I made my decision ) and see if I reach a different conclusion now that I’ve decided to do this whole ssd LVM thing, but I doubt it would change.
This might sound like a stupid question, and granted I should have been more careful when I ordered, but does anyone know if there is an easy(or at least easy-ish) way to test if that cheap chinese adapter I linked above is gonna send any data back to china?
Or should I just return it…?
LE: Nevermind I’ve decided to just return it rather then go through all the headaches and simply save up for something more decent like the Asus one below, or maybe just a new motherboard with more m.2 slots, haven’t decided yet.
Perhaps a backup system that supports deduplication would help?
Depending on the type of data, If the data isn’t already compressed, combined with deduplication it can dramatically reduce the storage requirements.
For HDDs made within the past 10 years or so that pack data much more densely, the magnetic fields are so strong that the ferrite bits can be readable for decades unless the drives are stored in a poor environment – e.g. heat speeds up magnetism loss.
(My long-term storage HDDs are in weatherproof containers with a desiccant pouch.)
A more likely issue is the electronics. Capacitors and resistors degrade over time, so they can drift too far off spec. Other than keeping the HDDs dry and cool, there’s not much that can be done about it.
Power-on time mainly impacts wear on the spindle motor and other moving parts. To “refresh” the data to prevent bit rot from magnetism loss, the data needs to be at least reread. Although rewriting is best, HDDs automatically do that when there’s a recoverable ECC error.
On a related note, NAND / flash memory hold an electrical charge that starts to degrade the moment data has been written. The most common recommendation is that data is generally readable for at least 5 years, but it’s going to depend a lot on the quality of the NAND and other factors.
Cool, thank you very much for the explanations in the first post.
As for the btrfs links you gave me they are interesting, and I don’t think I read them before, though with my memory I very well may have, however they do bring up more questions such as:
On the checksumming part I see the list with how many cycles each has per just 4 KiB and it scares the hell out of me!!!
I mean I know writes degrade ssds much faster the reads, but even reads supposedly cause problems since it seems there are these things called, and I quote:
‘read disturb’, where reading itself reduces the charge on the isolated gate. There’s also a thing called ‘neighbor disturb’ where reading an adjacent cell weakens the charge on a gate.
Not sure if I completely understand what that all means, but considering these ssds will be used mostly for seeding large chunks of data that are gonna be read a lot for at least 14-16 hours each day, won’t adding all those tens of thoudands of cycles from checksumming kill my ssds super extra fast?! I mean I’d like them to last at least half a decade for all the money I’m gonna have to pay for them…
Now on compression for the rest of my data(not the one I am actively seeding) how would that effect it’s usability? Say I want to watch a HD movie that is one of the hdds and that has been compressed with one of those methods can I just double click it, let VLC launch and watch it normally? Would that cause any stuttering or audio mismatch or some such problem? Or would I have to decompress it first before I could watch it?
I know this is gonna sound doubly stupid since my hdds already are NTFS(haven’t had the space necessary to reformat them) while the ssd that holds the OS is ext4, but ntfs is a win thing while ext4 and btrfs are Linux things.
So can I keep my OS ssd as ext4 and(should I decide to go that way) have the other ssds made as btrfs? Can I have both of those running at the same time in Linux, cause I’m seriously not gonna go through the hassle of reinstalling the OS…
Many onboard controllers in SSDs use deduplication, wear leveling, ECC, repeated reads and other techniques to compensate for read disturbances. When a flash cell’s charge is getting too low, the controller will rewrite the data in-place or to another data block (depending on the wear leveling).
Many SSDs also have DRAM caches for buffering frequently read data, and Linux/macOS/Windows all use some kind of RAM-based disk caching, so the amount of real-world reading/writing for a SSD is often a lot less than it might seem.
If you’re really concerned about wearing out a SSD due to write cycles, turn off journaling in ext3/ext4 (same idea applies to Btrfs, XFS and other journaling filesystems), but I don’t think it’s all that necessary – my home server’s OS drive (a 256GB Intel SSD) has clocked more than 55,000 power-on hours with over 9.1 TB of data written without a single hiccup.
At work, one of the production database servers keeps its data on SSDs. It handles over 42 million queries a day, constantly reading and writing data 24/7/365. The SSDs are just 2 weeks shy of 7 years old and none of them have needed to be replaced yet.
So if the majority of your workload is reading, the server the SSDs are in is more likely to hit the bit bucket first.
Coincidentally, exactly 11 years ago today, there was an article about the longevity of consumer-grade SSDs. What the authors found was that a 250GB Samsung SSD lasted thru more than 900TB of writes – that’s completely rewriting all 250GB once per day for 9.86 years!
(I’d include a link, but the article is no longer available from the original website or the Internet Archive’s Wayback Machine.)
Quick answers to your questions above – none, yes, no, no…
Btrfs’s inline compression is transparent to user-mode apps, they simply see what looks like uncompressed data. Most often you’ll only notice the difference in that a file manager and disk usage tool report wildly different results (e.g. a df -h on one of my disk volumes says 57GB is used, but it’s actually holding over 480GB of data – if I used deduplication, the disk usage would be even less).
Audio and video in an AV container include a timing track that aligns the audio with the video. Mismatches are most often due to a problem with the movie file itself; an underpowered system that cannot keep up with the video playback (e.g. 4K video on a dual-core PC); or software issues.
Since decompression includes both the audio and video streams, an underpowered computer might result in stuttering, but it’d have to be a pretty old computer because mainstream CPUs often have some sort of hardware-accelerated support instead of doing it all in software.
Unless your movies are uncompressed raw video instead of MP4, MKV or some other common AV container with MP3, AAC, H.264/265, etc., external compression from Btrfs isn’t likely to squeeze out much extra storage (it might even make the file occupy more storage).
Sure, mix and match as much as you wish. The only real hassle is dealing with permissions since you already know that NTFS isn’t compatible with Unix/Linux file permissions.
And if I was made of money I’d certainly consider it, alas I am not.
Cool @gadget that is a lot of very interesting/useful information, thank you.
In that case I guess I’ll stick with ext 4 for now because:
a) There truly is a lot to be said about simplicity, like @steppybug mentioned.
b) Most of the data held on these ssds will be .iso, .bin, .rar, etc. formats. In other words things that, unless I miss my guess, are already compressed so like you said I wouldn’t benefit from btrfs’s extra compression.
One of the problems I’ve been having in Linux is that sometimes when I fire up my pc in the morning a drive won’t get mounted and I have to restart Linux in order to fix that. Up until now this has only happened with a certain hdd which is kinda old, has been used intensely and already has quite a few dead sectors that QdiskInfo keeps warning me about, so I figured it was just an old drive that was dieing.
But since I bought my first high capacity(4 TB) ssd last week and started to transition the same thing has happened a couple of times with the new ssd. I fire up my pc and it doesn’t get mounted, I can still see it in KDE Partition Manager, it’s s.m.a.r.t. status reports as good, but I can’t use it until I restart the pc and it gets mounted properly.
So now finally comes the question: say I use LVM to combine multiple physical ssds into a single partition then one day when I fire up my pc one of them doesn’t get mounted will that ruing the entire partition and destroy all my data? Or can I just do my usual restart and everything will be fine after?
LE: As a matter of fact that old ssd sometimes gets unmounted after a while(without any system error warning or anything) even during regular pc usage, like it just happened now and I had to restart. That has not happened with the ssd yet, with the ssd only the failure to mount at startup in the morning happend a few times.
If it’s a simple logical volume spanning multiple physical drives, a drive in the volume group not responding would cause the system boot to stall or enter recovery mode (unless “nofail” is specified in the mount options for the volume) because the filesystem is missing a piece of itself.
A failed mount won’t destroy your data, so if a restart brings the missing drive back, it’ll be fine.
A drive suddenly dropping out of sight increases the odds of filesystem corruption. And if the drive is part of a LVM, it’s even worse.
Based on what’s been described, I’d suspect a hardware issue with the motherboard and/or power supply.
Given what’s been shared so far, your PC could be drawing too much power on one of its rails. Each time the PC starts, it’s pulling a lot of power to initialize the mainboard, spin up the HDDs, power up the SSDs plus other components.
(An ATX power supply tester is inexpensive and a great tool to have on hand. While a multimeter can be used, it’s safer and simpler to use a dedicated tester.)
It is pity, when I signed up I was impressed that there was still a place where users cared about others. where the problem solving was in the center without this gross language. Apparently you expecting from me to adapt me on your kind of wording and that I’m giving up my “values” while I try to use a language which others can understand?
I am not sorry being friendly and telling you in a simple but not unfriendly manner, what I think when I do read along in this forum. In your case I will let you do “whatever you want” give the others the chance telling you in an anonymous manner, how to use a unambiguous language that also makes sense when translated in different languages.
) and see if I reach a different conclusion now that I’ve decided to do this whole ssd LVM thing, but I doubt it would change.
thing while ext4 and btrfs are Linux things.
