Best media for long term data storage?

I thought I’d ask this in here as I expect people to have some good suggestions.

I have around 20GB of email backups I want to store in a safe for long term backup, but don’t want to carry it around taking up SSD space on laptop.

Can anyone suggest the best medium to use? I wish I could burn to DVD as that would be cheapest but I cant due to size and don’t fancy trying to split files up as that seems compicated (if even possible).

So the choice seems to be between USB stick (quality, Sandisk probably) or an older used HDD (Toshiba).

Which would be best to use for long term storage?!
thanks

Well, in general 321 backup strategy is valid ;). 3 copies, on 2 different media/devices (SDD/HDD/tapes), 1 off-site. If backup is encrypted then “1 off-site” could be a cloud storage. For the local storage I’d choose HDD from reputable brand. In any case, backup is worth nothing if not tested periodically.

3 Likes

As @arturasb said and also bear in mind that for archival take into account:

  1. You need to test the backups to ensure they have not decayed.
  2. Beware that the file system you use may be dropped if you keep the media long enough.

Copying archival media every few years help with both the above.

1 Like

Thanks both. Not sure what “archival” means in this context, or archival media.
It sounds like a reputable used HDD is preferred to a brand new solid state Sandisk 32GB stick or similar, is that true? Surprises me if so, but then that’s why I asked the question!

No! This is not what I meant :slight_smile: I’d never use used devices for backups myself.
I think that for having faster initial backup (to compress the output, maybe encrypt it right away) on server grade SSDs is OK. Next, you should validate the backup and if it is OK - make a copy to your HDD backup storage and to an off-site location. Copying already compressed and encrypted backup to slower HDD and to cloud storage over network should save some time, IMHO.

You might find this interesting.

Personally, I like to use mirrored (and encrypted) ZFS pools for my offline storage. Unfortunately, that is a super complicated setup. But the gist of the idea is that even if bit rot does occur on the devices, as long as the extent checksum on one of the devices is valid, the other devices can be “healed” by referencing the good copy. It is still possible that the same extent is corrupt on all the devices, but that is slightly less likely. Also, I run ZFS’s “scrub” command on the full device once per year which will detect and fix any bit rot that is present at the time it is run.

1 Like

On my NAS I sacrifice RAM for ZFS and disks for mirrored storage as well :slight_smile:
The setup is complicated a bit, but ZFS saved me from failures of 4 disks so far. 2 SSDs and 2 HDDs.

1 Like

Oh, one important point about ZFS – do not use the zfs-fuse package that is in Fedora’s default repositories. zfs-fuse is not maintained (it ought to be removed from Fedora’s repos).

Appreciate the replies, but I should have said, I don’t have option to buy any hardware for this. I understand the replies above are GOOD advice, but probably aimed at a higher level than I need (both in complexity and data integrity priority).

I just want to shove some email folders in the safe, but I can see it’s not that simple without taking some risk (bearing in mind life without risk is impossible).

One thing the above replies did was remind me of something…

On Mac I used a program called AES Crypt. I loved that thing, SO simple to use (no CLI), drag and drop files/folders onto the icon on dock, create a good strong password (put in Bitwarden!) and hit enter. Compression and encryption in one. I don’t know if it’s as strong as it could be (the encryption) but it was more than good enough for me, to stop prying eyes (family for example) opening a random folder on my machine and seeing business documents or confidential stuff from past work or whatever.

I was hoping to use this on Fedora but can’t see it in Software.

I would have liked to encrypt the data, and compression sure would help as it must surely come down in size dramatically given that it’s email data with very little AV media inside.

So, if I could rephrase the question…

As a dumber with no knowledge of CLI, I’d like to compress and ideally encrypt (easily) 20GB of data to put on a HDD in safe. Any suggestions?! :smiley:
thanks

Very interesting reading there (SSD needing power to keep it healthy, fascinating)

They can survive offline/unpowered for some time. I have some thumb drives that I only connect once per year and they seem to be fine after several years of such a usage pattern. But there are many factors that affect how long they will “keep”, including the brand of the device and the temperature they are stored at (and the temperature of the device at the time the data was written – if the SSD is hot, the charge that represents the data will write “deeper” and hold for longer, so it is best to write to the device when it is hot, but store it cold. :slightly_smiling_face:). Also, like I said, I run ZFS’s “scrub” command once per year and I like to think that the simple act of re-reading all the data on the device will also cause the data cells to be re-energized/recharged for another year (but I don’t actually know that that is how it works).

1 Like

haha, very intriguing this stuff. So I need to crank the heating up in my office when writing, but wait for winter to read? (Ok just to make sure, I AM actually joking there) :smiley:

I just had a thought which, in light of the above info, may be a better option (or an additional one at least).

I have an iMac which I use for business, it’s on 24/7, has two TimeMachine backup drives connected at all times, and I regularly clone the internal drive to a backup drive I keep in fire safe. It has half a TB spare, so I could just move this data to there, encrypt with AES Crypt, and use that as a backup, since the machine is well backed up routinely anyway.

I could then also put the stuff on a small HDD in the safe, just for the hell of it :smiley:

1 Like

But maybe to a different safe than the first one :smiley:

1 Like

Also, like I said, I run ZFS’s “scrub” command once per year and I like to think that the simple act of re-reading all the data on the device will also cause the data cells to be re-energized/recharged for another year (but I don’t actually know that that is how it works).

The ZFS scrub would correct for cells that do go bad by leveraging the redundancy in the filesystem

I draw a distinction between a backup that is a copy of my in-use data and moving data out of the in-use set of data, creating an archive.

Since you delete the data after copying it off it even more important how ypu manage the media its stored on is wgat I am getting at.

I know that. What I don’t know is if the simple act of accessing/reading a SSD cell will also “re-enforce” the charge on a cell (even if it hasn’t degraded to the point that the data is no longer correct).

Although the cells store 0s and 1s to represent the data, in truth, the actual value of the charge on the cell can be a much more continuous range of values (and that charge inevitably dissipates with time). It is entirely possible that the circuitry within the hardware device could detect that a cell’s charge is running low at the time it is read and automatically re-charge the cell if needed (but I don’t know if it actually does that and it might depend on the brand of the device). I think I remember reading somewhere that SIMMs/DIMMs actually do that sort of thing, but those are RAM technologies.

Good question. Edit: i did some search of the web to see what the answer might be.

It seems that powered on ssd refresh their stored data.
Powered off ssd should keep its data for 1 year when kept in temp <30C,
but only 3 months for “enterprise” ssd.

I’m sure the SSDs don’t suddenly lose all their charge/data after 1 year but I have no idea how much degradation there could be. FWIW, the cheap 8GB SanDisk Cruzer thumb drives that I’ve been accessing once per year (around tax season :slightly_smiling_face:) for the last few years have shown zero degradation (as indicated by ZFS scrub).

It is quite interesting that enterprise SSDs are only rated to hold data offline for 3 months.

If you are looking for a gui application to create encrypted backups, then take a look at this. Its available as flatpak, you can install it using GNOME software but make sure you have flathub enabled. And you can enable flathub with few clicks as mentioned here. There’s also “File roller”, if you just want to create compressed zip archives and its available in both flathub and fedora repository. You can install both of those applications in GNOME software, its just that flathub needs to be enabled for apps available only in flathub.

1 Like

Thank you. I’ve actually heard that mentioned on another thread when I was looking for something similar to Carbon Copy Cloner for Linux. Looks useful to me