What is the best way to store files in archive and preventing data loss due to bad sector?

Hello everyone, linux newbie here.

I want to store some files with encryption.

But first of all, I must say that I scare of data corruption due to bad sector data loss on HDD.
I don’t want to lose all data because of just a single byte loss etc.
If some bad sector occurs, I want to lose just some pixels/letters etc, not entire archive.

So, what is the best way to store files with password protection?
Is zip enough? Or should I use tar + gpg? Or something else?

1 Like

You are mixing different tasks:

  • The best way to protect data against loss/corruption is a backup.
    Better only a verified backup.

  • The best way to secure access to data is encryption.
    The optimal encryption method/type depends on your use case.

3 Likes

Yes, my plan is copying archives to two different HDD’s. Is it enough to protect from loss/corruption?
And in this case, is zip enough? Or should I use tar? I dont know which algorithm/program is better for protection from loss. Just I dont want to lose all data due to single bad sector.

You may inform yourself about btrfs and its featureset:

  • data and metadata checksums
  • inline compression
  • filesystem snapshots
  • sending of differential filesystem snapshots
  • RAID-like operation with self-healing capabilities

Therefore this filesystem alone (coming with Fedora 33 by default on installation) could serve your needs with some beforehand thought.

1 Like

Thank you for this information, I noted that.

In my case, I’m looking for more suitable / portable solution because I will backup files two different HDD’s. So I just want to know which archive algorithm better about rescue from bad sector

There’s no absolute best, there’s only better.
What is good enough for you may be not enough for someone else.

2 backups is better than 1 backup, which is better than no backups.
Independent storage for each backup is better than common storage for all backups, which is better than the same storage for both originals and backups that can barely be considered as backup.
That’s why compared to a real backup, file systems like Btrfs just slightly improve reliability.

Compression can help you save space at the cost of time.
Depending on the size and type of data, sometimes it’s worth making a deal, sometimes not.
Compression level and algorithm mostly depends on use case and personal preferences.

First of all, thank you for answering.

I’m lack of information about this: I believe (believe, not know), if I compress files, and if bad sector occurs; archive won’t decompress because it lost some bytes etc. So I prefer directly store without compress.

Am I right about this?

Yes, generally uncompressed data is easier to recover in case of emergency.

On the other hand, you may also be interested in the following:

Thank you for advices, but these are big solutions for me. I don’t want to encrypt all storage, I want just a single folder.

So I have new question;
About ease to recover from bad sector/data loss, is encrypted uncompressed archive same hard as encrypted compressed archive?

I don’t know zip handle data loss while decryption, so I wanted to ask this. Sorry if this is a silly question

Then EncFS looks like the right way for you:
https://www.howtoforge.com/encrypt-your-data-with-encfs-fedora-18

Compression can be considered as an additional layer of data obfuscation.
Thus, theoretically it should make data recovery more difficult.
However recovering damaged encrypted data is no easy task in any case.

1 Like

Understood, thanks so much for answers

1 Like

For me you are thinking in the wrong direction: the only thing you can do is to buy high quality hardware and monitor the SMART values of your storage regulary. Modern high-end hardware has also self-repair options and so you have the chance to save your data timely. Compressed or not compressed is not so important, encryption protects your privacy. Periodical backups are absolutely necessary. Using the proper filesystem is also a sound ground for healthy data. The strategy to using your hardware in a safe way is crucial. Avoid cheap and nasty hardware. :smiley:

1 Like

As @heliosstyx said, good quality storage devices used for backup would be best. I suspect your fears about data loss with a single byte or sector loss may be related to the antiquated use of tape storage which were definitely subject to loss.

Current technology with use of raid, backup to redundant HDDs stored separate from the original, etc. makes the loss of data on a copy of much less concern.
Have a good backup plan, follow it, monitor the status of devices used for the backup, and everything should be ok.

1 Like