Changed diacritical characters in the names of backup files and folders - Pika Backup

xracer · November 17, 2022, 7:20pm

Hi.

I backed up my files yesterday using Pika Backup software, installed via flathub on Fedora 37.
Today I tried to restore this backup using Pika Backup and noticed that all diacritics in the file and folder names are changed to some strange symbols. The problem also occurs when I open backup in Vorta.

The above names should look like this:
Hiszpańska zupa z ciecierzycą_1.jpg
Hiszpańska zupa z ciecierzycą_2.jpg
The same is true of diacritical marks from any other language.

I searched for a solution on the Internet, but found nothing similar.

Does anyone have an idea how to open the backup correctly so that the characters display correctly? or how to change all those symbols in the names of all files and folders to the correct letters. I have thousands of files and folders with corrupted diacriticals.

Your sincerely.

xracer · November 19, 2022, 10:21am

Ok.
I manually replaced the strange characters using the “Szyszka” tool available in the software section.

I’m still curious why the Pika backup made errors in the backup file names.
If anyone ever knows the answer, please write about it.

beastofcaerbannog · November 19, 2022, 10:48am

I guess that it doesn’t support all accented characters. I didn’t find a relevant open issue in the project’s GitLab, so I think it would be good if you could create a new one and let the developers know about it and fix it.

johnandmegh · November 19, 2022, 12:03pm

Just curious, what kind of device/under what system are you backing up to? Pika is based on BorgBackup, and in browsing the BorgBackup issues there was discussion of different systems having different encoding schemes by default (across Windows, macOS and Linux) - I experienced almost the exact same issue with certain characters when backing up to a drive connected to my Windows desktop, so I wonder if the issue occurs when backing up to the same device, and if it occurs backing up to another device of the same OS?

Possibly related?

github.com/borgbackup/borg

macOS: borg mount problem with unicode normalisation

opened 06:05PM - 30 Sep 19 UTC

aljungberg

help wanted os: macOS

## Have you checked borgbackup docs, FAQ, and open Github issues? Yes ## I…s this a BUG / ISSUE report or a QUESTION? BUG ## System information. For client/server mode post info for both machines. #### Your borg version (borg -V). borg 1.1.10 #### Operating system (distribution) and version. macos 10.14.6 #### Hardware / network configuration, and filesystems used. MBP, Mac OS Extended #### How much data is handled by borg? 1.5 TB #### Full borg commandline that lead to the problem (leave away excludes and passwords) ``` borg init -e none test.borg mkdir test filename=`printf 'la\xcc\x88.txt'` echo "hello" >test/$filename borg create test.borg::a test mkdir test2 borg mount test.borg::a test2 ls test # lä.txt ls test2/test # lä.txt cat test/lä.txt # hello cat test2/test/lä.txt # cat: test2/test/lä.txt: No such file or directory ``` So there is a file we can list but not open. I have the same problem with a real archive. #### Thoughts So the problem here is likely related to https://code.google.com/archive/p/macfuse/issues/139#c2: a mixup between precomposed and decomposed UTF encodings. In `fuse.py`, in `lookup`, we have this: ``` inode = self.contents[parent_inode].get(name) ``` Here we get name encoded as `b'l\xc3\xa4.txt'` (precomposed). But in the archive, in `self.contents[parent_inode]` we have `{b'la\xcc\x88.txt': 1000041}` (decomposed). Both forms are technically equivalent: ``` >>> os.fsdecode(b'l\xc3\xa4.txt') 'lä.txt' >>> os.fsdecode(b'la\xcc\x88.txt') 'lä.txt' >>> unicodedata.normalize("NFD", os.fsdecode(b'l\xc3\xa4.txt')) == unicodedata.normalize("NFD", os.fsdecode(b'la\xcc\x88.txt')) True ``` I would have liked to submit a patch but I honestly don't know the correct way to deal with this. If I add `name = os.fsencode(unicodedata.normalize("NFD", os.fsdecode(name)))` in `lookup`, it works for this particular error. But it would break if the archive used NFC instead. We could also `normalize` the encodings before writing them to `self.contents`. This would work in both cases. But what encoding is supposed to be used to begin with?