Deja-Dup in Fedora 30 unable to create or restore

Ask Fedora Ask in English
1

I had a question in the old forum about Deja-Dup failing to create or restore back-ups.
For some bizarre reason it managed to go unsanswered throughout the life of Fedora 29 - weird huh ?

I lost patience and have trashed that Fedora 29 installation. I’m now the proud owner of a brand new Fedora 30 PC with only a single change

  • I’ve installed Deja-Dup and have tried once again to get my data back onto this machine.

For about the entire year of being ignored the only suggestion was, and I paraphrase because … this is a new forum
“File a bug with the Deja-Dup developers because this can’t be a Fedora problem”

Filed the bug - no response other than " it might be an issue withe Fedora because other users aren’t seeing the same problem"

I tend to agree, mostly because I installed Deja-Dup on an MX Linux laptop and was able to run it, and restored the back-up on this external HDD.

So the steps taken to test whether it’s a problem with Fedora 30 have, thus far, consisted of.

  1. A new install of Fedora 30 (not an upgrade, the HDD was reformatted and repartitioned and the installer successfully completed.
  2. Opened the Software catalogue and installed “Backups” aka Deja-Dup
  3. Set the external drive as the back-up source
  4. Opted to Restore
  5. Waited until Deja-Dup found the back-ups on the drive.
  6. Selected the most recent
  7. Waited for a day while it restored.

I didn’t have to wait that long, after 12 hours I checked and it had failed - again with this error

Failed to read /home/saywot/.cache/deja-dup/tmp/duplicity-nf8HgH-tempdir/mktemp-62HNtZ-1: (<class ‘zlib.error’>, error(‘Error -3 while decompressing: invalid stored block lengths’,), <traceback object at 0x7f88d7846cf8>)

1 Like
2

Hello @saywot—welcome to the community. Please take a minute to glance over the #start-here here posts if you haven’t already.

This is a weird one. I see that there are multiple reports of this error on different tools, but in none of them has anyone been able to figure out the cause. Mostly, the error “just went away”.

https://duckduckgo.com/?q=zlib+invalid+stored+block+lengths&ia=web

Could you point us to the bugs that were filed? It needs to be narrowed down—it could be anything from a zlib bug to a bug in the use of zlib.

3

https://bugzilla.redhat.com/show_bug.cgi?id=1688096

Sent AFK

1 Like
4

Dagnabbit !
It hasn’t “just gone away”

Failed to read /home/saywot/.cache/deja-dup/tmp/duplicity-g_o7EP-tempdir/mktemp-ojBAFD-1: (<type 'exceptions.IOError'>, IOError('CRC check failed 0xe04db144 != 0x7cbda341L',), <traceback object at 0x7ff57cc16d40>)
5

just for a lark, to see if there’s something amiss with the backend (duplicity)
I attached an unused Toshiba 2TB HDD to the computer (it’s empty)
ran …

[saywot@Fedora28 ~]$ duplicity restore file:/run/media/saywot/Backup2/Fedora28/ /run/media/saywot/TOSHIBA\ EXT/ --force --no-encryption
Synchronising remote metadata to local cache...
Copying duplicity-full-signatures.20190201T030753Z.sigtar.gz to local cache.
Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1445, in do_backup
    sync_archive(col_stats)
  File "/usr/bin/duplicity", line 1230, in sync_archive
    copy_to_local(fn)
  File "/usr/bin/duplicity", line 1178, in copy_to_local
    gpg.GzipWriteFile(src_iter, tdp.name, size=sys.maxsize)
  File "/usr/lib64/python2.7/site-packages/duplicity/gpg.py", line 451, in GzipWriteFile
    outfile.write(new_block.data)
  File "/usr/lib64/python2.7/gzip.py", line 240, in write
    self.fileobj.write(self.compress.compress(data))
  File "/usr/lib64/python2.7/site-packages/duplicity/gpg.py", line 427, in write
    result = self.fileobj.write(buf)
 IOError: [Errno 28] No space left on device

This can’t be the case it’s an archive from a 1Tb machine !

The Toshiba drive doean’t have any hidden folders or files

6

So, the restore was being done from somewhere to the newly attached Toshiba device? The error could imply a number of things, from the looks of it:

The ENOSPC (“No space left on device”) error will be triggered in any situation in which the data or the metadata associated with an I/O operation can’t be written down anywhere because of lack of space. This doesn’t always mean disk space – it could mean physical disk space, logical space (e.g. maximum file length), space in a certain data structure or address space. For example you can get it if there isn’t space in the directory table (vfat) or there aren’t any inodes left. It roughly means “I can’t find where to write this down”.

reference: filesystems - Python causing: IOError: [Errno 28] No space left on device: '../results/32766.html' on disk with lots of space - Stack Overflow

This, however, is probably not related to the deja-dup issue.

7

Keep in mind these two lines:

duplicity is not copying to your 1TB drive, but copying to a local cache in a temporary directory. This may be ~/.cache/duplicity (if run directly), /tmp or /var/tmp or ~/.cache/deja-dup/tmp (if using deja-dup). Could there be anything special about one of those directories on your system?

Now, deja-dup is written in Vala, but the error you have from the first post is a Python traceback. duplicity is written in Python, so the error is either there or in how deja-dup is using it. Looking around, it seems you can set an environment variable to tell deja-dup to use a different temporary directory; you could try running it as:

DEJA_DUP_TEMPDIR=/run/media/saywot/TOSHIBA\ EXT/tmp deja-dup

and if that works, then the question is why are your ~/.cache directories broken…

1 Like
8

This installation of Fedora 30 is 5 days old, at the time this issue became apparent Deja-Dup was the only application installed apart from the defaults.
I have no clue why duplicity is doing what it did not why my /. cache directories are broken.

Tony Clifton

9

Now I have completely rebuilt the hardware (new CPU, more RAM and a shiny HDD) and installed Fedora 30
The only thing I have installed from the default repository or Software Catalogue is Deja-Dup
This is the only thing I have tried to do.
a RESTORE

so set the storage location for Deja-Dup
and clicked on the “Restore” option.
It found a back-up for 16/05/19
and started preparing, it prepared for so long that I went to bed.
Woke up this morning expecting to see all my documents, pictures, videos etc etc restored to their original location but - NO :frowning:
Failed with an unknown error.

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1498, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 733, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
    ITR(ropath.index, ropath)
  File "/usr/lib64/python2.7/site-packages/duplicity/lazy.py", line 344, in __call__
    last_branch.fast_process, args)
  File "/usr/lib64/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
    return function(*args)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 614, in fast_process
    ropath.copy(self.base_path.new_index(index))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 445, in copy
    other.writefileobj(self.open("rb"))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 627, in writefileobj
    buf = fin.read(_copy_blocksize)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 212, in read
    if not self.addtobuffer():
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 233, in addtobuffer
    self.buffer += fp.read()
  File "/usr/lib64/python2.7/tarfile.py", line 829, in read
    buf += self.fileobj.read()
  File "/usr/lib64/python2.7/tarfile.py", line 743, in read
    return self.readnormal(size)
  File "/usr/lib64/python2.7/tarfile.py", line 758, in readnormal
    return self.__read(size)
  File "/usr/lib64/python2.7/tarfile.py", line 748, in __read
    buf = self.fileobj.read(size)
  File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 209, in read
    return self.fileobj.read(length)
  File "/usr/lib64/python2.7/gzip.py", line 267, in read
    self._read(readsize)
  File "/usr/lib64/python2.7/gzip.py", line 331, in _read
    self._read_eof()
  File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
    hex(self.crc)))
 IOError: CRC check failed 0xa5587c60 != 0x5aa83c9aL
10

so just so it’s clear this was a day-old archive, a new hours-old Fedora 30/Deja-Dup installation.
There are no hardware issues.
And Deja-Dup remains useless.

11

just for a giggle I tried to restore a few missing files from Nautilus
Deja-Dup opened, it looked through the back-up, presented me with a list of files.
I ticked a few to restore them to the directory

Failed (of course)

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1498, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 733, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
    ITR(ropath.index, ropath)
  File "/usr/lib64/python2.7/site-packages/duplicity/lazy.py", line 344, in __call__
    last_branch.fast_process, args)
  File "/usr/lib64/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
    return function(*args)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 614, in fast_process
    ropath.copy(self.base_path.new_index(index))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 445, in copy
    other.writefileobj(self.open("rb"))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 627, in writefileobj
    buf = fin.read(_copy_blocksize)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 212, in read
    if not self.addtobuffer():
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 233, in addtobuffer
    self.buffer += fp.read()
  File "/usr/lib64/python2.7/tarfile.py", line 829, in read
    buf += self.fileobj.read()
  File "/usr/lib64/python2.7/tarfile.py", line 743, in read
    return self.readnormal(size)
  File "/usr/lib64/python2.7/tarfile.py", line 758, in readnormal
    return self.__read(size)
  File "/usr/lib64/python2.7/tarfile.py", line 748, in __read
    buf = self.fileobj.read(size)
  File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 209, in read
    return self.fileobj.read(length)
  File "/usr/lib64/python2.7/gzip.py", line 267, in read
    self._read(readsize)
  File "/usr/lib64/python2.7/gzip.py", line 331, in _read
    self._read_eof()
  File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
    hex(self.crc)))
 IOError: CRC check failed 0x5f995e46 != 0xecf64586L
12

Hi. I’ve had a similar issue in the past. It may not be the same issue that you’re facing but it sounds very similar.

I eventually figured out that it wasn’t complaining about any of my storage drives being full but my RAM (/tmp is tempfs on Fedora). Déjà-Dup uses a whole lot of memory when working with many objects and I ran out of memory before it could read some backup table, manifest, or whatever it was.

The solution was to set aside a 128 GB SSD I had laying around as a SWAP partition. It was incredibly slow but I did eventually manage to restore my Déjà-Dup folder.

Today I’m use Restic for my home directory and data partitions, and only use Déjà-Dup as a secondary backup for my ~/Blog and ~/Documents directories. (The Nautilus integration where I can restore previous copies of files still makes it useful.)

13

I think I’ve solved this, on my own, no help from anyone !!
It’s not a Deja-Dup problem, it’s sort of a Fedora issue though
I am halfway through completing this long-running restore from a back-up made before my Fedora 30 installation and the problem seems to be with — wait for it

Wayland

I booted into Gnome-Classic (not Gnome) and there ya go, Deja-Dup is working

14

Eh? Wayland? How so? No wonder no-one managed to diagnose it—that’s totally unexpected. How does the display system come into it? :thinking:

15

I haven’t any clue, no backing-up/restoring with ‘New Gnome’ but plenty if I boot to “Gnome Classic” .

I wouldn’t have found this except I needed to run Gedit as a super user

Sent AFK

16

Rebuild - new machine
tried to create a back-up (how many is this ? - 10,11,12 !)

Failed

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1549, in do_backup
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 664, in incremental_backup
    bytes_written = dummy_backup(tarblock_iter)
  File "/usr/bin/duplicity", line 227, in dummy_backup
    while tarblock_iter.next():
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 523, in next
    result = self.process(self.input_iter.next())
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 195, in get_delta_iter
    for new_path, sig_path in collated:
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 286, in collate2iters
    relem2 = riter2.next()
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 354, in combine_path_iters
    refresh_triple_list(triple_list)
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 341, in refresh_triple_list
    new_triple = get_triple(old_triple[1])
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 327, in get_triple
    path = path_iter_list[iter_index].next()
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 239, in sigtar2path_iter
    for tarinfo in tf:
  File "/usr/lib64/python2.7/tarfile.py", line 2510, in next
    tarinfo = self.tarfile.next()
  File "/usr/lib64/python2.7/tarfile.py", line 2350, in next
    self.fileobj.seek(self.offset - 1)
  File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 221, in seek
    return self.fileobj.seek(offset)
  File "/usr/lib64/python2.7/gzip.py", line 442, in seek
    self.read(1024)
  File "/usr/lib64/python2.7/gzip.py", line 267, in read
    self._read(readsize)
  File "/usr/lib64/python2.7/gzip.py", line 331, in _read
    self._read_eof()
  File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
    hex(self.crc)))
 IOError: CRC check failed 0x662d09a != 0xeba234a7L

What’s going on here ?

17

Not a clue, but something very specific to your setups. Not seen any of these errors on my multiple machines.

What exactly are you backing up? Are these files changing while you are backing up? That’s one thing that can cause the CRC check to fail.

18

it’s a ‘straight out of the box’ install, onto 2xHDD using LVM.

the only operation is the back-up

Sent AFK

19

But what files specifically are you backing up? Are these in use and therefore changing while the back up is in progress?

20

It’s backing up /home/saywot

  • and no the files aren’t changing (except for them being zipped-up and tarred into a compressed archive)

Sent AFK