Deja-Dup in Fedora 30 unable to create or restore

https://bugzilla.redhat.com/show_bug.cgi?id=1688096

Sent AFK

1 Like

Dagnabbit !
It hasn’t “just gone away”

Failed to read /home/saywot/.cache/deja-dup/tmp/duplicity-g_o7EP-tempdir/mktemp-ojBAFD-1: (<type 'exceptions.IOError'>, IOError('CRC check failed 0xe04db144 != 0x7cbda341L',), <traceback object at 0x7ff57cc16d40>)

just for a lark, to see if there’s something amiss with the backend (duplicity)
I attached an unused Toshiba 2TB HDD to the computer (it’s empty)
ran …

[saywot@Fedora28 ~]$ duplicity restore file:/run/media/saywot/Backup2/Fedora28/ /run/media/saywot/TOSHIBA\ EXT/ --force --no-encryption
Synchronising remote metadata to local cache...
Copying duplicity-full-signatures.20190201T030753Z.sigtar.gz to local cache.
Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1445, in do_backup
    sync_archive(col_stats)
  File "/usr/bin/duplicity", line 1230, in sync_archive
    copy_to_local(fn)
  File "/usr/bin/duplicity", line 1178, in copy_to_local
    gpg.GzipWriteFile(src_iter, tdp.name, size=sys.maxsize)
  File "/usr/lib64/python2.7/site-packages/duplicity/gpg.py", line 451, in GzipWriteFile
    outfile.write(new_block.data)
  File "/usr/lib64/python2.7/gzip.py", line 240, in write
    self.fileobj.write(self.compress.compress(data))
  File "/usr/lib64/python2.7/site-packages/duplicity/gpg.py", line 427, in write
    result = self.fileobj.write(buf)
 IOError: [Errno 28] No space left on device

This can’t be the case it’s an archive from a 1Tb machine !

The Toshiba drive doean’t have any hidden folders or files

So, the restore was being done from somewhere to the newly attached Toshiba device? The error could imply a number of things, from the looks of it:

The ENOSPC (“No space left on device”) error will be triggered in any situation in which the data or the metadata associated with an I/O operation can’t be written down anywhere because of lack of space. This doesn’t always mean disk space – it could mean physical disk space, logical space (e.g. maximum file length), space in a certain data structure or address space. For example you can get it if there isn’t space in the directory table (vfat) or there aren’t any inodes left. It roughly means “I can’t find where to write this down”.

reference: filesystems - Python causing: IOError: [Errno 28] No space left on device: '../results/32766.html' on disk with lots of space - Stack Overflow

This, however, is probably not related to the deja-dup issue.

Keep in mind these two lines:

duplicity is not copying to your 1TB drive, but copying to a local cache in a temporary directory. This may be ~/.cache/duplicity (if run directly), /tmp or /var/tmp or ~/.cache/deja-dup/tmp (if using deja-dup). Could there be anything special about one of those directories on your system?

Now, deja-dup is written in Vala, but the error you have from the first post is a Python traceback. duplicity is written in Python, so the error is either there or in how deja-dup is using it. Looking around, it seems you can set an environment variable to tell deja-dup to use a different temporary directory; you could try running it as:

DEJA_DUP_TEMPDIR=/run/media/saywot/TOSHIBA\ EXT/tmp deja-dup

and if that works, then the question is why are your ~/.cache directories broken…

1 Like

This installation of Fedora 30 is 5 days old, at the time this issue became apparent Deja-Dup was the only application installed apart from the defaults.
I have no clue why duplicity is doing what it did not why my /. cache directories are broken.

Tony Clifton

Now I have completely rebuilt the hardware (new CPU, more RAM and a shiny HDD) and installed Fedora 30
The only thing I have installed from the default repository or Software Catalogue is Deja-Dup
This is the only thing I have tried to do.
a RESTORE

so set the storage location for Deja-Dup
and clicked on the “Restore” option.
It found a back-up for 16/05/19
and started preparing, it prepared for so long that I went to bed.
Woke up this morning expecting to see all my documents, pictures, videos etc etc restored to their original location but - NO :frowning:
Failed with an unknown error.

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1498, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 733, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
    ITR(ropath.index, ropath)
  File "/usr/lib64/python2.7/site-packages/duplicity/lazy.py", line 344, in __call__
    last_branch.fast_process, args)
  File "/usr/lib64/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
    return function(*args)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 614, in fast_process
    ropath.copy(self.base_path.new_index(index))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 445, in copy
    other.writefileobj(self.open("rb"))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 627, in writefileobj
    buf = fin.read(_copy_blocksize)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 212, in read
    if not self.addtobuffer():
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 233, in addtobuffer
    self.buffer += fp.read()
  File "/usr/lib64/python2.7/tarfile.py", line 829, in read
    buf += self.fileobj.read()
  File "/usr/lib64/python2.7/tarfile.py", line 743, in read
    return self.readnormal(size)
  File "/usr/lib64/python2.7/tarfile.py", line 758, in readnormal
    return self.__read(size)
  File "/usr/lib64/python2.7/tarfile.py", line 748, in __read
    buf = self.fileobj.read(size)
  File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 209, in read
    return self.fileobj.read(length)
  File "/usr/lib64/python2.7/gzip.py", line 267, in read
    self._read(readsize)
  File "/usr/lib64/python2.7/gzip.py", line 331, in _read
    self._read_eof()
  File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
    hex(self.crc)))
 IOError: CRC check failed 0xa5587c60 != 0x5aa83c9aL

so just so it’s clear this was a day-old archive, a new hours-old Fedora 30/Deja-Dup installation.
There are no hardware issues.
And Deja-Dup remains useless.

just for a giggle I tried to restore a few missing files from Nautilus
Deja-Dup opened, it looked through the back-up, presented me with a list of files.
I ticked a few to restore them to the directory

Failed (of course)

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1498, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 733, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
    ITR(ropath.index, ropath)
  File "/usr/lib64/python2.7/site-packages/duplicity/lazy.py", line 344, in __call__
    last_branch.fast_process, args)
  File "/usr/lib64/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
    return function(*args)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 614, in fast_process
    ropath.copy(self.base_path.new_index(index))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 445, in copy
    other.writefileobj(self.open("rb"))
  File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 627, in writefileobj
    buf = fin.read(_copy_blocksize)
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 212, in read
    if not self.addtobuffer():
  File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 233, in addtobuffer
    self.buffer += fp.read()
  File "/usr/lib64/python2.7/tarfile.py", line 829, in read
    buf += self.fileobj.read()
  File "/usr/lib64/python2.7/tarfile.py", line 743, in read
    return self.readnormal(size)
  File "/usr/lib64/python2.7/tarfile.py", line 758, in readnormal
    return self.__read(size)
  File "/usr/lib64/python2.7/tarfile.py", line 748, in __read
    buf = self.fileobj.read(size)
  File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 209, in read
    return self.fileobj.read(length)
  File "/usr/lib64/python2.7/gzip.py", line 267, in read
    self._read(readsize)
  File "/usr/lib64/python2.7/gzip.py", line 331, in _read
    self._read_eof()
  File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
    hex(self.crc)))
 IOError: CRC check failed 0x5f995e46 != 0xecf64586L

Hi. I’ve had a similar issue in the past. It may not be the same issue that you’re facing but it sounds very similar.

I eventually figured out that it wasn’t complaining about any of my storage drives being full but my RAM (/tmp is tempfs on Fedora). Déjà-Dup uses a whole lot of memory when working with many objects and I ran out of memory before it could read some backup table, manifest, or whatever it was.

The solution was to set aside a 128 GB SSD I had laying around as a SWAP partition. It was incredibly slow but I did eventually manage to restore my Déjà-Dup folder.

Today I’m use Restic for my home directory and data partitions, and only use Déjà-Dup as a secondary backup for my ~/Blog and ~/Documents directories. (The Nautilus integration where I can restore previous copies of files still makes it useful.)

I think I’ve solved this, on my own, no help from anyone !!
It’s not a Deja-Dup problem, it’s sort of a Fedora issue though
I am halfway through completing this long-running restore from a back-up made before my Fedora 30 installation and the problem seems to be with — wait for it

Wayland

I booted into Gnome-Classic (not Gnome) and there ya go, Deja-Dup is working

Eh? Wayland? How so? No wonder no-one managed to diagnose it—that’s totally unexpected. How does the display system come into it? :thinking:

I haven’t any clue, no backing-up/restoring with ‘New Gnome’ but plenty if I boot to “Gnome Classic” .

I wouldn’t have found this except I needed to run Gedit as a super user

Sent AFK

Rebuild - new machine
tried to create a back-up (how many is this ? - 10,11,12 !)

Failed

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1549, in do_backup
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 664, in incremental_backup
    bytes_written = dummy_backup(tarblock_iter)
  File "/usr/bin/duplicity", line 227, in dummy_backup
    while tarblock_iter.next():
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 523, in next
    result = self.process(self.input_iter.next())
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 195, in get_delta_iter
    for new_path, sig_path in collated:
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 286, in collate2iters
    relem2 = riter2.next()
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 354, in combine_path_iters
    refresh_triple_list(triple_list)
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 341, in refresh_triple_list
    new_triple = get_triple(old_triple[1])
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 327, in get_triple
    path = path_iter_list[iter_index].next()
  File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 239, in sigtar2path_iter
    for tarinfo in tf:
  File "/usr/lib64/python2.7/tarfile.py", line 2510, in next
    tarinfo = self.tarfile.next()
  File "/usr/lib64/python2.7/tarfile.py", line 2350, in next
    self.fileobj.seek(self.offset - 1)
  File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 221, in seek
    return self.fileobj.seek(offset)
  File "/usr/lib64/python2.7/gzip.py", line 442, in seek
    self.read(1024)
  File "/usr/lib64/python2.7/gzip.py", line 267, in read
    self._read(readsize)
  File "/usr/lib64/python2.7/gzip.py", line 331, in _read
    self._read_eof()
  File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
    hex(self.crc)))
 IOError: CRC check failed 0x662d09a != 0xeba234a7L

What’s going on here ?

Not a clue, but something very specific to your setups. Not seen any of these errors on my multiple machines.

What exactly are you backing up? Are these files changing while you are backing up? That’s one thing that can cause the CRC check to fail.

it’s a ‘straight out of the box’ install, onto 2xHDD using LVM.

the only operation is the back-up

Sent AFK

But what files specifically are you backing up? Are these in use and therefore changing while the back up is in progress?

It’s backing up /home/saywot

  • and no the files aren’t changing (except for them being zipped-up and tarred into a compressed archive)

Sent AFK

If your user(s) are logged in while the files are being backed up (which they are in this situation), then lots of files being used by various applications are indeed changing. (~/.local, ~/.cache and lots of other “dotfolders” in ~ are always being accessed and written to by applications that are running).

So, maybe try backing up specific folders that you know are not being written to: ~/Documents or whatever and see how that goes?

PS: I don’t know how backup tools are meant to handle this particular case, other than being like gnome-software and backing things up when these files are not in use. Maybe someone else will know.

I’ve exclude all directories except “Videos”

still fails to complete.

There’s nothing fancy about this set-up apart from running old hardware.

As I said this back-up programme worked perfectly for me right up until Fedora 28 and since hasn’t served me at all.