https://bugzilla.redhat.com/show_bug.cgi?id=1688096
Sent AFK
Dagnabbit !
It hasn’t “just gone away”
Failed to read /home/saywot/.cache/deja-dup/tmp/duplicity-g_o7EP-tempdir/mktemp-ojBAFD-1: (<type 'exceptions.IOError'>, IOError('CRC check failed 0xe04db144 != 0x7cbda341L',), <traceback object at 0x7ff57cc16d40>)
just for a lark, to see if there’s something amiss with the backend (duplicity)
I attached an unused Toshiba 2TB HDD to the computer (it’s empty)
ran …
[saywot@Fedora28 ~]$ duplicity restore file:/run/media/saywot/Backup2/Fedora28/ /run/media/saywot/TOSHIBA\ EXT/ --force --no-encryption
Synchronising remote metadata to local cache...
Copying duplicity-full-signatures.20190201T030753Z.sigtar.gz to local cache.
Traceback (innermost last):
File "/usr/bin/duplicity", line 1581, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1567, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1419, in main
do_backup(action)
File "/usr/bin/duplicity", line 1445, in do_backup
sync_archive(col_stats)
File "/usr/bin/duplicity", line 1230, in sync_archive
copy_to_local(fn)
File "/usr/bin/duplicity", line 1178, in copy_to_local
gpg.GzipWriteFile(src_iter, tdp.name, size=sys.maxsize)
File "/usr/lib64/python2.7/site-packages/duplicity/gpg.py", line 451, in GzipWriteFile
outfile.write(new_block.data)
File "/usr/lib64/python2.7/gzip.py", line 240, in write
self.fileobj.write(self.compress.compress(data))
File "/usr/lib64/python2.7/site-packages/duplicity/gpg.py", line 427, in write
result = self.fileobj.write(buf)
IOError: [Errno 28] No space left on device
This can’t be the case it’s an archive from a 1Tb machine !
The Toshiba drive doean’t have any hidden folders or files
So, the restore was being done from somewhere to the newly attached Toshiba device? The error could imply a number of things, from the looks of it:
The
ENOSPC
(“No space left on device”) error will be triggered in any situation in which the data or the metadata associated with an I/O operation can’t be written down anywhere because of lack of space. This doesn’t always mean disk space – it could mean physical disk space, logical space (e.g. maximum file length), space in a certain data structure or address space. For example you can get it if there isn’t space in the directory table (vfat) or there aren’t any inodes left. It roughly means “I can’t find where to write this down”.
This, however, is probably not related to the deja-dup issue.
Keep in mind these two lines:
duplicity
is not copying to your 1TB drive, but copying to a local cache in a temporary directory. This may be ~/.cache/duplicity
(if run directly), /tmp
or /var/tmp
or ~/.cache/deja-dup/tmp
(if using deja-dup
). Could there be anything special about one of those directories on your system?
Now, deja-dup
is written in Vala, but the error you have from the first post is a Python traceback. duplicity
is written in Python, so the error is either there or in how deja-dup
is using it. Looking around, it seems you can set an environment variable to tell deja-dup
to use a different temporary directory; you could try running it as:
DEJA_DUP_TEMPDIR=/run/media/saywot/TOSHIBA\ EXT/tmp deja-dup
and if that works, then the question is why are your ~/.cache
directories broken…
This installation of Fedora 30 is 5 days old, at the time this issue became apparent Deja-Dup was the only application installed apart from the defaults.
I have no clue why duplicity is doing what it did not why my /. cache directories are broken.
Tony Clifton
Now I have completely rebuilt the hardware (new CPU, more RAM and a shiny HDD) and installed Fedora 30
The only thing I have installed from the default repository or Software Catalogue is Deja-Dup
This is the only thing I have tried to do.
a RESTORE
so set the storage location for Deja-Dup
and clicked on the “Restore” option.
It found a back-up for 16/05/19
and started preparing, it prepared for so long that I went to bed.
Woke up this morning expecting to see all my documents, pictures, videos etc etc restored to their original location but - NO
Failed with an unknown error.
Traceback (innermost last):
File "/usr/bin/duplicity", line 1581, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1567, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1419, in main
do_backup(action)
File "/usr/bin/duplicity", line 1498, in do_backup
restore(col_stats)
File "/usr/bin/duplicity", line 733, in restore
restore_get_patched_rop_iter(col_stats)):
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
ITR(ropath.index, ropath)
File "/usr/lib64/python2.7/site-packages/duplicity/lazy.py", line 344, in __call__
last_branch.fast_process, args)
File "/usr/lib64/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
return function(*args)
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 614, in fast_process
ropath.copy(self.base_path.new_index(index))
File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 445, in copy
other.writefileobj(self.open("rb"))
File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 627, in writefileobj
buf = fin.read(_copy_blocksize)
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 212, in read
if not self.addtobuffer():
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 233, in addtobuffer
self.buffer += fp.read()
File "/usr/lib64/python2.7/tarfile.py", line 829, in read
buf += self.fileobj.read()
File "/usr/lib64/python2.7/tarfile.py", line 743, in read
return self.readnormal(size)
File "/usr/lib64/python2.7/tarfile.py", line 758, in readnormal
return self.__read(size)
File "/usr/lib64/python2.7/tarfile.py", line 748, in __read
buf = self.fileobj.read(size)
File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 209, in read
return self.fileobj.read(length)
File "/usr/lib64/python2.7/gzip.py", line 267, in read
self._read(readsize)
File "/usr/lib64/python2.7/gzip.py", line 331, in _read
self._read_eof()
File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
hex(self.crc)))
IOError: CRC check failed 0xa5587c60 != 0x5aa83c9aL
so just so it’s clear this was a day-old archive, a new hours-old Fedora 30/Deja-Dup installation.
There are no hardware issues.
And Deja-Dup remains useless.
just for a giggle I tried to restore a few missing files from Nautilus
Deja-Dup opened, it looked through the back-up, presented me with a list of files.
I ticked a few to restore them to the directory
Failed (of course)
Traceback (innermost last):
File "/usr/bin/duplicity", line 1581, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1567, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1419, in main
do_backup(action)
File "/usr/bin/duplicity", line 1498, in do_backup
restore(col_stats)
File "/usr/bin/duplicity", line 733, in restore
restore_get_patched_rop_iter(col_stats)):
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
ITR(ropath.index, ropath)
File "/usr/lib64/python2.7/site-packages/duplicity/lazy.py", line 344, in __call__
last_branch.fast_process, args)
File "/usr/lib64/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
return function(*args)
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 614, in fast_process
ropath.copy(self.base_path.new_index(index))
File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 445, in copy
other.writefileobj(self.open("rb"))
File "/usr/lib64/python2.7/site-packages/duplicity/path.py", line 627, in writefileobj
buf = fin.read(_copy_blocksize)
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 212, in read
if not self.addtobuffer():
File "/usr/lib64/python2.7/site-packages/duplicity/patchdir.py", line 233, in addtobuffer
self.buffer += fp.read()
File "/usr/lib64/python2.7/tarfile.py", line 829, in read
buf += self.fileobj.read()
File "/usr/lib64/python2.7/tarfile.py", line 743, in read
return self.readnormal(size)
File "/usr/lib64/python2.7/tarfile.py", line 758, in readnormal
return self.__read(size)
File "/usr/lib64/python2.7/tarfile.py", line 748, in __read
buf = self.fileobj.read(size)
File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 209, in read
return self.fileobj.read(length)
File "/usr/lib64/python2.7/gzip.py", line 267, in read
self._read(readsize)
File "/usr/lib64/python2.7/gzip.py", line 331, in _read
self._read_eof()
File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
hex(self.crc)))
IOError: CRC check failed 0x5f995e46 != 0xecf64586L
Hi. I’ve had a similar issue in the past. It may not be the same issue that you’re facing but it sounds very similar.
I eventually figured out that it wasn’t complaining about any of my storage drives being full but my RAM (/tmp is tempfs on Fedora). Déjà-Dup uses a whole lot of memory when working with many objects and I ran out of memory before it could read some backup table, manifest, or whatever it was.
The solution was to set aside a 128 GB SSD I had laying around as a SWAP partition. It was incredibly slow but I did eventually manage to restore my Déjà-Dup folder.
Today I’m use Restic for my home directory and data partitions, and only use Déjà-Dup as a secondary backup for my ~/Blog and ~/Documents directories. (The Nautilus integration where I can restore previous copies of files still makes it useful.)
I think I’ve solved this, on my own, no help from anyone !!
It’s not a Deja-Dup problem, it’s sort of a Fedora issue though
I am halfway through completing this long-running restore from a back-up made before my Fedora 30 installation and the problem seems to be with — wait for it
Wayland
I booted into Gnome-Classic (not Gnome) and there ya go, Deja-Dup is working
Eh? Wayland? How so? No wonder no-one managed to diagnose it—that’s totally unexpected. How does the display system come into it?
I haven’t any clue, no backing-up/restoring with ‘New Gnome’ but plenty if I boot to “Gnome Classic” .
I wouldn’t have found this except I needed to run Gedit as a super user
Sent AFK
Rebuild - new machine
tried to create a back-up (how many is this ? - 10,11,12 !)
Failed
Traceback (innermost last):
File "/usr/bin/duplicity", line 1581, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1567, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1419, in main
do_backup(action)
File "/usr/bin/duplicity", line 1549, in do_backup
incremental_backup(sig_chain)
File "/usr/bin/duplicity", line 664, in incremental_backup
bytes_written = dummy_backup(tarblock_iter)
File "/usr/bin/duplicity", line 227, in dummy_backup
while tarblock_iter.next():
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 523, in next
result = self.process(self.input_iter.next())
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 195, in get_delta_iter
for new_path, sig_path in collated:
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 286, in collate2iters
relem2 = riter2.next()
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 354, in combine_path_iters
refresh_triple_list(triple_list)
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 341, in refresh_triple_list
new_triple = get_triple(old_triple[1])
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 327, in get_triple
path = path_iter_list[iter_index].next()
File "/usr/lib64/python2.7/site-packages/duplicity/diffdir.py", line 239, in sigtar2path_iter
for tarinfo in tf:
File "/usr/lib64/python2.7/tarfile.py", line 2510, in next
tarinfo = self.tarfile.next()
File "/usr/lib64/python2.7/tarfile.py", line 2350, in next
self.fileobj.seek(self.offset - 1)
File "/usr/lib64/python2.7/site-packages/duplicity/dup_temp.py", line 221, in seek
return self.fileobj.seek(offset)
File "/usr/lib64/python2.7/gzip.py", line 442, in seek
self.read(1024)
File "/usr/lib64/python2.7/gzip.py", line 267, in read
self._read(readsize)
File "/usr/lib64/python2.7/gzip.py", line 331, in _read
self._read_eof()
File "/usr/lib64/python2.7/gzip.py", line 353, in _read_eof
hex(self.crc)))
IOError: CRC check failed 0x662d09a != 0xeba234a7L
What’s going on here ?
Not a clue, but something very specific to your setups. Not seen any of these errors on my multiple machines.
What exactly are you backing up? Are these files changing while you are backing up? That’s one thing that can cause the CRC check to fail.
it’s a ‘straight out of the box’ install, onto 2xHDD using LVM.
the only operation is the back-up
Sent AFK
But what files specifically are you backing up? Are these in use and therefore changing while the back up is in progress?
It’s backing up /home/saywot
Sent AFK
If your user(s) are logged in while the files are being backed up (which they are in this situation), then lots of files being used by various applications are indeed changing. (~/.local
, ~/.cache
and lots of other “dotfolders” in ~
are always being accessed and written to by applications that are running).
So, maybe try backing up specific folders that you know are not being written to: ~/Documents
or whatever and see how that goes?
PS: I don’t know how backup tools are meant to handle this particular case, other than being like gnome-software and backing things up when these files are not in use. Maybe someone else will know.
I’ve exclude all directories except “Videos”
still fails to complete.
There’s nothing fancy about this set-up apart from running old hardware.
As I said this back-up programme worked perfectly for me right up until Fedora 28 and since hasn’t served me at all.