*** buffer overflow detected ***: terminated error message when using find . of android phone

Hello, I am using vmware with fedora 38 installed and mounted my galaxy s8 android to the vm. I am making a backup of the android phone. I changed directory to the root of the android phone using

cd “/run/user/1000/gvfs/mtp:host=SAMSUNG_SAMSUNG_Android_98897a48334d553957” which is the root of the galaxy s8. Then, I entered this command

find . -exec zip -0 “/home/username/Downloads/s8 phone backup 5-08-2023 (fedora).zip” {} +

After a few minutes I received an error:

*** buffer overflow detected ***: terminated
zip error: Interrupted (aborting)

How should I approach this problem ?

Try copying the files off the phone first, before zipping it.

1 Like

How can i copy the folders and files preserving the original timestampsv?

after mounting your phone’s storage, you can use rsync the data off, preserving timestamp, ownership and permissions using the archive mode (-a).

Example: rsync -arvz source destination (-r is for recursive, -z is for compression)

you can run this in a timer to automatically backup your phone, you can also do incremental backups (snapshots) using rsync.

I restarted the vmware again and ran the code once more, and it gave me the same message but nonetheless it started zipping althought it too hours and hours so I left my PC on overnight. This time the size comparison of the two phones is alot different. Is there something wrong with this? See screenshot below

Nothing at all wrong with that. A zipped file/backup is expected to be smaller (sometimes much smaller) than the original source.

1 Like

Do you know if zips created in fedora auto compress files?

A zip file is by definition compressed.

2 Likes

zip archives are compressed by default. But, it’s impossible for anyone to say if your zip archive is ok just by looking at the size. You’ll have to verify the contents yourself. An unverified backup is as good as no backup.

2 Likes

I’ve found out that when connecting the android phone and running the find command above, doesn’t zip hidden files and folders. But I am still receiving buffer error. Is the problem not enough ram to handle a large set of data ? Because it says buffer overflow detected, does that mean that data is too large to zip ? Also, how do I zip specific folders or subfolders within parent folders while excluding everything else?

Does anyone know why this is happening? any ideas or suggestions ?

Since we have little info about the phone or the data it contains it seems difficult to speculate about the cause of that buffer overflow error. Is the error on the source (the phone) or on your pc?

One workaround may be to simply copy the desired data from the phone to a subdirectory on your pc then zip the content of that subdirectory. You may even need to copy (or zip) smaller subsets of the full data content as individual units.

As far as the comment about excluding some data while zipping other parts I suggest you read up on how ‘find’ and the zip utility in use select what is being included or excluded. It is somewhat flexible but one needs to be familiar with its features as well.

It’s a samsung galaxy s8 with about 1500 pictures and about 100 videos in the DCIM/Camera folder and its on android 9. Total space is 64 GB and I have used 63.8/64 GB of storage.

That’s what I’m not sure about. How can I check which is giving the problem and what logs can I check ?

This is a long process because the data on the s8 is over 30 GB.

How can I do this via command line maybe making zips in smaller chunks avoids the I/O error ? I’m not sure…

This looks like it is related to your other post

If so then I suggest strongly that you first remove the sdcard from the phone and copy or archive the data from the DCIM directory off the card using a card reader on your pc. Once that is done and the data is deleted from the card then insert the card back into the phone and move the data from the phone DCIM directory to the card DCIM directory.

Repeat the above until all the necessary data is off the phone and you have additional space available to continue using the phone.

That tells me the phone has no space to assist in relocating the data.

All the more reason to do this process regularly and maintain free space on the phone.

I have always stored photos & videos only on the sd card to make it easy to transfer them to my PC and keep the phone memory open. You could if you wished use a 64GB sd card to allow more room for data.

1 Like

This is not a actual micro sdcard. The android’s internal storage is just called sdcard. It isn’t an actual micro sd card. I dont have a micro sd card inserted into my galaxy s8

even then I’m using a webdav app it isn’t making temporary files on the phone and rewriting it back into fedora

Since the device is connected to the pc one could simply use the file manager to move the files. It only would need the phone to read the files and allow the transfer. Once the file is written to the pc it could be removed from the phone.

This could also be done from the command line very easily with the mv command, or a combination of cp & rm.

I’m trying to understand why it doesn’t work with a simple webdav app. I could’ve done this easily if I had some space but knowing that I barely have anything, even making a small zip containing one file is not possible because it would first need to be saved on the phone

Having only a tiny remaining space on the phone causes a problem since many apps require the use of ram or temporary file space to function. Clearing out some of that space with a tool that requires minimal extra space is mandatory and the webdav app seems to require more than would be necessary with the command line tools.

Start with the command line, then after some space is freed up it may be that your preferred app could manage the rest…

Tools that do not require space on the already full file system seem best for the task.

I am somewhat surprised that android allows the file system to fill up that much (99.6%) since linux normally reserves 5% to avoid those issues you are seeing.

I guess so…knowing that my pixel 7 has about 50 GB of storage remaining (while using the find and zip command works with the pixel 7) vs. 200 MB (fine and zip command doesn’t work here) or less on my galaxy s8…I’m not sure if that would be as big as a problem but let’s try to free up some space anyway

Buffer overflow: It looks like zip is a bit picky about file naming:

zip /var/tmp/a.zip 02\ -\ Why\ Can’t\ the\ English\?.mp3 
*** buffer overflow detected ***: terminated


zip error: Interrupted (aborting)

cp 02\ -\ Why\ Can’t\ the\ English\?.mp3 02-WhyCanttheEnglish.mp3

zip /var/tmp/a.zip 02-WhyCanttheEnglish.mp3 
  adding: 02-WhyCanttheEnglish.mp3 (deflated 1%)


I’ve no idea yet what the reason is for zip to break out at this point, it’s not only the quote or question mark. Needs debugging.

Little debugging: routines local_to_wide string and local_to_utf8 string are involved, so
it has to do with the unusual characters in the file name.