Rsync inconsistent behavior possibly related to SRC versus DEST directory specification syntax

Ask Fedora
1

I’m using rsync to backup my main hard drive to external HDDs.
The main computer’s HDD has a btrfs file system. The external HDDs have ext4 file systems.

Here is the rsync command I first used on Fedora 41:

rsync -a -R -C -f'\''- /.cache'\'' 1> /home/rhimbo/.rsync/.rsync-messages.txt 2> /home/rhimbo/.rsync/.rsync-errors.txt \
      . /run/media/rhimbo/WD-disk2s1

I always ran the command from my home directory so I could back up my entire tree. And this worked fine for the first few times as far as I could tell. But then I tried specifying ~ as the source directory instead of ‘.’ as in the command above.

And I saw that the destination (the back up) was growing to twice the size of the source. I used ‘df’ to monitor. I canceled the rsync command.

After that, I went back to specifying ‘.’ as the source, and I executed the rsync command from my home directory. I also added the “–delete-before” command line option to delete any extraneous files on the destination.

Now I have done backups on two volumes on the back up drive (I have 3, 1TB file systems on it). But the sizes are a bit different. I can’t explain why.

I have not done an exhaustive check of the contents, for instance, a file-by-file comparison of the files on my main disk compared to the files on the back up volumes.

But I wanted to ask if the ~ has some special significance or interpretation by rsync. I have poured over the rsync man page and could not find anything.

In the ‘df’ output below, note that I did two backups immediately one after the other on WD-disk2s1 and WD-disk2s2, respectively. In the ‘df’ output below, neither seems correct. The size previously matched the size of the /home file system exactly (198144444). Additionally, I see a size difference between WD-disk2s1 and WD-disk2s2.

I don’t think that system log entries can explain the size difference. I can’t think of what else could be different in number of files or sizes thereof.

$ df
Filesystem      1K-blocks      Used Available Use% Mounted on
/dev/nvme0n1p3  975097856 198144444 776236484  21% /
devtmpfs             4096         0      4096   0% /dev
tmpfs            14212756     87752  14125004   1% /dev/shm
efivarfs              128        30        94  24% /sys/firmware/efi/efivars
tmpfs             5685104      2268   5682836   1% /run
tmpfs                1024         0      1024   0% /run/credentials/systemd-journald.service
tmpfs                1024         0      1024   0% /run/credentials/systemd-network-generator.service
tmpfs                1024         0      1024   0% /run/credentials/systemd-udev-load-credentials.service
tmpfs                1024         0      1024   0% /run/credentials/systemd-sysctl.service
tmpfs                1024         0      1024   0% /run/credentials/systemd-tmpfiles-setup-dev-early.service
tmpfs                1024         0      1024   0% /run/credentials/systemd-tmpfiles-setup-dev.service
/dev/nvme0n1p3  975097856 198144444 776236484  21% /home
tmpfs            14212756     11232  14201524   1% /tmp
/dev/nvme0n1p2     996780    354660    573308  39% /boot
tmpfs                1024         0      1024   0% /run/credentials/systemd-vconsole-setup.service
/dev/nvme0n1p1     613184     19796    593388   4% /boot/efi
tmpfs                1024         0      1024   0% /run/credentials/systemd-tmpfiles-setup.service
tmpfs                1024         0      1024   0% /run/credentials/systemd-resolved.service
tmpfs             2842548      6208   2836340   1% /run/user/1000
/dev/sda1      1030991000 214664324 763881544  22% /run/media/rhimbo/WD-disk2s1
/dev/sda3       686772600        28 651813008   1% /run/media/rhimbo/WD-disk2s3
/dev/sda2      1030992024 195702620 782844220  20% /run/media/rhimbo/WD-disk2s2
$ 
$
2

It’s just a shell expansion, see:

echo ~

diff -r /run/media/rhimbo/WD-disk2s{1,2}

You should use trailing slashes as explained in the manual:
rsync: a fast, versatile, remote (and local) file-copying tool | rsync Commands | Man Pages | ManKier

3

Yes, I was going to compare outputs of tar -t, but I’ll try ‘diff -r’. Thanks.

I’m going to read the rsync man page again. I’m wondering of putting “~/” as the source (SRC) versus “~” makes a difference to rsync.

I’ll try and report back here.

4

It certainly will.

“~” will copy the home directory by name and its contents.
Since ~ expands to /home/$USER that means using ~ would copy the directory $USER and its content – creating a directory $USER at the destination.

OTOH, using ~/ copies the content of the home directory and does not create the new $USER directory at the destination.

1 Like
5

@computersavvy Thank you, Jeff. Well, it finally hit home. The / character is the key element.

  1. If a trailing / is present it means: do not create a directory; just copy its contents.

  2. If no trailing / is present it means: create a directory for the last path component (the basename of the path argument), and then copy its contents.

I’ve attached a screen from the rsync man page where I found the explanation.

Well, I did read this previously. But I guess I wasn’t really thinking when I tried the ~ as the SRC argument on the rsync command line. There is an important difference between ‘.’ and ‘~’.

rsync-man-page
rsync-man-page1147×1386 180 KB