Backup hundreds of gigs of photos to the cloud from behind the Great Firewall

My mother passed away earlier this year.

On my bereavement trip to my parents’ home in my hometown, my father tasked me to backup and collect all the photos on both their phones to a central place so he can look at them when he needs them.

Both of them use Huawei HarmonyOS phones, which at that point was still kind of a fork of Android (Huawei has plans to break Android-compatibility in the near future). I have a big enough SanDisk “Dual Drive” USB stick with me that has both USB-A and USB-C plugs, so it’s perfect for the job (he can use the USB-C port on his phone and USB-A port on his laptop). So I just plugged it into my mom’s phone and started copying the photos to it. The phone’s builtin album/files app is ok for the job, despite being a bit confusing in the UI (it shows you the name of the directories altogether without telling you where the directories are from, so I ended up copying a directory on the USB stick to the USB stick again once before I realized the issue, and in the end I probably still have some duplicated photos/videos remaining there because of that). I did that for both their phones, and it ended up with about 200 GiB of photos and videos from their phones.

But that’s only the “collect” part, not the “backup” part. If I learnt anything from the years is that the photos and videos are non-reproducible files that’s most worthy for backups, and my dad is not tech-savy that if I don’t back them up at least a second copy he could have accidentally erase the USB stick and lose the files forever.

Now the problems about backups are:

1. I don’t have a second USB stick to make a second copy for the photos/videos
2. My phone’s free space is not enough to hold all the photos/videos
3. While I have cloud account with enough space (I have a 2TB Google Drive account mainly for backing up photos), my hometown in China has Great Firewall that the access to my cloud account at my parents’ home is unreliable at best

But I also had my laptop with me on the trip, and my phone and laptop had about 100 GiB of free space each, so they combined can hold a second copy of all the photos and videos. So I did some splitting and copied some to my phone and some to my laptop.

After that, I would still strongly prefer to get at least some of the photos/videos backed up to the cloud before I’m traveling back, because “traveling” is one of the most likely ways to lose your phone/laptop. Luckily I have a linux box home server running at my (US) home with big enough disk, so I want to get the photos/videos there from my parents’ home.

Initially I just used rsync with ssh to copy the files to the linux box, but because the connection is so unreliable it drops all the time, so in the end I wrote a simple shell script like this:

#!/bin/sh

while true
do
  if rsync -avz --progress ./* host:/home/fishy/photo.backup/; then
    break
  fi
done

What it does basically is to keep retrying rsync until it succeeds. If it failed because of broken connection, it will just rerun the same command again, and rsync can auto handling the skipping of already transferred files. So I just left it running over the night(s).

After the files are on my linux box, I just used rclone to sync them to the cloud (Google Drive, S3, etc.) for backups.