Using rclone and launchd to sync data to Google Drive on MacOS

Rclone is an open-source tool that we can copy our files or store on various cloud storage providers.

In this post, I'll use Rclone to sync data on MacOS with rclone + launchd (Mac daemons manager). You may ask why not just use built-in cloud sync? It's because I've been using Google Drive for a long time to back up my data and photos, and the Google Drive desktop app lacks flexibility in file exclusion.

Installation

I'm on Mac, so I'll use brew:

brew install rclone

To use Rclone, you need to configure it first, granting it access to the cloud service of your choice. These are the configuration process:

1. Open your terminal and run the following command to initiate the configuration setup:
   

   rclone config
   

2. The configuration will prompt you to create a new remote connection. Choose option n for a new remote.

1. Next, you'll be prompted to name your remote connection. Give it a descriptive name, such as "Google Drive" or any name you prefer.

2. Select the number corresponding to the cloud storage provider you want to use. It's 18 for my case. (You can refer to the official RClone documentation for specific details on each provider.)

1. Leave the client ID and secret empty when prompted, and choose y to allow RClone to use the Google Drive API without these credentials.

2. The browser will automatically open and authenticate RClone with the remote.

1. Confirm your settings and the remote connection should be successfully configured.

Now, you have set up rclone to access your cloud storage. You can repeat these steps if you want to configure additional remotes for different cloud providers.

Syncing Data with rclone and Launchd

Now, let's move on to syncing data using RClone and scheduling it with Launchd:

1. Create a file named 'exclude-file.txt' to specify the files that you want to ignore from syncing. Mine looks like this.

1. Create a simple bash script to sync your data. Let's call it sync.sh:
   

   #!/bin/bash
   
   rclone sync --config ~/path/to/secure/location/rclone.conf /path/to/local/folder remote:destination/folder
   rclone bisync --progress --exclude-from ~/.config/rclone/exclude-file.txt --resync --verbose /path/to/local/folder remote:destination/folder
   

   Replace /path/to/local/folder with the path to the local folder you want to sync and remote:destination/folder with the remote destination on your cloud storage.

2. Make the script executable:
   

   chmod +x sync.sh
   

3. Now, let's schedule the sync using Launchd. Create a new Launchd plist file, for example, com.example.rclone-sync.plist. Use a text editor to add the following content:
   

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
   <dict>
       <key>Label</key>
       <string>com.example.rclone-sync</string>
       <key>Program</key>
       <string>/path/to/your/sync.sh</string>
       <key>RunAtLoad</key>
       <true/>
       <key>StartInterval</key>
       <integer>3600</integer>
   </dict>
   </plist>
   

   Replace /path/to/your/sync.sh with the actual path to your sync.sh script.

4. Move the Launchd plist file to the appropriate directory:
   

   mv com.example.rclone-sync.plist ~/Library/LaunchAgents/
   

5. Load the Launchd job:
   

   launchctl load ~/Library/LaunchAgents/com.example.rclone-sync.plist
   

Now, your data will be automatically synced at the specified intervals using RClone and Launchd.