Configuration¶

This page shows where the transfer settings live and how to make the common changes: what to copy, copying to Amazon S3, and the schedule. Edit the files as root.

Where things live¶

Path	What it is
`/etc/rclone/rclone.env`	Source, destination, config and cache paths, one `KEY=value` per line.
`/etc/rclone/rclone.conf`	rclone remotes: `localdest`, and a commented-out `s3dest`.
`/var/lib/rclone/include`	Source folder.
`/var/lib/rclone/destination`	Default local destination.
`/var/lib/rclone/cache`	rclone's cache.
`/opt/sushi/bin/rclone-copy.sh`	Copies the source to the destination. Accepts only `--dry-run`.
`/opt/sushi/bin/rclone-check.sh`	Compares checksums from source to destination.

Both files in /etc/rclone are owned by root:rclone with mode 0640. First boot creates them once and keeps your edits afterwards.

The shipped rclone.env:

RCLONE_CONFIG=/etc/rclone/rclone.conf
RCLONE_CACHE_DIR=/var/lib/rclone/cache
RCLONE_SOURCE_PATH=/var/lib/rclone/include
RCLONE_DEST=localdest:/var/lib/rclone/destination

Each of these four keys must appear exactly once, and no other keys are allowed. Values are used exactly as written: do not quote them. Never put access keys in either file.

Copy rules¶

The scripts always run rclone copy --checksum. They never run sync, so files that exist only on the destination are left alone.
A file whose checksum differs is overwritten on the destination. The previous version is not kept. Turn on S3 bucket versioning if you need to recover overwritten objects.
rclone-check.sh checks one way: every source file must match on the destination, and extra destination files are not reported.

Choose what to copy¶

The source must stay inside /var/lib/rclone/include. You can point RCLONE_SOURCE_PATH at a subfolder of it, but not outside it. The scripts resolve symbolic links and refuse a source, local destination or cache that leads outside its own folder.

Copy files in as root and hand them to the rclone user, which must be able to read every file:

sudo cp -a <path> /var/lib/rclone/include/
sudo chown -R rclone:rclone /var/lib/rclone/include

Copy to Amazon S3¶

Create the bucket yourself, in an account you control. The configuration does not create buckets.
Create an IAM role for the instance that allows:
- s3:ListBucket and s3:ListBucketMultipartUploads on the bucket
- s3:GetObject, s3:PutObject, s3:AbortMultipartUpload and s3:ListMultipartUploadParts on the prefix
The copy never deletes, so it does not need s3:DeleteObject. Attach the role as the instance profile, alongside AmazonSSMManagedInstanceCore.
In /etc/rclone/rclone.conf, uncomment the s3dest block and set the bucket's region:
```
[s3dest]
type = s3
provider = AWS
env_auth = true
region = <bucket-region>
location_constraint =
no_check_bucket = true
```
For us-east-1, leave location_constraint empty. For any other region, set it to the same region. Leave acl unset: buckets with ACLs disabled reject ACL headers. env_auth = true takes credentials from the instance role.
In /etc/rclone/rclone.env, set the destination:
```
RCLONE_DEST=s3dest:<bucket>/<prefix>
```

Preview, copy and check:

sudo -u rclone /opt/sushi/bin/rclone-copy.sh --dry-run
sudo -u rclone /opt/sushi/bin/rclone-copy.sh
sudo -u rclone /opt/sushi/bin/rclone-check.sh

The instance needs outbound HTTPS to S3 and access to the instance metadata service for the role's credentials. Try it with an empty prefix before you depend on it.

Change the schedule¶

rclone-copy.timer runs once a day, with a random delay of up to 15 minutes, and catches up at boot if a run was missed while the instance was stopped. Each run is stopped after one hour. To change when it runs, override the timer:

sudo systemctl edit rclone-copy.timer

For example, to run at 02:30 every day:

[Timer]
OnCalendar=
OnCalendar=*-*-* 02:30:00

Then check the next run:

systemctl list-timers rclone-copy.timer

To run a copy now, start the service:

sudo systemctl start rclone-copy.service