Keeping up to date backups is very important when dealing with any type of online server. One can never predict when a hard drive will fail, a file system gets corrupt, or even when mother nature will ruin your day. Thus the reason to keep not only an on-site backup but a remote duplicate in a completely different location.
In this guide I will walk you though setting up a complete remote backup solution using rsync on just about any Linux distribution you would be using. Let’s get started!
Here’s what I will be covering in this guide:
- Setting up a password-less login with RSA keys.
- Choosing our rsync command
- Adding optional email notification
- Scheduling our command via cron
Automatic SSH Login
It’s important, especially for automating the process, that our system does not need to prompt for a password when initiating the SSH connection to transfer our files to the remote location. Thankfully SSH is a wonderful protocol and we can use RSA keys to securely automate the authentication process.
Step 1 – Create RSA Key Pair
Run the following command on your client that contains the files you wish to backup.
ssh-keygen -t rsa
You will be prompted for a location, for the most part, this can be left as is. So just press enter. If you are prompted to include a pass-phrase with your key, just press enter. Do not set a pass-phrase, otherwise you will be unable to automate the backup process.
Now you should have a generated SSH RSA Key in /root/.ssh/id_rsa
Step 2 – Copy the key
Now that we have generated the key, we must copy it to the destination server. To do this, just use the following command. Be sure to replace the user and IP to match your setup.
ssh-copy-id [email protected]
You will be prompted for a password for the destination machine. Enter the password and the key will automatically be copied into the correct location. Your output should look something like the following:
The authenticity of host '12.34.56.78 (12.34.56.78)' can't be established. RSA key fingerprint is b1:2d:33:67:ce:35:4d:5f:f3:a8:cd:c0:c4:48:86:12. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '12.34.56.78' (RSA) to the list of known hosts. [email protected]'s password: Now try logging into the machine, with "ssh '[email protected]'", and check in: ~/.ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting.
Now you should be all set! You can now log into the destination machine without the need for entering a password.
Setting Up rsync
rsync is the amazing Linux tool that we will be using to automate the remote backups. You may want to use a slightly different set of switches so I would recommend reading over the rsyn man page. I will be using the following command. It gets the job done just fine!
rsync -avh -delete --progress --bwlimit=10240 -e ssh /location/of/files/to/backup [email protected]:~/backup/server42/
-a
Archive Mode
-v
Verbose Mode
-h
Preserve hard links
-delete
Delete files on the destination that don’t exist on the source.
–progress
Shows the transfer progress in real time
–bwlimit=10240
Limits by backup transfer to only using a set about of bandwidth measured in KBPS. Useful to prevent network interruptions/slowness for live services.
-e ssh
For the SSH transfer
/location/of/files/to/backup
The source. The locations of the files that you are backing up to the remote location.
[email protected]:~/backup/server42/
The destination. The location on the remote server where the files will be transferred to.
Optional: Setup Simple Email Notifications
I personally like to get notified once my remote backups are complete. A simple way to accomplish this is by piping the rsync output right into an email! This can be done by adding the following to the end of your rsync command.
| mail -s 'Backup Log - Server42' [email protected]
This will send you the output of the rsync command right to your email! Just note that it will probably end up in a spam/junk folder. So just add the domain to your contact list once you get your first email.
Your final rsync command will looks something like the following:
rsync -avh -delete --progress --bwlimit=10240 -e ssh /source/ [email protected]:~/destination/ | mail -s 'Backup Log' [email protected]
Automating With Cron
At this point you should be able to manually run the above command and your remote backup process will begin! This is great but having to manually run the command every time you want to backup your system is a bit of a pain, especially if you have many systems that need to be backed up. The solution to this is to simple let cron automatically run the command for you!
Type the following while logged into your source machine.
crontab -e
This should bring up your users cron job list. We are going to add our job to the end of this file. The following line will run our backup rsync job every day at 6 in the morning.
0 6 * * * rsync -avh -delete --progress --bwlimit=10240 -e ssh /source/ [email protected]:~/destination/ | mail -s 'Backup Log' [email protected]
Once that line as been appended to the bottom of your crontab you can save the file and your job will be scheduled! Now all that’s left is to wait until the next morning to the arrival of an email of your completed backup log.
Conclusion
Linux is awesome. Backups are awesome. rsync is awesome!
Keeping a backup of your files is extremely important. With this simple job, you will never have to worry about manually keeping an offsite backup of your systems files. I personally use this in combination with a server running Proxmox. I use the built in Proxmox backup system to backup specific containers to its local storage on a daily basis while keeping an archive of the last three backups. I then have the scheduled rsync command run every morning to transfer the latest backup to a remote location. It’s a great setup and combined with the automatic email notifications from both Proxmox and my automatic rsync command, I always know the status of my latest backups.
If you run into any issues or have some questions about setting this up on your system, feel free to ask me in the comments. I’m happy to help!
