Notes

Use rsync to mirror websites

Edit on GitHub

System Administration
2 minutes
backups commands

tl;dr

 1apt update && apt install rsync
 2
 3#Access via remote shell:
 4#  Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
 5#  Push: rsync [OPTION...] SRC... [USER@]HOST:DEST
 6
 7# Pull: sync remote with local
 8rsync -vhaze ssh user@server.example.com:/var/www/ /var/www
 9rsync -vPhaze "ssh -p 1234 -i /root/.ssh/id_rsa" root@server.example.com:/var/www/cakebox.me/public_html/ /var/www/cakebox.me/public_html
10rsync -qaze "ssh -p 1234 -i /root/.ssh/id_rsa" root@server.example.com:/etc/letsencrypt/archive :/etc/letsencrypt/live :/etc/letsencrypt/renewal /etc/letsencrypt/

-v, --verbose               increase verbosity
-h, --human-readable        output numbers in a human-readable format
-q, --quiet                 suppress non-error messages
-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
-z, --compress              compress file data during the transfer
-e, --rsh=COMMAND           specify the remote shell to use
    --progress              show progress during transfer
-P                          same as --partial --progress
  • explainshell for the rsync command.
  • -e is basically the shell to use, e.g. ssh -p 2234 -i /user/.ssh/id_rsa, followed by the command
  • -a is used for preservation (ownership, permissions, soft links etc.)
  • You can specify custom SSH port and SSH key to use for the connection. The path for the key file needs to be absolute path, ~ will not expand, so you have to use /userhome
  • Make sure which way you’re syncing. If note sure, you could end up overwriting imprtant files. For example config files with database connection details for example, this has happened.
  • An easy way to determine which way you are syncing is the -e flag. -e determines the remote shell, so if you’re using the option, you’re pulling, i.e. syncing remote changes to local.
  • -e can not be used when you’re syncing local changes to remote

Create an unprivileged user for the sake of transferring files

1useradd -d /home/rsyncuser -m -s /bin/bash rsyncuser
2passwd rsyncuser

Create and copy an SSH key for password-less access

1# Generate, copy and connect with an SSH key
2ssh-keygen -t rsa
3ssh-copy-id -i /home/
4rsyncuser/.ssh/id_rsa.pub rsyncuser@webserver.example.com
5ssh rsyncuser@webserver.example.com

Create a cronjob to automate the whole thing

1crontab –e
2*/5 *  *  * * rsync -vhaze "ssh -p 1234 -i /root/.ssh/id_rsa" root@server.example.com:/var/www/cakebox.me/public_html/ /var/www/cakebox.me/public_html

Related

Browse all topics