rsync-backup.pl

You can download rsync-backup.pl here.

NAME

rsync-backup.pl -- manage backups of remote systems via rsync

SYNOPSIS

rsync-backup.pl [ SWITCHES ] [ -f /path/to/config_file ] [ label label... ]

DESCRIPTION

rsync-backup.pl uses rsync over ssh to perform backups of remote systems. It supports multiple host definitions, allowing you to specify unique remote paths, exclusions, local backup targets and so on. You can even mount a filesystem before starting the backup, and unmount it upon completion (dangerous for multiuser environments, but handy for toasters).

rsync-backup.pl is designed to be run from cron, for multiple daily backups and optional archiving of daily, weekly and monthly snapshots. Snapshots are done via hard links, so disk usage is minimal, and since rsync only transfers changes since the last run, and uses compression to boot, bandwidth requirements are light too.

PREREQUISITES

This script requires the following packages:

Note: On BSD systems, install the coreutils port to get gnu cp and tar.

COMMAND-LINE SWITCHES

The following switches are supported:

-h      Display short help summary

-f      Path to the configuration file

-v      Enable verbose logging

-D      Enable debug logging (implies -v)

-t      Run configuration tests; no transfers

-n      The number of backup operations to perform at once

labels  Execute only the named backup configurations

CONFIGURATION

rsync-backup.pl requires a configuration file containing one or more ``config blocks'', which define a remote host targeted for backup. Here's a sample config block:

example {
        hostname            eg.mydomain.com
        path                /

        snapshots-hourly    4   
        snapshots-daily     7   
        snapshots-weekly    4   
        snapshots-monthly   1

        snapshot-path       /mnt/backups

        excludes /backups/:/proc/:/dev/:tmp/:/usr/src/:/var/db/mysql/

        mount-dev           /dev/da0s1a
        mount-point         /mnt/backups
        mount-type          ufs
        mount-flags         -fu
        mount-on-startup    yes
        umount-on-shutdown  yes

        create-tarballs     yes
        tarball-size        4000m
}

This block tells rsync-backup.pl to backup the entire contents of host 'eg.mydomain.com' 4 times daily. This configuration would preseve one monthly backup, plus the most recent 4 weeks and the last 7 days. It would be advisable with this setup to archive the monthly backup to permanent media, before it is overwritten the following month. If your snapshots are small (or your disks are large), you might save 12 monthly backups, giving you a year's history at a glance.

You may have as many such config blocks in your config file as you like; rsync-backup.pl will process each one in turn. Note that each block must begin with a label, used to identify this backup configuration.

A description of the configuration options follow:

USAGE NOTES

Local Configuration

Before running rsync-backup.pl, edit the script and alter the values of the *_cmd variables to match your specific system layout. The defaults are:

my $cp_cmd        = '/usr/local/bin/cp -alf';
my $touch_cmd     = '/usr/bin/touch';
my $ssh_cmd       = '/usr/bin/ssh';
my $mount_cmd     = '/sbin/mount';
my $umount_cmd    = '/sbin/umount';
my $tar_cmd       = '/usr/local/bin/bin/tar';
my $find_cmd      = '/usr/bin/find';

cron

rsync-backup.pl is designed to run from cron. Furthermore, to properly manage weekly and monthly snapshots, the script needs to run at least on sundays, and on the first of every month. Thus it is recommended that you create a cron job to run the script daily, as many times as is needed by the highest value of snapshots-daily in your config file. For example, the config block shown above would suggest the following crontab entry:

0,6,12,18 * * *  /path/to/rsync-backup.pl -f conf_file

See crontab(5) for details.

Logging

As of version 2.0, a seperate log file (called backup.log) is created in the snapshot directory for each host. The log files are truncated at each run, so no rotating is necessary.

By default no output is sent to STDOUT. You may override this behaviour by specifying the -v switch; this will cause a copy of entries in each host's log file to be echoed to STDOUT.

Also new with version 2.0 is support for logging to syslog via Sys::Syslog. By default only errors are directed there; this can be overridden by enabling debug output (via the -D switch). Doing so will cause all output to be copied to syslog in addition to STDOUT/STDERR and the host log files, as well as enabling various debug-only messages.

Fatal errors are always sent to syslog and to STDERR.

SSH and rsync

Unless you want to hang around and enter a password every time rsync-backup.pl launches rsync to back up a remote host, you're going to want to use certificate-based authentication for the ssh user.

Additionally, if you want to do full system backups with rsync, you're probably going to need to run rsync-backup.pl as root, and allow root to ssh into the remote host and run rsync. Allowing remote logins by root can be dangerous, however. What follows is an overview of my solution to this problem; I strongly recommend you familiarize yourself with the security implications of this setup before blindly charging forth. The author will accept no responsibility for your being foolish, yadda yadda yadda.

  1. Allow root SSH for authorized commands only

    To do this, simply set PermitRootLogin to forced-commands-only in your remote host's sshd_config. Now the root user will be permitted to login via SSH, but may only execute the command you specify in the authorized_keys file.

  2. Configure root's authorized commands

    Edit root's authorized_keys file on the remote host, and modify the line containing your backup host's key thusly:

    command=``/root/bin/ssh_allowed.sh'', ssh-dss  ...  root@backup-host

    This will force every root login from backup-host to run the shell script ssh_allowed.sh. By interrogating the $SSH_ORIGINAL_COMMAND environment variable in this script, we can decide whether or not to permit the command to be executed. Here's a simple ssh_allowed.sh:

      #!/bin/sh 
  # 
  # spawned by ssh to execute valid commands remotely 
  # 
  case "$SSH_ORIGINAL_COMMAND" in 
      *\&*) 
          echo "Rejected" 
      ;; 
      *\;*) 
          echo "Rejected" 
      ;; 
      rsync\ --server\ --sender\ -logDtprRz\ .\ /*) 
          $SSH_ORIGINAL_COMMAND 
      ;; 
      *) 
          echo "$SSH_ORIGINAL_COMMAND" >> /var/log/root_ssh_rejected.log 
          echo "Rejected"
      ;; 
  esac

    Note: depending on your calling parameters and rsync version, the exact sequence of arguments on the rsync --server line may or may not match this example; if your rsyncs are failing, check the rejected log to see what args are bing passed and modify the script accordingly.

    And of course, ensure your ssh_allowed.sh's permissions are set to 500.

Restoring a split tarball

If you find yourself in the position of needing to restore a backup from a tarball which has been split into chunks, simply copy all the pieces of the tarball into a directory, and execute:

     % cat tarball.tgz_* | gnu-tar --preserve -xzf -

Backing up mysql databases

Trying to rsync mysql databases while mysql is running on the remote host will result in broken tables (and kvetching lusers). It is recommended the remote host run mysqlhotcopy from a cron job some time before the rsync backup is scheduled, such that rsync can backup copies of the databases rather than the databases themselves. Such a crontab entry might look like this:

    2  3 * * * mysqlhotcopy --addtodest -u user --password=... dbname /path/to/backups

Consult the mysql documentation for details.

CAVEATS

At this time, rsync-backup.pl has no brains for checking disk space before engaging in possibly-dangerous things like creating multiple gigantic tarballs of whole filesystems. Be thou therefore careful with thine tars.

VERSION

This is version 2.1 of rsync-backup.pl.

CHANGES SINCE 2.0

CHANGES SINCE 1.7

CHANGES SINCE 1.6

CHANGES SINCE 1.5

CHANGES SINCE 1.4

CHANGES SINCE 1.3

Modified behaviour of the mount options. We can now:

CHANGES SINCE 1.2

CHANGES SINCE 1.1

CHANGES SINCE 1.0

TO DO

AUTHOR

rsync-backup.pl was written by Greg Boyington  < greg [at] automagick.us >.

ACKNOWLEDGEMENTS

The basic structure of the backup scheme isn't mine; it belongs to Stu Sheldon, < stu [at] actusa.net >, whose mirror script I found linked on Mike Rubel's excellent article, ``Easy Automated Snapshot-Style Backups with Linux And Rsync.'' You can read the article here: http://www.mikerubel.org/computers/rsync_snapshots/

License

Creative Commons License
All source code, tools and scripts on http://automagick.us is Copyright © 2007 - 2010 Greg Boyington, and licensed under a
Creative Commons Attribution-Share Alike 3.0 United States License, except where otherwise noted.

 

Valid HTML 4.01 Transitional  |  Valid CSS
/tools/rsync-backup.mhtml Last Updated Fri Jul 16 16:25:44 2010