Fork me on GitHub

This is an old revision of the document!


Monitoring Backups

Once you have configured multiple backups, you will probably start looking for a way to monitor all these log files. Although, LBackup includes some basic email reporting features, you may require more advanced reporting options so that you are more easily able to customize the reporting of backups and how notifications of backup errors or warnings will be reported.

That is where the scripts listed below will save you from writing custom scripts from scratch :

Monitoring Multiple Backup Log Script Information Links Download Links
updateloglist.rb download
backupscan.rb download
scan_server.rb download
list_contab_backup_information.rb download

Note : Scripts listed in bold above are the primary scripts for basic setups. Other scripts listed are useful if required for your specific configuration or reporting requirements.

There are plans to publish these scripts on-line with installers and documentation. In the mean time you are welcome to contact us and we will send you a copy of the latest development versions (which are also accessible via GitHub).

Please also feel free to contact us if you are interested in being involved with the development of the LBackup monitoring system, which is currently under development and will most likely leverage these scripts. Also if you have comments or suggestions regarding these scripts or the monitoring system then please contact us.



Further information on monitoring and reporting with regards to LBackup is available from the reporting page.





Monitoring Multiple Backup Script Details

This section outlines the use of the scripts which may be downloaded from the links listed above. This is the first iteration of documentation relating to the use of these scripts. Your contributions to the improvement of the LBackup project are welcome. To discuss development and use of these scripts please sign up on the discussion mailing list.



List of backup monitoring scripts currently available on-line :

Other related monitoring scripts available online :


updateloglist.rb

The updateloglist.rb script will generate two output files. One will contain a list of backup configurations which are scheduled in the crontab. The other file will contain a list related log files (one for each of the backup configuration files). This script must be executed as the backup user so that it will be able to see the backups listed in the crontab. This file may be used by the backupscan.rb script to check the status of the backups which have been loaded into the crontab of a backup server. To schedule this process on your backup server, copy the script to your backup server and then add it to your backup users crontab at the desired interval. Alternatively, you may specify a wrapper script which will first execute the updateloglist.rb and then execute the backupscan.rb script in sequence.



Default file names which will be output by this script :

  • backuploglist.txt (list of backup log file paths)
  • backupconfiglist.txt (list of backup configuration file paths)


Example of the output from this script if you have backups scheduled with cron (backuploglist.txt):

/Users/backup/backup_configurations/svnserver/svnserver_backup.log
/Users/backup/backup_configurations/mailserver/mailserver_backup.log
/Users/backup/backup_configurations/fileserver/fileserver_backup.log


You may specify custom output file names and paths if you like.

./updateloglist.rb -l output_loglist.txt -c output_configlist.txt

Assuming you are in the same directory as the updateloglist.rb script and the script is executable, then the example above will generate some custom files with the names you provided as arguments. In this case “output_loglist.txt” and “output_configlist.txt”.



backupscan.rb

The backupscan.rb script is used to check multiple log files on a single backup server. By default it will parse a file called backuploglist.txt and output_configlist.txt. Each line of the backuploglist.txt file must contain a single path to a backup log, which will be checked for current errors. The output_configlist.txt file is at this stage is used to provide additional error reporting. Each line of the output_configlist.txt must contain a single path to a backup configuration. Both of these files must be present for the script to run successfully.


It is assumed that each line of the respective file relates to the same backup. To clarify an confusion on this point, line one of each file is assumed to be the path to the backup log and configuration respectively.


In addition to checking for current errors. The backupscan.rb script will by default check that limits on backup initiation time and backup duration are adhered to or errors are reported.



Example of calling the script :

./backupscan.rb -l backup_log_file_list.txt -c backup_configuration_file_list.txt



scan_server.rb

The scan_server.rb script will allow you to scan multiple backup servers from a single system. By default the script will parse a file called host_list.txt for backup servers and the command which should be executed on that backup server. This command will most likely be to execute the backupscan.rb script which will already be installed on the backup server.



The format of this input file is as follows :

server_to_connect_to_via_ssh <tab> command_to_execute_on_the_server



Example of the host_list.txt file :

your_lbackup_server_hostname	cd /path/to/backupscan/directory; ./backupscan.rb
lbackup.your.company.com	cd /var/lbackup/backupscan; ./backupscan.rb

Note : If you have specified custom output files to the updateloglist.rb script then add the appropriate arguments to the backupscan.rb command as arguments within this configuration file. If you require assistance then please post a message on the mailing lists.



Example script to scan multiple servers :

#!/bin/bash

#
# Version 1.0
# (C)2008 Lucid Information Systems
# Licenced under the GNU GPL
# www.lucidsystems.org
#

# This script is using SSHKeychain as the Authentication Agent
# This script assumes that you have placed the scan_server.rb script into the direcotry
# ~/bin/scanbackupservers 

# Load SSH Keychain as the SSH Key Agent
SSH_AUTH_SOCK=/tmp/`id -u`/SSHKeychain.socket

# Move into the scan backup servers directory
cd ~/bin/scanbackupservers/

# Scan Some Servers
./scan_server.rb



list_contab_backup_information.rb

The list_contab_backup_information.rb script will allow you to pull various information from backups which are scheduled on a system using cron. It is really a template that you will want to edit to pull the information about scheduled backups you require, presented in a way that you that suites your requirements.


In order to run this script make sure it is executable and that you are in the correct working directory, then issue the following command : list_contab_backup_information.rb


  • Note : When you run the list_contab_backup_information.rb command, keep in mind that the crontab of the user you are will be searched. You should run the command once for each user, for whom you wish to gather contab information.



Example

Summary Log File Update

The example below assumes you have downloaded the 'backupscan.rb' script to the directory '/Users/Shared/backupscan/', this script is executable and that a fils 'backuploglist.txt' and 'backupconfiglist.txt' generated by the 'updateloglist.rb' also exists in this directory.

cd /Users/Shared/backupscan/; ./backupscan.rb > ./backupsummary.log

As an example the command above could be added as a cron job on your system so that a log file summary is regularly updated. Visit the reporting page for a possible example of how you may schedule regular delivery of this summary output to a number recipients.



Helpful Links