Fork me on GitHub

Migrating Backup Sets (Mac OS X)

This page is part of the developer section. As such there may be problems with this information. This page is intended to be the basis for yet to be developed backupset a migration tool, which will assist with migrating / moving backups between volumes. This tool would then be included with LBackup.

In the mean time this page contains information regarding migrating a backupset from one volume to another.

The steps involved with migration will probably be different depending upon how you store your backup set. Below are instructions for migrating each and every file of a backup set to a new volume. If you have stored your backup set within a virtual file system (such as a disk image) then you may be able to simply copy the disk image from one location to another and will not need the instructions listed below.



Overview

  1. Determine the path to backup set you wish to migrate
    1. Check the size of this backup set
    • du -hs /path/to/backup/set
  2. Determine the path to the directory where you will migrating this backup set
    1. Check the destination partition has sufficient disk space and inodes available
    • df -hi | grep | paritionname
    1. Check the destination partition has permissions enabled
    2. Check the destination partition has access control lists enabled
  3. Lock the backup configuration
    • touch /path/to/backup/configuration/directory/backup_in_progress.lock
  4. Migrate the backup set
  5. Check the migration was successful
  6. Alter the configuration to so the destination is set to the new location
  7. Remove the lock file
    • rm /path/to/backup/configuration/directory/backup_in_progress.lock
  8. Perform a backup to the newly migrated backup set
  9. Check that the backup is working as expected and that hard linking is working correctly.
  10. Remove or rename the old backup set so that it is clear that it not currently in use.



Introduction

If you require more space for your backup sets then one option is to invest in a larger storage device or storage array. Some storage systems such as Drobo or XSan will allow you up to a certain point dynamically increase a volumes size when required. In addition, some virtual file systems will allow you to dynamically increase the volume size. However, even with such options available, you may need to migrate your backup set to another device for archival purposes or for some other reason such as migrating your backupsets to a Drobo.

This page should provide tips and information pertaining to the migration of data including the hard links which make up an LBackup backupset.


Preparation

The first thing you should determine is the size of your backup set. If you have a dedicated volume for your backup set then the command df -h will provide you some quick feed back regarding the size of the backup set. However, if you have multiple backup sets on a volume or other data residing on the same volume as your backup set, then you should first determine the size of the backup set which you are planning to migrate. In order to determine the size of an individual backup set the command sudo du -hs /path/to/backup/set. However, be forewarned that if you have a lot of files in your backup set this command may take quite some time to complete.


Additional Steps

While you migrate the backup set it is recommended that you lock the backup set by creating a lock file within the backup configuration directory. This is particularly important if you have scheduled automated backups. Once your data has been migrated successfully you may then remove the backup lock file.

The lock file name is typically set to :

  backup_in_progress.lock

You may create this lock file using the command below :

touch /path/to/backup/configuration/dir/backup_in_progress.lock

Once you have completed the migration then you may remove the lock file using the following command :

rm /path/to/backup/configuration/dir/backup_in_progress.lock



Solutions


NOTE: If you backup was generated as root then you should perform the migration as root.

rsync

If you decide to use rsync for the migration of your backup data set the most important things to keep in mind is that you will likely want to preserve hard links (space saving) and also the backup directory structure. In addition, you will also want to preserve any meta-data which is specific to your backup set.

Note : When migrating data from one volume to another it is important that the destination file system and mount options are capable of supporting the attributes of the backup set which you are moving (such as file size, permissions, extended-attributes, ACLs, etc…).

On Mac OS X if you have compiled the latest version of rsync with the Mac OS X patachs applied flags listed in the example below may be used to migrate your backup set(s) to a new volume.


The disadvantage of using rsync for this kind of copy is that depending on how much data you have it may take a while to copy. However, the advantage is that the process can be broken into multiple executions of rysnc. See the notes below for further details regarding executing rsync multiple times.

/path/to/rsync -aNHAXEx --protect-args --fileflags --force-change /path/to/backupset /path/to/dest/dir

The command above will let you migrate a backup set between volumes. This method should preserve the the hard links in the source backup volume. It is possible to specify multiple sources for you backup sets provided the destination for all backup sets is going to be the same.

Keep in mind that if you are doing this over multiple runs (large backup sets that take multiple days to migrate) and the backup set source is changing during the runs then you may need to add the delete flag to ensure that any changes made between runs are accurately migrated to the new volume. The example below includes the delete flag which should be added if you are splitting the migration into multiple rsync executions.

/path/to/rsync -aNHAXEx --delete --protect-args --fileflags --force-change /path/to/backupset /path/to/dest/dir



rsync via ssh

As with rsync running on a local machine, if you use rsync via SSH for the migration of your backup data set the most important things to keep in mind is that you will likely want to preserve hard links (space saving) and also the backup directory structure. In addition, you will also want to preserve any meta-data which is specific to your backup set.

If you wish to migrate to a remote machine via the network, consider adding the following option to perform the migration via SSH :

--rsync-path=/path/on/remote/machine/to/rsync

Finally, for the destination replace /path/to/backupset with the following command :

user@server.your.domain:/path/to/backupset



The links below may not cover not all of these rsync options listed on this page. This is because some of the options listed on this page are provided by rsync patches. In order to have these options available in rsync follow the instructions described on the this developer page.



Relevant / Related Notes

It is also possible to specify the remote path to rsync with the following option :

--rsync-path=/path/to/rsync


It is also possible to specify that you would like to see the progress of the copy with the following option :

--progress


It is also possible to gather human readable statistics by adding the following options :

--stats ---human-readable


For help and listing of rsync commands from your installed version, issue the following command :

/path/to/rsync --help