MySQL High Availability- P10

To start a MySQL instance and use a backup of your data, execute a command like the following: mysqld -defaults-file=/home/cbell/backup.cnf The innobackup script The innobackup file is a

Restoring data with ibbackup

Restoring data requires applying the log described in the previous section This ation essentially points the MySQL instance to the backup copies of the database It

oper-does not copy the files to the normal MySQL datadir location If you want to do this, you must copy the files manually or use the innobackup script The reason for this is that the ibbackup utility is designed to refuse to overwrite any data To start a MySQL

instance and use a backup of your data, execute a command like the following:

mysqld -defaults-file=/home/cbell/backup.cnf

The innobackup script

The innobackup file is a Perl script designed to automate many of the operations of ibbackup It can create backups; restore data; start a MySQL instance using the data

from a backup; or copy data, index, and logfiles from a backup directory back to theiroriginal locations

The innobackup script is currently not available for use on Windows.

Unlike the ibbackup utility, you do not need to specify the commands in a separate configuration file for innobackup Rather, you specify the parameters for the backup or

restore using command-line options These options are described in Table 12-2

Table 12-2 innobackup options

help Displays a list of all options.

version Displays the version of the script.

apply-log Applies the backup log to the backup in preparation for starting a MySQL server with the backup files.

copy-back Copies data and index files from the backup location to their original locations.

use-memory=MB Passed to ibbackup , this option controls how much memory is used during restoration.

sleep=MS Passed to ibbackup , this option causes the utility to pause after every 1 Mb of data is copied.

compress=LEVEL Passed to ibbackup , this option provides the compression level to use.

include=REGEXP Passed to ibbackup , this option instructs the process to back up only those table files that match

the regular expression This is used for a selective backup.

uncompress Passed to ibbackup , this option uncompresses a compressed backup.

user=NAME Username to use for connecting to the server.

password=PWD Password for the user.

port=PORT Port for the server.

socket=SOCK Socket of the server.

Performing a backup with innobackup

To create a backup, you need only two options: the configuration file of the server andthe location for your backup files:

perl innobackup /etc/mysql/my.cnf /home/cbell/backup

If you want a consistent backup, specify the apply-log option as well:

perl innobackup apply-log /etc/mysql/my.cnf /home/cbell/backup

Restoring data with innobackup

To restore data, apply the log and use the copy-back option to copy the files to theoriginal location We show a sample of these commands below You must stop theserver before the copy and then restart it afterward

perl innobackup apply-log /etc/mysql/my.cnf /home/cbell/backup mysqladmin -uroot shutdown

perl innobackup copy-file /etc/mysql/my.cnf /home/cbell/backup /etc/init.d/mysql start

Physical File Copy

The easiest and most basic form of backup for MySQL is a simple file copy nately, this requires you to stop the server for best results To perform a file copy, simplystop your server and copy the data directory and any setup files on the server Onecommon method for this is to use the Unix tar command to create an archive You canthen move this archive to another system and restore the data directory

Unfortu-The following is a typical sequence of tar commands that backs up the data from onedatabase server and restores it on another system Execute the following command on

the server you want to back up, where backup_2009_09_09.tar.gz is the file you want

to create and /usr/loca/mysql/data is the path for the data directory:

tar -czf backup_2009_09_09.tar.gz /usr/loca/mysql/data/*

The backup_2009_09_09.tar.gz file must reside on a directory shared by the two servers

(or you must physically copy it to the new server) Now, on the server where you want

to restore the data, change to the root installation of your new installation of MySQL.Delete the existing data directory, if it exists, then execute the following:

Other than Cygwin or another Unix-on-Windows package, the closest equivalent tothis command on Windows is the handy folder archive feature of the Explorer or anarchive program such as WinZip To do this, open Windows Explorer and navigate toyour data directory But instead of opening the directory, right-click the data directoryand choose an option that compresses the data, which will have a label such as SendTo→Compressed (zipped) Folder, and then provide a name for the zip file

While a physical file copy is the quickest and easiest form of backup, it does requirethat you shut down the server But that isn’t necessary if you are careful to ensure thereare no updates occurring during the file copy To do this, you must lock all tables andperform a flush tables command, then take your server offline before making the filecopy

This is similar to the process for cloning a slave See Chapter 2 for more details and an example of cloning a slave using file copy.

Additionally, depending on the size of the data, your server must be offline not onlyfor the time to copy the files, but also for any additional data loads like cache entries,the use of memory tables for fast lookups, etc For this reason, physical copy backupmay not be feasible for some installations

Fortunately, there is a Perl script, created by Tim Bunce, to automate this process The

name of the script is mysqlhotcopy.sh and it is located in the /scripts folder of your

MySQL installation It allows you to make hot copies of databases However, you canuse it only to back up MyISAM or Archive storage engines and it works only on Unixand Netware operating systems

The mysqlhotcopy.sh utility also includes customization features You can find more

information about it at http://dev.mysql.com/doc/refman/5.4/en/mysqlhotcopy.html

The mysqldump Utility

The most popular alternative to the physical file copy feature is the mysqldump clientapplication It has been part of the MySQL installation for some time and was originallydonated to MySQL by Igor Romanenko mysqldump creates a set of SQL statements thatre-create the databases when you rerun them For example, when you run a backup,the output contains all of the CREATE statements needed to create the databases and thetables they contain, as well as all the INSERT statements needed to re-create the data inthose tables

This can be very handy if you need to do a search-and-replace operation in the text ofyour data Simply back up your database, edit the resulting file with a text editor, thenrestore the database to effect the changes Many MySQL users use this technique tocorrect all sorts of errors caused by making batch edits to the data You will find thismuch easier than writing, say, 1,000 UPDATE statements with complicated WHERE clauses.The drawbacks of using mysqldump are that it takes a lot more time than the binarycopies made by file-level (physical) backups like InnoDB Hot Backup, LVM, or a simpleoffline file copy, and it requires a lot more storage space This cost in time can besignificant if you make frequent backups, want to restore a database quickly after asystem failure, or need to transfer the backup file across a network

You can use mysqldump to back up all your databases, a specific subset of databases, oreven particular tables within a given database The following examples show each ofthese options:

mysqldump -uroot -all-databases mysqldump -uroot db1, db2 mysqldump -uroot my_db t1

You can also use mysqldump to do a hot backup of InnoDB tables The single-trans action option issues a BEGIN statement at the start of the backup, which signals theInnoDB storage engine to read the tables as a consistent read Thus, any changes youmake are applied to the tables, but the data is frozen at the time of backup However,

no other connection should use data definition language (DDL) statements like ALTER TABLE, DROP TABLE, RENAME TABLE, TRUNCATE TABLE This is because a consistent read isnot isolated from DDL changes

The single-transaction option and the lock-tables option are mutually exclusive because LOCK TABLES issues an implicit commit.

The utility has several options that control the backup as well as what is included.Table 12-3 describes some of the more important options See the online MySQL Ref-erence Manual at http://dev.mysql.com/doc/refman/5.4/en/mysqldump.html for a com-plete set of options.

Table 12-3 mysqlbackup options

add-drop-database Includes a DROP DATABASE statement before each database.

add-drop-table Includes a DROP TABLE statement before each table.

add-locks Surrounds each included table with LOCK TABLES and UNLOCK TABLES

all-databases Includes all databases.

create-options Includes all MySQL-specific table options in the CREATE TABLE statements.

databases Includes a list of databases only.

delete-master-logs On a master, deletes the binary logs after performing the backup.

events Backs up events from the included databases.

extended-insert Uses the alternative INSERT syntax that includes each row as a VALUES clause.

flush-logs Flushes the logfiles before starting the backup.

flush-privileges Includes a FLUSH PRIVILEGES statement after backing up the mysql database.

ignore-table=db.tbl Does not back up the specified table.

lock-all-tables Locks all tables across all databases during the dump.

lock-tables Locks all tables before including them.

log-error=filename Appends warnings and errors to the specified file.

master-data[=value] Includes the binlog filename and position in the output.

no-data Does not write any table row information (only CREATE statements).

password[=password] The password to use when connecting to the server.

port=port_num The TCP/IP port number to use for the connection.

result-file=filename Outputs to a specific file.

routines Includes stored routines (procedures and functions).

single-transaction Issues a BEGIN SQL statement before dumping data from the server This allows for a

consistent snapshot of the InnoDB tables.

tables Overrides the databases option.

triggers Includes triggers.

where='condition' Includes only rows selected by the condition.

xml Produces XML output.

You can also include these options in a MySQL configuration file under the heading [mysqldump] In most cases, you can specify the option sim- ply by removing the initial dashes For example, to always produce XML output, include xml in your configuration file.

One very handy feature of mysqldump is the ability to dump a database schema You cannormally do this using a set of the CREATE commands to re-create all of the objectswithout the INSERT statements that include the data This usage can be very useful forkeeping a historical record of the changes to your schema If you use the no-data

option along with the options to include all of the objects (e.g., routines,

triggers), you can use mysqldump to create a database schema

Notice the option master-data This option can be very helpful for performing PITRbecause it saves the binary log information like InnoDB Hot Backup does

There are many more options that allow you to control how the utility works If creating

a backup in the form of SQL statements sounds like the best option for you, feel free

to explore the rest of the options for making mysqldump work for you

In an effort to create a hot backup solution for XtraDB, Percona has created XtraBackup.This tool is optimized for InnoDB and XtraDB, but can also back up and restoreMyISAM tables It provides many of the features expected of backup solutions, in-cluding compression and incremental backup

You can download and build XtraBackup by getting the source code from Launchpad

at https://launchpad.net/percona-xtrabackup You can compile and execute XtraBackup

on most platforms It is compatible with MySQL versions 5.0 and 5.1

The online manual for XtraBackup is located at http://www.percona.com/docs/wiki/per cona-xtrabackup:xtrabackup_manual

Logical Volume Manager Snapshots

Most Linux and some Unix systems provide another powerful method of backing up

your MySQL database It makes use of a technology called the logical volume ager (LVM).

man-Microsoft Windows has a similar technology called Volume Shadow

Copy Unfortunately, there are no generic utilities to make a snapshot

of a random partition or folder structure as there are for LVM You can, however, make snapshots of an entire drive, which can be useful if your database directory is the only thing on that drive See the Microsoft online documentation for more information.

An LVM is a disk subsystem that gives you a lot of administrative power to create,remove, and resize volumes easily and quickly without using the older, often compli-cated and unforgiving disk tools

The added benefit for backup is the concept of taking a snapshot—that is, a copy of an

active volume—without disrupting the applications that access the data on that ume The idea is to take a snapshot, which is a relatively fast operation, and then back

vol-up the snapshot instead of the original volume Deep inside LVM, a snapshot is aged using a mechanism that keeps track of the changes since you took the snapshot,

man-so that it stores only the disk segments that have changed Thus, a snapshot takes upless space than a complete copy of the volume and when the backup is made, the LVMcopies the files as they existed at the time of the snapshot Snapshots effectively freezethe data

Another benefit of using LVM and snapshots for backing up database systems lies inhow you use the volumes The best practice is to use a separate volume for each of yourMySQL installations so that all of the data is on the same volume, allowing you to create

a backup quickly using a snapshot Of course, it is also possible to use multiple logicalvolumes in some situations, such as using one logical volume for each tablespace oreven different logical volumes for MyISAM and InnoDB tables

Getting started with LVM

If your Linux installation does not have LVM installed, you can install it using yourpackage manager For example, on Ubuntu you can install LVM using the followingcommand:

sudo apt-get install lvm2

Although not all LVM systems are the same, the following procedure is based on atypical Debian distribution and works well on systems like Ubuntu We don’t mean towrite a complete tutorial on LVM but just to give you an idea of the complexity of usingLVM for making database backups Consult your operating system documentation forspecifics about the type of LVM your system supports, or simply browse the many how-

to documents available on the Web

Before we get started with the details, let’s take a moment to understand the basicconcepts of LVM There is a hierarchy of levels to the LVM implementation At the

lowest level is the disk itself On top of that are partitions, which allow us to nicate with the disk On top of the partition we create a physical volume, which is the

commu-control mechanism that the LVM provides You can add a physical volume to a volume group (which can contain multiple physical volumes), and a volume group can contain one or more logical volumes Figure 12-1 depicts the relationship among filesystems,volume groups, physical volumes, and block devices.

Figure 12-1 Anatomy of LVM

A logical volume can act as either a normal mounted filesystem or a snapshot Thecreation of a snapshot logical volume is the key to using snapshots for backup Thefollowing sections describe how you can get started experimenting with LVM andmaking backups of your data

There are several useful commands that you should become familiar with The ing list contains the most frequently used commands and their uses Be sure to consultthe documentation for more information about these commands:

Unmounts a logical volume

To use LVM, you need to have either a new disk or a disk device that you can logicallyunmount The process is as follows (the output here was generated on a laptop runningUbuntu version 9.04):

1 Create a backup of an existing MySQL data directory

tar -czf ~/my_backups/backup.tar.gz /dev/mysql/datadir

2 Partition the drive

sudo parted select /dev/sdb mklabel msdos mkpart test quit

3 Create a physical volume for the drive

sudo pvcreate /dev/sdb

4 Create a volume group

sudo vgcreate /dev/sdb mysql

5 Create a logical volume for your data Here we create a 20 GB volume

sudo lvcreate -L20G -ndatadir mysql

6 Create a filesystem on the logical volume

mke2fs /dev/mysql/datadir

7 Mount the logical volume

sudo mkdir /mnt sudo mount /dev/mysql/datadir /mnt

8 Copy the archive and restore your data to the logical volume

sudo cp ~/my_backups/backup.tar.gz sudo tar -xvf backup.tar.gz

9 Create an instance of a MySQL server and use datadir to point to the folder onthe logical volume

./mysqld console -uroot datadir=/mnt

If you want to experiment with LVM, we recommend you use a disk whose data you can afford to lose A good, cheap option is a small USB hard drive.

That’s all you need to get started with a logical volume Take some time to experimentwith the LVM tools, until you are certain you can work with them effectively, beforeyou start using them for your production systems

LVM in a backup and restore

To do your backup, you need to flush and temporarily lock all of the tables, take thesnapshot, and then unlock the tables The lock is necessary to ensure all of your ongoingtransactions are finished The process is shown here along with the shell-level com-mands that perform the operations:

1 Issue a FLUSH TABLES WITH READ LOCK command in a MySQL client

2 Create a snapshot of your logical volume (the -s option specifies a snapshot)

sudo lvcreate -L20M -s -n backup /dev/mysql/datadir

3 Issue an UNLOCK TABLES command in a MySQL client (your server can now resumeits operations)

4 Mount the snapshot

sudo mkdir /mnts sudo mount /dev/mysql/backup /mnts

5 Perform a backup of the snapshot

tar -[FIXTHIS]f snapshot.tar.gz /mnts

Of course, the best use of the snapshot is to initiate a copy periodically so that you can

do another backup There are scripts available from volunteers on the Web to automatethis process, but the tried and true mechanism is to remove the snapshot and re-create

it using the following procedure:

1 Unmount the snapshot

sudo umount /mnts

2 Remove the snapshot (logical volume)

sudo lvremove /dev/mysql/backup

You can then re-create the snapshot and perform your backup If you create your ownscript, we recommend adding the snapshot removal after you have verified the backuparchive was created This will ensure your script performs proper cleanup

If you need to restore the snapshot, simply restore the data The real benefit of LVM isthat all of the operations for creating the snapshot and the backup using the tar utilityallow you to create a customized script that you can run periodically (such as a cronjob), which can help you automate your backups

LVM in ZFS

The procedure for performing a backup using Sun Microsystems’ ZFS filesystem able in Solaris 10) is very similar to the Linux LVM procedure We describe the differ-ences here for those of you using Solaris

(avail-In ZFS, you store your logical volumes (which Sun calls filesystems for read/write andsnapshots for read-only copies) in a pool (similar to the volume group) To make a copy

or backup, simply create a snapshot of a filesystem

Issue the following commands to create a ZFS filesystem that you can manage:

zpool create -f mypool c0d0s5 zfs create mypool/mydataUse the following command to make a backup (take a snapshot of the new filesystem):zfs snapshot mypool/mydata@backup_12_Dec_2009

Use the following commands to restore the filesystem to a specific backup:

cd /mypool/mydata zfs rollback mypool/mydata@backup_12_Dec_2009ZFS provides not only full volume (filesystem) backups, but also supports selective filerestore For more information about ZFS and performing backup and restore, visit http: //dlc.sun.com/osol/docs/content/ZFSADMIN/gavvx.html

Comparison of Backup Methods

InnoDB Hot Backup, mysqldump, and third-party backup options differ along a number

of important dimensions, and there is almost no end to the nuances of how each methodworks We provide a comparison of backup methods here based on whether they allowfor hot backups, their cost, the speed of the backup, the speed of the restore, the type

of backup (logical or physical), platform restrictions (operating system), and supportedstorage engines Table 12-4 lists each of the backup methods mentioned in this chapteralong with a column for each comparison item

Table 12-4 Comparison of backup methods

InnoDB Hot Backup mysqldump Physical copy XtraBackup LVM/ZFS snapshot Hot backup? Yes (InnoDB only) Yes (InnoDB only re-

quires transaction )

single-No Yes (InnoDB and

XtraDB only) Yes (requires tableflush with lock)

Cost Paid license Free Free Free Free

Type Physical Logical Physical Physical Physical

InnoDB Hot Backup mysqldump Physical copy XtraBackup LVM/ZFS snapshot

Engines InnoDB, MyISAM All All InnoDB, XtraDB,

MyISAM All

InnoDB Hot Backup is not supported fully on Windows The Perl script does not execute on some Windows configurations See the online doc- umentation for more details.

Table 12-4 can help you plan your data recovery procedures by allowing you to findthe best tool for the job given your needs For example, if you need a hot backup foryour InnoDB database and cost is not a factor, the Innobase InnoDB Hot Backup ap-plication is the best choice On the other hand, if speed is a factor, you need somethingthat can back up all databases (all storage engines), and you work on a Linux system,LVM is a good choice

Backup and MySQL Replication

There are two ways to use backup with MySQL replication In previous chapters, youlearned about MySQL replication and its many uses for scale-out and high availability

In this chapter, we examine two more common uses for MySQL replication involvingbackup These include using replication to create a backup copy of the data and usingbackups taken previously for PITR

Backup and recovery

Keeping an extra server around for creating backups is very common; it allows you

to create your backups without disturbing the main server at all, since you can takethe backup server offline and do whatever you like with it

PITR

Even if you create your backups regularly, you may have to restore the server to anexact point in time By administering your backups properly, you can actuallyrestore the server to the granularity of a specific second This can be very useful inrecovering from human error—such as mistyping commands or entering incorrectdata—or reverting changes that are not needed anymore The possibilities areendless, but they require the existence of proper backups

Backup and Recovery with Replication

One shortcoming of backups in general is that they are created at a specific time (usuallylate at night to avoid disturbing other operations) If there is a problem that requiresyou to restore the master to some point after you created the backup, you will be out

of luck, right? Nope! As it turns out, this is indeed not only possible, but quite easy ifyou combine the backups with the binary logs.

The binary log records all changes that are made to the database while the database isrunning, so by restoring the correct backup and playing back the binary log up to theappropriate second, you can actually restore a server to a precise moment in time.The most important step in a recovery procedure is, of course, recovery So let’s focus

on performing recovery before outlining the procedure for performing a backup

PITR

The most frequent use for backup in replication is PITR, the ability to recover from anerror (such as data loss or hardware failure) by restoring the system to a state as close

as possible to the most recent correct state, thus minimizing the loss of data For this

to work, you must have performed at least one backup

Once you repair the server, you can restore the latest backup image and apply the binarylog using that binary log name and position as the starting point

The following describes one procedure you can use to perform PITR using the backupsystem:

1 Return your server to an operational state after the event

2 Find the latest backup for the databases you need to restore

3 Restore the latest backup image

4 Apply the binary log using the mysqlbinlog utility using the starting position (orstarting date/time) from the last backup

At this point, you may be wondering, “Which binary log do I use for PITR after abackup?” The answer depends on how you performed the backup If you flushed thebinary log prior to running the backup, you need to use the name and position of thecurrent log (the newly opened logfile) If you did not flush the binary log prior to run-ning the backup, use the name and position of the previous log

For easier PITR, always flush the logs prior to a backup The starting point is then at the start of the file.

Restoring after an error is replicated

Now let’s see how a backup can help you recover unintentional changes in a replicationtopology Suppose one of your users has made a catastrophic (yet valid) change that isreplicated to all of your slaves Replication cannot help you here, but the backup systemcan come to the rescue

Perform the following steps to recover from unintentional changes to data in a cation topology:

repli-1 Drop the databases on the master

2 Stop replication

3 Restore the latest backup image before the event on the master

4 Record the master’s current binlog position

5 Restore the latest backup image before the event on the slaves

6 Perform a PITR on the master, as described in the previous section

7 Restart replication from the recorded position and allow the slaves to sync

In short, a good backup strategy is not only a necessary protection against data loss,but an important tool in your replication toolbox

Recovery example

Now, let’s look at a concrete example Assume you’re creating backups regularly everymorning at 2:00 A.M and save the backup images away somewhere for later usage.For this example, let’s assume all binary logs are available and that none are removed

In reality, you will prune the binlog files regularly to keep the disk space down, but let’sconsider how to handle that later

You have been tasked with recovering the database to its state at 2009-12-19 12:54:23,because that’s when the manager’s favorite pictures were accidentally deleted by hisoverzealous assistant, who took his “Could you please clean my desk?” request to in-clude the computer desktop as well

1 Locate the backup image that was taken before 2009-12-19 12:54:23

It does not actually make any difference which one you pick, but to save on therecovery time, you should probably pick the one closest, which would then be thebackup image dated in the morning of the same day

2 Restore the backup image on the machine to create an exact copy of the database

at 2009-12-19 02:00:00

3 Locate all the binlog files that include the entire range from 2009-12-19 02:00:00

to 2009-12-19 12:54:23 It does not matter if there are events from before the starttime or after the end time, but it is critical that the binlog files cover the entire rangeyou want

4 Play the binlog files back using the mysqlbinlog utility and give a start time of2009-12-19 02:00:00 and an end time of 2009-12-19 12:54:23

You can now tell your manager that his favorite pictures are back

To automate this, it is necessary to do some bookkeeping This will give you an cation of what you need to save when doing the backup, so let’s go through the infor-mation that you will require when doing a recovery

indi-• To use the backup images correctly, it is critical to label each with the start andend time it represents This will help you determine which image to pick.

• You also need the binlog position of the backup This is necessary because the time

is not sufficiently precise in deciding where to start playing back the binlog files

• You also need to keep information about what range each binlog file represents.Strictly speaking, this is not required, but it can be quite helpful because it helpsyou avoid processing all binlog files of a recovery image This is not something thatthe MySQL server does automatically, so you have to handle it yourself

• You cannot keep these files around forever, so it is important to sort all the mation, backup images, and binlog files in such a way that you can easily archivethem when you need to free up some disk space

infor-Recovery images

To help you administer all the information about your backups in manageable chunks,

we introduce the concept of a recovery image The recovery image is just a virtual

con-tainer and is not a physical entity: it contains only information about where all thenecessary pieces are to be able to perform recovery

Figure 12-2 shows a sequence of recovery images and the contents of each The final

recovery image in the sequence is special and is called the open recovery image This is

the recovery image that you are still adding changes to, hence it does not have an end

time yet The other recovery images are called closed recovery images and these have an

end time

Figure 12-2 A sequence of recovery images and contents

Each recovery image has some pieces of information that are necessary to perform arecovery:

A backup image

The backup image is required for restoring the database

A set of binlog files

Binlog files must cover the entire range for the recovery image

Start time and an optional end time

These are the start and end time that the recovery image represents If this is theopen recovery image, it does not have an end time This recovery image is notpractical to archive, so for our purposes, the end time of this recovery image is thecurrent time

The binlog files usually contain events outside the range of start and end times, but all events in that range should be in the recovery image.

A list of the name and start time for each binlog file

To extract the correct binlog files from, say, an archive, you need the names andthe start and end times for each file You can extract the start time for the binlogfile using mysqlbinlog

If you used an offline backup tool, the backup time is the time when you lockedthe tables, and the binlog position is the position given by SHOW MASTER STATUS afteryou locked the database

2 Create a new open recovery Imagen with the following parameters:

• Backup(Imagen) is now the backup image from step 1

• Position(Imagen) is the position from step 1

• BinlogFiles(Imagen) is unknown, but it starts with the filename from the positionfrom step 1

• StartTime(Imagen) is the start time of the image, which is taken from the event

at the binlog position from step 1

3 Close Imagen–1, noting the following:

• BinlogFiles(Imagen–1) is now the binlog files from Position(Imagen–1) to tion(Imagen)

Posi-• EndTime(Imagen–1) is now the same as StartTime(Imagen)

This method will use the backup image for the backup and restore it on server

By using a class to represent a backup method in this manner, it is easy to replace thebackup method with any other method as long as it has these methods

Example 12-1 Class for representing the physical backup method

class BackupImage(object):

"Class for representing a backup image"

def init (self, backup_url):

self.url = urlparse.urlparse(backup_url) def backup_server(self, server, db):

"Backup databases from a server and add them to the backup image."

pass def restore_server(self, server):

"Restore the databases in an image on the server"

pass class PhysicalBackup(BackupImage):

"A physical backup of a database"

def backup_server(self, server, db="*"):

path = self.url.path server.ssh(["tar", "zpscf", path, "-C", datadir] + db)

if server.host != "localhost":

subprocess.call(["scp", server.host + ":" + path, self.url.path]) server.sql("UNLOCK TABLES")

return position def restore_server(self, server):

if server.host == "localhost":

path = self.url.path else:

path = basename(self.url.path) datadir = server.fetch_config().get('datadir') try:

Creates a new open recovery image by creating a backup of server and collectsinformation about the backup

RecoveryImage.restore_to(server, datetime)

Restores the recovery image on the server so that all changes up to and including

datetime are applied This assumes datetime is in the range for the recovery image

If datetime is before the recovery image’s start time, nothing will be applied, and

if it is after the recovery image’s end time, all will be applied

Example 12-2 A representation of a recovery image

class RecoveryImage(object):

def init (self, backup_method):

self.backup_method = backup_method self.backup_position = None self.start_time = None self.end_time = None self.binlog_files = []

self.binlog_datetime = {}

def backup_from(self, server, datetime):

self.backup_position = backup_method.backup_from(server) def restore_to(self, server):

backup_method.restore_on(server) def contains(self, datetime):

if self.end_time:

return self.start_time <= datetime < self.end_time else:

return self.start_time <= datetime

Because managing the recovery images requires manipulating several images, we troduce the RecoveryImageManager class in Example 12-3 The class contains two meth-ods in addition to the constructor:

Example 12-3 RecoveryImageManager class

class RecoveryImageManager(object):

def init (self, backup_method):

self. images = []

self. backup_method = backup_method

def point_in_time_recovery(server, datetime):

from itertools import takewhile from subprocess import Popen, PIPE for im in images:

if im.contains(datetime):

image = im break image.restore_on(server) def before(file):

return image.binlog_datetime(file) < datetime files = takewhile(before, image.binlog_files) command = ["mysqlbinlog",

" start-position=%s" % (image.backup_position.pos), " stop-datetime=%s" % (datetime)]

mysqlbinlog_proc = Popen(mysqlbinlog_command + files, stdout=PIPE) mysql_command = ["mysql",

" host=%s" % (server.host), " user=%s" % (server.sql_user.name), " password=%s" % (server.sql_user.password)]

mysql_proc = Popen(mysql_command, stdin=mysqlbinlog_proc.stdout) output = mysql_proc.communicate()[0]

def point_in_time_backup(self, server):

new_image = RecoveryImage(self. backup_method) new_image.backup_position = image.backup_from(server) new_image.start_time = event_datetime(new_image.backup_position) prev_image = self. images[-1].binlog_files

prev_image.binlog_files = binlog_range(prev_image.backup_position.file, new_image.backup_position.file) prev_image.end_time = new_image.start_time

self. images.append(new_image)

Automating Backups

It is fairly easy to automate backups In the previous section, we demonstrated how to

do a backup and recovery with replication In this section, we generalize the procedure

to make it easier to do nonreplication-related backup and restore

The only issue you may encounter is providing a mechanism to automatically name thebackup image file There are many ways to do this, and Example 12-4 shows a methodthat names the file using the backup time You can add this backup method to thePython library to complement your replication methods This is the same library shown

in previous chapters

Example 12-4 Backup script

#!/usr/bin/python import MySQLdb, optparse

#

# Parse arguments and read Configuration

# parser = optparse.OptionParser() parser.add_option("-u", " user", dest="user", help="User to connect to server with") parser.add_option("-p", " password", dest="password", help="Password to use when connecting to server") parser.add_option("-d", " database", dest="database",

help="Database to connect to") (opts, args) = parser.parse_args()

if not opts.password or not opts.user or not opts.database:

parser.error("You have to supply user, password, and database") try:

print "Connecting to server "

# # Connect to server #

dbh = MySQLdb.connect(host="localhost", port=3306, unix_socket="/tmp/mysql.sock", user=opts.user, passwd=opts.password, db=opts.database)

# # Perform the restore #

from datetime import datetime filename = datetime.time().strftime("backup_%Y-%m-%d_%H-%M-%S.bak")

dbh.cursor().execute("[BACKUP COMMAND]%s'" % filename)

print "\nBACKUP complete."

except MySQLdb.Error, (n, e):

print 'CRITICAL: Connect failed with reason:', e

Substitute the executable command for your backup solution for

[BACKUP COMMAND].

Automating restores is a bit easier if you consider that there is no need to create a backupimage name However, depending on your installation, usage, and configuration, youmay need to add commands to ensure there is no destructive interaction with activities

by other applications or users Example 12-5 shows a typical restore method that youcan add to the Python library to complement your replication methods

Example 12-5 Restore script

#!/usr/bin/python import MySQLdb, optparse

#

# Parse arguments and read Configuration

# parser = optparse.OptionParser() parser.add_option("-u", " user", dest="user", help="User to connect to server with") parser.add_option("-p", " password", dest="password", help="Password to use when connecting to server") parser.add_option("-d", " database", dest="database",

help="Database to connect to") (opts, args) = parser.parse_args()

if not opts.password or not opts.user or not opts.database:

parser.error("You have to supply user, password, and database") try:

print "Connecting to server "

# # Connect to server #

dbh = MySQLdb.connect(host="localhost", port=3306, unix_socket="/tmp/mysql.sock", user=opts.user, passwd=opts.password, db=opts.database)

# # Perform the restore #

from datetime import datetime filename = datetime.time().strftime("backup_%Y-%m-%d_%H-%M-%S.bak") dbh.cursor().execute("[RESTORE COMMAND]%S'",

(database, filename)) print "\nRestore complete."

except MySQLdb.Error, (n, e):

print 'CRITICAL: Connect failed with reason:', e

Substitute the executable command for your backup solution for

[RESTORE COMMAND].

As you can see in Example 12-5, you can automate a restore However, most peopleprefer to execute the restore manually One scenario in which an automated restoremight be helpful is in a testing environment where you want to start with a baselineand restore a database to a known state Another use is in a development system whereyou want to preserve a certain environment for each project

In this chapter, we studied IA and drilled down to the aspects that most affect you as

an IT professional We saw the importance of disaster recovery planning and how tomake your own disaster recovery plan, and we saw how the database system is anintegral part of disaster recovery Finally, we examined several ways you can protectyour MySQL data by making regular backups

In the next few chapters, we will examine more advanced MySQL topics, includingMySQL Enterprise, cloud computing, and MySQL Cluster

Joel glanced at his terminal window and issued another command to check on hisdatabase backups He had set up a recurring backup script to back up all of his databasesand felt confident this simple operation was working He lamented that it was only asmall start toward getting his disaster recovery plan written He looked forward toexperimenting with more scripts and scheduling his first disaster planning meeting Healready had several of his new colleagues in mind for the team A sharp rap on his doorstartled him from his thoughts

“Did you get that media order in yet, Joel? What about that plan thing?” Mr merson asked

Sum-Joel smiled and said, “Yes, and I’ve determined I can back up the entire database withjust a…”

“Hey, that’s great, Joel I don’t need the details; just keep us off of the auditor’s radar.OK?”

Joel smiled and nodded as his boss disappeared on his way to execute a drive-by tasking

on another employee He wondered if his boss really understood the amount of work

it would take to reach that goal He opened his email application and started composing

a message to request more personnel and resources

CHAPTER 13

MySQL Enterprise

Joel clicked open another terminal window and read the output He rubbed his eyesuntil his vision cleared The numbers were beginning to run together as he tried to keeptabs on all of his servers spread out over three sites His report was overdue

He had tried using a spreadsheet to tabulate the data, which had worked when he hadonly a few servers, but it was getting tedious now that he had more than 30 to monitor.His friends had tried to convince him to buy an enterprise monitoring suite, but hisboss had a reputation of vetoing any purchases that didn’t immediately contribute tothe company’s ability to produce revenue

“Hey, Joel.”

Joel looked up to see his friend Doug from customer support standing at his door with

a stained coffee cup in hand “Hey,” Joel replied

“You look like you could use a break.”

“No time I’ve got to get a report together that shows the status of all of the servers and

I can’t write it until I finish checking each server.”

“Wow, that’s a real hands-on approach.”

“I know I’ve read about these enterprise suites that make this all easier to do, but Idon’t know which one to buy or even if Mr Summerson will sign off on it.”

“Well, if you could show him how much time it takes you to do all this…” Doug said,with a dangerous wave of his mug

Joel thought for a moment “If only I could show him the difference between what I’vehad to slog through and what it would be like to use a good tool….”

“Good plan Now how about a cup? My treat.”

Joel followed Doug to the break room and chatted with him about a recent Dave thews Band concert they’d attended

Mat-When Joel returned to his office, he started reading about the MySQL Enterprise fering from Oracle