I’d like to focus this blog on using MySQL in the Enterprise and kickoff with a series of posts on “Enterprise Backup” building on the new features in both MySQL Enterprise Backup (MEB) and MySQL Enterprise Monitor (MEM). The new features in MEB 3.6 provide the capabilities to stream backups directly to another server, interface with backup media management software, and take advantage of tape encryption. MEM 2.3.5 now has a Backup Advisor that helps monitor your backups. In this and a subsequent blog post, I’ll go through a progression of backups building up on a fairly straightforward vanilla single file backup with MEB as follows:
- Backing up to a Single File
- Add streaming to your Single File backup
- Stream your Single File Backup to a Media Management System
- Encrypt your Backup Tapes using your Media Management System
- Monitoring your Backup with MySQL Enterprise Monitor or queries
MEB is a backup tool included with an enterprise subscription from Oracle/MySQL. If you are interested in a trying it out, it can be downloaded as a trial from http://edelivery.oracle.com. MEB was previously known as InnoDB Hot Backup, and provides hot, non-blocking backups for InnoDB tables, and “warm” backups for MyISAM tables. MEM is also included in the enterprise subscription from Oracle/MySQL and can be downloaded for a trial at the same site. It provides proactive monitoring for your MySQL databases.
This post includes the MEB related single file backup, streaming the single file, interfacing with a media management system, and encrypting tapes. My next post will cover the MEM Backup Advisor and the tables behind the scene.
The following abbreviations are used in this blog:
MEB MySQL Enterprise Backup
MEM MySQL Enterprise Monitor
MMS Media Management System (software for managing tapes and backups)
OSB Oracle Secure Backup
SBT Secure Backup to Tape Interface
Step 1: Backup to a Single File
You can now backup into a single file, which simplifies moving backup data around, especially if you have a large number of files to keep track of. You do this by using the backup-to-image option of MySQL Enterprise Backup:
mysqlbackup
–backup-image= hr.mbi
–backup-dir=/backup-tmp
–user lynn
–password
backup-to-image
This places my single file backup in the file hr.mbi. Since some small work files are still used, you need to indicate where these should go with the –backup-dir option. If you are interested in these files, see the documentation in the “Files that Are Backed Up” section of the MySQL Enterprise Backup User’s Guide http://dev.mysql.com/doc/mysql-enterprise-backup/3.5/en/meb-files-overview.html . There are also copies of these files inside your successful single file backup, so you are not required to keep them. For example, backup_variables.txt contains the start and end log sequence numbers plus information indicating whether this was a compressed, partial, or incremental backup. The file backup_create.xml lists the command line arguments and the environment that the backup was created in.
You can also convert an existing backup directory to a single file backup as follows:
mysqlbackup
— backup-image=/backup/my.mbi
–backup-dir=/var/mysql/backup
— user lynn
— password
backup-dir-to-image
Use list-image to list the contents of a single file backup (leaving out user and password for simplicity):
mysqlbackup
–backup-image=/backup/my.mbi
list-image
You will see a list of the contents of the single file backup with each database and table listed. Here’s an example of the contents of a single file backup named MondayApril112022.mbi:
______________________________________________
mysqlbackup –backup_image=Sept2011.mbi list-image
IMPORTANT: Please check that mysqlbackup run completes successfully.
At the end of a successful ‘list-image’ run mysqlbackup
prints “mysqlbackup completed OK!”.
mysqlbackup: INFO: Backup Image MEB version string: 3.6.0 [01.07.2011 ]
[File]: [Size: 197]: backup-my.cnf
[File]: [Size: 5578]: meta\backup_create.xml
[File]: [Size: 16777216]: datadir\ibdata1
[File]: [Size: 2097152]: datadir\ibdata1.$_append_$.1
[Dir]: datadir\crazybase
[File]: [Size: 65]: datadir\crazybase\db.opt
[Dir]: datadir\crazybase3
[File]: [Size: 65]: datadir\crazybase3\db.opt
[Dir]: datadir\mysql
[File]: [Size: 35]: datadir\mysql\backup_history.CSM
[File]: [Size: 5557]: datadir\mysql\backup_history.CSV
[File]: [Size: 71260]: datadir\mysql\backup_history.frm
[File]: [Size: 35]: datadir\mysql\backup_progress.CSM
[File]: [Size: 5423]: datadir\mysql\backup_progress.CSV
[File]: [Size: 33370]: datadir\mysql\backup_progress.frm
[File]: [Size: 0]: datadir\mysql\columns_priv.MYD
[File]: [Size: 4096]: datadir\mysql\columns_priv.MYI
[File]: [Size: 8820]: datadir\mysql\columns_priv.frm
[File]: [Size: 1320]: datadir\mysql\db.MYD
[File]: [Size: 5120]: datadir\mysql\db.MYI
[File]: [Size: 9582]: datadir\mysql\db.frm
[File]: [Size: 0]: datadir\mysql\event.MYD
[File]: [Size: 2048]: datadir\mysql\event.MYI
[File]: [Size: 10223]: datadir\mysql\event.frm
[File]: [Size: 0]: datadir\mysql\func.MYD
[File]: [Size: 1024]: datadir\mysql\func.MYI
[File]: [Size: 8665]: datadir\mysql\func.frm
[File]: [Size: 35]: datadir\mysql\general_log.CSM
[File]: [Size: 0]: datadir\mysql\general_log.CSV
[File]: [Size: 8776]: datadir\mysql\general_log.frm
[File]: [Size: 22078]: datadir\mysql\help_category.MYD
[File]: [Size: 3072]: datadir\mysql\help_category.MYI
[File]: [Size: 8700]: datadir\mysql\help_category.frm
[File]: [Size: 89241]: datadir\mysql\help_keyword.MYD
[File]: [Size: 16384]: datadir\mysql\help_keyword.MYI
[File]: [Size: 8612]: datadir\mysql\help_keyword.frm
[File]: [Size: 8928]: datadir\mysql\help_relation.MYD
[File]: [Size: 18432]: datadir\mysql\help_relation.MYI
[File]: [Size: 8630]: datadir\mysql\help_relation.frm
[File]: [Size: 418976]: datadir\mysql\help_topic.MYD
[File]: [Size: 20480]: datadir\mysql\help_topic.MYI
[File]: [Size: 8770]: datadir\mysql\help_topic.frm
[File]: [Size: 0]: datadir\mysql\host.MYD
[File]: [Size: 2048]: datadir\mysql\host.MYI
[File]: [Size: 9510]: datadir\mysql\host.frm
[File]: [Size: 84]: datadir\mysql\inventory.MYD
[File]: [Size: 2048]: datadir\mysql\inventory.MYI
[File]: [Size: 8592]: datadir\mysql\inventory.frm
[File]: [Size: 0]: datadir\mysql\ndb_binlog_index.MYD
[File]: [Size: 1024]: datadir\mysql\ndb_binlog_index.MYI
[File]: [Size: 8778]: datadir\mysql\ndb_binlog_index.frm
[File]: [Size: 0]: datadir\mysql\plugin.MYD
[File]: [Size: 1024]: datadir\mysql\plugin.MYI
[File]: [Size: 8586]: datadir\mysql\plugin.frm
[File]: [Size: 0]: datadir\mysql\proc.MYD
[File]: [Size: 2048]: datadir\mysql\proc.MYI
[File]: [Size: 9996]: datadir\mysql\proc.frm
[File]: [Size: 0]: datadir\mysql\procs_priv.MYD
[File]: [Size: 4096]: datadir\mysql\procs_priv.MYI
[File]: [Size: 8875]: datadir\mysql\procs_priv.frm
[File]: [Size: 693]: datadir\mysql\proxies_priv.MYD
[File]: [Size: 5120]: datadir\mysql\proxies_priv.MYI
[File]: [Size: 8800]: datadir\mysql\proxies_priv.frm
[File]: [Size: 0]: datadir\mysql\servers.MYD
[File]: [Size: 1024]: datadir\mysql\servers.MYI
[File]: [Size: 8838]: datadir\mysql\servers.frm
[File]: [Size: 35]: datadir\mysql\slow_log.CSM
[File]: [Size: 0]: datadir\mysql\slow_log.CSV
[File]: [Size: 8976]: datadir\mysql\slow_log.frm
[File]: [Size: 0]: datadir\mysql\tables_priv.MYD
[File]: [Size: 4096]: datadir\mysql\tables_priv.MYI
[File]: [Size: 8955]: datadir\mysql\tables_priv.frm
[File]: [Size: 0]: datadir\mysql\time_zone.MYD
[File]: [Size: 8192]: datadir\mysql\time_zone.MYI
[File]: [Size: 8636]: datadir\mysql\time_zone.frm
[File]: [Size: 312]: datadir\mysql\time_zone_leap_second.MYD
[File]: [Size: 2048]: datadir\mysql\time_zone_leap_second.MYI
[File]: [Size: 8624]: datadir\mysql\time_zone_leap_second.frm
[File]: [Size: 111896]: datadir\mysql\time_zone_name.MYD
[File]: [Size: 12288]: datadir\mysql\time_zone_name.MYI
[File]: [Size: 8606]: datadir\mysql\time_zone_name.frm
[File]: [Size: 658733]: datadir\mysql\time_zone_transition.MYD
[File]: [Size: 733184]: datadir\mysql\time_zone_transition.MYI
[File]: [Size: 8686]: datadir\mysql\time_zone_transition.frm
[File]: [Size: 99788]: datadir\mysql\time_zone_transition_type.MYD
[File]: [Size: 38912]: datadir\mysql\time_zone_transition_type.MYI
[File]: [Size: 8748]: datadir\mysql\time_zone_transition_type.frm
[File]: [Size: 376]: datadir\mysql\user.MYD
[File]: [Size: 2048]: datadir\mysql\user.MYI
[File]: [Size: 10630]: datadir\mysql\user.frm
[Dir]: datadir\performance_schema
[File]: [Size: 8624]: datadir\performance_schema\cond_instances.frm
[File]: [Size: 61]: datadir\performance_schema\db.opt
[File]: [Size: 9220]: datadir\performance_schema\events_waits_current.frm
[File]: [Size: 9220]: datadir\performance_schema\events_waits_history.frm
[File]: [Size: 9220]:datadir\performance_schema\events_waits_history_long.frm
[File)]:[Size: 8878]: datadir\performance_schema\events_waits_summary_by_instance.frm
[File]: [Size: 8854]: datadir\performance_schema\events_waits_summary_by_thread_by_event_name.frm
[File]: [Size: 8814]: datadir\performance_schema\events_waits_summary_global_by_event_name.frm
[File]: [Size: 8654]: datadir\performance_schema\file_instances.frm
[File]: [Size: 8800]: datadir\performance_schema\file_summary_by_event_name.frm
[File]: [Size: 8840]: datadir\performance_schema\file_summary_by_instance.frm
[File]: [Size: 8684]: datadir\performance_schema\mutex_instances.frm
[File]: [Size: 8776]: datadir\performance_schema\performance_timers.frm
[File]: [Size: 8758]: datadir\performance_schema\rwlock_instances.frm
[File]: [Size: 8605]: datadir\performance_schema\setup_consumers.frm
[File]: [Size: 8637]: datadir\performance_schema\setup_instruments.frm
[File]: [Size: 8650]: datadir\performance_schema\setup_timers.frm
[File]: [Size: 8650]: datadir\performance_schema\threads.frm
[Dir]: datadir\pets
[File]: [Size: 65]: datadir\pets\db.opt
[Dir]: datadir\test
[File]: [Size: 8560]: datadir\test\names.frm
[Dir]: datadir\world
[File]: [Size: 8652]: datadir\world\bartstations.frm
[File]: [Size: 8710]: datadir\world\city.frm
[File]: [Size: 8630]: datadir\world\citychild.frm
[File]: [Size: 8646]: datadir\world\citylist2.frm
[File]: [Size: 9172]: datadir\world\country.frm
[File]: [Size: 8702]: datadir\world\countrylanguage.frm
[File]: [Size: 8590]: datadir\world\countrylist2.frm
[File]: [Size: 8590]: datadir\world\countryparent.frm
[File]: [Size: 65]: datadir\world\db.opt
[File]: [Size: 741]: datadir\world\europe_view.frm
[File]: [Size: 3584]: datadir\ibbackup_logfile
[File]: [Size: 176]: meta\backup_variables.txt
[File]: [Size: 38562]: meta\backup_content.xml
[File]: [Size: 15236]: meta\image_files.xml
mysqlbackup: INFO: Backup image contents listed successfully.
Source Image Path= C:\temp\temp\Sep42011.mbi
mysqlbackup completed OK!
______________________________________________
Step 2: Add Streaming to your Single File Backup
Streaming allows you to write the backup to a different server without ever storing it locally. This limits the storage space you need on the local database server, and can be faster than copying it locally and then moving the backup to a different server. You build on the single file option by using it in combination with OS features like pipes, ssh/scp, etc and take your input from standard output.
mysqlbackup
— backup-image=-backup-to-image | ssh user@host command arg1 arg2 …
where command is the combination of command, device, etc used during normal archiving (such as dd or tar)
Step 3: Stream your Single File Backup to a Media Management System
You can backup to tape with media management software (MMS) like Oracle Secure Backup (OSB), Legato, Netbackup, etc. The MMS must support version 2 or higher of the System Backup to Tape (SBT) Interface. To see a list of Oracle partners who use the SBT interface, go to
http://www.oracle.com/technetwork/database/features/availability/bsp-088814.html
This interface was originally developed by Oracle as a standard way for third party backup media providers to integrate their solutions with RMAN, the Oracle Database Recovery Manager and Backup tool. MySQL Enterprise Backup (MEB) 3.6 now supports this interface so if you are already using a media management solution like Oracle Secure Backup (OSB) or Netbackup, you can stream directly from MEB to the MMS. Some MySQL sites have hundreds of servers backed up to many physical or virtual tape devices and can produce thousands of backup tapes on a regular basis. These tapes may be maintained at different locations for various time periods. An MMS gives you better control over the process and may add capabilities like backup policies, tape vaulting control, and tape encryption. To interface with a media management system like OSB, you once again build on the single file backup command, but use a prefix on your image filename. This special prefix, sbt: sends the backup to the MMS instead of a local file
–backup-image=sbt:name
Your mysqlbackup command would look something like this:
mysqlbackup
–port=3306
–protocol=tcp
–user=lynn
–password
–backup-image=sbt:backup-hr-2011-09-06
–backup-dir=/backup
backup-to-image
Mysqlbackup defaults to the normal operating system paths and environment variables to locate the Secure Backup to Tape (SBT) library it needs to accomplish this. What if you have more than one MMS? No problem. You just specify the –sbt-lib-path parameter of the mysqlbackup command with the correct path and filename for your MMS.
Step 4: Encrypt your Backup Tapes using your Media Management System
Encryption adds an extra layer of security to tapes in case they are misplaced or stolen. MEB does not in itself do encryption, but you can use OSB or another encryption enabled MMS to encrypt MySQL tapes. In OSB you control encryption by defining a storage selector that applies specific features (like encryption) to a particular backup. You define the storage selector only once. Then OSB will automatically select the appropriate database backup storage selector for the backup job.
In OSB a storage selector contains the database name, the hostname, and the name of the media to use for backups. Optionally you can indicate whether encryption should be used, the type of backup (full, incremental) and restrictions on tape devices. When OSB receives a backup command through MEB, it examines the defined database backup storage selectors to determine whether a backup storage selector matches the attributes of the just received backup job. OSB uses the database name and backup type (ie full or incremental etc) to select the most appropriate backup storage selector.
OSB storage selectors are created either through Oracle Enterprise Manager or the command line interface to OSB. In Oracle Enterprise Manager, storage selectors are defined from within the Maintenance tab under Backup Settings.
If you are not using Enterprise Manager, use the OSB command line interface (obtool) to define your storage selector. Here is an example of a command which creates a storage selector called MySQLworld with encryption for the database worlddb:
mkssel (stands for make storage selector)
–dbname worlddb
–host myserver2
–content full
–encryption on
MySQLworld
To restore MySQL data from tape , you use the –sbt-backup-name parameter as part of the restore operation.
A few notes about using MEB with OSB: You must pre-authorize user access for MySQL for the backup to work with OSB. For instructions, please see section 4.2.2 of the OSB Administrator’s Guide “Creating a Preauthorized Oracle Secure Backup User” http://download.oracle.com/docs/cd/E14812_01/doc/doc.103/e12834/osb_rman_backup.htm#BDCCCIIA
In this entry I’ve covered how to use some of the new features in MySQL Enterprise Backup to stream backups and interface with media management systems. Next week I will cover the new Backup Advisor in MySQL Enteprise Monitor.
Lynn Ferrante has worked with databases in the enterprise for her whole career at MySQL, Oracle, Sybase, and Ingres. She also worked on an open source project called GenMapp (University of California, San Francisco), and contributed to the development of many database applications in the fields of energy and environment
