public class DbBackup
extends java.lang.Object
Backing up without DbBackup
Because JE has an append only log file architecture, it is always possible to do a hot backup without the use of DbBackup by copying all log files (.jdb files) to your archival location. As long as the log files are copied in alphabetical order, (numerical in effect) and all log files are copied, the environment can be successfully backed up without any need to stop database operations or background activity. This means that your backup operation must do a loop to check for the creation of new log files before deciding that the backup is finished. For example:
time files in activity environment t0 000000001.jdb Backup starts copying file 1 000000003.jdb 000000004.jdb t1 000000001.jdb JE log cleaner migrates portion of file 3 to newly 000000004.jdb created file 5 and deletes file 3. Backup finishes 000000005.jdb file 1, starts copying file 4. Backup MUST include file 5 for a consistent backup! t2 000000001.jdb Backup finishes copying file 4, starts and 000000004.jdb finishes file 5, has caught up. Backup ends. 000000005.jdb
In the example above, the backup operation must be sure to copy file 5, which came into existence after the backup had started. If the backup stopped operations at file 4, the backup set would include only file 1 and 4, omitting file 3, which would be an inconsistent set.
Also note that log file 5 may not have filled up before it was copied to archival storage. On the next backup, there might be a newer, larger version of file 5, and that newer version should replace the older file 5 in archive storage.
Using the approach above, as opposed to using DbBackup, will copy all files
including reserved files
as well
as active files
. A large number of
reserved files may be present in an HA Environment, and they are essentially
wasted space in a backup. Using DbBackup is strongly recommended for this
reason, as well as to reduce the complexity of file copying.
Backing up with DbBackup
DbBackup helps simplify application backup by defining the set of active files
that must be copied for each
backup operation. If the environment directory has read/write protection,
the application must pass DbBackup an open, read/write environment handle.
When entering backup mode, JE determines the set of active files needed for a consistent backup, and freezes all changes to those files. The application can copy that defined set of files and finish operation without checking for the ongoing creation of new files. Also, there will be no need to check for a newer version of the last file on the next backup.
In the example above, if DbBackup was used at t0, the application would only have to copy files 1, 3 and 4 to back up. On a subsequent backup, the application could start its copying at file 5. There would be no need to check for a newer version of file 4.
When it is important to minimize the time that it takes to recover using a
backup, a checkpoint should be performed immediately before calling startBackup()
. This will reduce recovery time when opening the environment
with the restored log files. A checkpoint is performed explicitly by
calling Environment.checkpoint(com.sleepycat.je.CheckpointConfig)
using a config object for which
setForce(true)
has been called.
Performing simple/full backups
The following examples shows how to perform a full backup. A checkpoint is performed to minimize recovery time.
void myBackup(Environment env, File destDir) { DbBackup backupHelper = new DbBackup(env); // Optional: Do a checkpoint to reduce recovery time after a restore. env.checkpoint(new CheckpointConfig().setForce(true)); // Start backup, find out what needs to be copied. backupHelper.startBackup(); try { // Copy the necessary files to archival storage. String[] filesToCopy = backupHelper.getLogFilesInBackupSet(); myCopyFiles(env, backupHelper, filesToCopy, destDir); } finally { // Remember to exit backup mode, or the JE cleaner cannot delete // log files and disk usage will grow without bounds. backupHelper.endBackup(); } } void myCopyFiles( Environment env, DbBackup backupHelper, String[] filesToCopy, File destDir) { for (String fileName : filesToCopy) { // Copy fileName to destDir. // SeeWhen copying files to the backup directory, it is critical that each file is verified before or during the copy. If a file is copied that is corrupt (due to an earlier disk failure that went unnoticed, for example), the backup will be invalid and provide a false sense of security.LogVerificationReadableByteChannel
and //LogVerificationInputStream
. .... // Remove protection to allow file to be deleted in order to reclaim // disk space. backupHelper.removeFileProtection(fileName); } }
The example here
shows how to implement
the myCopyFiles
method using LogVerificationInputStream
. A LogVerificationReadableByteChannel
could also be used for higher performance copying. A filter input stream is
used to verify the file efficiently as it is being read. If you choose to
use a script for copying files, the DbVerifyLog
command line tool
can be used instead.
Assuming that the full backup copied files into an empty directory, to restore you can simply copy these files back into another empty directory.
Always start with an empty directory as the destination for a full backup or
a restore, to ensure that no unused files are present. Unused files --
perhaps the residual of an earlier environment or an earlier backup -- will
take up space, and they will never be deleted by the JE log cleaner. Also
note that such files will not be used by JE for calculating utilization and
will not appear in the DbSpace
output.
Performing incremental backups
Incremental backups are used to reduce the number of files copied during each backup. Compared to a full backup, there are two additional pieces of information needed for an incremental backup: the number of the last file in the previous backup, and a list of the active files in the environment directory at the time of the current backup, i.e., the current snapshot. Their purpose is explained below.
The number of the last file in the previous backup is used to avoid copying files that are already present in the backup set. This file number must be obtained before beginning the backup, either by checking the backup archive, or getting this value from a stored location. For example, the last file number could be written to a special file in the backup set at the time of a backup, and then read from the special file before starting the next backup.
The list of files in the current snapshot, which should be obtained by
calling getLogFilesInSnapshot()
(after calling startBackup()
),
is used to avoid unused files after a restore, and may also be used to
reduce the size of the backup set. How to use this list is described below.
Some applications need the ability to restore to the point in time of any of the incremental backups that were made in the past, and other applications only need to restore to the point in time of the most recent backup. Accordingly, the list of current files (that is made at the time of the backup), should be used in one of two ways.
The following two examples shows how to perform an incremental backup. In the first example, the list of current files is used to delete files from the backup set that are no longer needed.
void myBackup(Environment env, File destDir) { // Get the file number of the last file in the previous backup. long lastFileInPrevBackup = ... DbBackup backupHelper = new DbBackup(env, lastFileInPrevBackup); // Optional: Do a checkpoint to reduce recovery time after a restore. env.checkpoint(new CheckpointConfig().setForce(true)); // Start backup, find out what needs to be copied. backupHelper.startBackup(); try { // Copy the necessary files to archival storage. String[] filesToCopy = backupHelper.getLogFilesInBackupSet(); myCopyFiles(env, backupHelper, filesToCopy, destDir); // Delete files that are no longer needed. // WARNING: This should only be done after copying all new files. String[] filesInSnapshot = backupHelper.getLogFilesInSnapshot(); myDeleteUnusedFiles(destDir, filesInSnapshot); // Update knowledge of last file saved in the backup set. lastFileInPrevBackup = backupHelper.getLastFileInBackupSet(); // Save lastFileInPrevBackup persistently here ... } finally { // Remember to exit backup mode, or the JE cleaner cannot delete // log files and disk usage will grow without bounds. backupHelper.endBackup(); } } void myDeleteUnusedFiles(File destDir, String[] filesInSnapshot) { // For each file in destDir that is NOT in filesInSnapshot, it should // be deleted from destDir to save disk space in the backup set, and to // ensure that unused files will not be restored. } See myCopyFiles further above.
When performing backups as shown in the first example above, to restore you can simply copy all files from the backup set into an empty directory.
In the second example below, the list of current files is saved with the backup set so it can be used during a restore. The backup set will effectively hold multiple backups that can be used to restore to different points in time.
void myBackup(Environment env, File destDir) { // Get the file number of the last file in the previous backup. long lastFileInPrevBackup = ... DbBackup backupHelper = new DbBackup(env, lastFileInPrevBackup); // Optional: Do a checkpoint to reduce recovery time after a restore. env.checkpoint(new CheckpointConfig().setForce(true)); // Start backup, find out what needs to be copied. backupHelper.startBackup(); try { // Copy the necessary files to archival storage. String[] filesToCopy = backupHelper.getLogFilesInBackupSet(); myCopyFiles(env, backupHelper, filesToCopy, destDir); // Save current list of files with backup data set. String[] filesInSnapshot = backupHelper.getLogFilesInSnapshot(); // Save filesInSnapshot persistently here ... // Update knowledge of last file saved in the backup set. lastFileInPrevBackup = backupHelper.getLastFileInBackupSet(); // Save lastFileInPrevBackup persistently here ... } finally { // Remember to exit backup mode, or the JE cleaner cannot delete // log files and disk usage will grow without bounds. backupHelper.endBackup(); } } See myCopyFiles further above.
When performing backups as shown in the second example above, to restore you must choose one of the file lists that was saved. You may choose the list written by the most recent backup, or a list written by an earlier backup. To restore, the files in the list should be copied into an empty destination directory.
As described in the sections above, the restore procedure is to copy the files from a backup set into an empty directory. Depending on the type of backup that was performed (see above), either all files from the backup set are copied, or only the files on a list that was created during the backup.
There is one additional consideration when performing a restore, under the following condition:
If the above condition holds, after copying the files an additional step is
needed. To enable the creation of future incremental backups using the
restored files, the EnvironmentConfig.ENV_RECOVERY_FORCE_NEW_FILE
parameter
should be set to true when opening the JE Environment for the first time
after the restore. When this parameter is set to true, the last .jdb file
restored will not be modified when opening the Environment, and the next
.jdb file will be created and will become the end-of-log file.
WARNING: When the above special condition is true and this property is not set to true when opening the environment for the first time after a restore, then the backup set that was restored may not be used as the basis for future incremental backups. If a future incremental backup were performed based on this backup set, it would be incomplete and data would be lost if that incremental backup were restored.
When JE 6.3 or later is used to create the backup, and the backup is created
in a read-write environment (the usual case), this extra step is
unnecessary. In this case, startBackup()
will have added an
"immutable file" marker to the last file in the backup and this will prevent
that file from being modified, just as if the
ENV_RECOVERY_FORCE_NEW_FILE
parameter were set to true.
Constructor and Description |
---|
DbBackup(Environment env)
Creates a DbBackup helper for a full backup.
|
DbBackup(Environment env,
long lastFileInPrevBackup)
Creates a DbBackup helper for an incremental backup.
|
Modifier and Type | Method and Description |
---|---|
void |
endBackup()
End backup mode, thereby re-enabling normal deletion of log files by the
JE log cleaner.
|
long |
getLastFileInBackupSet()
Can only be called in backup mode, after startBackup() has been called.
|
java.lang.String[] |
getLogFilesInBackupSet()
Get the minimum list of files that must be copied for this backup.
|
java.lang.String[] |
getLogFilesInBackupSet(long lastFileInPrevBackup)
Deprecated.
replaced by
getLogFilesInBackupSet() ; pass
lastFileInPrevBackup to the DbBackup(Environment,long)
constructor. |
java.lang.String[] |
getLogFilesInSnapshot()
Get the list of all active files that are needed for the environment at
the point of time when backup mode started, i.e., the current snapshot.
|
void |
removeFileProtection(java.lang.String fileName)
Removes protection for a file in the backup set.
|
void |
startBackup()
Start backup mode in order to determine the definitive backup set needed
at this point in time.
|
public DbBackup(Environment env) throws DatabaseException
This is equivalent to using DbBackup(Environment,long)
and
passing -1
for the lastFileInPrevBackup
parameter.
env
- with an open, valid environment handle. If the environment
directory has read/write permissions, the environment handle must be
configured for read/write.java.lang.IllegalArgumentException
- if the environment directory has
read/write permissions, but the environment handle is not configured for
read/write.DatabaseException
public DbBackup(Environment env, long lastFileInPrevBackup)
env
- with an open, valid environment handle. If the environment
directory has read/write permissions, the environment handle must be
configured for read/write.lastFileInPrevBackup
- the last file in the previous backup set
when performing an incremental backup, or -1
to perform a full
backup. The first file in this backup set will be the file following
lastFileInPrevBackup
.EnvironmentFailureException
- if an unexpected, internal or
environment-wide failure occurs.java.lang.IllegalArgumentException
- if the environment directory has
read/write permissions, but the environment handle is not configured for
read/write.public void startBackup() throws DatabaseException
This method determines the last file in the backup set, which is the last log file in the environment at this point in time. Following this method call, all new data will be written to other, new log files. In other words, the last file in the backup set will not be modified after this method returns.
WARNING: After calling this method, deletion of log files in
the backup set by the JE log cleaner will be disabled until endBackup()
is called. To prevent unbounded growth of disk usage, be
sure to call endBackup()
to re-enable log file deletion.
Additionally, the Environment can't be closed until endBackup() is
called.
LogOverwriteException
- if a replication
operation is overwriting log files. The backup can not proceed because
files may be invalid. The backup may be attempted at a later time.EnvironmentFailureException
- if an unexpected, internal or
environment-wide failure occurs.java.lang.IllegalStateException
- if a backup is already in progressDatabaseException
public void endBackup()
LogOverwriteException
- if a replication
operation has overwritten log files. Any copied files should be
considered invalid and discarded. The backup may be attempted at a
later time.EnvironmentFailureException
- if an unexpected,
internal or environment-wide failure occurs.java.lang.IllegalStateException
- if a backup has not been started.public long getLastFileInBackupSet()
java.lang.IllegalStateException
- if a backup has not been started.public java.lang.String[] getLogFilesInBackupSet()
The file numbers returned are in the range from the constructor
parameter lastFileInPrevBackup + 1
to the last log file at the
time that startBackup()
was called.
EnvironmentFailureException
- if an unexpected, internal or
environment-wide failure occurs.java.lang.IllegalStateException
- if a backup has not been started.@Deprecated public java.lang.String[] getLogFilesInBackupSet(long lastFileInPrevBackup)
getLogFilesInBackupSet()
; pass
lastFileInPrevBackup to the DbBackup(Environment,long)
constructor.lastFileInPrevBackup
- file number of last file copied in the last
backup session, obtained from getLastFileInBackupSet().EnvironmentFailureException
- if an unexpected, internal or
environment-wide failure occurs.java.lang.IllegalStateException
- if a backup has not been started.public java.lang.String[] getLogFilesInSnapshot()
When performing an incremental backup, this method is called to determine the files that would needed for a restore. As described in the examples at the top of this class, this list can be used to avoid unused files after a restore, and may also be used to reduce the size of the backup set.
When performing a full backup this method is normally not needed,
since in that case it returns the same set of files that is returned by
getLogFilesInBackupSet()
.
EnvironmentFailureException
- if an unexpected, internal or
environment-wide failure occurs.java.lang.IllegalStateException
- if a backup has not been started.public void removeFileProtection(java.lang.String fileName)
fileName
- a file name that has already been copied, in the format
returned by getLogFilesInBackupSet()
.Copyright (c) 2002, 2017 Oracle and/or its affiliates. All rights reserved.