Package org.apache.hadoop.hbase.backup.impl

Class IncrementalTableBackupClient

java.lang.Object

org.apache.hadoop.hbase.backup.impl.TableBackupClient

org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient

@Private
public class IncrementalTableBackupClient
extends TableBackupClient

Incremental backup implementation. See the execute method.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.backup.impl.TableBackupClient

TableBackupClient.Stage

Field Summary

Fields

Modifier and Type	Field	Description
private static final org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hbase.backup.impl.TableBackupClient

BACKUP_CLIENT_IMPL_CLASS, BACKUP_TEST_MODE_STAGE, backupId, backupInfo, backupManager, conf, conn, fs, newTimestamps, tableList

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	IncrementalTableBackupClient()
	IncrementalTableBackupClient(Connection conn, String backupId, BackupRequest request)

Method Summary

Modifier and Type	Method	Description
private static boolean	areCfsCompatible(ColumnFamilyDescriptor[] currentCfs, ColumnFamilyDescriptor[] backupCfs)
protected void	convertWALsToHFiles()
protected void	deleteBulkLoadDirectory()
void	execute()	Backup request execution.
protected List<String>	filterMissingFiles(List<String> incrBackupFileList)
protected org.apache.hadoop.fs.Path	getBulkOutputDir()
protected org.apache.hadoop.fs.Path	getBulkOutputDirForTable(TableName table)
private Map<TableName,String>	getFullBackupIds()
protected static int	getIndex(TableName tbl, List<TableName> sTableList)
private org.apache.hadoop.fs.Path	getTargetDirForTable(TableName table)
protected List<BulkLoad>	handleBulkLoad(List<TableName> tablesToBackup)	Reads bulk load records from backup table, iterates through the records and forms the paths for bulk loaded hfiles.
private void	incrementalCopyBulkloadHFiles(org.apache.hadoop.fs.FileSystem tgtFs, TableName tn)
protected void	incrementalCopyHFiles(String[] files, String backupDest)
protected boolean	isActiveWalPath(org.apache.hadoop.fs.Path p)	Check if a given path is belongs to active WAL directory
private void	mergeSplitAndCopyBulkloadedHFiles(List<String> activeFiles, List<String> archiveFiles, TableName tn, org.apache.hadoop.fs.FileSystem tgtFs)
private void	mergeSplitAndCopyBulkloadedHFiles(List<String> files, TableName tn, org.apache.hadoop.fs.FileSystem tgtFs)
private void	setupRegionLocator()
protected boolean	tableExists(TableName table, Connection conn)
void	updateFileLists(List<String> activeFiles, List<String> archiveFiles)
private void	verifyCfCompatibility(Set<TableName> tables, Map<TableName,String> tablesToFullBackupId)	Verifies that the current table descriptor CFs matches the descriptor CFs of the last full backup for the tables.
protected void	walToHFiles(List<String> dirPaths, List<String> tableList)

Methods inherited from class org.apache.hadoop.hbase.backup.impl.TableBackupClient

addManifest, beginBackup, cleanupAndRestoreBackupSystem, cleanupExportSnapshotLog, cleanupTargetDir, completeBackup, deleteSnapshots, failBackup, failStageIf, getAncestors, getMessage, getTestStage, init, obtainBackupMetaDataStr

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

Constructor Details

IncrementalTableBackupClient

protected IncrementalTableBackupClient()

IncrementalTableBackupClient

public IncrementalTableBackupClient(Connection conn,
 String backupId,
 BackupRequest request)
                             throws IOException

Throws:

IOException

Method Details

filterMissingFiles

protected List<String> filterMissingFiles(List<String> incrBackupFileList)
                                   throws IOException

Throws:

IOException

isActiveWalPath

protected boolean isActiveWalPath(org.apache.hadoop.fs.Path p)

Check if a given path is belongs to active WAL directory

Parameters:

p - path

Returns:

true, if yes

getIndex

protected static int getIndex(TableName tbl,
 List<TableName> sTableList)

handleBulkLoad

protected List<BulkLoad> handleBulkLoad(List<TableName> tablesToBackup)
                                 throws IOException

Reads bulk load records from backup table, iterates through the records and forms the paths for bulk loaded hfiles. Copies the bulk loaded hfiles to backup destination. This method does NOT clean up the entries in the bulk load system table. Those entries should not be cleaned until the backup is marked as complete.

Parameters:

tablesToBackup - list of tables to be backed up

Throws:

IOException

mergeSplitAndCopyBulkloadedHFiles

private void mergeSplitAndCopyBulkloadedHFiles(List<String> activeFiles,
 List<String> archiveFiles,
 TableName tn,
 org.apache.hadoop.fs.FileSystem tgtFs)
                                        throws IOException

Throws:

IOException

mergeSplitAndCopyBulkloadedHFiles

private void mergeSplitAndCopyBulkloadedHFiles(List<String> files,
 TableName tn,
 org.apache.hadoop.fs.FileSystem tgtFs)
                                        throws IOException

Throws:

IOException

updateFileLists

public void updateFileLists(List<String> activeFiles,
 List<String> archiveFiles)
                     throws IOException

Throws:

IOException

execute

public void execute()
             throws IOException,
ColumnFamilyMismatchException

Description copied from class: TableBackupClient

Backup request execution.

Specified by:

execute in class TableBackupClient

Throws:

IOException - If the execution of the backup fails

ColumnFamilyMismatchException - If the column families of the current table do not match the column families for the last full backup. In which case, a full backup should be taken

incrementalCopyHFiles

protected void incrementalCopyHFiles(String[] files,
 String backupDest)
                              throws IOException

Throws:

IOException

deleteBulkLoadDirectory

protected void deleteBulkLoadDirectory()
                                throws IOException

Throws:

IOException

convertWALsToHFiles

protected void convertWALsToHFiles()
                            throws IOException

Throws:

IOException

tableExists

protected boolean tableExists(TableName table,
 Connection conn)
                       throws IOException

Throws:

IOException

walToHFiles

protected void walToHFiles(List<String> dirPaths,
 List<String> tableList)
                    throws IOException

Throws:

IOException

incrementalCopyBulkloadHFiles

private void incrementalCopyBulkloadHFiles(org.apache.hadoop.fs.FileSystem tgtFs,
 TableName tn)
                                    throws IOException

Throws:

IOException

getBulkOutputDirForTable

protected org.apache.hadoop.fs.Path getBulkOutputDirForTable(TableName table)

getBulkOutputDir

protected org.apache.hadoop.fs.Path getBulkOutputDir()

getTargetDirForTable

private org.apache.hadoop.fs.Path getTargetDirForTable(TableName table)

setupRegionLocator

private void setupRegionLocator()
                         throws IOException

Throws:

IOException

getFullBackupIds

private Map<TableName,String> getFullBackupIds()
                                        throws IOException

Throws:

IOException

verifyCfCompatibility

private void verifyCfCompatibility(Set<TableName> tables,
 Map<TableName,String> tablesToFullBackupId)
                            throws IOException,
ColumnFamilyMismatchException

Verifies that the current table descriptor CFs matches the descriptor CFs of the last full backup for the tables. This ensures CF compatibility across incremental backups. If a mismatch is detected, a full table backup should be taken, rather than an incremental one

Throws:

IOException

ColumnFamilyMismatchException

areCfsCompatible

private static boolean areCfsCompatible(ColumnFamilyDescriptor[] currentCfs,
 ColumnFamilyDescriptor[] backupCfs)