Package org.apache.hadoop.hbase.backup

Class IntegrationTestBackupRestoreBase

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.backup.IntegrationTestBackupRestoreBase

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

IntegrationTestBackupRestore, IntegrationTestContinuousBackupRestore

public abstract class IntegrationTestBackupRestoreBase
extends IntegrationTestBase

An abstract base class that is used to run backup, restore, and delete integration tests. This class performs both full backups and incremental backups. Both continuous backup and non-continuous backup test cases are supported. The number of incremental backups performed depends on the number of iterations defined by the user. The class performs the backup/restore in a separate thread, where one thread is created per table. The number of tables is user-defined, along with other various configurations.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	IntegrationTestBackupRestoreBase.BackupAndRestoreThread

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field	Description
protected String	backupRootDir
protected static final String	COLUMN_NAME
protected static String	DEFAULT_BACKUP_ROOT_DIR
protected static final int	DEFAULT_NUM_ITERATIONS
protected static final int	DEFAULT_NUMBER_OF_TABLES
protected static final int	DEFAULT_REGION_COUNT
protected static final int	DEFAULT_REGIONSERVER_COUNT
protected static final int	DEFAULT_ROWS_IN_ITERATION
protected org.apache.hadoop.fs.FileSystem	fs
protected static final org.slf4j.Logger	LOG
protected static final String	NUM_ITERATIONS_KEY
protected static final String	NUMBER_OF_TABLES_KEY
protected int	numIterations
protected int	numTables
protected static final String	REGION_COUNT_KEY
protected int	regionsCountPerServer
protected static final String	REGIONSERVER_COUNT_KEY
protected int	regionServerCount
protected static final String	ROWS_PER_ITERATION_KEY
protected int	rowsInIteration
protected static final long	SLEEP_TIME_DEFAULT
protected static final String	SLEEP_TIME_KEY
protected long	sleepTime
protected static org.apache.hadoop.hbase.TableName[]	tableNames

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor	Description
IntegrationTestBackupRestoreBase()

Method Summary

Modifier and Type	Method	Description
protected void	addOptions()
private boolean	areAllTimestampsPastThreshold(Map<org.apache.hadoop.hbase.ServerName,Long> timestamps, long threshold, org.apache.hadoop.hbase.TableName tableName)	Checks if all timestamps in a map with ServerName and timestamp key-values pairs are past the provided timestamp threshold.
private String	backup(org.apache.hadoop.hbase.backup.BackupRequest request, org.apache.hadoop.hbase.backup.BackupAdmin client, List<String> backupIds)
protected boolean	checkSucceeded(String backupId)	Returns status of backup
private void	cleanUpBackupDir()
protected void	createAndSetBackupWalDir()	Creates a directory specified by backupWALDir and sets this directory to CONF_CONTINUOUS_BACKUP_WAL_DIR in the configuration.
org.apache.hadoop.hbase.backup.RestoreRequest	createRestoreRequest(String backupRootDir, String backupId, boolean check, org.apache.hadoop.hbase.TableName[] fromTables, org.apache.hadoop.hbase.TableName[] toTables, boolean isOverwrite)	Get restore request.
private void	createTable(org.apache.hadoop.hbase.TableName tableName)
protected void	createTables(String tableBaseName)
private void	delete(String[] backupIds, org.apache.hadoop.hbase.backup.BackupAdmin client)
private void	deleteMostRecentIncrementalBackup(List<String> backupIds, org.apache.hadoop.hbase.backup.BackupAdmin client)	Performs the delete command for the most recently taken incremental backup, and also removes this backup from the list of backup IDs.
private void	deleteTablesIfAny()
private String[]	getAllIncrementalBackupIds(List<String> backupIds)	Uses the list of all backup IDs to return a sublist of incremental backup IDs.
private org.apache.hadoop.hbase.backup.BackupInfo	getBackupInfo(String backupId)
protected Set<String>	getColumnFamilies()	Provides the name of the CFs that are protected from random Chaos monkey activity (alter)
private long	getLatestPutTimestamp(org.apache.hadoop.hbase.client.Table table)	Scans the backed-up table and returns the timestamp (ms) of the latest Put operation on the table.
org.apache.hadoop.hbase.TableName	getTablename()	Provides the name of the table that is protected from random Chaos monkey activity
protected void	initializeTestParameters()	This method is useful for ensuring important test parameters are initialized when running integration tests in IntelliJ or in the command line with Maven.
private void	loadData(org.apache.hadoop.hbase.TableName table, int numRows)
private void	merge(String[] backupIds, org.apache.hadoop.hbase.backup.BackupAdmin client)
protected void	processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)
private void	removeNonexistentBackups(List<String> backupIds)
private void	restore(org.apache.hadoop.hbase.backup.RestoreRequest request, org.apache.hadoop.hbase.backup.BackupAdmin client)
private void	restoreTableAndVerifyRowCount(org.apache.hadoop.hbase.client.Connection conn, org.apache.hadoop.hbase.backup.BackupAdmin client, org.apache.hadoop.hbase.TableName table, String backupId, long expectedRows)	Restores a table using the provided backup ID and ensure the table has the correct row count after
protected void	runTestMulti(boolean isContinuousBackupEnabled)	This is the main driver method used by tests that extend this abstract base class.
private void	runTestSingle(org.apache.hadoop.hbase.TableName tableName, boolean isContinuousBackupEnabled)	This method is what performs the actual backup, restore, merge, and delete operations.
void	setUpCluster()
void	setUpMonkey()
void	tearDown()
private void	verifyAllBackupsExist(List<String> backupIds)	Verifies all backups in the list of backup IDs actually exist on the filesystem.
private void	verifyBackupDoesNotExist(String backupId)
private void	verifyBackupExistenceAfterMerge(List<String> backupIds)	Iterates through the list of backups and verifies the full backup and latest incremental backup still exist, while also verifying all other backups no longer exist.
private void	verifyBackupExists(String backupId)
private void	verifyBackupWALFiles(org.apache.hadoop.fs.Path walPartitionDir)	Verifies the WAL partition directory contains a backup WAL file.
private void	verifyNoBackupsExist(List<String> backupIds)	Verifies zero backups in the list of backup IDs exist on the filesystem.
private void	verifySnapshotExists(org.apache.hadoop.hbase.TableName tableName, String backupId)	Verifies a snapshot's "data.manifest" file exists after a full backup has been performed for a table.
private org.apache.hadoop.fs.Path	verifyWALPartitionDirExists(org.apache.hadoop.fs.Path backupWALs)	Waits for a WAL partition directory to exist inside the backup WAL directory.
private org.apache.hadoop.fs.Path	verifyWALsDirectoryExists()	Verifies the .../backupWALDir/WALs directory exists and returns its Path
private void	waitForCheckpointTimestampsToUpdate(org.apache.hadoop.hbase.client.Connection conn, long latestPutTimestamp, org.apache.hadoop.hbase.TableName tableName)	Waits for the replication checkpoint timestamp of each region server to meet or pass the timestamp of the latest Put operation on the backed-up table.

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, runTestFromCommandLine, setUp, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

protected static final org.slf4j.Logger LOG

NUMBER_OF_TABLES_KEY

protected static final String NUMBER_OF_TABLES_KEY

See Also:

• Constant Field Values

COLUMN_NAME

protected static final String COLUMN_NAME

See Also:

• Constant Field Values

REGION_COUNT_KEY

protected static final String REGION_COUNT_KEY

See Also:

• Constant Field Values

REGIONSERVER_COUNT_KEY

protected static final String REGIONSERVER_COUNT_KEY

See Also:

• Constant Field Values

ROWS_PER_ITERATION_KEY

protected static final String ROWS_PER_ITERATION_KEY

See Also:

• Constant Field Values

NUM_ITERATIONS_KEY

protected static final String NUM_ITERATIONS_KEY

See Also:

• Constant Field Values

DEFAULT_REGION_COUNT

protected static final int DEFAULT_REGION_COUNT

See Also:

• Constant Field Values

DEFAULT_REGIONSERVER_COUNT

protected static final int DEFAULT_REGIONSERVER_COUNT

See Also:

• Constant Field Values

DEFAULT_NUMBER_OF_TABLES

protected static final int DEFAULT_NUMBER_OF_TABLES

See Also:

• Constant Field Values

DEFAULT_NUM_ITERATIONS

protected static final int DEFAULT_NUM_ITERATIONS

See Also:

• Constant Field Values

DEFAULT_ROWS_IN_ITERATION

protected static final int DEFAULT_ROWS_IN_ITERATION

See Also:

• Constant Field Values

SLEEP_TIME_KEY

protected static final String SLEEP_TIME_KEY

See Also:

• Constant Field Values

SLEEP_TIME_DEFAULT

protected static final long SLEEP_TIME_DEFAULT

See Also:

• Constant Field Values

DEFAULT_BACKUP_ROOT_DIR

protected static String DEFAULT_BACKUP_ROOT_DIR

rowsInIteration

protected int rowsInIteration

regionsCountPerServer

protected int regionsCountPerServer

regionServerCount

protected int regionServerCount

numIterations

protected int numIterations

numTables

protected int numTables

sleepTime

protected long sleepTime

tableNames

protected static org.apache.hadoop.hbase.TableName[] tableNames

fs

protected org.apache.hadoop.fs.FileSystem fs

backupRootDir

protected String backupRootDir

Constructor Details

IntegrationTestBackupRestoreBase

public IntegrationTestBackupRestoreBase()

Method Details

tearDown

@AfterEach
public void tearDown()
              throws IOException

Throws:

IOException

setUpMonkey

public void setUpMonkey()
                 throws Exception

Overrides:

setUpMonkey in class IntegrationTestBase

Throws:

Exception

deleteTablesIfAny

private void deleteTablesIfAny()
                        throws IOException

Throws:

IOException

createTables

protected void createTables(String tableBaseName)
                     throws Exception

Throws:

Exception

createAndSetBackupWalDir

protected void createAndSetBackupWalDir()
                                 throws IOException

Creates a directory specified by backupWALDir and sets this directory to CONF_CONTINUOUS_BACKUP_WAL_DIR in the configuration.

Throws:

IOException

cleanUpBackupDir

private void cleanUpBackupDir()
                       throws IOException

Throws:

IOException

runTestMulti

protected void runTestMulti(boolean isContinuousBackupEnabled)

This is the main driver method used by tests that extend this abstract base class. This method starts one backup and restore thread per table.

Parameters:

isContinuousBackupEnabled - Boolean flag used to specify if the backups should have continuous backup enabled.

runTestSingle

private void runTestSingle(org.apache.hadoop.hbase.TableName tableName,
 boolean isContinuousBackupEnabled)
                    throws IOException,
InterruptedException

This method is what performs the actual backup, restore, merge, and delete operations. This method is run in a separate thread. It first performs a full backup. After, it iteratively performs a series of incremental backups and restores. Later, it deletes the backups.

Parameters:

tableName - The table the backups are performed on

isContinuousBackupEnabled - Boolean flag used to indicate if the backups should have continuous backup enabled.

Throws:

IOException

InterruptedException

createTable

private void createTable(org.apache.hadoop.hbase.TableName tableName)
                  throws Exception

Throws:

Exception

loadData

private void loadData(org.apache.hadoop.hbase.TableName table,
 int numRows)
               throws IOException

Throws:

IOException

backup

private String backup(org.apache.hadoop.hbase.backup.BackupRequest request,
 org.apache.hadoop.hbase.backup.BackupAdmin client,
 List<String> backupIds)
               throws IOException

Throws:

IOException

restore

private void restore(org.apache.hadoop.hbase.backup.RestoreRequest request,
 org.apache.hadoop.hbase.backup.BackupAdmin client)
              throws IOException

Throws:

IOException

merge

private void merge(String[] backupIds,
 org.apache.hadoop.hbase.backup.BackupAdmin client)
            throws IOException

Throws:

IOException

delete

private void delete(String[] backupIds,
 org.apache.hadoop.hbase.backup.BackupAdmin client)
             throws IOException

Throws:

IOException

verifySnapshotExists

private void verifySnapshotExists(org.apache.hadoop.hbase.TableName tableName,
 String backupId)
                           throws IOException

Verifies a snapshot's "data.manifest" file exists after a full backup has been performed for a table. The "data.manifest" file's path will look like the following: .../backupRootDir/backup_1760572298945/default/TABLE_NAME/.hbase-snapshot/ snapshot_1760572306407_default_TABLE_NAME/data.manifest

Throws:

IOException

verifyWALsDirectoryExists

private org.apache.hadoop.fs.Path verifyWALsDirectoryExists()
                                                     throws IOException

Verifies the .../backupWALDir/WALs directory exists and returns its Path

Throws:

IOException

verifyWALPartitionDirExists

private org.apache.hadoop.fs.Path verifyWALPartitionDirExists(org.apache.hadoop.fs.Path backupWALs)
                                                       throws IOException,
InterruptedException

Waits for a WAL partition directory to exist inside the backup WAL directory. The directory should be something like: .../backupWALDir/WALs/2025-10-17. The directory's existence is either eventually asserted, or an assertion error is thrown if it does not exist past the wait deadline. This verification is to be used for full backups with continuous backup enabled.

Parameters:

backupWALs - The directory that should contain the partition directory. i.e. .../backupWALDir/WALs

Returns:

The Path to the WAL partition directory

Throws:

IOException

InterruptedException

verifyBackupWALFiles

private void verifyBackupWALFiles(org.apache.hadoop.fs.Path walPartitionDir)
                           throws IOException

Verifies the WAL partition directory contains a backup WAL file. The WAL file's path will look something like the following: .../backupWALDir/WALs/2025-10-17/wal_file.1760738249595.1880be89-0b69-4bad-8d0e-acbf25c63b7e

Parameters:

walPartitionDir - The date directory for a backup WAL i.e. .../backupWALDir/WALs/2025-10-17

Throws:

IOException

areAllTimestampsPastThreshold

private boolean areAllTimestampsPastThreshold(Map<org.apache.hadoop.hbase.ServerName,Long> timestamps,
 long threshold,
 org.apache.hadoop.hbase.TableName tableName)

Checks if all timestamps in a map with ServerName and timestamp key-values pairs are past the provided timestamp threshold.

Parameters:

timestamps - Map with ServerName and timestamp key-value pairs

threshold - Timestamp to check all timestamp values in the map against

Returns:

True if all timestamps values in the map have met or are path the timestamp threshold; False otherwise

waitForCheckpointTimestampsToUpdate

private void waitForCheckpointTimestampsToUpdate(org.apache.hadoop.hbase.client.Connection conn,
 long latestPutTimestamp,
 org.apache.hadoop.hbase.TableName tableName)
                                          throws IOException,
InterruptedException

Waits for the replication checkpoint timestamp of each region server to meet or pass the timestamp of the latest Put operation on the backed-up table.

Parameters:

conn - Minicluster connection

latestPutTimestamp - Timestamp of the latest Put operation on the backed-up table

Throws:

IOException

InterruptedException

getLatestPutTimestamp

private long getLatestPutTimestamp(org.apache.hadoop.hbase.client.Table table)
                            throws IOException

Scans the backed-up table and returns the timestamp (ms) of the latest Put operation on the table.

Parameters:

table - The backed-up table to scan

Returns:

Timestamp of the most recent Put on the backed-up table

Throws:

IOException

restoreTableAndVerifyRowCount

private void restoreTableAndVerifyRowCount(org.apache.hadoop.hbase.client.Connection conn,
 org.apache.hadoop.hbase.backup.BackupAdmin client,
 org.apache.hadoop.hbase.TableName table,
 String backupId,
 long expectedRows)
                                    throws IOException

Restores a table using the provided backup ID and ensure the table has the correct row count after

Throws:

IOException

getAllIncrementalBackupIds

private String[] getAllIncrementalBackupIds(List<String> backupIds)

Uses the list of all backup IDs to return a sublist of incremental backup IDs. This method assumes the first backup in the list is a full backup, followed by incremental backups.

checkSucceeded

protected boolean checkSucceeded(String backupId)
                          throws IOException

Returns status of backup

Throws:

IOException

getBackupInfo

private org.apache.hadoop.hbase.backup.BackupInfo getBackupInfo(String backupId)
                                                         throws IOException

Throws:

IOException

createRestoreRequest

public org.apache.hadoop.hbase.backup.RestoreRequest createRestoreRequest(String backupRootDir,
 String backupId,
 boolean check,
 org.apache.hadoop.hbase.TableName[] fromTables,
 org.apache.hadoop.hbase.TableName[] toTables,
 boolean isOverwrite)

Get restore request.

Parameters:

backupId - backup ID

check - check the backup

fromTables - table names to restore from

toTables - new table names to restore to

isOverwrite - overwrite the table(s)

Returns:

an instance of RestoreRequest

verifyBackupExistenceAfterMerge

private void verifyBackupExistenceAfterMerge(List<String> backupIds)
                                      throws IOException

Iterates through the list of backups and verifies the full backup and latest incremental backup still exist, while also verifying all other backups no longer exist. This method is meant to be run after all incremental backups have been merged.

Throws:

IOException

removeNonexistentBackups

private void removeNonexistentBackups(List<String> backupIds)
                               throws IOException

Throws:

IOException

deleteMostRecentIncrementalBackup

private void deleteMostRecentIncrementalBackup(List<String> backupIds,
 org.apache.hadoop.hbase.backup.BackupAdmin client)
                                        throws IOException

Performs the delete command for the most recently taken incremental backup, and also removes this backup from the list of backup IDs.

Throws:

IOException

verifyAllBackupsExist

private void verifyAllBackupsExist(List<String> backupIds)
                            throws IOException

Verifies all backups in the list of backup IDs actually exist on the filesystem.

Throws:

IOException

verifyNoBackupsExist

private void verifyNoBackupsExist(List<String> backupIds)
                           throws IOException

Verifies zero backups in the list of backup IDs exist on the filesystem.

Throws:

IOException

verifyBackupExists

private void verifyBackupExists(String backupId)
                         throws IOException

Throws:

IOException

verifyBackupDoesNotExist

private void verifyBackupDoesNotExist(String backupId)
                               throws IOException

Throws:

IOException

setUpCluster

public void setUpCluster()
                  throws Exception

Specified by:

setUpCluster in class IntegrationTestBase

Throws:

Exception

getTablename

public org.apache.hadoop.hbase.TableName getTablename()

Description copied from class: IntegrationTestBase

Provides the name of the table that is protected from random Chaos monkey activity

Specified by:

getTablename in class IntegrationTestBase

Returns:

table to not delete.

getColumnFamilies

protected Set<String> getColumnFamilies()

Description copied from class: IntegrationTestBase

Provides the name of the CFs that are protected from random Chaos monkey activity (alter)

Specified by:

getColumnFamilies in class IntegrationTestBase

Returns:

set of cf names to protect.

initializeTestParameters

protected void initializeTestParameters()

This method is useful for ensuring important test parameters are initialized when running integration tests in IntelliJ or in the command line with Maven. Tests executed in the command line via the bin/hbase command have their parameters set in IntegrationTestBase.processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine)

addOptions

protected void addOptions()

Overrides:

addOptions in class IntegrationTestBase

processOptions

protected void processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)

Overrides:

processOptions in class IntegrationTestBase