Package org.apache.hadoop.hbase.backup

Class TestIncrementalBackupManager

java.lang.Object

org.apache.hadoop.hbase.backup.TestBackupBase

org.apache.hadoop.hbase.backup.TestIncrementalBackupManager

@Tag("org.apache.hadoop.hbase.testclassification.LargeTests")
public class TestIncrementalBackupManager
extends TestBackupBase

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.backup.TestBackupBase

TestBackupBase.FullTableBackupClientForTest, TestBackupBase.IncrementalTableBackupClientForTest

Field Summary

Fields inherited from class org.apache.hadoop.hbase.backup.TestBackupBase

autoRestoreOnFailure, BACKUP_REMOTE_ROOT_DIR, BACKUP_ROOT_DIR, conf1, conf2, famName, NB_ROWS_IN_BATCH, provider, qualName, secure, table1, table1_restore, table1Desc, table2, table2_restore, table3, table3_restore, table4, TEST_UTIL, TEST_UTIL2, useSecondCluster

Constructor Summary

Constructors

Constructor	Description
TestIncrementalBackupManager()

Method Summary

Modifier and Type	Method	Description
private static org.apache.hadoop.fs.Path	archivedWAL(org.apache.hadoop.fs.Path archiveDir, long ts)
private static void	rollAllLiveRegionServers()
private static List<String>	runIncrementalBackup(org.apache.hadoop.hbase.client.Connection conn, List<org.apache.hadoop.hbase.TableName> tables, String backupId)
static void	setUp()
private static org.apache.hadoop.fs.Path	splittingDir(org.apache.hadoop.fs.Path walRootDir, org.apache.hadoop.hbase.ServerName serverName)
private void	testCollectWALFiles(boolean separateOldLogDir)
void	testCollectWALFilesFromFlatOldWALDirectory()
void	testCollectWALFilesFromRegionServerDirectories()
void	testDeadServerKeepsBoundaryWhileOldWALIsStillSplitting()	A dead region server can keep an old WAL, already covered by its boundary, in its -splitting directory across several incremental backups.
void	testOlderWALsOfDeadServerAreNotSkippedWhenNewerWALIsArchivedFirst()	WAL splitting archives a dead region server's WALs in parallel, so a newer WAL can be archived while older ones are still in the -splitting directory.
void	testOutOfOrderArchivedWALDoesNotSkipOlderWAL()	WALs can be archived out of order, so a region server that took part in the log roll can have an archived WAL newer than its roll result while an older WAL is still in the WALs directory.
void	testWALOfDeadServerStillSplittingIsBackedUpAfterArchiving()	A dead region server's WALs stay in its -splitting directory until WAL splitting finishes.
private static String	walName(org.apache.hadoop.hbase.ServerName serverName, long ts)

Methods inherited from class org.apache.hadoop.hbase.backup.TestBackupBase

backupTables, backupTables, checkFailed, checkSucceeded, createBackupRequest, createBackupRequest, createBackupRequest, createTables, deleteContinuousBackupReplicationPeerIfExists, dumpBackupDir, fullTableBackup, getBackupAdmin, getLatestBackupManifest, getListOfWALFiles, incrementalTableBackup, insertIntoTable, loadTable, setUpHelper, tearDown, toList

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TestIncrementalBackupManager

public TestIncrementalBackupManager()

Method Details

setUp

@BeforeAll
public static void setUp()
                  throws Exception

Throws:

Exception

testCollectWALFilesFromRegionServerDirectories

@Test
public void testCollectWALFilesFromRegionServerDirectories()
                                                    throws Exception

Throws:

Exception

testCollectWALFilesFromFlatOldWALDirectory

@Test
public void testCollectWALFilesFromFlatOldWALDirectory()
                                                throws Exception

Throws:

Exception

testCollectWALFiles

private void testCollectWALFiles(boolean separateOldLogDir)
                          throws Exception

Throws:

Exception

testOutOfOrderArchivedWALDoesNotSkipOlderWAL

@Test
public void testOutOfOrderArchivedWALDoesNotSkipOlderWAL()
                                                  throws Exception

WALs can be archived out of order, so a region server that took part in the log roll can have an archived WAL newer than its roll result while an older WAL is still in the WALs directory. The newer archived WAL must be deferred to a later backup instead of being backed up twice, and the older WAL must still end up in a backup.

Throws:

Exception

testWALOfDeadServerStillSplittingIsBackedUpAfterArchiving

@Test
public void testWALOfDeadServerStillSplittingIsBackedUpAfterArchiving()
                                                               throws Exception

A dead region server's WALs stay in its -splitting directory until WAL splitting finishes. An incremental backup that runs during the split must not lose them once they are archived.

Throws:

Exception

testOlderWALsOfDeadServerAreNotSkippedWhenNewerWALIsArchivedFirst

@Test
public void testOlderWALsOfDeadServerAreNotSkippedWhenNewerWALIsArchivedFirst()
                                                                       throws Exception

WAL splitting archives a dead region server's WALs in parallel, so a newer WAL can be archived while older ones are still in the -splitting directory. Backing up the newer WAL must not move the boundary past the older ones.

Throws:

Exception

testDeadServerKeepsBoundaryWhileOldWALIsStillSplitting

@Test
public void testDeadServerKeepsBoundaryWhileOldWALIsStillSplitting()
                                                            throws Exception

A dead region server can keep an old WAL, already covered by its boundary, in its -splitting directory across several incremental backups. Its boundary must survive those backups, or a later backup would include that WAL again and could bring back deleted data.

Throws:

Exception

splittingDir

private static org.apache.hadoop.fs.Path splittingDir(org.apache.hadoop.fs.Path walRootDir,
 org.apache.hadoop.hbase.ServerName serverName)

archivedWAL

private static org.apache.hadoop.fs.Path archivedWAL(org.apache.hadoop.fs.Path archiveDir,
 long ts)

walName

private static String walName(org.apache.hadoop.hbase.ServerName serverName,
 long ts)

rollAllLiveRegionServers

private static void rollAllLiveRegionServers()
                                      throws Exception

Throws:

Exception

runIncrementalBackup

private static List<String> runIncrementalBackup(org.apache.hadoop.hbase.client.Connection conn,
 List<org.apache.hadoop.hbase.TableName> tables,
 String backupId)
                                          throws Exception

Throws:

Exception