Package org.apache.hadoop.hbase.backup

Class TestBackupOfflineRS

java.lang.Object

org.apache.hadoop.hbase.backup.TestBackupBase

org.apache.hadoop.hbase.backup.TestBackupOfflineRS

@Tag("org.apache.hadoop.hbase.testclassification.LargeTests")
public class TestBackupOfflineRS
extends TestBackupBase

Tests that WAL files from offline/inactive RegionServers are handled correctly during backup. Specifically verifies that WALs from an offline RS are:

1. Backed up once in the first backup after the RS goes offline
2. NOT re-backed up in subsequent backups

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TestBackupOfflineRS.FullTableBackupClientWithHook	Full backup client that runs afterSnapshotHook once the table snapshots exist.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.backup.TestBackupBase

TestBackupBase.FullTableBackupClientForTest, TestBackupBase.IncrementalTableBackupClientForTest

Field Summary

Fields

Modifier and Type	Field	Description
private static Runnable	afterSnapshotHook
private static final org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hbase.backup.TestBackupBase

autoRestoreOnFailure, BACKUP_REMOTE_ROOT_DIR, BACKUP_ROOT_DIR, conf1, conf2, famName, NB_ROWS_IN_BATCH, provider, qualName, secure, table1, table1_restore, table1Desc, table2, table2_restore, table3, table3_restore, table4, TEST_UTIL, TEST_UTIL2, useSecondCluster

Constructor Summary

Constructors

Constructor	Description
TestBackupOfflineRS()

Method Summary

Modifier and Type	Method	Description
private static Long	boundary(org.apache.hadoop.hbase.backup.impl.BackupSystemTable sysTable, String host)
private static void	moveRegionAndWait(org.apache.hadoop.hbase.TableName table, org.apache.hadoop.hbase.regionserver.HRegionServer destination)
private static void	restore(String backupId, org.apache.hadoop.hbase.TableName sourceTable, org.apache.hadoop.hbase.TableName restoredTable)
private static void	restoreAndAssertAllRowsPresent(String backupId, String restoredTableName, String message)
static void	setUp()
private static void	stopRegionServerAndWait(org.apache.hadoop.hbase.ServerName serverName)
void	testBackupWithOfflineRS()	Tests that when a full backup is taken while an RS is offline (with WALs in oldlogs), the offline host's timestamps are recorded so subsequent incremental backups don't reinclude those WALs.
void	testDeletedRowIsNotResurrectedByOfflineRSWALs()	Tests that a row deleted before a full backup is not brought back by the WALs of a region server that went offline between two backups.
void	testRSGoesOfflineAfterFullBackupBeforeIncremental()	Tests that WALs written to an RS after a full backup are correctly included in the subsequent incremental backup, even if that RS has gone offline before the incremental runs.
void	testRSStartedDuringFullBackup()	Tests that edits written during a full backup, after its log roll and table snapshot, to an RS that started after that log roll, are included in the next incremental backup.
void	testTransientRSAfterFullBackupBeforeIncremental()	Tests that WALs from an RS that comes online and goes offline between a full backup and an incremental backup are correctly included in the incremental backup and not re-included in subsequent incremental backups.
void	testTransientRSBeforeFullBackup()	Tests that a brand-new RS that comes online and goes offline before any backup correctly has its WALs covered by the full backup.
private static List<String>	walsOfHost(org.apache.hadoop.hbase.backup.impl.BackupSystemTable sysTable, String backupId, String host)

Methods inherited from class org.apache.hadoop.hbase.backup.TestBackupBase

backupTables, backupTables, checkFailed, checkSucceeded, createBackupRequest, createBackupRequest, createBackupRequest, createTables, deleteContinuousBackupReplicationPeerIfExists, dumpBackupDir, fullTableBackup, getBackupAdmin, getLatestBackupManifest, getListOfWALFiles, incrementalTableBackup, insertIntoTable, loadTable, setUpHelper, tearDown, toList

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

afterSnapshotHook

private static Runnable afterSnapshotHook

Constructor Details

TestBackupOfflineRS

public TestBackupOfflineRS()

Method Details

setUp

@BeforeAll
public static void setUp()
                  throws Exception

Throws:

Exception

stopRegionServerAndWait

private static void stopRegionServerAndWait(org.apache.hadoop.hbase.ServerName serverName)
                                     throws Exception

Throws:

Exception

restore

private static void restore(String backupId,
 org.apache.hadoop.hbase.TableName sourceTable,
 org.apache.hadoop.hbase.TableName restoredTable)
                     throws Exception

Throws:

Exception

restoreAndAssertAllRowsPresent

private static void restoreAndAssertAllRowsPresent(String backupId,
 String restoredTableName,
 String message)
                                            throws Exception

Throws:

Exception

boundary

private static Long boundary(org.apache.hadoop.hbase.backup.impl.BackupSystemTable sysTable,
 String host)
                      throws IOException

Throws:

IOException

walsOfHost

private static List<String> walsOfHost(org.apache.hadoop.hbase.backup.impl.BackupSystemTable sysTable,
 String backupId,
 String host)
                                throws IOException

Throws:

IOException

moveRegionAndWait

private static void moveRegionAndWait(org.apache.hadoop.hbase.TableName table,
 org.apache.hadoop.hbase.regionserver.HRegionServer destination)
                               throws Exception

Throws:

Exception

testBackupWithOfflineRS

@Test
public void testBackupWithOfflineRS()
                             throws Exception

Tests that when a full backup is taken while an RS is offline (with WALs in oldlogs), the offline host's timestamps are recorded so subsequent incremental backups don't reinclude those WALs.

Throws:

Exception

testRSGoesOfflineAfterFullBackupBeforeIncremental

@Test
public void testRSGoesOfflineAfterFullBackupBeforeIncremental()
                                                       throws Exception

Tests that WALs written to an RS after a full backup are correctly included in the subsequent incremental backup, even if that RS has gone offline before the incremental runs.

Throws:

Exception

testTransientRSBeforeFullBackup

@Test
public void testTransientRSBeforeFullBackup()
                                     throws Exception

Tests that a brand-new RS that comes online and goes offline before any backup correctly has its WALs covered by the full backup.

Throws:

Exception

testTransientRSAfterFullBackupBeforeIncremental

@Test
public void testTransientRSAfterFullBackupBeforeIncremental()
                                                     throws Exception

Tests that WALs from an RS that comes online and goes offline between a full backup and an incremental backup are correctly included in the incremental backup and not re-included in subsequent incremental backups.

Throws:

Exception

testRSStartedDuringFullBackup

@Test
public void testRSStartedDuringFullBackup()
                                   throws Exception

Tests that edits written during a full backup, after its log roll and table snapshot, to an RS that started after that log roll, are included in the next incremental backup.

Throws:

Exception

testDeletedRowIsNotResurrectedByOfflineRSWALs

@Test
public void testDeletedRowIsNotResurrectedByOfflineRSWALs()
                                                   throws Exception

Tests that a row deleted before a full backup is not brought back by the WALs of a region server that went offline between two backups. The row and its delete marker are compacted away before the full backup, so only the offline RS's old WAL, which still holds the put, could bring it back.

Throws:

Exception