Package org.apache.hadoop.hbase.snapshot
Class TestRestoreSnapshotHelper
java.lang.Object
org.apache.hadoop.hbase.snapshot.TestRestoreSnapshotHelper
- Direct Known Subclasses:
TestMobRestoreSnapshotHelper
Test the restore/clone operation from a file-system point of view.
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected org.apache.hadoop.fs.Path
static final HBaseClassTestRule
protected org.apache.hadoop.conf.Configuration
protected org.apache.hadoop.fs.FileSystem
private static final org.slf4j.Logger
protected org.apache.hadoop.fs.Path
protected static final String
protected static final HBaseTestingUtil
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprivate void
checkNoHFileLinkInTableDir
(org.apache.hadoop.hbase.TableName tableName) private void
createAndAssertSnapshot
(org.apache.hadoop.hbase.TableName tableName, String snapshotName) protected SnapshotTestingUtils.SnapshotMock
protected void
createTableAndSnapshot
(org.apache.hadoop.hbase.TableName tableName, String snapshotName) private void
flipCompactions
(boolean isEnable) private org.apache.hadoop.hbase.procedure2.ProcedureExecutor<org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv>
private org.apache.hadoop.fs.Path
getReferredToFile
(String referenceName) private org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper
getRestoreHelper
(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path snapshotDir, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription sd, org.apache.hadoop.hbase.client.TableDescriptor htdClone) Initialize the restore helper, based on the snapshot and table information provided.private boolean
hasHFileLink
(org.apache.hadoop.fs.Path tableDir) private void
mergeRegions
(org.apache.hadoop.hbase.TableName tableName, int mergeCount) private void
restoreAndVerify
(String snapshotName, String tableName) void
setup()
static void
protected void
setupConf
(org.apache.hadoop.conf.Configuration conf) void
tearDown()
static void
void
void
Test scenario for HBASE-29346, which addresses the issue where restoring snapshots after region merge operations could lead to missing store file references, potentially resulting in data loss.void
void
private void
testRestore
(org.apache.hadoop.fs.Path snapshotDir, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription sd, org.apache.hadoop.hbase.client.TableDescriptor htdClone) Execute the restore operationvoid
void
private void
verifyRestore
(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.hbase.client.TableDescriptor sourceHtd, org.apache.hadoop.hbase.client.TableDescriptor htdClone)
-
Field Details
-
CLASS_RULE
-
LOG
-
TEST_UTIL
-
TEST_HFILE
- See Also:
-
conf
-
archiveDir
-
fs
-
rootDir
-
-
Constructor Details
-
TestRestoreSnapshotHelper
public TestRestoreSnapshotHelper()
-
-
Method Details
-
setupConf
-
setupCluster
- Throws:
Exception
-
tearDownCluster
- Throws:
Exception
-
setup
- Throws:
Exception
-
tearDown
- Throws:
Exception
-
createSnapshotMock
- Throws:
IOException
-
testRestore
- Throws:
IOException
-
testRestoreWithNamespace
- Throws:
IOException
-
testNoHFileLinkInRootDir
- Throws:
IOException
-
testSkipReplayAndUpdateSeqId
- Throws:
Exception
-
testCopyExpiredSnapshotForScanner
- Throws:
IOException
InterruptedException
-
testMultiSnapshotRestoreWithMerge
Test scenario for HBASE-29346, which addresses the issue where restoring snapshots after region merge operations could lead to missing store file references, potentially resulting in data loss.This test performs the following steps:
- Creates a table with multiple regions.
- Inserts data into each region and flushes to create store files.
- Takes snapshot of the table and performs restore.
- Disable compactions, merge regions, create a new snapshot, and restore that snapshot on the same restore path.
- Verifies data integrity by scanning all data post region re-open.
- Throws:
IOException
InterruptedException
-
createAndAssertSnapshot
private void createAndAssertSnapshot(org.apache.hadoop.hbase.TableName tableName, String snapshotName) throws org.apache.hadoop.hbase.snapshot.SnapshotCreationException, IllegalArgumentException, IOException - Throws:
org.apache.hadoop.hbase.snapshot.SnapshotCreationException
IllegalArgumentException
IOException
-
flipCompactions
-
mergeRegions
private void mergeRegions(org.apache.hadoop.hbase.TableName tableName, int mergeCount) throws IOException - Throws:
IOException
-
getMasterProcedureExecutor
private org.apache.hadoop.hbase.procedure2.ProcedureExecutor<org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv> getMasterProcedureExecutor() -
createTableAndSnapshot
protected void createTableAndSnapshot(org.apache.hadoop.hbase.TableName tableName, String snapshotName) throws IOException - Throws:
IOException
-
checkNoHFileLinkInTableDir
private void checkNoHFileLinkInTableDir(org.apache.hadoop.hbase.TableName tableName) throws IOException - Throws:
IOException
-
hasHFileLink
- Throws:
IOException
-
restoreAndVerify
- Throws:
IOException
-
verifyRestore
private void verifyRestore(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.hbase.client.TableDescriptor sourceHtd, org.apache.hadoop.hbase.client.TableDescriptor htdClone) throws IOException - Throws:
IOException
-
testRestore
private void testRestore(org.apache.hadoop.fs.Path snapshotDir, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription sd, org.apache.hadoop.hbase.client.TableDescriptor htdClone) throws IOException Execute the restore operation- Parameters:
snapshotDir
- The snapshot directory to use as "restore source"sd
- The snapshot descriptorhtdClone
- The HTableDescriptor of the table to restore/clone.- Throws:
IOException
-
getRestoreHelper
private org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper getRestoreHelper(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path snapshotDir, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription sd, org.apache.hadoop.hbase.client.TableDescriptor htdClone) throws IOException Initialize the restore helper, based on the snapshot and table information provided.- Throws:
IOException
-
getReferredToFile
-