Package org.apache.hadoop.hbase.mob

Class TestMobCompactionWithDefaults

java.lang.Object

org.apache.hadoop.hbase.mob.TestMobCompactionWithDefaults

Direct Known Subclasses:

TestMobCompactionOptMode, TestMobCompactionOptRegionBatchMode, TestMobCompactionRegularRegionBatchMode

@Tag("org.apache.hadoop.hbase.testclassification.LargeTests")
public class TestMobCompactionWithDefaults
extends Object

Mob file compaction base test. 1. Enables batch mode for regular MOB compaction, Sets batch size to 7 regions. (Optional) 2. Disables periodic MOB compactions, sets minimum age to archive to 10 sec 3. Creates MOB table with 20 regions 4. Loads MOB data (randomized keys, 1000 rows), flushes data. 5. Repeats 4. two more times 6. Verifies that we have 20 *3 = 60 mob files (equals to number of regions x 3) 7. Runs major MOB compaction. 8. Verifies that number of MOB files in a mob directory is 20 x4 = 80 9. Waits for a period of time larger than minimum age to archive 10. Runs Mob cleaner chore 11 Verifies that number of MOB files in a mob directory is 20. 12 Runs scanner and checks all 3 * 1000 rows.

Field Summary

Fields

Modifier and Type	Field	Description
protected org.apache.hadoop.hbase.client.Admin	admin
protected static org.apache.hadoop.conf.Configuration	conf
protected static final byte[]	fam
private org.apache.hadoop.hbase.client.ColumnFamilyDescriptor	familyDescriptor
protected static final String	famStr
protected HBaseTestingUtil	HTU
private static final org.slf4j.Logger	LOG
protected static long	minAgeToArchive
protected static final long	mobLen
protected static final byte[]	mobVal
protected int	numRegions
protected static final byte[]	qualifier
protected int	rows
protected org.apache.hadoop.hbase.TableName	table
protected org.apache.hadoop.hbase.client.TableDescriptor	tableDescriptor
private String	testMethodName
protected Boolean	useFileBasedSFT

Constructor Summary

Constructors

Constructor	Description
TestMobCompactionWithDefaults(Boolean useFileBasedSFT)

Method Summary

Modifier and Type	Method	Description
protected void	additonalConfigSetup()
void	baseTestMobFileCompaction()
protected void	cleanupAndVerifyCounts(org.apache.hadoop.hbase.TableName table, String family, int rows)
protected String	description()
protected void	disableCompactions()
protected void	enableCompactions()
protected long	getNumberOfFilesInMobFamilyDir(org.apache.hadoop.hbase.client.TableDescriptor descriptor, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDesc, String family)
protected long	getNumberOfStoreFiles(org.apache.hadoop.hbase.client.TableDescriptor descriptor, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDesc, String family)
protected void	htuStart()
protected void	loadAndFlushThreeTimes(int rows, org.apache.hadoop.hbase.TableName table, String family)
private void	loadData(org.apache.hadoop.hbase.TableName tableName, int num)
protected void	mobCompact(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor)	compact the given table and return once it is done.
protected void	mobCompactImpl(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor)	Call the API for compaction specific to the test set.
static Stream<org.junit.jupiter.params.provider.Arguments>	parameters()
protected long	scanTable(org.apache.hadoop.hbase.TableName tableName)
void	setUp(org.junit.jupiter.api.TestInfo testInfo)
void	tearDown()
void	testMobFileCompactionAfterSnapshotClone()
void	testMobFileCompactionAfterSnapshotCloneAndFlush()
protected void	waitUntilCompactionIsComplete(org.apache.hadoop.hbase.TableName table)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

HTU

protected HBaseTestingUtil HTU

conf

protected static org.apache.hadoop.conf.Configuration conf

minAgeToArchive

protected static long minAgeToArchive

famStr

protected static final String famStr

See Also:

• Constant Field Values

fam

protected static final byte[] fam

qualifier

protected static final byte[] qualifier

mobLen

protected static final long mobLen

See Also:

• Constant Field Values

mobVal

protected static final byte[] mobVal

testMethodName

private String testMethodName

tableDescriptor

protected org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor

familyDescriptor

private org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor

admin

protected org.apache.hadoop.hbase.client.Admin admin

table

protected org.apache.hadoop.hbase.TableName table

numRegions

protected int numRegions

rows

protected int rows

useFileBasedSFT

protected Boolean useFileBasedSFT

Constructor Details

TestMobCompactionWithDefaults

public TestMobCompactionWithDefaults(Boolean useFileBasedSFT)

Method Details

parameters

public static Stream<org.junit.jupiter.params.provider.Arguments> parameters()

htuStart

protected void htuStart()
                 throws Exception

Throws:

Exception

additonalConfigSetup

protected void additonalConfigSetup()

setUp

@BeforeEach
public void setUp(org.junit.jupiter.api.TestInfo testInfo)
           throws Exception

Throws:

Exception

loadData

private void loadData(org.apache.hadoop.hbase.TableName tableName,
 int num)

tearDown

@AfterEach
public void tearDown()
              throws Exception

Throws:

Exception

baseTestMobFileCompaction

@TestTemplate
public void baseTestMobFileCompaction()
                               throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

testMobFileCompactionAfterSnapshotClone

@TestTemplate
public void testMobFileCompactionAfterSnapshotClone()
                                             throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

getNumberOfFilesInMobFamilyDir

protected long getNumberOfFilesInMobFamilyDir(org.apache.hadoop.hbase.client.TableDescriptor descriptor,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDesc,
 String family)
                                       throws IOException

Throws:

IOException

testMobFileCompactionAfterSnapshotCloneAndFlush

@TestTemplate
public void testMobFileCompactionAfterSnapshotCloneAndFlush()
                                                     throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

loadAndFlushThreeTimes

protected void loadAndFlushThreeTimes(int rows,
 org.apache.hadoop.hbase.TableName table,
 String family)
                               throws IOException

Throws:

IOException

description

protected String description()

enableCompactions

protected void enableCompactions()
                          throws IOException

Throws:

IOException

disableCompactions

protected void disableCompactions()
                           throws IOException

Throws:

IOException

mobCompact

protected void mobCompact(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor)
                   throws IOException,
InterruptedException

compact the given table and return once it is done. should presume compactions are disabled when called. should ensure compactions are disabled before returning.

Throws:

IOException

InterruptedException

mobCompactImpl

protected void mobCompactImpl(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor)
                       throws IOException,
InterruptedException

Call the API for compaction specific to the test set. should not wait for compactions to finish. may assume compactions are enabled when called.

Throws:

IOException

InterruptedException

waitUntilCompactionIsComplete

protected void waitUntilCompactionIsComplete(org.apache.hadoop.hbase.TableName table)
                                      throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

cleanupAndVerifyCounts

protected void cleanupAndVerifyCounts(org.apache.hadoop.hbase.TableName table,
 String family,
 int rows)
                               throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

scanTable

protected long scanTable(org.apache.hadoop.hbase.TableName tableName)

getNumberOfStoreFiles

protected long getNumberOfStoreFiles(org.apache.hadoop.hbase.client.TableDescriptor descriptor,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDesc,
 String family)
                              throws IOException

Throws:

IOException