org.apache.hadoop.hbase.mob

Class TestMobCompactionWithDefaults

java.lang.Object

org.apache.hadoop.hbase.mob.TestMobCompactionWithDefaults

Direct Known Subclasses:

TestMobCompactionOptMode, TestMobCompactionOptRegionBatchMode, TestMobCompactionRegularRegionBatchMode

public class TestMobCompactionWithDefaults
extends Object

Mob file compaction base test. 1. Enables batch mode for regular MOB compaction, Sets batch size to 7 regions. (Optional) 2. Disables periodic MOB compactions, sets minimum age to archive to 10 sec 3. Creates MOB table with 20 regions 4. Loads MOB data (randomized keys, 1000 rows), flushes data. 5. Repeats 4. two more times 6. Verifies that we have 20 *3 = 60 mob files (equals to number of regions x 3) 7. Runs major MOB compaction. 8. Verifies that number of MOB files in a mob directory is 20 x4 = 80 9. Waits for a period of time larger than minimum age to archive 10. Runs Mob cleaner chore 11 Verifies that number of MOB files in a mob directory is 20. 12 Runs scanner and checks all 3 * 1000 rows.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.hbase.client.Admin	admin
static HBaseClassTestRule	CLASS_RULE
protected org.apache.hadoop.hbase.mob.MobFileCleanerChore	cleanerChore
protected static org.apache.hadoop.conf.Configuration	conf
protected static byte[]	fam
protected org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder.ModifyableColumnFamilyDescriptor	familyDescriptor
protected static String	famStr
protected static HBaseTestingUtility	HTU
private static org.slf4j.Logger	LOG
protected static long	minAgeToArchive
protected static long	mobLen
protected static byte[]	mobVal
protected int	numRegions
protected static byte[]	qualifier
protected int	rows
protected org.apache.hadoop.hbase.TableName	table
protected org.apache.hadoop.hbase.client.TableDescriptorBuilder.ModifyableTableDescriptor	tableDescriptor
org.junit.rules.TestName	test
protected Boolean	useFileBasedSFT

Constructor Summary

Constructors

Constructor and Description
TestMobCompactionWithDefaults(Boolean useFileBasedSFT)

Method Summary

Modifier and Type	Method and Description
protected void	additonalConfigSetup()
void	baseTestMobFileCompaction()
protected void	cleanupAndVerifyCounts(org.apache.hadoop.hbase.TableName table, String family, int rows)
static Collection<Boolean>	data()
protected String	description()
protected void	disableCompactions()
protected void	enableCompactions()
protected long	getNumberOfMobFiles(org.apache.hadoop.hbase.TableName tableName, String family)
protected void	htuStart()
protected void	loadAndFlushThreeTimes(int rows, org.apache.hadoop.hbase.TableName table, String family)
private void	loadData(org.apache.hadoop.hbase.TableName tableName, int num)
protected void	mobCompact(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor) compact the given table and return once it is done.
protected void	mobCompactImpl(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor) Call the API for compaction specific to the test set.
protected long	scanTable(org.apache.hadoop.hbase.TableName tableName)
void	setUp()
void	tearDown()
void	testMobFileCompactionAfterSnapshotClone()
void	testMobFileCompactionAfterSnapshotCloneAndFlush()
protected void	waitUntilCompactionIsComplete(org.apache.hadoop.hbase.TableName table)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

HTU

protected static HBaseTestingUtility HTU

conf

protected static org.apache.hadoop.conf.Configuration conf

minAgeToArchive

protected static long minAgeToArchive

famStr

protected static final String famStr

See Also:

Constant Field Values

fam

protected static final byte[] fam

qualifier

protected static final byte[] qualifier

mobLen

protected static final long mobLen

See Also:

Constant Field Values

mobVal

protected static final byte[] mobVal

test

public org.junit.rules.TestName test

tableDescriptor

protected org.apache.hadoop.hbase.client.TableDescriptorBuilder.ModifyableTableDescriptor tableDescriptor

familyDescriptor

protected org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder.ModifyableColumnFamilyDescriptor familyDescriptor

admin

protected org.apache.hadoop.hbase.client.Admin admin

table

protected org.apache.hadoop.hbase.TableName table

numRegions

protected int numRegions

rows

protected int rows

cleanerChore

protected org.apache.hadoop.hbase.mob.MobFileCleanerChore cleanerChore

useFileBasedSFT

protected Boolean useFileBasedSFT

Constructor Detail

TestMobCompactionWithDefaults

public TestMobCompactionWithDefaults(Boolean useFileBasedSFT)

Method Detail

data

public static Collection<Boolean> data()

htuStart

protected void htuStart()
                 throws Exception

Throws:

Exception

additonalConfigSetup

protected void additonalConfigSetup()

setUp

public void setUp()
           throws Exception

Throws:

Exception

loadData

private void loadData(org.apache.hadoop.hbase.TableName tableName,
                      int num)

tearDown

public void tearDown()
              throws Exception

Throws:

Exception

baseTestMobFileCompaction

public void baseTestMobFileCompaction()
                               throws InterruptedException,
                                      IOException

Throws:

InterruptedException

IOException

testMobFileCompactionAfterSnapshotClone

public void testMobFileCompactionAfterSnapshotClone()
                                             throws InterruptedException,
                                                    IOException

Throws:

InterruptedException

IOException

testMobFileCompactionAfterSnapshotCloneAndFlush

public void testMobFileCompactionAfterSnapshotCloneAndFlush()
                                                     throws InterruptedException,
                                                            IOException

Throws:

InterruptedException

IOException

loadAndFlushThreeTimes

protected void loadAndFlushThreeTimes(int rows,
                                      org.apache.hadoop.hbase.TableName table,
                                      String family)
                               throws IOException

Throws:

IOException

description

protected String description()

enableCompactions

protected void enableCompactions()
                          throws IOException

Throws:

IOException

disableCompactions

protected void disableCompactions()
                           throws IOException

Throws:

IOException

mobCompact

protected void mobCompact(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor,
                          org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor)
                   throws IOException,
                          InterruptedException

compact the given table and return once it is done. should presume compactions are disabled when called. should ensure compactions are disabled before returning.

Throws:

IOException

InterruptedException

mobCompactImpl

protected void mobCompactImpl(org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor,
                              org.apache.hadoop.hbase.client.ColumnFamilyDescriptor familyDescriptor)
                       throws IOException,
                              InterruptedException

Call the API for compaction specific to the test set. should not wait for compactions to finish. may assume compactions are enabled when called.

Throws:

IOException

InterruptedException

waitUntilCompactionIsComplete

protected void waitUntilCompactionIsComplete(org.apache.hadoop.hbase.TableName table)
                                      throws IOException,
                                             InterruptedException

Throws:

IOException

InterruptedException

cleanupAndVerifyCounts

protected void cleanupAndVerifyCounts(org.apache.hadoop.hbase.TableName table,
                                      String family,
                                      int rows)
                               throws InterruptedException,
                                      IOException

Throws:

InterruptedException

IOException

getNumberOfMobFiles

protected long getNumberOfMobFiles(org.apache.hadoop.hbase.TableName tableName,
                                   String family)
                            throws IOException

Throws:

IOException

scanTable

protected long scanTable(org.apache.hadoop.hbase.TableName tableName)