org.apache.hadoop.hbase

Class IntegrationTestMobCompaction

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestMobCompaction

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class IntegrationTestMobCompaction
extends IntegrationTestBase

An integration test to detect regressions in HBASE-22749. Test creates MOB-enabled table, and runs in parallel, the following tasks: loads data, runs MOB compactions, runs MOB cleaning chore. The failure injections into MOB compaction cycle is implemented via specific sub-class of DefaultMobStoreCompactor - FaultyMobStoreCompactor. The probability of failure is controlled by command-line argument 'failprob'.

See Also:

HBASE-22749

Sample usage:

 hbase org.apache.hadoop.hbase.IntegrationTestMobCompaction -Dservers=10 -Drows=1000000
 -Dfailprob=0.2
      

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	IntegrationTestMobCompaction.CleanMobAndArchive
(package private) static class	IntegrationTestMobCompaction.MajorCompaction
(package private) class	IntegrationTestMobCompaction.WriteData

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field and Description
private static org.apache.hadoop.hbase.client.Admin	admin
private static org.apache.hadoop.hbase.mob.MobFileCleanerChore	chore
private static org.apache.hadoop.conf.Configuration	conf
protected static double	DEFAULT_FAILURE_PROB
protected static int	DEFAULT_REGIONSERVER_COUNT
protected static int	DEFAULT_ROWS_COUNT
protected static String	FAILURE_PROB_KEY
protected static double	failureProb
protected static byte[]	fam
protected static String	famStr
private static org.apache.hadoop.hbase.HColumnDescriptor	hcd
private static org.apache.hadoop.hbase.HTableDescriptor	hdt
protected static org.slf4j.Logger	LOG
protected static long	mobLen
protected static byte[]	mobVal
protected static byte[]	qualifier
protected static String	REGIONSERVER_COUNT_KEY
protected static int	regionServerCount
protected static String	ROWS_COUNT_KEY
protected static long	rowsToLoad
private static boolean	run
private static org.apache.hadoop.hbase.client.Table	table

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor and Description
IntegrationTestMobCompaction()

Method Summary

Modifier and Type	Method and Description
protected void	addOptions()
private void	createTestTable()
private void	deleteTablesIfAny()
protected Set<String>	getColumnFamilies() Provides the name of the CFs that are protected from random Chaos monkey activity (alter)
private long	getNumberOfMobFiles(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.TableName tableName, String family)
org.apache.hadoop.hbase.TableName	getTablename() Provides the name of the table that is protected from random Chaos monkey activity
private static void	initConf(org.apache.hadoop.conf.Configuration conf)
static void	main(String[] args)
void	printStats(long loaded)
protected void	processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)
int	runTestFromCommandLine() Returns status of CLI execution
private void	scanTable()
void	setUp()
void	setUpCluster()
void	setUpMonkey()
void	tearDown()
void	testMobCompaction()

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

REGIONSERVER_COUNT_KEY

protected static final String REGIONSERVER_COUNT_KEY

See Also:

Constant Field Values

ROWS_COUNT_KEY

protected static final String ROWS_COUNT_KEY

See Also:

Constant Field Values

FAILURE_PROB_KEY

protected static final String FAILURE_PROB_KEY

See Also:

Constant Field Values

DEFAULT_REGIONSERVER_COUNT

protected static final int DEFAULT_REGIONSERVER_COUNT

See Also:

Constant Field Values

DEFAULT_ROWS_COUNT

protected static final int DEFAULT_ROWS_COUNT

See Also:

Constant Field Values

DEFAULT_FAILURE_PROB

protected static final double DEFAULT_FAILURE_PROB

See Also:

Constant Field Values

regionServerCount

protected static int regionServerCount

rowsToLoad

protected static long rowsToLoad

failureProb

protected static double failureProb

famStr

protected static String famStr

fam

protected static byte[] fam

qualifier

protected static byte[] qualifier

mobLen

protected static long mobLen

mobVal

protected static byte[] mobVal

conf

private static org.apache.hadoop.conf.Configuration conf

hdt

private static org.apache.hadoop.hbase.HTableDescriptor hdt

hcd

private static org.apache.hadoop.hbase.HColumnDescriptor hcd

admin

private static org.apache.hadoop.hbase.client.Admin admin

table

private static org.apache.hadoop.hbase.client.Table table

chore

private static org.apache.hadoop.hbase.mob.MobFileCleanerChore chore

run

private static volatile boolean run

Constructor Detail

IntegrationTestMobCompaction

public IntegrationTestMobCompaction()

Method Detail

setUp

public void setUp()
           throws Exception

Overrides:

setUp in class IntegrationTestBase

Throws:

Exception

createTestTable

private void createTestTable()
                      throws IOException

Throws:

IOException

tearDown

public void tearDown()
              throws IOException

Throws:

IOException

setUpMonkey

public void setUpMonkey()
                 throws Exception

Overrides:

setUpMonkey in class IntegrationTestBase

Throws:

Exception

deleteTablesIfAny

private void deleteTablesIfAny()
                        throws IOException

Throws:

IOException

setUpCluster

public void setUpCluster()
                  throws Exception

Specified by:

setUpCluster in class IntegrationTestBase

Throws:

Exception

runTestFromCommandLine

public int runTestFromCommandLine()
                           throws Exception

Returns status of CLI execution

Specified by:

runTestFromCommandLine in class IntegrationTestBase

Throws:

Exception

getTablename

public org.apache.hadoop.hbase.TableName getTablename()

Description copied from class: IntegrationTestBase

Provides the name of the table that is protected from random Chaos monkey activity

Specified by:

getTablename in class IntegrationTestBase

Returns:

table to not delete.

getColumnFamilies

protected Set<String> getColumnFamilies()

Description copied from class: IntegrationTestBase

Provides the name of the CFs that are protected from random Chaos monkey activity (alter)

Specified by:

getColumnFamilies in class IntegrationTestBase

Returns:

set of cf names to protect.

addOptions

protected void addOptions()

Overrides:

addOptions in class IntegrationTestBase

processOptions

protected void processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)

Overrides:

processOptions in class IntegrationTestBase

initConf

private static void initConf(org.apache.hadoop.conf.Configuration conf)

testMobCompaction

public void testMobCompaction()
                       throws InterruptedException,
                              IOException

Throws:

InterruptedException

IOException

getNumberOfMobFiles

private long getNumberOfMobFiles(org.apache.hadoop.conf.Configuration conf,
                                 org.apache.hadoop.hbase.TableName tableName,
                                 String family)
                          throws IOException

Throws:

IOException

printStats

public void printStats(long loaded)

scanTable

private void scanTable()

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception