Package org.apache.hadoop.hbase

Class IntegrationTestIngest

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestIngest

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

IntegrationTestIngestStripeCompactions, IntegrationTestIngestWithACL, IntegrationTestIngestWithEncryption, IntegrationTestIngestWithMOB, IntegrationTestIngestWithTags, IntegrationTestIngestWithVisibilityLabels, IntegrationTestRegionReplicaReplication, IntegrationTestTimeBoundedRequestsWithRegionReplicas

public class IntegrationTestIngest
extends IntegrationTestBase

A base class for tests that do something with the cluster while running LoadTestTool to write and verify some data.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field	Description
protected HBaseClusterInterface	cluster
protected static final long	DEFAULT_NUM_KEYS_PER_SERVER
protected static final int	DEFAULT_NUM_READ_THREADS
protected static final int	DEFAULT_NUM_WRITE_THREADS
protected static final long	DEFAULT_RUN_TIME
static final char	HIPHEN
protected static final long	JUNIT_RUN_TIME
protected String[]	LOAD_TEST_TOOL_INIT_ARGS
protected org.apache.hadoop.hbase.util.LoadTestTool	loadTool
protected static final org.slf4j.Logger	LOG
protected static final String	NUM_KEYS_PER_SERVER_KEY
protected static final String	NUM_READ_THREADS_KEY
protected static final String	NUM_WRITE_THREADS_KEY
protected static final String	RUN_TIME_KEY	A soft limit on how long we should run
private static final int	SERVER_COUNT

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor	Description
IntegrationTestIngest()

Method Summary

Modifier and Type	Method	Description
private void	deleteTableIfNecessary()
protected String[]	getArgsForLoadTestTool(String mode, String modeSpecificArg, long startKey, long numKeys)
protected String[]	getArgsForLoadTestToolInitTable()
protected Set<String>	getColumnFamilies()	Provides the name of the CFs that are protected from random Chaos monkey activity (alter)
private String	getColumnFamiliesAsString()
protected int	getMinServerCount()
protected long	getNumKeys(long keysPerServer)	Estimates a data size based on the cluster size
org.apache.hadoop.hbase.TableName	getTablename()	Provides the name of the table that is protected from random Chaos monkey activity
protected void	initTable()
protected void	internalRunIngestTest(long runTime)
static void	main(String[] args)
protected void	runIngestTest(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads)
int	runTestFromCommandLine()
void	setUpCluster()
void	testIngest()

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

addOptions, cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, processOptions, setUp, setUpMonkey, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

HIPHEN

public static final char HIPHEN

See Also:

• Constant Field Values

SERVER_COUNT

private static final int SERVER_COUNT

See Also:

• Constant Field Values

DEFAULT_RUN_TIME

protected static final long DEFAULT_RUN_TIME

See Also:

• Constant Field Values

JUNIT_RUN_TIME

protected static final long JUNIT_RUN_TIME

See Also:

• Constant Field Values

RUN_TIME_KEY

protected static final String RUN_TIME_KEY

A soft limit on how long we should run

See Also:

• Constant Field Values

NUM_KEYS_PER_SERVER_KEY

protected static final String NUM_KEYS_PER_SERVER_KEY

See Also:

• Constant Field Values

DEFAULT_NUM_KEYS_PER_SERVER

protected static final long DEFAULT_NUM_KEYS_PER_SERVER

See Also:

• Constant Field Values

NUM_WRITE_THREADS_KEY

protected static final String NUM_WRITE_THREADS_KEY

See Also:

• Constant Field Values

DEFAULT_NUM_WRITE_THREADS

protected static final int DEFAULT_NUM_WRITE_THREADS

See Also:

• Constant Field Values

NUM_READ_THREADS_KEY

protected static final String NUM_READ_THREADS_KEY

See Also:

• Constant Field Values

DEFAULT_NUM_READ_THREADS

protected static final int DEFAULT_NUM_READ_THREADS

See Also:

• Constant Field Values

LOG

protected static final org.slf4j.Logger LOG

cluster

protected HBaseClusterInterface cluster

loadTool

protected org.apache.hadoop.hbase.util.LoadTestTool loadTool

LOAD_TEST_TOOL_INIT_ARGS

protected String[] LOAD_TEST_TOOL_INIT_ARGS

Constructor Details

IntegrationTestIngest

public IntegrationTestIngest()

Method Details

setUpCluster

public void setUpCluster()
                  throws Exception

Specified by:

setUpCluster in class IntegrationTestBase

Throws:

Exception

getMinServerCount

protected int getMinServerCount()

initTable

protected void initTable()
                  throws IOException

Throws:

IOException

runTestFromCommandLine

public int runTestFromCommandLine()
                           throws Exception

Specified by:

runTestFromCommandLine in class IntegrationTestBase

Throws:

Exception

testIngest

public void testIngest()
                throws Exception

Throws:

Exception

internalRunIngestTest

protected void internalRunIngestTest(long runTime)
                              throws Exception

Throws:

Exception

getTablename

public org.apache.hadoop.hbase.TableName getTablename()

Description copied from class: IntegrationTestBase

Provides the name of the table that is protected from random Chaos monkey activity

Specified by:

getTablename in class IntegrationTestBase

Returns:

table to not delete.

getColumnFamilies

protected Set<String> getColumnFamilies()

Description copied from class: IntegrationTestBase

Provides the name of the CFs that are protected from random Chaos monkey activity (alter)

Specified by:

getColumnFamilies in class IntegrationTestBase

Returns:

set of cf names to protect.

deleteTableIfNecessary

private void deleteTableIfNecessary()
                             throws IOException

Throws:

IOException

runIngestTest

protected void runIngestTest(long defaultRunTime,
 long keysPerServerPerIter,
 int colsPerKey,
 int recordSize,
 int writeThreads,
 int readThreads)
                      throws Exception

Throws:

Exception

getArgsForLoadTestToolInitTable

protected String[] getArgsForLoadTestToolInitTable()

getArgsForLoadTestTool

protected String[] getArgsForLoadTestTool(String mode,
 String modeSpecificArg,
 long startKey,
 long numKeys)

getColumnFamiliesAsString

private String getColumnFamiliesAsString()

getNumKeys

protected long getNumKeys(long keysPerServer)
                   throws IOException

Estimates a data size based on the cluster size

Throws:

IOException

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception