Package org.apache.hadoop.hbase.test

Class IntegrationTestTimeBoundedRequestsWithRegionReplicas

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestIngest

org.apache.hadoop.hbase.test.IntegrationTestTimeBoundedRequestsWithRegionReplicas

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

IntegrationTestTimeBoundedMultiGetRequestsWithRegionReplicas

public class IntegrationTestTimeBoundedRequestsWithRegionReplicas
extends IntegrationTestIngest

An IntegrationTest for doing reads with a timeout, to a read-only table with region replicas. ChaosMonkey is run which kills the region servers and master, but ensures that meta region server is not killed, and at most 2 region servers are dead at any point in time. The expected behavior is that all reads with stale mode true will return before the timeout (5 sec by default). The test fails if the read requests does not finish in time.

This test uses LoadTestTool to read and write the data from a single client but multiple threads. The data is written first, then we allow the region replicas to catch up. Then we start the reader threads doing get requests with stale mode true. Chaos Monkey is started after some delay (20 sec by default) after the reader threads are started so that there is enough time to fully cache meta. These parameters (and some other parameters from LoadTestTool) can be used to control behavior, given values are default:

 -Dhbase.IntegrationTestTimeBoundedRequestsWithRegionReplicas.runtime=600000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_regions_per_server=5
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.get_timeout_ms=5000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_keys_per_server=2500
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.region_replication=3
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_read_threads=20
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_write_threads=20
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_regions_per_server=5
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.chaos_monkey_delay=20000
 

Use this test with "serverKilling" ChaosMonkey. Sample usage:

 hbase org.apache.hadoop.hbase.test.IntegrationTestTimeBoundedRequestsWithRegionReplicas
 -Dhbase.IntegrationTestTimeBoundedRequestsWithRegionReplicas.runtime=600000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_write_threads=40
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_read_threads=40
 -Dhbase.ipc.client.allowsInterrupt=true --monkey serverKilling
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	IntegrationTestTimeBoundedRequestsWithRegionReplicas.TimeBoundedMultiThreadedReader

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field	Description
protected static final String	CHAOS_MONKEY_DELAY_KEY
protected static final long	DEFAUL_CHAOS_MONKEY_DELAY
protected static final long	DEFAULT_GET_TIMEOUT
protected static final int	DEFAULT_REGION_REPLICATION
protected static final String	GET_TIMEOUT_KEY
private static final org.slf4j.Logger	LOG
private static final String	TEST_NAME

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

cluster, DEFAULT_NUM_KEYS_PER_SERVER, DEFAULT_NUM_READ_THREADS, DEFAULT_NUM_WRITE_THREADS, DEFAULT_RUN_TIME, HIPHEN, JUNIT_RUN_TIME, LOAD_TEST_TOOL_INIT_ARGS, loadTool, NUM_KEYS_PER_SERVER_KEY, NUM_READ_THREADS_KEY, NUM_WRITE_THREADS_KEY, RUN_TIME_KEY

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor	Description
IntegrationTestTimeBoundedRequestsWithRegionReplicas()

Method Summary

Modifier and Type	Method	Description
protected String[]	getArgsForLoadTestTool(String mode, String modeSpecificArg, long startKey, long numKeys)
protected MonkeyFactory	getDefaultMonkeyFactory()
static void	main(String[] args)
protected void	runIngestTest(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads)
void	setConf(org.apache.hadoop.conf.Configuration conf)
protected void	startMonkey()
protected void	writeData(int colsPerKey, int recordSize, int writeThreads, long startKey, long numKeys)

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

getArgsForLoadTestToolInitTable, getColumnFamilies, getMinServerCount, getNumKeys, getTablename, initTable, internalRunIngestTest, runTestFromCommandLine, setUpCluster, testIngest

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

addOptions, cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getTestingUtil, loadMonkeyProperties, processBaseOptions, processOptions, setUp, setUpMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

TEST_NAME

private static final String TEST_NAME

DEFAULT_GET_TIMEOUT

protected static final long DEFAULT_GET_TIMEOUT

See Also:

• Constant Field Values

GET_TIMEOUT_KEY

protected static final String GET_TIMEOUT_KEY

See Also:

• Constant Field Values

DEFAUL_CHAOS_MONKEY_DELAY

protected static final long DEFAUL_CHAOS_MONKEY_DELAY

See Also:

• Constant Field Values

CHAOS_MONKEY_DELAY_KEY

protected static final String CHAOS_MONKEY_DELAY_KEY

See Also:

• Constant Field Values

DEFAULT_REGION_REPLICATION

protected static final int DEFAULT_REGION_REPLICATION

See Also:

• Constant Field Values

Constructor Details

IntegrationTestTimeBoundedRequestsWithRegionReplicas

public IntegrationTestTimeBoundedRequestsWithRegionReplicas()

Method Details

startMonkey

protected void startMonkey()
                    throws Exception

Overrides:

startMonkey in class IntegrationTestBase

Throws:

Exception

getDefaultMonkeyFactory

protected MonkeyFactory getDefaultMonkeyFactory()

Overrides:

getDefaultMonkeyFactory in class IntegrationTestBase

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class org.apache.hadoop.hbase.util.AbstractHBaseTool

writeData

protected void writeData(int colsPerKey,
 int recordSize,
 int writeThreads,
 long startKey,
 long numKeys)
                  throws IOException

Throws:

IOException

runIngestTest

protected void runIngestTest(long defaultRunTime,
 long keysPerServerPerIter,
 int colsPerKey,
 int recordSize,
 int writeThreads,
 int readThreads)
                      throws Exception

Overrides:

runIngestTest in class IntegrationTestIngest

Throws:

Exception

getArgsForLoadTestTool

protected String[] getArgsForLoadTestTool(String mode,
 String modeSpecificArg,
 long startKey,
 long numKeys)

Overrides:

getArgsForLoadTestTool in class IntegrationTestIngest

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception