org.apache.hadoop.hbase.test

Class IntegrationTestTimeBoundedRequestsWithRegionReplicas

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestIngest

org.apache.hadoop.hbase.test.IntegrationTestTimeBoundedRequestsWithRegionReplicas

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

IntegrationTestTimeBoundedMultiGetRequestsWithRegionReplicas

public class IntegrationTestTimeBoundedRequestsWithRegionReplicas
extends IntegrationTestIngest

An IntegrationTest for doing reads with a timeout, to a read-only table with region replicas. ChaosMonkey is run which kills the region servers and master, but ensures that meta region server is not killed, and at most 2 region servers are dead at any point in time. The expected behavior is that all reads with stale mode true will return before the timeout (5 sec by default). The test fails if the read requests does not finish in time.

This test uses LoadTestTool to read and write the data from a single client but multiple threads. The data is written first, then we allow the region replicas to catch up. Then we start the reader threads doing get requests with stale mode true. Chaos Monkey is started after some delay (20 sec by default) after the reader threads are started so that there is enough time to fully cache meta. These parameters (and some other parameters from LoadTestTool) can be used to control behavior, given values are default:

 -Dhbase.IntegrationTestTimeBoundedRequestsWithRegionReplicas.runtime=600000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_regions_per_server=5
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.get_timeout_ms=5000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_keys_per_server=2500
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.region_replication=3
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_read_threads=20
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_write_threads=20
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_regions_per_server=5
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.chaos_monkey_delay=20000
 

Use this test with "serverKilling" ChaosMonkey. Sample usage:

 hbase org.apache.hadoop.hbase.test.IntegrationTestTimeBoundedRequestsWithRegionReplicas
 -Dhbase.IntegrationTestTimeBoundedRequestsWithRegionReplicas.runtime=600000
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_write_threads=40
 -DIntegrationTestTimeBoundedRequestsWithRegionReplicas.num_read_threads=40
 -Dhbase.ipc.client.allowsInterrupt=true --monkey serverKilling
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	IntegrationTestTimeBoundedRequestsWithRegionReplicas.TimeBoundedMultiThreadedReader

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	CHAOS_MONKEY_DELAY_KEY
protected static long	DEFAUL_CHAOS_MONKEY_DELAY
protected static long	DEFAULT_GET_TIMEOUT
protected static int	DEFAULT_REGION_REPLICATION
protected static String	GET_TIMEOUT_KEY
private static org.slf4j.Logger	LOG
private static String	TEST_NAME

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

cluster, DEFAULT_NUM_KEYS_PER_SERVER, DEFAULT_NUM_READ_THREADS, DEFAULT_NUM_WRITE_THREADS, DEFAULT_RUN_TIME, HIPHEN, JUNIT_RUN_TIME, LOAD_TEST_TOOL_INIT_ARGS, loadTool, NUM_KEYS_PER_SERVER_KEY, NUM_READ_THREADS_KEY, NUM_WRITE_THREADS_KEY, RUN_TIME_KEY, util

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor and Description
IntegrationTestTimeBoundedRequestsWithRegionReplicas()

Method Summary

Modifier and Type	Method and Description
protected String[]	getArgsForLoadTestTool(String mode, String modeSpecificArg, long startKey, long numKeys)
protected MonkeyFactory	getDefaultMonkeyFactory()
static void	main(String[] args)
protected void	runIngestTest(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads)
void	setConf(org.apache.hadoop.conf.Configuration conf)
protected void	startMonkey()
protected void	writeData(int colsPerKey, int recordSize, int writeThreads, long startKey, long numKeys)

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

getArgsForLoadTestToolInitTable, getColumnFamilies, getMinServerCount, getNumKeys, getTablename, initTable, internalRunIngestTest, runTestFromCommandLine, setUpCluster, testIngest

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

addOptions, cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getTestingUtil, processBaseOptions, processOptions, setUp, setUpMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

TEST_NAME

private static final String TEST_NAME

DEFAULT_GET_TIMEOUT

protected static final long DEFAULT_GET_TIMEOUT

See Also:

Constant Field Values

GET_TIMEOUT_KEY

protected static final String GET_TIMEOUT_KEY

See Also:

Constant Field Values

DEFAUL_CHAOS_MONKEY_DELAY

protected static final long DEFAUL_CHAOS_MONKEY_DELAY

See Also:

Constant Field Values

CHAOS_MONKEY_DELAY_KEY

protected static final String CHAOS_MONKEY_DELAY_KEY

See Also:

Constant Field Values

DEFAULT_REGION_REPLICATION

protected static final int DEFAULT_REGION_REPLICATION

See Also:

Constant Field Values

Constructor Detail

IntegrationTestTimeBoundedRequestsWithRegionReplicas

public IntegrationTestTimeBoundedRequestsWithRegionReplicas()

Method Detail

startMonkey

protected void startMonkey()
                    throws Exception

Overrides:

startMonkey in class IntegrationTestBase

Throws:

Exception

getDefaultMonkeyFactory

protected MonkeyFactory getDefaultMonkeyFactory()

Overrides:

getDefaultMonkeyFactory in class IntegrationTestBase

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class org.apache.hadoop.hbase.util.AbstractHBaseTool

writeData

protected void writeData(int colsPerKey,
                         int recordSize,
                         int writeThreads,
                         long startKey,
                         long numKeys)
                  throws IOException

Throws:

IOException

runIngestTest

protected void runIngestTest(long defaultRunTime,
                             long keysPerServerPerIter,
                             int colsPerKey,
                             int recordSize,
                             int writeThreads,
                             int readThreads)
                      throws Exception

Overrides:

runIngestTest in class IntegrationTestIngest

Throws:

Exception

getArgsForLoadTestTool

protected String[] getArgsForLoadTestTool(String mode,
                                          String modeSpecificArg,
                                          long startKey,
                                          long numKeys)

Overrides:

getArgsForLoadTestTool in class IntegrationTestIngest

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception