Package org.apache.hadoop.hbase
Class IntegrationTestRegionReplicaReplication
java.lang.Object
org.apache.hadoop.hbase.util.AbstractHBaseTool
org.apache.hadoop.hbase.IntegrationTestBase
org.apache.hadoop.hbase.IntegrationTestIngest
org.apache.hadoop.hbase.IntegrationTestRegionReplicaReplication
- All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
,org.apache.hadoop.util.Tool
Integration test for testing async wal replication to secondary region replicas. Sets up a table
with given region replication (default 2), and uses LoadTestTool client writer, updater and
reader threads for writes and reads and verification. It uses a delay queue with a given delay
("read_delay_ms", default 5000ms) between the writer/updater and reader threads to make the
written items available to readers. This means that a reader will only start reading from a row
written by the writer / updater after 5secs has passed. The reader thread performs the reads from
the given region replica id (default 1) to perform the reads. Async wal replication has to finish
with the replication of the edits before read_delay_ms to the given region replica id so that the
read and verify will not fail. The job will run for at least given runtime (default 10min)
by running a concurrent writer and reader workload followed by a concurrent updater and reader
workload for num_keys_per_server.
Example usage:
hbase org.apache.hadoop.hbase.IntegrationTestRegionReplicaReplication -DIntegrationTestRegionReplicaReplication.num_keys_per_server=10000 -Dhbase.IntegrationTestRegionReplicaReplication.runtime=600000 -DIntegrationTestRegionReplicaReplication.read_delay_ms=5000 -DIntegrationTestRegionReplicaReplication.region_replication=3 -DIntegrationTestRegionReplicaReplication.region_replica_id=2 -DIntegrationTestRegionReplicaReplication.num_read_threads=100 -DIntegrationTestRegionReplicaReplication.num_write_threads=100
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic class
This extends MultiThreadedWriter to add a configurable delay to the keys written by the writer threads to become available to the MultiThradedReader threads.static class
This extends MultiThreadedWriter to add a configurable delay to the keys written by the writer threads to become available to the MultiThradedReader threads.Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool
org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator
-
Field Summary
Modifier and TypeFieldDescriptionprivate static final String[]
private static final int
private static final String
private static final int
private static final String
Fields inherited from class org.apache.hadoop.hbase.IntegrationTestIngest
cluster, DEFAULT_NUM_KEYS_PER_SERVER, DEFAULT_NUM_READ_THREADS, DEFAULT_NUM_WRITE_THREADS, DEFAULT_RUN_TIME, HIPHEN, JUNIT_RUN_TIME, LOAD_TEST_TOOL_INIT_ARGS, loadTool, LOG, NUM_KEYS_PER_SERVER_KEY, NUM_READ_THREADS_KEY, NUM_WRITE_THREADS_KEY, RUN_TIME_KEY
Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase
CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util
Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool
cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionprotected int
static void
protected void
runIngestTest
(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads) void
setConf
(org.apache.hadoop.conf.Configuration conf) void
Methods inherited from class org.apache.hadoop.hbase.IntegrationTestIngest
getArgsForLoadTestTool, getArgsForLoadTestToolInitTable, getColumnFamilies, getNumKeys, getTablename, initTable, internalRunIngestTest, runTestFromCommandLine, setUpCluster
Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase
addOptions, cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, processOptions, setUp, setUpMonkey, startMonkey
Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool
addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run
-
Field Details
-
TEST_NAME
-
OPT_READ_DELAY_MS
- See Also:
-
DEFAULT_REGION_REPLICATION
- See Also:
-
SERVER_COUNT
- See Also:
-
DEFAULT_COLUMN_FAMILIES
-
-
Constructor Details
-
IntegrationTestRegionReplicaReplication
-
-
Method Details
-
getMinServerCount
- Overrides:
getMinServerCount
in classIntegrationTestIngest
-
setConf
- Specified by:
setConf
in interfaceorg.apache.hadoop.conf.Configurable
- Overrides:
setConf
in classorg.apache.hadoop.hbase.util.AbstractHBaseTool
-
testIngest
- Overrides:
testIngest
in classIntegrationTestIngest
- Throws:
Exception
-
runIngestTest
protected void runIngestTest(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads) throws Exception - Overrides:
runIngestTest
in classIntegrationTestIngest
- Throws:
Exception
-
main
- Throws:
Exception
-