org.apache.hadoop.hbase.test

Class IntegrationTestReplication

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList

org.apache.hadoop.hbase.test.IntegrationTestReplication

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class IntegrationTestReplication
extends IntegrationTestBigLinkedList

This is an integration test for replication. It is derived off IntegrationTestBigLinkedList that creates a large circular linked list in one cluster and verifies that the data is correct in a sink cluster. The test handles creating the tables and schema and setting up the replication.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	IntegrationTestReplication.ClusterID Wrapper around an HBase ClusterID allowing us to get admin connections and configurations for it
protected class	IntegrationTestReplication.VerifyReplicationLoop The main runner loop for the test.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList

IntegrationTestBigLinkedList.CINode, IntegrationTestBigLinkedList.Generator, IntegrationTestBigLinkedList.Loop, IntegrationTestBigLinkedList.Search, IntegrationTestBigLinkedList.Verify, IntegrationTestBigLinkedList.WalkerBase

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field and Description
private int	DEFAULT_GENERATE_VERIFY_GAP
private int	DEFAULT_NUM_ITERATIONS
private int	DEFAULT_NUM_MAPPERS
private int	DEFAULT_NUM_NODES
private int	DEFAULT_NUM_REDUCERS
private int	DEFAULT_WIDTH
private int	DEFAULT_WRAP_MULTIPLIER
private String	DEST_CLUSTER_OPT
private String	GENERATE_VERIFY_GAP_OPT The gap (in seconds) from when data is finished being generated at the source to when it can be verified.
protected int	generateVerifyGap
private String	ITERATIONS_OPT
private String	NO_REPLICATION_SETUP_OPT
protected boolean	noReplicationSetup
private String	NUM_MAPPERS_OPT
private String	NUM_NODES_OPT The number of nodes in the test setup.
private String	NUM_REDUCERS_OPT
protected int	numIterations
protected int	numMappers
protected long	numNodes
protected int	numReducers
private String	OUTPUT_DIR_OPT
protected String	outputDir
protected String	sinkClusterIdString
private String	SOURCE_CLUSTER_OPT
protected String	sourceClusterIdString
protected Integer	width
private String	WIDTH_OPT The width of the linked list.
private String	WRAP_MULTIPLIER_OPT The number of rows after which the linked list points to the first row.
protected Integer	wrapMultiplier

Fields inherited from class org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList

COLUMN_CLIENT, COLUMN_COUNT, COLUMN_PREV, DEFAULT_TABLE_NAME, FAMILY_NAME, NO_KEY, NUM_SLAVES_BASE, otherArgs, TABLE_NAME_KEY, toRun, util

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor and Description
IntegrationTestReplication()

Method Summary

Modifier and Type	Method and Description
protected void	addOptions()
static void	main(String[] args)
protected void	processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)
int	runTestFromCommandLine()

Methods inherited from class org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList

cleanUpCluster, getColumnFamilies, getTablename, getTableName, setJobScannerConf, setUpCluster, testContinuousIngest

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

cleanUp, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, setUp, setUpMonkey, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sourceClusterIdString

protected String sourceClusterIdString

sinkClusterIdString

protected String sinkClusterIdString

numIterations

protected int numIterations

numMappers

protected int numMappers

numNodes

protected long numNodes

outputDir

protected String outputDir

numReducers

protected int numReducers

generateVerifyGap

protected int generateVerifyGap

width

protected Integer width

wrapMultiplier

protected Integer wrapMultiplier

noReplicationSetup

protected boolean noReplicationSetup

SOURCE_CLUSTER_OPT

private final String SOURCE_CLUSTER_OPT

See Also:

Constant Field Values

DEST_CLUSTER_OPT

private final String DEST_CLUSTER_OPT

See Also:

Constant Field Values

ITERATIONS_OPT

private final String ITERATIONS_OPT

See Also:

Constant Field Values

NUM_MAPPERS_OPT

private final String NUM_MAPPERS_OPT

See Also:

Constant Field Values

OUTPUT_DIR_OPT

private final String OUTPUT_DIR_OPT

See Also:

Constant Field Values

NUM_REDUCERS_OPT

private final String NUM_REDUCERS_OPT

See Also:

Constant Field Values

NO_REPLICATION_SETUP_OPT

private final String NO_REPLICATION_SETUP_OPT

See Also:

Constant Field Values

GENERATE_VERIFY_GAP_OPT

private final String GENERATE_VERIFY_GAP_OPT

The gap (in seconds) from when data is finished being generated at the source to when it can be verified. This is the replication lag we are willing to tolerate

See Also:

Constant Field Values

WIDTH_OPT

private final String WIDTH_OPT

The width of the linked list. See IntegrationTestBigLinkedList for more details

See Also:

Constant Field Values

WRAP_MULTIPLIER_OPT

private final String WRAP_MULTIPLIER_OPT

The number of rows after which the linked list points to the first row. See IntegrationTestBigLinkedList for more details

See Also:

Constant Field Values

NUM_NODES_OPT

private final String NUM_NODES_OPT

The number of nodes in the test setup. This has to be a multiple of WRAP_MULTIPLIER * WIDTH in order to ensure that the linked list can is complete. See IntegrationTestBigLinkedList for more details

See Also:

Constant Field Values

DEFAULT_NUM_MAPPERS

private final int DEFAULT_NUM_MAPPERS

See Also:

Constant Field Values

DEFAULT_NUM_REDUCERS

private final int DEFAULT_NUM_REDUCERS

See Also:

Constant Field Values

DEFAULT_NUM_ITERATIONS

private final int DEFAULT_NUM_ITERATIONS

See Also:

Constant Field Values

DEFAULT_GENERATE_VERIFY_GAP

private final int DEFAULT_GENERATE_VERIFY_GAP

See Also:

Constant Field Values

DEFAULT_WIDTH

private final int DEFAULT_WIDTH

See Also:

Constant Field Values

DEFAULT_WRAP_MULTIPLIER

private final int DEFAULT_WRAP_MULTIPLIER

See Also:

Constant Field Values

DEFAULT_NUM_NODES

private final int DEFAULT_NUM_NODES

See Also:

Constant Field Values

Constructor Detail

IntegrationTestReplication

public IntegrationTestReplication()

Method Detail

addOptions

protected void addOptions()

Overrides:

addOptions in class IntegrationTestBase

processOptions

protected void processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)

Overrides:

processOptions in class IntegrationTestBigLinkedList

runTestFromCommandLine

public int runTestFromCommandLine()
                           throws Exception

Overrides:

runTestFromCommandLine in class IntegrationTestBigLinkedList

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception