org.apache.hadoop.hbase

Class IntegrationTestDDLMasterFailover

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class IntegrationTestDDLMasterFailover
extends IntegrationTestBase

Integration test that verifies Procedure V2. DDL operations should go through (rollforward or rollback) when primary master is killed by ChaosMonkey (default MASTER_KILLING).

Multiple Worker threads are started to randomly do the following Actions in loops: Actions generating and populating tables:

• CreateTableAction
• DisableTableAction
• EnableTableAction
• DeleteTableAction
• AddRowAction

Actions performing column family DDL operations:

• AddColumnFamilyAction
• AlterColumnFamilyVersionsAction
• AlterColumnFamilyEncodingAction
• DeleteColumnFamilyAction

Actions performing namespace DDL operations:

• AddNamespaceAction
• AlterNamespaceAction
• DeleteNamespaceAction

The threads run for a period of time (default 20 minutes) then are stopped at the end of runtime. Verification is performed towards those checkpoints:

1. No Actions throw Exceptions.
2. No inconsistencies are detected in hbck.

This test should be run by the hbase user since it invokes hbck at the end

Usage: hbase org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover -Dhbase.IntegrationTestDDLMasterFailover.runtime=1200000 -Dhbase.IntegrationTestDDLMasterFailover.numThreads=20 -Dhbase.IntegrationTestDDLMasterFailover.numRegions=50 --monkey masterKilling

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	IntegrationTestDDLMasterFailover.ACTION
private class	IntegrationTestDDLMasterFailover.AddColumnFamilyAction
private class	IntegrationTestDDLMasterFailover.AddRowAction
private class	IntegrationTestDDLMasterFailover.AlterFamilyEncodingAction
private class	IntegrationTestDDLMasterFailover.AlterFamilyVersionsAction
private class	IntegrationTestDDLMasterFailover.ColumnAction
private class	IntegrationTestDDLMasterFailover.CreateNamespaceAction
private class	IntegrationTestDDLMasterFailover.CreateTableAction
private class	IntegrationTestDDLMasterFailover.DeleteColumnFamilyAction
private class	IntegrationTestDDLMasterFailover.DeleteNamespaceAction
private class	IntegrationTestDDLMasterFailover.DeleteTableAction
private class	IntegrationTestDDLMasterFailover.DisableTableAction
private class	IntegrationTestDDLMasterFailover.EnableTableAction
private class	IntegrationTestDDLMasterFailover.MasterAction
private class	IntegrationTestDDLMasterFailover.ModifyNamespaceAction
private class	IntegrationTestDDLMasterFailover.NamespaceAction
private class	IntegrationTestDDLMasterFailover.TableAction
private class	IntegrationTestDDLMasterFailover.Worker

Field Summary

Fields

Modifier and Type	Field and Description
protected HBaseCluster	cluster
protected org.apache.hadoop.hbase.client.Connection	connection
protected AtomicBoolean	create_table
protected static int	DEFAULT_NUM_REGIONS
protected static int	DEFAULT_NUM_THREADS
protected static long	DEFAULT_RUN_TIME
(package private) ConcurrentHashMap<org.apache.hadoop.hbase.TableName,org.apache.hadoop.hbase.client.TableDescriptor>	deletedTables
(package private) ConcurrentHashMap<org.apache.hadoop.hbase.TableName,org.apache.hadoop.hbase.client.TableDescriptor>	disabledTables
(package private) ConcurrentHashMap<org.apache.hadoop.hbase.TableName,org.apache.hadoop.hbase.client.TableDescriptor>	enabledTables
private boolean	keepObjectsAtTheEnd
private static org.slf4j.Logger	LOG
(package private) ConcurrentHashMap<String,org.apache.hadoop.hbase.NamespaceDescriptor>	namespaceMap
protected static String	NUM_REGIONS_KEY
protected static String	NUM_THREADS_KEY
protected int	numRegions
protected int	numThreads
protected static String	RUN_TIME_KEY A soft limit on how long we should run
protected AtomicBoolean	running
private static int	SERVER_COUNT

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor and Description
IntegrationTestDDLMasterFailover()

Method Summary

Modifier and Type	Method and Description
private void	checkException(List<IntegrationTestDDLMasterFailover.Worker> workers)
void	cleanUpCluster()
protected Set<String>	getColumnFamilies() Provides the name of the CFs that are protected from random Chaos monkey activity (alter)
protected org.apache.hadoop.hbase.client.Connection	getConnection()
protected int	getMinServerCount()
org.apache.hadoop.hbase.TableName	getTablename() Provides the name of the table that is protected from random Chaos monkey activity
static void	main(String[] args)
private int	runTest()
int	runTestFromCommandLine()
protected void	setConnection(org.apache.hadoop.hbase.client.Connection connection)
void	setUpCluster()
void	testAsUnitTest()
protected void	verifyNamespaces()
protected void	verifyTables()

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

addOptions, cleanUp, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, processOptions, setUp, setUpMonkey, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

SERVER_COUNT

private static final int SERVER_COUNT

See Also:

Constant Field Values

DEFAULT_RUN_TIME

protected static final long DEFAULT_RUN_TIME

See Also:

Constant Field Values

DEFAULT_NUM_THREADS

protected static final int DEFAULT_NUM_THREADS

See Also:

Constant Field Values

DEFAULT_NUM_REGIONS

protected static final int DEFAULT_NUM_REGIONS

See Also:

Constant Field Values

keepObjectsAtTheEnd

private boolean keepObjectsAtTheEnd

cluster

protected HBaseCluster cluster

connection

protected org.apache.hadoop.hbase.client.Connection connection

RUN_TIME_KEY

protected static final String RUN_TIME_KEY

A soft limit on how long we should run

See Also:

Constant Field Values

NUM_THREADS_KEY

protected static final String NUM_THREADS_KEY

See Also:

Constant Field Values

NUM_REGIONS_KEY

protected static final String NUM_REGIONS_KEY

See Also:

Constant Field Values

running

protected AtomicBoolean running

create_table

protected AtomicBoolean create_table

numThreads

protected int numThreads

numRegions

protected int numRegions

namespaceMap

ConcurrentHashMap<String,org.apache.hadoop.hbase.NamespaceDescriptor> namespaceMap

enabledTables

ConcurrentHashMap<org.apache.hadoop.hbase.TableName,org.apache.hadoop.hbase.client.TableDescriptor> enabledTables

disabledTables

ConcurrentHashMap<org.apache.hadoop.hbase.TableName,org.apache.hadoop.hbase.client.TableDescriptor> disabledTables

deletedTables

ConcurrentHashMap<org.apache.hadoop.hbase.TableName,org.apache.hadoop.hbase.client.TableDescriptor> deletedTables

Constructor Detail

IntegrationTestDDLMasterFailover

public IntegrationTestDDLMasterFailover()

Method Detail

setUpCluster

public void setUpCluster()
                  throws Exception

Specified by:

setUpCluster in class IntegrationTestBase

Throws:

Exception

cleanUpCluster

public void cleanUpCluster()
                    throws Exception

Overrides:

cleanUpCluster in class IntegrationTestBase

Throws:

Exception

getMinServerCount

protected int getMinServerCount()

setConnection

protected void setConnection(org.apache.hadoop.hbase.client.Connection connection)

getConnection

protected org.apache.hadoop.hbase.client.Connection getConnection()

verifyNamespaces

protected void verifyNamespaces()
                         throws IOException

Throws:

IOException

verifyTables

protected void verifyTables()
                     throws IOException

Throws:

IOException

testAsUnitTest

public void testAsUnitTest()
                    throws Exception

Throws:

Exception

runTestFromCommandLine

public int runTestFromCommandLine()
                           throws Exception

Specified by:

runTestFromCommandLine in class IntegrationTestBase

Throws:

Exception

checkException

private void checkException(List<IntegrationTestDDLMasterFailover.Worker> workers)

runTest

private int runTest()
             throws Exception

Throws:

Exception

getTablename

public org.apache.hadoop.hbase.TableName getTablename()

Description copied from class: IntegrationTestBase

Provides the name of the table that is protected from random Chaos monkey activity

Specified by:

getTablename in class IntegrationTestBase

Returns:

table to not delete.

getColumnFamilies

protected Set<String> getColumnFamilies()

Description copied from class: IntegrationTestBase

Provides the name of the CFs that are protected from random Chaos monkey activity (alter)

Specified by:

getColumnFamilies in class IntegrationTestBase

Returns:

set of cf names to protect.

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception