Package org.apache.hadoop.hbase

Class IntegrationTestingUtility

java.lang.Object

org.apache.hadoop.hbase.HBaseCommonTestingUtil

org.apache.hadoop.hbase.HBaseZKTestingUtil

org.apache.hadoop.hbase.HBaseTestingUtil

org.apache.hadoop.hbase.IntegrationTestingUtility

public class IntegrationTestingUtility
extends HBaseTestingUtil

Facility for integration/system tests. This extends HBaseTestingUtil and adds-in the functionality needed by integration and system tests. This class understands distributed and pseudo-distributed/local cluster deployments, and abstracts those from the tests in this module.

IntegrationTestingUtility is constructed and used by the integration tests, but the tests themselves should not assume a particular deployment. They can rely on the methods in this class and HBaseCluster. Before the testing begins, the test should initialize the cluster by calling initializeCluster(int).

The cluster that is used defaults to a mini cluster, but it can be forced to use a distributed cluster by calling setUseDistributedCluster(Configuration). This method is invoked by test drivers (maven, IntegrationTestsDriver, etc) before initializing the cluster via initializeCluster(int). Individual tests should not directly call setUseDistributedCluster(Configuration).

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.HBaseTestingUtil

HBaseTestingUtil.SeenRowTracker

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.HBaseCommonTestingUtil

HBaseCommonTestingUtil.PortAllocator

Field Summary

Fields

Modifier and Type	Field	Description
private static final Class<? extends ClusterManager>	DEFAULT_HBASE_CLUSTER_MANAGER_CLASS
static final int	DEFAULT_REGIONS_PER_SERVER	The default number of regions per regionserver when creating a pre-split table.
static final String	HBASE_CLUSTER_MANAGER_CLASS	Config for pluggable hbase cluster manager.
static final String	IS_DISTRIBUTED_CLUSTER	Configuration that controls whether this utility assumes a running/deployed cluster.
private static final org.slf4j.Logger	LOG
static final boolean	PRESPLIT_TEST_TABLE
static final String	PRESPLIT_TEST_TABLE_KEY
static final String	REGIONS_PER_SERVER_KEY

Fields inherited from class org.apache.hadoop.hbase.HBaseTestingUtil

BLOOM_AND_COMPRESSION_COMBINATIONS, COLUMNS, fam1, fam2, fam3, FIRST_CHAR, KEYS, KEYS_FOR_HBA_CREATE_TABLE, LAST_CHAR, MEMSTORETS_TAGS_PARAMETRIZED, ROWS, START_KEY, START_KEY_BYTES

Fields inherited from class org.apache.hadoop.hbase.HBaseZKTestingUtil

clusterTestDir, zooKeeperWatcher

Fields inherited from class org.apache.hadoop.hbase.HBaseCommonTestingUtil

BASE_TEST_DIRECTORY_KEY, BOOLEAN_PARAMETERIZED, COMPRESSION_ALGORITHMS, COMPRESSION_ALGORITHMS_PARAMETERIZED, conf, DEFAULT_BASE_TEST_DIRECTORY

Constructor Summary

Constructors

Constructor	Description
IntegrationTestingUtility()
IntegrationTestingUtility(org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method	Description
void	checkNodeCount(int numSlaves)	Checks whether we have more than numSlaves nodes.
void	createDistributedHBaseCluster()
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor td, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor[] cds, int numRegionsPerServer)	Creates a pre-split table for load testing.
void	initializeCluster(int numSlaves)	Initializes the state of the cluster.
boolean	isDistributedCluster()	Returns whether we are interacting with a distributed cluster as opposed to and in-process mini cluster or a local cluster.
void	restoreCluster()	Restores the cluster to the initial state if it is a distributed cluster, otherwise, shutdowns the mini cluster.
static void	setUseDistributedCluster(org.apache.hadoop.conf.Configuration conf)	Sets the configuration property to use a distributed cluster for the integration tests.

Methods inherited from class org.apache.hadoop.hbase.HBaseTestingUtil

assertKVListsEqual, assertRegionOnlyOnServer, assertRegionOnServer, assignRegion, available, await, checksumRows, cleanupDataTestDirOnTestFS, cleanupDataTestDirOnTestFS, closeConnection, closeRegionAndWAL, closeRegionAndWAL, compact, compact, countRows, countRows, countRows, countRows, countRows, countRows, countRows, createLocalHRegion, createLocalHRegion, createLocalHRegion, createLocalHRegion, createLocalHRegionWithInMemoryFlags, createMockRegionServerService, createMockRegionServerService, createMockRegionServerService, createModifyableTableDescriptor, createModifyableTableDescriptor, createMultiRegionsInMeta, createMultiRegionTable, createMultiRegionTable, createMultiRegionTable, createMultiRegionTable, createMultiRegionTable, createRandomTable, createRegionAndWAL, createRegionAndWAL, createRegionAndWAL, createRegionAndWAL, createRegionDir, createRegionDir, createRootDir, createRootDir, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTable, createTableDescriptor, createTableDescriptor, createTableDescriptor, createTableDescriptor, createTestRegion, createTestRegion, createWal, createWALRootDir, deleteNumericRows, deleteTable, deleteTableData, deleteTableIfAny, ensureSomeNonStoppedRegionServersAvailable, ensureSomeRegionServersAvailable, expireMasterSession, expireRegionServerSession, expireSession, expireSession, explainTableAvailability, explainTableState, findLastTableState, flush, flush, generateColumnDescriptors, generateColumnDescriptors, getAdmin, getAllOnlineRegions, getAsyncConnection, getAsyncConnection, getClosestRowBefore, getClusterKey, getConfiguration, getConnection, getConnection, getDataTestDirOnTestFS, getDataTestDirOnTestFS, getDefaultRootDirPath, getDefaultRootDirPath, getDFSCluster, getDifferentUser, getFromStoreFile, getFromStoreFile, getHBaseCluster, getHBaseClusterInterface, getHbck, getMetaRSPort, getMiniHBaseCluster, getNumHFiles, getNumHFilesForRS, getOtherRegionServer, getRegionSplitStartKeys, getRpcConnnectionURI, getRSForFirstRegionInTable, getSplittableRegion, getSupportedCompressionAlgorithms, getTestFileSystem, getZkConnectionURI, invalidateConnection, isNewVersionBehaviorEnabled, isReadShortCircuitOn, killMiniHBaseCluster, loadNumericRows, loadRandomRows, loadRegion, loadRegion, loadRegion, loadTable, loadTable, loadTable, loadTable, loadTable, memStoreTSTagsAndOffheapCombination, moveRegionAndWait, predicateNoRegionsInTransition, predicateTableAvailable, predicateTableDisabled, predicateTableEnabled, randomFreePort, randomMultiCastAddress, restartHBaseCluster, restartHBaseCluster, restartHBaseCluster, safeGetAsStr, setDFSCluster, setDFSCluster, setFileSystemURI, setHBaseCluster, setMaxRecoveryErrorCount, setReplicas, setReplicas, setupDataTestDir, setupMiniKdc, shutdownMiniCluster, shutdownMiniDFSCluster, shutdownMiniHBaseCluster, shutdownMiniMapReduceCluster, startMiniCluster, startMiniCluster, startMiniCluster, startMiniDFSCluster, startMiniDFSCluster, startMiniDFSCluster, startMiniDFSCluster, startMiniDFSClusterForTestWAL, startMiniHBaseCluster, startMiniHBaseCluster, startMiniHBaseCluster, startMiniHBaseCluster, startMiniHBaseCluster, startMiniMapReduceCluster, startSecureMiniCluster, truncateTable, truncateTable, unassignRegion, unassignRegion, unassignRegionByRow, unassignRegionByRow, verifyNumericRows, verifyNumericRows, verifyNumericRows, verifyNumericRows, verifyNumericRows, verifyTableDescriptorIgnoreTableName, waitForHostPort, waitLabelAvailable, waitTableAvailable, waitTableAvailable, waitTableAvailable, waitTableDisabled, waitTableDisabled, waitTableDisabled, waitTableEnabled, waitTableEnabled, waitTableEnabled, waitUntilAllRegionsAssigned, waitUntilAllRegionsAssigned, waitUntilAllSystemRegionsAssigned, waitUntilNoRegionsInTransition, waitUntilNoRegionsInTransition

Methods inherited from class org.apache.hadoop.hbase.HBaseZKTestingUtil

cleanupTestDir, getClusterTestDir, getZkCluster, getZooKeeperWatcher, setupClusterTestDir, setZkCluster, shutdownMiniZKCluster, startMiniZKCluster, startMiniZKCluster

Methods inherited from class org.apache.hadoop.hbase.HBaseCommonTestingUtil

cleanupTestDir, createSubDir, deleteDir, deleteOnExit, getDataTestDir, getDataTestDir, getRandomDir, getRandomUUID, waitFor, waitFor, waitFor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

IS_DISTRIBUTED_CLUSTER

public static final String IS_DISTRIBUTED_CLUSTER

Configuration that controls whether this utility assumes a running/deployed cluster. This is different than "hbase.cluster.distributed" since that parameter indicates whether the cluster is in an actual distributed environment, while this shows that there is a deployed (distributed or pseudo-distributed) cluster running, and we do not need to start a mini-cluster for tests.

See Also:

• Constant Field Values

HBASE_CLUSTER_MANAGER_CLASS

public static final String HBASE_CLUSTER_MANAGER_CLASS

Config for pluggable hbase cluster manager. Pass fully-qualified class name as property value. Drop the '.class' suffix.

See Also:

• Constant Field Values

DEFAULT_HBASE_CLUSTER_MANAGER_CLASS

private static final Class<? extends ClusterManager> DEFAULT_HBASE_CLUSTER_MANAGER_CLASS

REGIONS_PER_SERVER_KEY

public static final String REGIONS_PER_SERVER_KEY

See Also:

• Constant Field Values

DEFAULT_REGIONS_PER_SERVER

public static final int DEFAULT_REGIONS_PER_SERVER

The default number of regions per regionserver when creating a pre-split table.

See Also:

• Constant Field Values

PRESPLIT_TEST_TABLE_KEY

public static final String PRESPLIT_TEST_TABLE_KEY

See Also:

• Constant Field Values

PRESPLIT_TEST_TABLE

public static final boolean PRESPLIT_TEST_TABLE

See Also:

• Constant Field Values

Constructor Details

IntegrationTestingUtility

public IntegrationTestingUtility()

IntegrationTestingUtility

public IntegrationTestingUtility(org.apache.hadoop.conf.Configuration conf)

Method Details

initializeCluster

public void initializeCluster(int numSlaves)
                       throws Exception

Initializes the state of the cluster. It starts a new in-process mini cluster, OR if we are given an already deployed distributed cluster it initializes the state.

Parameters:

numSlaves - Number of slaves to start up if we are booting a mini cluster. Otherwise we check whether this many nodes are available and throw an exception if not.

Throws:

Exception

checkNodeCount

public void checkNodeCount(int numSlaves)
                    throws Exception

Checks whether we have more than numSlaves nodes. Throws an exception otherwise.

Throws:

Exception

restoreCluster

public void restoreCluster()
                    throws IOException

Restores the cluster to the initial state if it is a distributed cluster, otherwise, shutdowns the mini cluster.

Throws:

IOException

setUseDistributedCluster

public static void setUseDistributedCluster(org.apache.hadoop.conf.Configuration conf)

Sets the configuration property to use a distributed cluster for the integration tests. Test drivers should use this to enforce cluster deployment.

isDistributedCluster

public boolean isDistributedCluster()

Returns whether we are interacting with a distributed cluster as opposed to and in-process mini cluster or a local cluster.

See Also:

• setUseDistributedCluster(Configuration)

createDistributedHBaseCluster

public void createDistributedHBaseCluster()
                                   throws IOException

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor td,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor[] cds,
 int numRegionsPerServer)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException