Package org.apache.hadoop.hbase

Class HBaseTestingUtil

java.lang.Object

org.apache.hadoop.hbase.HBaseCommonTestingUtil

org.apache.hadoop.hbase.HBaseZKTestingUtil

org.apache.hadoop.hbase.HBaseTestingUtil

Direct Known Subclasses:

IntegrationTestingUtility

@LimitedPrivate("Phoenix")
@Evolving
public class HBaseTestingUtil
extends HBaseZKTestingUtil

Facility for testing HBase. Replacement for old HBaseTestCase and HBaseClusterTestCase functionality. Create an instance and keep it around testing HBase.

This class is meant to be your one-stop shop for anything you might need testing. Manages one cluster at a time only. Managed cluster can be an in-process SingleProcessHBaseCluster, or a deployed cluster of type DistributedHBaseCluster. Not all methods work with the real cluster.

Depends on log4j being on classpath and hbase-site.xml for logging and test-run configuration.

It does not set logging levels.

In the configuration properties, default values for master-info-port and region-server-port are overridden such that a random port will be assigned (thus avoiding port contention if another local HBase instance is already running).

To preserve test data directories, pass the system property "hbase.testing.preserve.testdir" setting it to true.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static final class	HBaseTestingUtil.FsDatasetAsyncDiskServiceFixer
static class	HBaseTestingUtil.SeenRowTracker	A tracker for tracking and validating table rows generated with loadTable(Table, byte[])

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.HBaseCommonTestingUtil

HBaseCommonTestingUtil.PortAllocator

Field Summary

Fields

Modifier and Type	Field	Description
private final AtomicReference<org.apache.hadoop.hbase.client.AsyncClusterConnection>	asyncConnection
static final Collection<Object[]>	BLOOM_AND_COMPRESSION_COMBINATIONS
static final byte[][]	COLUMNS
private org.apache.hadoop.fs.Path	dataTestDirOnTestFS	Directory on test filesystem where we put the data for this instance of HBaseTestingUtility
static final int	DEFAULT_REGIONS_PER_SERVER	The default number of regions per regionserver when creating a pre-split table.
private org.apache.hadoop.hdfs.MiniDFSCluster	dfsCluster
private HBaseTestingUtil.FsDatasetAsyncDiskServiceFixer	dfsClusterFixer
static final byte[]	fam1
static final byte[]	fam2
static final byte[]	fam3
static final char	FIRST_CHAR
private static String	FS_URI	Filesystem URI used for map-reduce mini-cluster setup
private String	hadoopLogDir
private org.apache.hadoop.hbase.client.Admin	hbaseAdmin
private HBaseClusterInterface	hbaseCluster
static final byte[][]	KEYS
static final byte[][]	KEYS_FOR_HBA_CREATE_TABLE
static final char	LAST_CHAR
private static final int	MAXVERSIONS
static final List<Object[]>	MEMSTORETS_TAGS_PARAMETRIZED	This is for unit tests parameterized with a single boolean.
private boolean	miniClusterRunning	If there is a mini cluster running for this testing utility instance.
private org.apache.hadoop.mapred.MiniMRCluster	mrCluster
static final boolean	PRESPLIT_TEST_TABLE
static final String	PRESPLIT_TEST_TABLE_KEY
static final String	REGIONS_PER_SERVER_KEY
static final byte[][]	ROWS	All the row values for the data loaded by loadTable(Table, byte[])
static final String	START_KEY
static final byte[]	START_KEY_BYTES

Fields inherited from class org.apache.hadoop.hbase.HBaseZKTestingUtil

clusterTestDir, zooKeeperWatcher

Fields inherited from class org.apache.hadoop.hbase.HBaseCommonTestingUtil

BASE_TEST_DIRECTORY_KEY, BOOLEAN_PARAMETERIZED, COMPRESSION_ALGORITHMS, COMPRESSION_ALGORITHMS_PARAMETERIZED, conf, DEFAULT_BASE_TEST_DIRECTORY, LOG

Constructor Summary

Constructors

Constructor	Description
HBaseTestingUtil()	Create an HBaseTestingUtility using a default configuration.
HBaseTestingUtil(org.apache.hadoop.conf.Configuration conf)	Create an HBaseTestingUtility using a given configuration.

Method Summary

Modifier and Type	Method	Description
static void	assertKVListsEqual(String additionalMsg, List<? extends org.apache.hadoop.hbase.Cell> expected, List<? extends org.apache.hadoop.hbase.Cell> actual)
void	assertRegionOnlyOnServer(org.apache.hadoop.hbase.client.RegionInfo hri, org.apache.hadoop.hbase.ServerName server, long timeout)	Check to make sure the region is open on the specified region server, but not on any other one.
void	assertRegionOnServer(org.apache.hadoop.hbase.client.RegionInfo hri, org.apache.hadoop.hbase.ServerName server, long timeout)	Due to async racing issue, a region may not be in the online region list of a region server yet, after the assignment znode is deleted and the new assignment is recorded in master.
boolean	assignRegion(org.apache.hadoop.hbase.client.RegionInfo regionInfo)	Uses directly the assignment manager to assign the region.
static boolean	available(int port)	Checks to see if a specific port is available.
static void	await(long sleepMillis, BooleanSupplier condition)	Await the successful return of condition, sleeping sleepMillis between invocations.
private static List<Object[]>	bloomAndCompressionCombinations()	Create all combinations of Bloom filters and compression algorithms for testing.
String	checksumRows(org.apache.hadoop.hbase.client.Table table)	Return an md5 digest of the entire contents of a table.
private void	cleanup()
boolean	cleanupDataTestDirOnTestFS()	Cleans the test data directory on the test filesystem.
boolean	cleanupDataTestDirOnTestFS(String subdirName)	Cleans a subdirectory under the test data directory on the test filesystem.
void	closeConnection()
static void	closeRegionAndWAL(org.apache.hadoop.hbase.regionserver.HRegion r)	Close both the HRegion r and it's underlying WAL.
static void	closeRegionAndWAL(org.apache.hadoop.hbase.regionserver.Region r)	Close both the region r and it's underlying WAL.
void	compact(boolean major)	Compact all regions in the mini hbase cluster
void	compact(org.apache.hadoop.hbase.TableName tableName, boolean major)	Compact all of a table's reagion in the mini hbase cluster
static int	countRows(org.apache.hadoop.hbase.client.Table table)	Return the number of rows in the given table.
static int	countRows(org.apache.hadoop.hbase.client.Table table, byte[]... families)
static int	countRows(org.apache.hadoop.hbase.client.Table table, org.apache.hadoop.hbase.client.Scan scan)
static int	countRows(org.apache.hadoop.hbase.regionserver.InternalScanner scanner)
static int	countRows(org.apache.hadoop.hbase.regionserver.Region region)
static int	countRows(org.apache.hadoop.hbase.regionserver.Region region, org.apache.hadoop.hbase.client.Scan scan)
int	countRows(org.apache.hadoop.hbase.TableName tableName)	Return the number of rows in the given table.
private String	createDirAndSetProperty(String property)
private String	createDirAndSetProperty(String relPath, String property)
private void	createDirsAndSetProperties()	This is used before starting HDFS and map-reduce mini-clusters Run something like the below to check for the likes of '/tmp' references -- i.e.
org.apache.hadoop.hbase.regionserver.HRegion	createLocalHRegion(org.apache.hadoop.hbase.client.RegionInfo info, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor desc, org.apache.hadoop.hbase.wal.WAL wal)	Create an HRegion that writes to the local tmp dirs with specified wal
org.apache.hadoop.hbase.regionserver.HRegion	createLocalHRegion(org.apache.hadoop.hbase.client.RegionInfo info, org.apache.hadoop.hbase.client.TableDescriptor desc)	Create an HRegion that writes to the local tmp dirs.
org.apache.hadoop.hbase.regionserver.HRegion	createLocalHRegion(org.apache.hadoop.hbase.client.TableDescriptor desc, byte[] startKey, byte[] endKey)	Create an HRegion that writes to the local tmp dirs
org.apache.hadoop.hbase.regionserver.HRegion	createLocalHRegion(org.apache.hadoop.hbase.TableName tableName, byte[] startKey, byte[] stopKey, org.apache.hadoop.conf.Configuration conf, boolean isReadOnly, org.apache.hadoop.hbase.client.Durability durability, org.apache.hadoop.hbase.wal.WAL wal, byte[]... families)
org.apache.hadoop.hbase.regionserver.HRegion	createLocalHRegionWithInMemoryFlags(org.apache.hadoop.hbase.TableName tableName, byte[] startKey, byte[] stopKey, org.apache.hadoop.conf.Configuration conf, boolean isReadOnly, org.apache.hadoop.hbase.client.Durability durability, org.apache.hadoop.hbase.wal.WAL wal, boolean[] compactedMemStore, byte[]... families)
org.apache.hadoop.hbase.regionserver.RegionServerServices	createMockRegionServerService()	Create a stubbed out RegionServerService, mainly for getting FS.
org.apache.hadoop.hbase.regionserver.RegionServerServices	createMockRegionServerService(org.apache.hadoop.hbase.ipc.RpcServerInterface rpc)	Create a stubbed out RegionServerService, mainly for getting FS.
org.apache.hadoop.hbase.regionserver.RegionServerServices	createMockRegionServerService(org.apache.hadoop.hbase.ServerName name)	Create a stubbed out RegionServerService, mainly for getting FS.
org.apache.hadoop.hbase.client.TableDescriptorBuilder	createModifyableTableDescriptor(String name)
org.apache.hadoop.hbase.client.TableDescriptorBuilder	createModifyableTableDescriptor(org.apache.hadoop.hbase.TableName name, int minVersions, int versions, int ttl, org.apache.hadoop.hbase.KeepDeletedCells keepDeleted)
List<org.apache.hadoop.hbase.client.RegionInfo>	createMultiRegionsInMeta(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor htd, byte[][] startKeys)	Create rows in hbase:meta for regions of the specified table with the specified start keys.
org.apache.hadoop.hbase.client.Table	createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName, byte[] family)	Create a table with multiple regions.
org.apache.hadoop.hbase.client.Table	createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families)	Create a table with multiple regions.
org.apache.hadoop.hbase.client.Table	createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int numVersions)	Create a table with multiple regions.
org.apache.hadoop.hbase.client.Table	createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName, byte[] family, int numRegions)	Create a table with multiple regions.
org.apache.hadoop.hbase.client.Table	createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName, int replicaCount, byte[][] families)	Create a table with multiple regions.
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor desc, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor hcd)	Creates a pre-split table for load testing.
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor desc, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor[] hcds, int numRegionsPerServer)	Creates a pre-split table for load testing.
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor td, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor[] cds, org.apache.hadoop.hbase.util.RegionSplitter.SplitAlgorithm splitter, int numRegionsPerServer)	Creates a pre-split table for load testing.
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor desc, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor hcd, int numRegionsPerServer)	Creates a pre-split table for load testing.
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.TableName tableName, byte[][] columnFamilies, org.apache.hadoop.hbase.io.compress.Compression.Algorithm compression, org.apache.hadoop.hbase.io.encoding.DataBlockEncoding dataBlockEncoding, int numRegionsPerServer, int regionReplication, org.apache.hadoop.hbase.client.Durability durability)	Creates a pre-split table for load testing.
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.TableName tableName, byte[] columnFamily, org.apache.hadoop.hbase.io.compress.Compression.Algorithm compression, org.apache.hadoop.hbase.io.encoding.DataBlockEncoding dataBlockEncoding)	Creates a pre-split table for load testing.
static int	createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.TableName tableName, byte[] columnFamily, org.apache.hadoop.hbase.io.compress.Compression.Algorithm compression, org.apache.hadoop.hbase.io.encoding.DataBlockEncoding dataBlockEncoding, int numRegionsPerServer, int regionReplication, org.apache.hadoop.hbase.client.Durability durability)	Creates a pre-split table for load testing.
org.apache.hadoop.hbase.client.Table	createRandomTable(org.apache.hadoop.hbase.TableName tableName, Collection<String> families, int maxVersions, int numColsPerRow, int numFlushes, int numRegions, int numRowsPerFlush)	Creates a random table with the given parameters
static org.apache.hadoop.hbase.regionserver.HRegion	createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor htd)	Create a region with it's own WAL.
static org.apache.hadoop.hbase.regionserver.HRegion	createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor htd, boolean initialize)	Create a region with it's own WAL.
static org.apache.hadoop.hbase.regionserver.HRegion	createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor htd, org.apache.hadoop.hbase.io.hfile.BlockCache blockCache)	Create a region with it's own WAL.
static org.apache.hadoop.hbase.regionserver.HRegion	createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor htd, org.apache.hadoop.hbase.mob.MobFileCache mobFileCache)	Create a region with it's own WAL.
org.apache.hadoop.fs.Path	createRootDir()	Same as createRootDir(boolean create) except that create flag is false.
org.apache.hadoop.fs.Path	createRootDir(boolean create)	Creates an hbase rootdir in user home directory.
private void	createSubDirAndSystemProperty(String propertyName, org.apache.hadoop.fs.Path parent, String subDirName)
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.client.TableDescriptor htd, byte[][] splitRows)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.client.TableDescriptor htd, byte[][] families, byte[][] splitKeys, org.apache.hadoop.conf.Configuration c)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.client.TableDescriptor htd, byte[][] families, byte[][] splitKeys, org.apache.hadoop.hbase.regionserver.BloomType type, int blockSize, org.apache.hadoop.conf.Configuration c)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.client.TableDescriptor htd, byte[][] families, org.apache.hadoop.conf.Configuration c)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[] family)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, byte[][] splitKeys)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, byte[][] splitKeys, int replicaCount)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, byte[][] splitKeys, int replicaCount, org.apache.hadoop.conf.Configuration c)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int numVersions)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int[] numVersions)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int numVersions, byte[][] splitKeys)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int numVersions, byte[] startKey, byte[] endKey, int numRegions)
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int numVersions, int blockSize)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int numVersions, int blockSize, String cpName)
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[] family, byte[][] splitRows)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, byte[] family, int numVersions)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, String family)	Create a table.
org.apache.hadoop.hbase.client.Table	createTable(org.apache.hadoop.hbase.TableName tableName, String[] families)	Create a table.
org.apache.hadoop.hbase.client.TableDescriptor	createTableDescriptor(org.apache.hadoop.hbase.TableName name)	Create a table of name name.
org.apache.hadoop.hbase.client.TableDescriptor	createTableDescriptor(org.apache.hadoop.hbase.TableName tableName, byte[] family)
org.apache.hadoop.hbase.client.TableDescriptor	createTableDescriptor(org.apache.hadoop.hbase.TableName tableName, byte[][] families, int maxVersions)
org.apache.hadoop.hbase.client.TableDescriptor	createTableDescriptor(org.apache.hadoop.hbase.TableName name, int minVersions, int versions, int ttl, org.apache.hadoop.hbase.KeepDeletedCells keepDeleted)
org.apache.hadoop.hbase.regionserver.HRegion	createTestRegion(String tableName, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor cd)
org.apache.hadoop.hbase.regionserver.HRegion	createTestRegion(String tableName, org.apache.hadoop.hbase.client.ColumnFamilyDescriptor cd, org.apache.hadoop.hbase.io.hfile.BlockCache blockCache)
static org.apache.hadoop.hbase.wal.WAL	createWal(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.hbase.client.RegionInfo hri)	Create an unmanaged WAL.
org.apache.hadoop.fs.Path	createWALRootDir()	Creates a hbase walDir in the user's home directory.
private void	decrementMinRegionServerCount()
private void	decrementMinRegionServerCount(org.apache.hadoop.conf.Configuration conf)
void	deleteNumericRows(org.apache.hadoop.hbase.client.Table t, byte[] f, int startRow, int endRow)
void	deleteTable(org.apache.hadoop.hbase.TableName tableName)	Drop an existing table
org.apache.hadoop.hbase.client.Table	deleteTableData(org.apache.hadoop.hbase.TableName tableName)	Provide an existing table name to truncate.
void	deleteTableIfAny(org.apache.hadoop.hbase.TableName tableName)	Drop an existing table
private void	enableShortCircuit()	Enable the short circuit read, unless configured differently.
boolean	ensureSomeNonStoppedRegionServersAvailable(int num)	Make sure that at least the specified number of region servers are running.
boolean	ensureSomeRegionServersAvailable(int num)	Make sure that at least the specified number of region servers are running
void	expireMasterSession()	Expire the Master's session
void	expireRegionServerSession(int index)	Expire a region server's session
void	expireSession(org.apache.hadoop.hbase.zookeeper.ZKWatcher nodeZK)
void	expireSession(org.apache.hadoop.hbase.zookeeper.ZKWatcher nodeZK, boolean checkStatus)	Expire a ZooKeeper session as recommended in ZooKeeper documentation http://hbase.apache.org/book.html#trouble.zookeeper
String	explainTableAvailability(org.apache.hadoop.hbase.TableName tableName)
String	explainTableState(org.apache.hadoop.hbase.TableName table, org.apache.hadoop.hbase.client.TableState.State state)
org.apache.hadoop.hbase.client.TableState	findLastTableState(org.apache.hadoop.hbase.TableName table)
void	flush()	Flushes all caches in the mini hbase cluster
void	flush(org.apache.hadoop.hbase.TableName tableName)	Flushes all caches in the mini hbase cluster
private void	forceChangeTaskLogDir()	Tasktracker has a bug where changing the hadoop.log.dir system property will not change its internal static LOG_DIR variable.
static List<org.apache.hadoop.hbase.client.ColumnFamilyDescriptor>	generateColumnDescriptors()	Create a set of column descriptors with the combination of compression, encoding, bloom codecs available.
static List<org.apache.hadoop.hbase.client.ColumnFamilyDescriptor>	generateColumnDescriptors(String prefix)	Create a set of column descriptors with the combination of compression, encoding, bloom codecs available.
org.apache.hadoop.hbase.client.Admin	getAdmin()	Returns an Admin instance which is shared between HBaseTestingUtility instance users.
static NavigableSet<String>	getAllOnlineRegions(SingleProcessHBaseCluster cluster)
org.apache.hadoop.hbase.client.AsyncClusterConnection	getAsyncConnection()	Get a shared AsyncClusterConnection to the cluster.
org.apache.hadoop.hbase.client.AsyncClusterConnection	getAsyncConnection(org.apache.hadoop.hbase.security.User user)	Get a assigned AsyncClusterConnection to the cluster.
private org.apache.hadoop.fs.Path	getBaseTestDirOnTestFS()
org.apache.hadoop.hbase.client.Result	getClosestRowBefore(org.apache.hadoop.hbase.regionserver.Region r, byte[] row, byte[] family)
String	getClusterKey()	Deprecated. since 2.7.0, will be removed in 4.0.0.
org.apache.hadoop.conf.Configuration	getConfiguration()	Returns this classes's instance of Configuration.
org.apache.hadoop.hbase.client.Connection	getConnection()	Get a shared Connection to the cluster.
org.apache.hadoop.hbase.client.Connection	getConnection(org.apache.hadoop.hbase.security.User user)	Get a assigned Connection to the cluster.
org.apache.hadoop.fs.Path	getDataTestDirOnTestFS()	Returns a Path in the test filesystem, obtained from getTestFileSystem() to write temporary test data.
org.apache.hadoop.fs.Path	getDataTestDirOnTestFS(String subdirName)	Returns a Path in the test filesystem, obtained from getTestFileSystem() to write temporary test data.
org.apache.hadoop.fs.Path	getDefaultRootDirPath()	Same as {getDefaultRootDirPath(boolean create) except that create flag is false.
org.apache.hadoop.fs.Path	getDefaultRootDirPath(boolean create)	Returns the path to the default root dir the minicluster uses.
org.apache.hadoop.hdfs.MiniDFSCluster	getDFSCluster()
static org.apache.hadoop.hbase.security.User	getDifferentUser(org.apache.hadoop.conf.Configuration c, String differentiatingSuffix)	This method clones the passed c configuration setting a new user into the clone.
static List<org.apache.hadoop.hbase.Cell>	getFromStoreFile(org.apache.hadoop.hbase.regionserver.HStore store, byte[] row, NavigableSet<byte[]> columns)	Do a small get/scan against one store.
static List<org.apache.hadoop.hbase.Cell>	getFromStoreFile(org.apache.hadoop.hbase.regionserver.HStore store, org.apache.hadoop.hbase.client.Get get)	Do a small get/scan against one store.
SingleProcessHBaseCluster	getHBaseCluster()	Get the Mini HBase cluster.
HBaseClusterInterface	getHBaseClusterInterface()	Returns the HBaseCluster instance.
org.apache.hadoop.hbase.client.Hbck	getHbck()	Returns an Hbck instance.
static int	getMetaRSPort(org.apache.hadoop.hbase.client.Connection connection)
SingleProcessHBaseCluster	getMiniHBaseCluster()	Returns current mini hbase cluster.
private org.apache.hadoop.fs.Path	getNewDataTestDirOnTestFS()	Sets up a new path in test filesystem to be used by tests.
int	getNumHFiles(org.apache.hadoop.hbase.TableName tableName, byte[] family)
int	getNumHFilesForRS(org.apache.hadoop.hbase.regionserver.HRegionServer rs, org.apache.hadoop.hbase.TableName tableName, byte[] family)
org.apache.hadoop.hbase.regionserver.HRegionServer	getOtherRegionServer(org.apache.hadoop.hbase.regionserver.HRegionServer rs)	Find any other region server which is different from the one identified by parameter
byte[][]	getRegionSplitStartKeys(byte[] startKey, byte[] endKey, int numRegions)	Create region split keys between startkey and endKey
String	getRpcConnnectionURI()
org.apache.hadoop.hbase.regionserver.HRegionServer	getRSForFirstRegionInTable(org.apache.hadoop.hbase.TableName tableName)	Tool to get the reference to the region server object that holds the region of the specified user table.
org.apache.hadoop.hbase.regionserver.HRegion	getSplittableRegion(org.apache.hadoop.hbase.TableName tableName, int maxAttempts)	Retrieves a splittable region randomly from tableName
static org.apache.hadoop.hbase.io.compress.Compression.Algorithm[]	getSupportedCompressionAlgorithms()	Get supported compression algorithms.
org.apache.hadoop.fs.FileSystem	getTestFileSystem()
String	getZkConnectionURI()
void	invalidateConnection()	Resets the connections so that the next time getConnection() is called, a new connection is created.
boolean	isNewVersionBehaviorEnabled()	Check whether the tests should assume NEW_VERSION_BEHAVIOR when creating new column families.
boolean	isReadShortCircuitOn()	Get the HBase setting for dfs.client.read.shortcircuit from the conf or a system property.
private boolean	isTargetTable(byte[] inRow, org.apache.hadoop.hbase.Cell c)
void	killMiniHBaseCluster()	Abruptly Shutdown HBase mini cluster.
void	loadNumericRows(org.apache.hadoop.hbase.client.Table t, byte[] f, int startRow, int endRow)
void	loadRandomRows(org.apache.hadoop.hbase.client.Table t, byte[] f, int rowSize, int totalRows)
int	loadRegion(org.apache.hadoop.hbase.regionserver.HRegion r, byte[] f)
int	loadRegion(org.apache.hadoop.hbase.regionserver.HRegion r, byte[] f, boolean flush)	Load region with rows from 'aaa' to 'zzz'.
int	loadRegion(org.apache.hadoop.hbase.regionserver.Region r, byte[] f)
int	loadTable(org.apache.hadoop.hbase.client.Table t, byte[] f)	Load table with rows from 'aaa' to 'zzz'.
int	loadTable(org.apache.hadoop.hbase.client.Table t, byte[][] f)	Load table of multiple column families with rows from 'aaa' to 'zzz'.
int	loadTable(org.apache.hadoop.hbase.client.Table t, byte[][] f, byte[] value)	Load table of multiple column families with rows from 'aaa' to 'zzz'.
int	loadTable(org.apache.hadoop.hbase.client.Table t, byte[][] f, byte[] value, boolean writeToWAL)	Load table of multiple column families with rows from 'aaa' to 'zzz'.
int	loadTable(org.apache.hadoop.hbase.client.Table t, byte[] f, boolean writeToWAL)	Load table with rows from 'aaa' to 'zzz'.
private static List<Object[]>	memStoreTSAndTagsCombination()	Create combination of memstoreTS and tags
static List<Object[]>	memStoreTSTagsAndOffheapCombination()
void	moveRegionAndWait(org.apache.hadoop.hbase.client.RegionInfo destRegion, org.apache.hadoop.hbase.ServerName destServer)	Move region to destination server and wait till region is completely moved and online
Waiter.ExplainingPredicate<IOException>	predicateNoRegionsInTransition()	Returns a Waiter.Predicate for checking that there are no regions in transition in master
Waiter.Predicate<IOException>	predicateTableAvailable(org.apache.hadoop.hbase.TableName tableName)	Returns a Waiter.Predicate for checking that table is enabled
Waiter.Predicate<IOException>	predicateTableDisabled(org.apache.hadoop.hbase.TableName tableName)	Returns a Waiter.Predicate for checking that table is enabled
Waiter.Predicate<IOException>	predicateTableEnabled(org.apache.hadoop.hbase.TableName tableName)	Returns a Waiter.Predicate for checking that table is enabled
static int	randomFreePort()
static String	randomMultiCastAddress()
void	restartHBaseCluster(int servers)	Starts the hbase cluster up again after shutting it down previously in a test.
void	restartHBaseCluster(int servers, List<Integer> ports)
void	restartHBaseCluster(StartTestingClusterOption option)
static <T> String	safeGetAsStr(List<T> lst, int i)
void	setDFSCluster(org.apache.hadoop.hdfs.MiniDFSCluster cluster)
void	setDFSCluster(org.apache.hadoop.hdfs.MiniDFSCluster cluster, boolean requireDown)	Set the MiniDFSCluster
static void	setFileSystemURI(String fsURI)
private void	setFs()
void	setHBaseCluster(HBaseClusterInterface hbaseCluster)
private void	setHBaseFsTmpDir()
static void	setMaxRecoveryErrorCount(OutputStream stream, int max)	Set maxRecoveryErrorCount in DFSClient.
static void	setReplicas(org.apache.hadoop.hbase.client.Admin admin, org.apache.hadoop.hbase.TableName table, int replicaCount)	Set the number of Region replicas.
static void	setReplicas(org.apache.hadoop.hbase.client.AsyncAdmin admin, org.apache.hadoop.hbase.TableName table, int replicaCount)	Set the number of Region replicas.
protected org.apache.hadoop.fs.Path	setupDataTestDir()	Home our data in a dir under HBaseCommonTestingUtil.DEFAULT_BASE_TEST_DIRECTORY.
private void	setupDataTestDirOnTestFS()	Sets up a path in test filesystem to be used by tests.
org.apache.hadoop.minikdc.MiniKdc	setupMiniKdc(File keytabFile)	Sets up MiniKdc for testing security.
void	shutdownMiniCluster()	Stops mini hbase, zk, and hdfs clusters.
void	shutdownMiniDFSCluster()	Shuts down instance created by call to startMiniDFSCluster(int) or does nothing.
void	shutdownMiniHBaseCluster()	Shutdown HBase mini cluster.Does not shutdown zk or dfs if running.
void	shutdownMiniMapReduceCluster()	Stops the previously started MiniMRCluster.
SingleProcessHBaseCluster	startMiniCluster()	Start up a minicluster of hbase, dfs and zookeeper all using default options.
SingleProcessHBaseCluster	startMiniCluster(int numSlaves)	Start up a minicluster of hbase, dfs and zookeeper clusters with given slave node number.
SingleProcessHBaseCluster	startMiniCluster(StartTestingClusterOption option)	Start up a mini cluster of hbase, optionally dfs and zookeeper if needed.
org.apache.hadoop.hdfs.MiniDFSCluster	startMiniDFSCluster(int servers)	Start a minidfscluster.
org.apache.hadoop.hdfs.MiniDFSCluster	startMiniDFSCluster(int servers, String[] hosts)	Start a minidfscluster.
org.apache.hadoop.hdfs.MiniDFSCluster	startMiniDFSCluster(int servers, String[] racks, String[] hosts)
org.apache.hadoop.hdfs.MiniDFSCluster	startMiniDFSCluster(String[] hosts)	Start a minidfscluster.
org.apache.hadoop.hdfs.MiniDFSCluster	startMiniDFSClusterForTestWAL(int namenodePort)
SingleProcessHBaseCluster	startMiniHBaseCluster()	Starts up mini hbase cluster using default options.
SingleProcessHBaseCluster	startMiniHBaseCluster(int numMasters, int numRegionServers)	Deprecated. since 2.2.0 and will be removed in 4.0.0.
SingleProcessHBaseCluster	startMiniHBaseCluster(int numMasters, int numRegionServers, List<Integer> rsPorts)	Deprecated. since 2.2.0 and will be removed in 4.0.0.
SingleProcessHBaseCluster	startMiniHBaseCluster(int numMasters, int numRegionServers, List<Integer> rsPorts, Class<? extends org.apache.hadoop.hbase.master.HMaster> masterClass, Class<? extends SingleProcessHBaseCluster.MiniHBaseClusterRegionServer> rsClass, boolean createRootDir, boolean createWALDir)	Deprecated. since 2.2.0 and will be removed in 4.0.0.
SingleProcessHBaseCluster	startMiniHBaseCluster(StartTestingClusterOption option)	Starts up mini hbase cluster.
org.apache.hadoop.mapred.MiniMRCluster	startMiniMapReduceCluster()	Starts a MiniMRCluster with a default number of TaskTracker's.
private void	startMiniMapReduceCluster(int servers)	Starts a MiniMRCluster.
org.apache.hadoop.hbase.client.Table	truncateTable(org.apache.hadoop.hbase.TableName tableName)	Truncate a table using the admin command.
org.apache.hadoop.hbase.client.Table	truncateTable(org.apache.hadoop.hbase.TableName tableName, boolean preserveRegions)	Truncate a table using the admin command.
void	unassignRegion(byte[] regionName)	Unassign the named region.
void	unassignRegion(String regionName)	Unassign the named region.
void	unassignRegionByRow(byte[] row, org.apache.hadoop.hbase.client.RegionLocator table)	Closes the region containing the given row.
void	unassignRegionByRow(String row, org.apache.hadoop.hbase.client.RegionLocator table)	Closes the region containing the given row.
void	verifyNumericRows(org.apache.hadoop.hbase.client.Table table, byte[] f, int startRow, int endRow, int replicaId)
void	verifyNumericRows(org.apache.hadoop.hbase.regionserver.HRegion region, byte[] f, int startRow, int endRow)
void	verifyNumericRows(org.apache.hadoop.hbase.regionserver.HRegion region, byte[] f, int startRow, int endRow, boolean present)
void	verifyNumericRows(org.apache.hadoop.hbase.regionserver.Region region, byte[] f, int startRow, int endRow)
void	verifyNumericRows(org.apache.hadoop.hbase.regionserver.Region region, byte[] f, int startRow, int endRow, boolean present)
void	verifyTableDescriptorIgnoreTableName(org.apache.hadoop.hbase.client.TableDescriptor ltd, org.apache.hadoop.hbase.client.TableDescriptor rtd)
static void	waitForHostPort(String host, int port)
void	waitLabelAvailable(long timeoutMillis, String... labels)	Wait until labels is ready in VisibilityLabelsCache.
void	waitTableAvailable(byte[] table, long timeoutMillis)	Wait until all regions in a table have been assigned
void	waitTableAvailable(org.apache.hadoop.hbase.TableName table)	Wait until all regions in a table have been assigned.
void	waitTableAvailable(org.apache.hadoop.hbase.TableName table, long timeoutMillis)
void	waitTableDisabled(byte[] table)	Waits for a table to be 'disabled'.
void	waitTableDisabled(byte[] table, long timeoutMillis)	Waits for a table to be 'disabled'.
void	waitTableDisabled(org.apache.hadoop.hbase.TableName table, long millisTimeout)
void	waitTableEnabled(byte[] table, long timeoutMillis)	Waits for a table to be 'enabled'.
void	waitTableEnabled(org.apache.hadoop.hbase.TableName table)	Waits for a table to be 'enabled'.
void	waitTableEnabled(org.apache.hadoop.hbase.TableName table, long timeoutMillis)
void	waitUntilAllRegionsAssigned(org.apache.hadoop.hbase.TableName tableName)	Wait until all regions for a table in hbase:meta have a non-empty info:server, up to a configuable timeout value (default is 60 seconds) This means all regions have been deployed, master has been informed and updated hbase:meta with the regions deployed server.
void	waitUntilAllRegionsAssigned(org.apache.hadoop.hbase.TableName tableName, long timeout)	Wait until all regions for a table in hbase:meta have a non-empty info:server, or until timeout.
void	waitUntilAllSystemRegionsAssigned()	Waith until all system table's regions get assigned
void	waitUntilNoRegionsInTransition()	Wait until no regions in transition.
void	waitUntilNoRegionsInTransition(long timeout)	Wait until no regions in transition.

Methods inherited from class org.apache.hadoop.hbase.HBaseZKTestingUtil

cleanupTestDir, getClusterTestDir, getZkCluster, getZooKeeperWatcher, setupClusterTestDir, setZkCluster, shutdownMiniZKCluster, startMiniZKCluster, startMiniZKCluster

Methods inherited from class org.apache.hadoop.hbase.HBaseCommonTestingUtil

cleanupTestDir, createSubDir, deleteDir, deleteOnExit, getDataTestDir, getDataTestDir, getRandomDir, getRandomUUID, waitFor, waitFor, waitFor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

REGIONS_PER_SERVER_KEY

public static final String REGIONS_PER_SERVER_KEY

See Also:

• Constant Field Values

DEFAULT_REGIONS_PER_SERVER

public static final int DEFAULT_REGIONS_PER_SERVER

The default number of regions per regionserver when creating a pre-split table.

See Also:

• Constant Field Values

PRESPLIT_TEST_TABLE_KEY

public static final String PRESPLIT_TEST_TABLE_KEY

See Also:

• Constant Field Values

PRESPLIT_TEST_TABLE

public static final boolean PRESPLIT_TEST_TABLE

See Also:

• Constant Field Values

dfsCluster

private org.apache.hadoop.hdfs.MiniDFSCluster dfsCluster

dfsClusterFixer

private HBaseTestingUtil.FsDatasetAsyncDiskServiceFixer dfsClusterFixer

hbaseCluster

private volatile HBaseClusterInterface hbaseCluster

mrCluster

private org.apache.hadoop.mapred.MiniMRCluster mrCluster

miniClusterRunning

private volatile boolean miniClusterRunning

If there is a mini cluster running for this testing utility instance.

hadoopLogDir

private String hadoopLogDir

dataTestDirOnTestFS

private org.apache.hadoop.fs.Path dataTestDirOnTestFS

Directory on test filesystem where we put the data for this instance of HBaseTestingUtility

asyncConnection

private final AtomicReference<org.apache.hadoop.hbase.client.AsyncClusterConnection> asyncConnection

FS_URI

private static String FS_URI

Filesystem URI used for map-reduce mini-cluster setup

MEMSTORETS_TAGS_PARAMETRIZED

public static final List<Object[]> MEMSTORETS_TAGS_PARAMETRIZED

This is for unit tests parameterized with a single boolean.

BLOOM_AND_COMPRESSION_COMBINATIONS

public static final Collection<Object[]> BLOOM_AND_COMPRESSION_COMBINATIONS

fam1

public static final byte[] fam1

fam2

public static final byte[] fam2

fam3

public static final byte[] fam3

COLUMNS

public static final byte[][] COLUMNS

MAXVERSIONS

private static final int MAXVERSIONS

See Also:

• Constant Field Values

FIRST_CHAR

public static final char FIRST_CHAR

See Also:

• Constant Field Values

LAST_CHAR

public static final char LAST_CHAR

See Also:

• Constant Field Values

START_KEY_BYTES

public static final byte[] START_KEY_BYTES

START_KEY

public static final String START_KEY

ROWS

public static final byte[][] ROWS

All the row values for the data loaded by loadTable(Table, byte[])

KEYS

public static final byte[][] KEYS

KEYS_FOR_HBA_CREATE_TABLE

public static final byte[][] KEYS_FOR_HBA_CREATE_TABLE

hbaseAdmin

private org.apache.hadoop.hbase.client.Admin hbaseAdmin

Constructor Details

HBaseTestingUtil

public HBaseTestingUtil()

Create an HBaseTestingUtility using a default configuration.

Initially, all tmp files are written to a local test data directory. Once startMiniDFSCluster(int) is called, either directly or via startMiniCluster(), tmp data will be written to the DFS directory instead.

HBaseTestingUtil

public HBaseTestingUtil(@Nullable
 org.apache.hadoop.conf.Configuration conf)

Create an HBaseTestingUtility using a given configuration.

Initially, all tmp files are written to a local test data directory. Once startMiniDFSCluster(int) is called, either directly or via startMiniCluster(), tmp data will be written to the DFS directory instead.

Parameters:

conf - The configuration to use for further operations

Method Details

available

public static boolean available(int port)

Checks to see if a specific port is available.

Parameters:

port - the port number to check for availability

Returns:

true if the port is available, or false if not

bloomAndCompressionCombinations

private static List<Object[]> bloomAndCompressionCombinations()

Create all combinations of Bloom filters and compression algorithms for testing.

memStoreTSAndTagsCombination

private static List<Object[]> memStoreTSAndTagsCombination()

Create combination of memstoreTS and tags

memStoreTSTagsAndOffheapCombination

public static List<Object[]> memStoreTSTagsAndOffheapCombination()

closeRegionAndWAL

public static void closeRegionAndWAL(org.apache.hadoop.hbase.regionserver.Region r)
                              throws IOException

Close both the region r and it's underlying WAL. For use in tests.

Throws:

IOException

closeRegionAndWAL

public static void closeRegionAndWAL(org.apache.hadoop.hbase.regionserver.HRegion r)
                              throws IOException

Close both the HRegion r and it's underlying WAL. For use in tests.

Throws:

IOException

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

Returns this classes's instance of Configuration. Be careful how you use the returned Configuration since Connection instances can be shared. The Map of Connections is keyed by the Configuration. If say, a Connection was being used against a cluster that had been shutdown, see shutdownMiniCluster(), then the Connection will no longer be wholesome. Rather than use the return direct, its usually best to make a copy and use that. Do Configuration c = new Configuration(INSTANCE.getConfiguration());

Overrides:

getConfiguration in class HBaseCommonTestingUtil

Returns:

Instance of Configuration.

setHBaseCluster

public void setHBaseCluster(HBaseClusterInterface hbaseCluster)

setupDataTestDir

protected org.apache.hadoop.fs.Path setupDataTestDir()

Home our data in a dir under HBaseCommonTestingUtil.DEFAULT_BASE_TEST_DIRECTORY. Give it a random name so can have many concurrent tests running if we need to. Moding a System property is not the way to do concurrent instances -- another instance could grab the temporary value unintentionally -- but not anything can do about it at moment; single instance only is how the minidfscluster works. We also create the underlying directory names for hadoop.log.dir, mapreduce.cluster.local.dir and hadoop.tmp.dir, and set the values in the conf, and as a system property for hadoop.tmp.dir (We do not create them!).

Overrides:

setupDataTestDir in class HBaseCommonTestingUtil

Returns:

The calculated data test build directory, if newly-created.

createSubDirAndSystemProperty

private void createSubDirAndSystemProperty(String propertyName,
 org.apache.hadoop.fs.Path parent,
 String subDirName)

getBaseTestDirOnTestFS

private org.apache.hadoop.fs.Path getBaseTestDirOnTestFS()
                                                  throws IOException

Returns:

Where to write test data on the test filesystem; Returns working directory for the test filesystem by default

Throws:

IOException

See Also:

• setupDataTestDirOnTestFS()
• getTestFileSystem()

getDataTestDirOnTestFS

public org.apache.hadoop.fs.Path getDataTestDirOnTestFS()
                                                 throws IOException

Returns a Path in the test filesystem, obtained from getTestFileSystem() to write temporary test data. Call this method after setting up the mini dfs cluster if the test relies on it.

Returns:

a unique path in the test filesystem

Throws:

IOException

getDataTestDirOnTestFS

public org.apache.hadoop.fs.Path getDataTestDirOnTestFS(String subdirName)
                                                 throws IOException

Returns a Path in the test filesystem, obtained from getTestFileSystem() to write temporary test data. Call this method after setting up the mini dfs cluster if the test relies on it.

Parameters:

subdirName - name of the subdir to create under the base test dir

Returns:

a unique path in the test filesystem

Throws:

IOException

setupDataTestDirOnTestFS

private void setupDataTestDirOnTestFS()
                               throws IOException

Sets up a path in test filesystem to be used by tests. Creates a new directory if not already setup.

Throws:

IOException

getNewDataTestDirOnTestFS

private org.apache.hadoop.fs.Path getNewDataTestDirOnTestFS()
                                                     throws IOException

Sets up a new path in test filesystem to be used by tests.

Throws:

IOException

cleanupDataTestDirOnTestFS

public boolean cleanupDataTestDirOnTestFS()
                                   throws IOException

Cleans the test data directory on the test filesystem.

Returns:

True if we removed the test dirs

Throws:

IOException

cleanupDataTestDirOnTestFS

public boolean cleanupDataTestDirOnTestFS(String subdirName)
                                   throws IOException

Cleans a subdirectory under the test data directory on the test filesystem.

Returns:

True if we removed child

Throws:

IOException

startMiniDFSCluster

public org.apache.hadoop.hdfs.MiniDFSCluster startMiniDFSCluster(int servers)
                                                          throws Exception

Start a minidfscluster.

Parameters:

servers - How many DNs to start.

Returns:

The mini dfs cluster created.

Throws:

Exception

See Also:

• shutdownMiniDFSCluster()

startMiniDFSCluster

public org.apache.hadoop.hdfs.MiniDFSCluster startMiniDFSCluster(String[] hosts)
                                                          throws Exception

Start a minidfscluster. This is useful if you want to run datanode on distinct hosts for things like HDFS block location verification. If you start MiniDFSCluster without host names, all instances of the datanodes will have the same host name.

Parameters:

hosts - hostnames DNs to run on.

Returns:

The mini dfs cluster created.

Throws:

Exception

See Also:

• shutdownMiniDFSCluster()

startMiniDFSCluster

public org.apache.hadoop.hdfs.MiniDFSCluster startMiniDFSCluster(int servers,
 String[] hosts)
                                                          throws Exception

Start a minidfscluster. Can only create one.

Parameters:

servers - How many DNs to start.

hosts - hostnames DNs to run on.

Returns:

The mini dfs cluster created.

Throws:

Exception

See Also:

• shutdownMiniDFSCluster()

setFs

private void setFs()
            throws IOException

Throws:

IOException

startMiniDFSCluster

public org.apache.hadoop.hdfs.MiniDFSCluster startMiniDFSCluster(int servers,
 String[] racks,
 String[] hosts)
                                                          throws Exception

Throws:

Exception

startMiniDFSClusterForTestWAL

public org.apache.hadoop.hdfs.MiniDFSCluster startMiniDFSClusterForTestWAL(int namenodePort)
                                                                    throws IOException

Throws:

IOException

createDirsAndSetProperties

private void createDirsAndSetProperties()
                                 throws IOException

This is used before starting HDFS and map-reduce mini-clusters Run something like the below to check for the likes of '/tmp' references -- i.e. references outside of the test data dir -- in the conf.

 Configuration conf = TEST_UTIL.getConfiguration();
 for (Iterator<Map.Entry<String, String>> i = conf.iterator(); i.hasNext();) {
   Map.Entry<String, String> e = i.next();
   assertFalse(e.getKey() + " " + e.getValue(), e.getValue().contains("/tmp"));
 }
 

Throws:

IOException

isNewVersionBehaviorEnabled

public boolean isNewVersionBehaviorEnabled()

Check whether the tests should assume NEW_VERSION_BEHAVIOR when creating new column families. Default to false.

isReadShortCircuitOn

public boolean isReadShortCircuitOn()

Get the HBase setting for dfs.client.read.shortcircuit from the conf or a system property. This allows to specify this parameter on the command line. If not set, default is true.

enableShortCircuit

private void enableShortCircuit()

Enable the short circuit read, unless configured differently. Set both HBase and HDFS settings, including skipping the hdfs checksum checks.

createDirAndSetProperty

private String createDirAndSetProperty(String property)

createDirAndSetProperty

private String createDirAndSetProperty(String relPath,
 String property)

shutdownMiniDFSCluster

public void shutdownMiniDFSCluster()
                            throws IOException

Shuts down instance created by call to startMiniDFSCluster(int) or does nothing.

Throws:

IOException

startMiniCluster

public SingleProcessHBaseCluster startMiniCluster(int numSlaves)
                                           throws Exception

Start up a minicluster of hbase, dfs and zookeeper clusters with given slave node number. All other options will use default values, defined in StartTestingClusterOption.Builder.

Parameters:

numSlaves - slave node number, for both HBase region server and HDFS data node.

Throws:

Exception

See Also:

• startMiniCluster(StartTestingClusterOption option)
• shutdownMiniDFSCluster()

startMiniCluster

public SingleProcessHBaseCluster startMiniCluster()
                                           throws Exception

Start up a minicluster of hbase, dfs and zookeeper all using default options. Option default value can be found in StartTestingClusterOption.Builder.

Throws:

Exception

See Also:

• startMiniCluster(StartTestingClusterOption option)
• shutdownMiniDFSCluster()

startMiniCluster

public SingleProcessHBaseCluster startMiniCluster(StartTestingClusterOption option)
                                           throws Exception

Start up a mini cluster of hbase, optionally dfs and zookeeper if needed. It modifies Configuration. It homes the cluster data directory under a random subdirectory in a directory under System property test.build.data, to be cleaned up on exit.

Throws:

Exception

See Also:

• shutdownMiniDFSCluster()

startMiniHBaseCluster

public SingleProcessHBaseCluster startMiniHBaseCluster(StartTestingClusterOption option)
                                                throws IOException,
InterruptedException

Starts up mini hbase cluster. Usually you won't want this. You'll usually want startMiniCluster(). This is useful when doing stepped startup of clusters.

Returns:

Reference to the hbase mini hbase cluster.

Throws:

IOException

InterruptedException

See Also:

• startMiniCluster(StartTestingClusterOption)
• shutdownMiniHBaseCluster()

startMiniHBaseCluster

public SingleProcessHBaseCluster startMiniHBaseCluster()
                                                throws IOException,
InterruptedException

Starts up mini hbase cluster using default options. Default options can be found in StartTestingClusterOption.Builder.

Throws:

IOException

InterruptedException

See Also:

• startMiniHBaseCluster(StartTestingClusterOption)
• shutdownMiniHBaseCluster()

startMiniHBaseCluster

@Deprecated
public SingleProcessHBaseCluster startMiniHBaseCluster(int numMasters,
 int numRegionServers)
                                                throws IOException,
InterruptedException

Deprecated.

since 2.2.0 and will be removed in 4.0.0. Use startMiniHBaseCluster(StartTestingClusterOption) instead.

Starts up mini hbase cluster. Usually you won't want this. You'll usually want startMiniCluster(). All other options will use default values, defined in StartTestingClusterOption.Builder.

Parameters:

numMasters - Master node number.

numRegionServers - Number of region servers.

Returns:

The mini HBase cluster created.

Throws:

IOException

InterruptedException

See Also:

• shutdownMiniHBaseCluster()
• startMiniHBaseCluster(StartTestingClusterOption)
• HBASE-21071

startMiniHBaseCluster

@Deprecated
public SingleProcessHBaseCluster startMiniHBaseCluster(int numMasters,
 int numRegionServers,
 List<Integer> rsPorts)
                                                throws IOException,
InterruptedException

Deprecated.

since 2.2.0 and will be removed in 4.0.0. Use startMiniHBaseCluster(StartTestingClusterOption) instead.

Starts up mini hbase cluster. Usually you won't want this. You'll usually want startMiniCluster(). All other options will use default values, defined in StartTestingClusterOption.Builder.

Parameters:

numMasters - Master node number.

numRegionServers - Number of region servers.

rsPorts - Ports that RegionServer should use.

Returns:

The mini HBase cluster created.

Throws:

IOException

InterruptedException

See Also:

• shutdownMiniHBaseCluster()
• startMiniHBaseCluster(StartTestingClusterOption)
• HBASE-21071

startMiniHBaseCluster

@Deprecated
public SingleProcessHBaseCluster startMiniHBaseCluster(int numMasters,
 int numRegionServers,
 List<Integer> rsPorts,
 Class<? extends org.apache.hadoop.hbase.master.HMaster> masterClass,
 Class<? extends SingleProcessHBaseCluster.MiniHBaseClusterRegionServer> rsClass,
 boolean createRootDir,
 boolean createWALDir)
                                                throws IOException,
InterruptedException

Deprecated.

since 2.2.0 and will be removed in 4.0.0. Use startMiniHBaseCluster(StartTestingClusterOption) instead.

Starts up mini hbase cluster. Usually you won't want this. You'll usually want startMiniCluster(). All other options will use default values, defined in StartTestingClusterOption.Builder.

Parameters:

numMasters - Master node number.

numRegionServers - Number of region servers.

rsPorts - Ports that RegionServer should use.

masterClass - The class to use as HMaster, or null for default.

rsClass - The class to use as HRegionServer, or null for default.

createRootDir - Whether to create a new root or data directory path.

createWALDir - Whether to create a new WAL directory.

Returns:

The mini HBase cluster created.

Throws:

IOException

InterruptedException

See Also:

• shutdownMiniHBaseCluster()
• startMiniHBaseCluster(StartTestingClusterOption)
• HBASE-21071

restartHBaseCluster

public void restartHBaseCluster(int servers)
                         throws IOException,
InterruptedException

Starts the hbase cluster up again after shutting it down previously in a test. Use this if you want to keep dfs/zk up and just stop/start hbase.

Parameters:

servers - number of region servers

Throws:

IOException

InterruptedException

restartHBaseCluster

public void restartHBaseCluster(int servers,
 List<Integer> ports)
                         throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

restartHBaseCluster

public void restartHBaseCluster(StartTestingClusterOption option)
                         throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

getMiniHBaseCluster

public SingleProcessHBaseCluster getMiniHBaseCluster()

Returns current mini hbase cluster. Only has something in it after a call to startMiniCluster().

See Also:

• startMiniCluster()

shutdownMiniCluster

public void shutdownMiniCluster()
                         throws IOException

Stops mini hbase, zk, and hdfs clusters.

Throws:

IOException

See Also:

• startMiniCluster(int)

shutdownMiniHBaseCluster

public void shutdownMiniHBaseCluster()
                              throws IOException

Shutdown HBase mini cluster.Does not shutdown zk or dfs if running.

Throws:

IOException - in case command is unsuccessful

killMiniHBaseCluster

public void killMiniHBaseCluster()
                          throws IOException

Abruptly Shutdown HBase mini cluster. Does not shutdown zk or dfs if running.

Throws:

IOException - throws in case command is unsuccessful

cleanup

private void cleanup()
              throws IOException

Throws:

IOException

getDefaultRootDirPath

public org.apache.hadoop.fs.Path getDefaultRootDirPath(boolean create)
                                                throws IOException

Returns the path to the default root dir the minicluster uses. If create is true, a new root directory path is fetched irrespective of whether it has been fetched before or not. If false, previous path is used. Note: this does not cause the root dir to be created.

Returns:

Fully qualified path for the default hbase root dir

Throws:

IOException

getDefaultRootDirPath

public org.apache.hadoop.fs.Path getDefaultRootDirPath()
                                                throws IOException

Same as {getDefaultRootDirPath(boolean create) except that create flag is false. Note: this does not cause the root dir to be created.

Returns:

Fully qualified path for the default hbase root dir

Throws:

IOException

createRootDir

public org.apache.hadoop.fs.Path createRootDir(boolean create)
                                        throws IOException

Creates an hbase rootdir in user home directory. Also creates hbase version file. Normally you won't make use of this method. Root hbasedir is created for you as part of mini cluster startup. You'd only use this method if you were doing manual operation.

Parameters:

create - This flag decides whether to get a new root or data directory path or not, if it has been fetched already. Note : Directory will be made irrespective of whether path has been fetched or not. If directory already exists, it will be overwritten

Returns:

Fully qualified path to hbase root dir

Throws:

IOException

createRootDir

public org.apache.hadoop.fs.Path createRootDir()
                                        throws IOException

Same as createRootDir(boolean create) except that create flag is false.

Returns:

Fully qualified path to hbase root dir

Throws:

IOException

createWALRootDir

public org.apache.hadoop.fs.Path createWALRootDir()
                                           throws IOException

Creates a hbase walDir in the user's home directory. Normally you won't make use of this method. Root hbaseWALDir is created for you as part of mini cluster startup. You'd only use this method if you were doing manual operation.

Returns:

Fully qualified path to hbase root dir

Throws:

IOException

setHBaseFsTmpDir

private void setHBaseFsTmpDir()
                       throws IOException

Throws:

IOException

flush

public void flush()
           throws IOException

Flushes all caches in the mini hbase cluster

Throws:

IOException

flush

public void flush(org.apache.hadoop.hbase.TableName tableName)
           throws IOException

Flushes all caches in the mini hbase cluster

Throws:

IOException

compact

public void compact(boolean major)
             throws IOException

Compact all regions in the mini hbase cluster

Throws:

IOException

compact

public void compact(org.apache.hadoop.hbase.TableName tableName,
 boolean major)
             throws IOException

Compact all of a table's reagion in the mini hbase cluster

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 String family)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 String[] families)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[] family)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createMultiRegionTable

public org.apache.hadoop.hbase.client.Table createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName,
 byte[] family,
 int numRegions)
                                                            throws IOException

Create a table with multiple regions.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createMultiRegionTable

public org.apache.hadoop.hbase.client.Table createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families)
                                                            throws IOException

Create a table with multiple regions.

Returns:

A Table instance for the created table.

Throws:

IOException

createMultiRegionTable

public org.apache.hadoop.hbase.client.Table createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName,
 int replicaCount,
 byte[][] families)
                                                            throws IOException

Create a table with multiple regions.

Parameters:

replicaCount - replica count.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 byte[][] splitKeys)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 byte[][] splitKeys,
 int replicaCount)
                                                 throws IOException

Create a table.

Parameters:

tableName - the table name

families - the families

splitKeys - the splitkeys

replicaCount - the region replica count

Returns:

A Table instance for the created table.

Throws:

IOException - throws IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int numVersions,
 byte[] startKey,
 byte[] endKey,
 int numRegions)
                                                 throws IOException

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.client.TableDescriptor htd,
 byte[][] families,
 org.apache.hadoop.conf.Configuration c)
                                                 throws IOException

Create a table.

Parameters:

c - Configuration to use

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.client.TableDescriptor htd,
 byte[][] families,
 byte[][] splitKeys,
 org.apache.hadoop.conf.Configuration c)
                                                 throws IOException

Create a table.

Parameters:

htd - table descriptor

families - array of column families

splitKeys - array of split keys

c - Configuration to use

Returns:

A Table instance for the created table.

Throws:

IOException - if getAdmin or createTable fails

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.client.TableDescriptor htd,
 byte[][] families,
 byte[][] splitKeys,
 org.apache.hadoop.hbase.regionserver.BloomType type,
 int blockSize,
 org.apache.hadoop.conf.Configuration c)
                                                 throws IOException

Create a table.

Parameters:

htd - table descriptor

families - array of column families

splitKeys - array of split keys

type - Bloom type

blockSize - block size

c - Configuration to use

Returns:

A Table instance for the created table.

Throws:

IOException - if getAdmin or createTable fails

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.client.TableDescriptor htd,
 byte[][] splitRows)
                                                 throws IOException

Create a table.

Parameters:

htd - table descriptor

splitRows - array of split keys

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 byte[][] splitKeys,
 int replicaCount,
 org.apache.hadoop.conf.Configuration c)
                                                 throws IOException

Create a table.

Parameters:

tableName - the table name

families - the families

splitKeys - the split keys

replicaCount - the replica count

c - Configuration to use

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[] family,
 int numVersions)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int numVersions)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int numVersions,
 byte[][] splitKeys)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createMultiRegionTable

public org.apache.hadoop.hbase.client.Table createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int numVersions)
                                                            throws IOException

Create a table with multiple regions.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int numVersions,
 int blockSize)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int numVersions,
 int blockSize,
 String cpName)
                                                 throws IOException

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int[] numVersions)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createTable

public org.apache.hadoop.hbase.client.Table createTable(org.apache.hadoop.hbase.TableName tableName,
 byte[] family,
 byte[][] splitRows)
                                                 throws IOException

Create a table.

Returns:

A Table instance for the created table.

Throws:

IOException

createMultiRegionTable

public org.apache.hadoop.hbase.client.Table createMultiRegionTable(org.apache.hadoop.hbase.TableName tableName,
 byte[] family)
                                                            throws IOException

Create a table with multiple regions.

Returns:

A Table instance for the created table.

Throws:

IOException

setReplicas

public static void setReplicas(org.apache.hadoop.hbase.client.Admin admin,
 org.apache.hadoop.hbase.TableName table,
 int replicaCount)
                        throws IOException,
InterruptedException

Set the number of Region replicas.

Throws:

IOException

InterruptedException

setReplicas

public static void setReplicas(org.apache.hadoop.hbase.client.AsyncAdmin admin,
 org.apache.hadoop.hbase.TableName table,
 int replicaCount)
                        throws ExecutionException,
IOException,
InterruptedException

Set the number of Region replicas.

Throws:

ExecutionException

IOException

InterruptedException

deleteTable

public void deleteTable(org.apache.hadoop.hbase.TableName tableName)
                 throws IOException

Drop an existing table

Parameters:

tableName - existing table

Throws:

IOException

deleteTableIfAny

public void deleteTableIfAny(org.apache.hadoop.hbase.TableName tableName)
                      throws IOException

Drop an existing table

Parameters:

tableName - existing table

Throws:

IOException

createModifyableTableDescriptor

public org.apache.hadoop.hbase.client.TableDescriptorBuilder createModifyableTableDescriptor(String name)

createTableDescriptor

public org.apache.hadoop.hbase.client.TableDescriptor createTableDescriptor(org.apache.hadoop.hbase.TableName name,
 int minVersions,
 int versions,
 int ttl,
 org.apache.hadoop.hbase.KeepDeletedCells keepDeleted)

createModifyableTableDescriptor

public org.apache.hadoop.hbase.client.TableDescriptorBuilder createModifyableTableDescriptor(org.apache.hadoop.hbase.TableName name,
 int minVersions,
 int versions,
 int ttl,
 org.apache.hadoop.hbase.KeepDeletedCells keepDeleted)

createTableDescriptor

public org.apache.hadoop.hbase.client.TableDescriptor createTableDescriptor(org.apache.hadoop.hbase.TableName name)

Create a table of name name.

Parameters:

name - Name to give table.

Returns:

Column descriptor.

createTableDescriptor

public org.apache.hadoop.hbase.client.TableDescriptor createTableDescriptor(org.apache.hadoop.hbase.TableName tableName,
 byte[] family)

createTableDescriptor

public org.apache.hadoop.hbase.client.TableDescriptor createTableDescriptor(org.apache.hadoop.hbase.TableName tableName,
 byte[][] families,
 int maxVersions)

createLocalHRegion

public org.apache.hadoop.hbase.regionserver.HRegion createLocalHRegion(org.apache.hadoop.hbase.client.TableDescriptor desc,
 byte[] startKey,
 byte[] endKey)
                                                                throws IOException

Create an HRegion that writes to the local tmp dirs

Parameters:

desc - a table descriptor indicating which table the region belongs to

startKey - the start boundary of the region

endKey - the end boundary of the region

Returns:

a region that writes to local dir for testing

Throws:

IOException

createLocalHRegion

public org.apache.hadoop.hbase.regionserver.HRegion createLocalHRegion(org.apache.hadoop.hbase.client.RegionInfo info,
 org.apache.hadoop.hbase.client.TableDescriptor desc)
                                                                throws IOException

Create an HRegion that writes to the local tmp dirs. Creates the WAL for you. Be sure to call closeRegionAndWAL(HRegion) when you're finished with it.

Throws:

IOException

createLocalHRegion

public org.apache.hadoop.hbase.regionserver.HRegion createLocalHRegion(org.apache.hadoop.hbase.client.RegionInfo info,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor desc,
 org.apache.hadoop.hbase.wal.WAL wal)
                                                                throws IOException

Create an HRegion that writes to the local tmp dirs with specified wal

Parameters:

info - regioninfo

conf - configuration

desc - table descriptor

wal - wal for this region.

Returns:

created hregion

Throws:

IOException

createLocalHRegion

public org.apache.hadoop.hbase.regionserver.HRegion createLocalHRegion(org.apache.hadoop.hbase.TableName tableName,
 byte[] startKey,
 byte[] stopKey,
 org.apache.hadoop.conf.Configuration conf,
 boolean isReadOnly,
 org.apache.hadoop.hbase.client.Durability durability,
 org.apache.hadoop.hbase.wal.WAL wal,
 byte[]... families)
                                                                throws IOException

Returns:

A region on which you must call closeRegionAndWAL(HRegion) when done.

Throws:

IOException

createLocalHRegionWithInMemoryFlags

public org.apache.hadoop.hbase.regionserver.HRegion createLocalHRegionWithInMemoryFlags(org.apache.hadoop.hbase.TableName tableName,
 byte[] startKey,
 byte[] stopKey,
 org.apache.hadoop.conf.Configuration conf,
 boolean isReadOnly,
 org.apache.hadoop.hbase.client.Durability durability,
 org.apache.hadoop.hbase.wal.WAL wal,
 boolean[] compactedMemStore,
 byte[]... families)
                                                                                 throws IOException

Throws:

IOException

deleteTableData

public org.apache.hadoop.hbase.client.Table deleteTableData(org.apache.hadoop.hbase.TableName tableName)
                                                     throws IOException

Provide an existing table name to truncate. Scans the table and issues a delete for each row read.

Parameters:

tableName - existing table

Returns:

HTable to that new table

Throws:

IOException

truncateTable

public org.apache.hadoop.hbase.client.Table truncateTable(org.apache.hadoop.hbase.TableName tableName,
 boolean preserveRegions)
                                                   throws IOException

Truncate a table using the admin command. Effectively disables, deletes, and recreates the table.

Parameters:

tableName - table which must exist.

preserveRegions - keep the existing split points

Returns:

HTable for the new table

Throws:

IOException

truncateTable

public org.apache.hadoop.hbase.client.Table truncateTable(org.apache.hadoop.hbase.TableName tableName)
                                                   throws IOException

Truncate a table using the admin command. Effectively disables, deletes, and recreates the table. For previous behavior of issuing row deletes, see deleteTableData. Expressly does not preserve regions of existing table.

Parameters:

tableName - table which must exist.

Returns:

HTable for the new table

Throws:

IOException

loadTable

public int loadTable(org.apache.hadoop.hbase.client.Table t,
 byte[] f)
              throws IOException

Load table with rows from 'aaa' to 'zzz'.

Parameters:

t - Table

f - Family

Returns:

Count of rows loaded.

Throws:

IOException

loadTable

public int loadTable(org.apache.hadoop.hbase.client.Table t,
 byte[] f,
 boolean writeToWAL)
              throws IOException

Load table with rows from 'aaa' to 'zzz'.

Parameters:

t - Table

f - Family

Returns:

Count of rows loaded.

Throws:

IOException

loadTable

public int loadTable(org.apache.hadoop.hbase.client.Table t,
 byte[][] f)
              throws IOException

Load table of multiple column families with rows from 'aaa' to 'zzz'.

Parameters:

t - Table

f - Array of Families to load

Returns:

Count of rows loaded.

Throws:

IOException

loadTable

public int loadTable(org.apache.hadoop.hbase.client.Table t,
 byte[][] f,
 byte[] value)
              throws IOException

Load table of multiple column families with rows from 'aaa' to 'zzz'.

Parameters:

t - Table

f - Array of Families to load

value - the values of the cells. If null is passed, the row key is used as value

Returns:

Count of rows loaded.

Throws:

IOException

loadTable

public int loadTable(org.apache.hadoop.hbase.client.Table t,
 byte[][] f,
 byte[] value,
 boolean writeToWAL)
              throws IOException

Load table of multiple column families with rows from 'aaa' to 'zzz'.

Parameters:

t - Table

f - Array of Families to load

value - the values of the cells. If null is passed, the row key is used as value

Returns:

Count of rows loaded.

Throws:

IOException

loadRegion

public int loadRegion(org.apache.hadoop.hbase.regionserver.HRegion r,
 byte[] f)
               throws IOException

Throws:

IOException

loadRegion

public int loadRegion(org.apache.hadoop.hbase.regionserver.Region r,
 byte[] f)
               throws IOException

Throws:

IOException

loadRegion

public int loadRegion(org.apache.hadoop.hbase.regionserver.HRegion r,
 byte[] f,
 boolean flush)
               throws IOException

Load region with rows from 'aaa' to 'zzz'.

Parameters:

r - Region

f - Family

flush - flush the cache if true

Returns:

Count of rows loaded.

Throws:

IOException

loadNumericRows

public void loadNumericRows(org.apache.hadoop.hbase.client.Table t,
 byte[] f,
 int startRow,
 int endRow)
                     throws IOException

Throws:

IOException

loadRandomRows

public void loadRandomRows(org.apache.hadoop.hbase.client.Table t,
 byte[] f,
 int rowSize,
 int totalRows)
                    throws IOException

Throws:

IOException

verifyNumericRows

public void verifyNumericRows(org.apache.hadoop.hbase.client.Table table,
 byte[] f,
 int startRow,
 int endRow,
 int replicaId)
                       throws IOException

Throws:

IOException

verifyNumericRows

public void verifyNumericRows(org.apache.hadoop.hbase.regionserver.Region region,
 byte[] f,
 int startRow,
 int endRow)
                       throws IOException

Throws:

IOException

verifyNumericRows

public void verifyNumericRows(org.apache.hadoop.hbase.regionserver.HRegion region,
 byte[] f,
 int startRow,
 int endRow)
                       throws IOException

Throws:

IOException

verifyNumericRows

public void verifyNumericRows(org.apache.hadoop.hbase.regionserver.Region region,
 byte[] f,
 int startRow,
 int endRow,
 boolean present)
                       throws IOException

Throws:

IOException

verifyNumericRows

public void verifyNumericRows(org.apache.hadoop.hbase.regionserver.HRegion region,
 byte[] f,
 int startRow,
 int endRow,
 boolean present)
                       throws IOException

Throws:

IOException

deleteNumericRows

public void deleteNumericRows(org.apache.hadoop.hbase.client.Table t,
 byte[] f,
 int startRow,
 int endRow)
                       throws IOException

Throws:

IOException

countRows

public static int countRows(org.apache.hadoop.hbase.client.Table table)
                     throws IOException

Return the number of rows in the given table.

Parameters:

table - to count rows

Returns:

count of rows

Throws:

IOException

countRows

public static int countRows(org.apache.hadoop.hbase.client.Table table,
 org.apache.hadoop.hbase.client.Scan scan)
                     throws IOException

Throws:

IOException

countRows

public static int countRows(org.apache.hadoop.hbase.client.Table table,
 byte[]... families)
                     throws IOException

Throws:

IOException

countRows

public int countRows(org.apache.hadoop.hbase.TableName tableName)
              throws IOException

Return the number of rows in the given table.

Throws:

IOException

countRows

public static int countRows(org.apache.hadoop.hbase.regionserver.Region region)
                     throws IOException

Throws:

IOException

countRows

public static int countRows(org.apache.hadoop.hbase.regionserver.Region region,
 org.apache.hadoop.hbase.client.Scan scan)
                     throws IOException

Throws:

IOException

countRows

public static int countRows(org.apache.hadoop.hbase.regionserver.InternalScanner scanner)
                     throws IOException

Throws:

IOException

checksumRows

public String checksumRows(org.apache.hadoop.hbase.client.Table table)
                    throws Exception

Return an md5 digest of the entire contents of a table.

Throws:

Exception

createMultiRegionsInMeta

public List<org.apache.hadoop.hbase.client.RegionInfo> createMultiRegionsInMeta(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor htd,
 byte[][] startKeys)
                                                                         throws IOException

Create rows in hbase:meta for regions of the specified table with the specified start keys. The first startKey should be a 0 length byte array if you want to form a proper range of regions.

Returns:

list of region info for regions added to meta

Throws:

IOException

createWal

public static org.apache.hadoop.hbase.wal.WAL createWal(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.hbase.client.RegionInfo hri)
                                                 throws IOException

Create an unmanaged WAL. Be sure to close it when you're through.

Throws:

IOException

createRegionAndWAL

public static org.apache.hadoop.hbase.regionserver.HRegion createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor htd)
                                                                       throws IOException

Create a region with it's own WAL. Be sure to call closeRegionAndWAL(HRegion) to clean up all resources.

Throws:

IOException

createRegionAndWAL

public static org.apache.hadoop.hbase.regionserver.HRegion createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor htd,
 org.apache.hadoop.hbase.io.hfile.BlockCache blockCache)
                                                                       throws IOException

Create a region with it's own WAL. Be sure to call closeRegionAndWAL(HRegion) to clean up all resources.

Throws:

IOException

createRegionAndWAL

public static org.apache.hadoop.hbase.regionserver.HRegion createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor htd,
 org.apache.hadoop.hbase.mob.MobFileCache mobFileCache)
                                                                       throws IOException

Create a region with it's own WAL. Be sure to call closeRegionAndWAL(HRegion) to clean up all resources.

Throws:

IOException

createRegionAndWAL

public static org.apache.hadoop.hbase.regionserver.HRegion createRegionAndWAL(org.apache.hadoop.hbase.client.RegionInfo info,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor htd,
 boolean initialize)
                                                                       throws IOException

Create a region with it's own WAL. Be sure to call closeRegionAndWAL(HRegion) to clean up all resources.

Throws:

IOException

getOtherRegionServer

public org.apache.hadoop.hbase.regionserver.HRegionServer getOtherRegionServer(org.apache.hadoop.hbase.regionserver.HRegionServer rs)

Find any other region server which is different from the one identified by parameter

Returns:

another region server

getRSForFirstRegionInTable

public org.apache.hadoop.hbase.regionserver.HRegionServer getRSForFirstRegionInTable(org.apache.hadoop.hbase.TableName tableName)
                                                                              throws IOException,
InterruptedException

Tool to get the reference to the region server object that holds the region of the specified user table.

Parameters:

tableName - user table to lookup in hbase:meta

Returns:

region server that holds it, null if the row doesn't exist

Throws:

IOException

InterruptedException

startMiniMapReduceCluster

public org.apache.hadoop.mapred.MiniMRCluster startMiniMapReduceCluster()
                                                                 throws IOException

Starts a MiniMRCluster with a default number of TaskTracker's.

Throws:

IOException - When starting the cluster fails.

forceChangeTaskLogDir

private void forceChangeTaskLogDir()

Tasktracker has a bug where changing the hadoop.log.dir system property will not change its internal static LOG_DIR variable.

startMiniMapReduceCluster

private void startMiniMapReduceCluster(int servers)
                                throws IOException

Starts a MiniMRCluster. Call setFileSystemURI(String) to use a different filesystem.

Parameters:

servers - The number of TaskTracker's to start.

Throws:

IOException - When starting the cluster fails.

shutdownMiniMapReduceCluster

public void shutdownMiniMapReduceCluster()

Stops the previously started MiniMRCluster.

createMockRegionServerService

public org.apache.hadoop.hbase.regionserver.RegionServerServices createMockRegionServerService()
                                                                                        throws IOException

Create a stubbed out RegionServerService, mainly for getting FS.

Throws:

IOException

createMockRegionServerService

public org.apache.hadoop.hbase.regionserver.RegionServerServices createMockRegionServerService(org.apache.hadoop.hbase.ipc.RpcServerInterface rpc)
                                                                                        throws IOException

Create a stubbed out RegionServerService, mainly for getting FS. This version is used by TestTokenAuthentication

Throws:

IOException

createMockRegionServerService

public org.apache.hadoop.hbase.regionserver.RegionServerServices createMockRegionServerService(org.apache.hadoop.hbase.ServerName name)
                                                                                        throws IOException

Create a stubbed out RegionServerService, mainly for getting FS. This version is used by TestOpenRegionHandler

Throws:

IOException

expireMasterSession

public void expireMasterSession()
                         throws Exception

Expire the Master's session

Throws:

Exception

expireRegionServerSession

public void expireRegionServerSession(int index)
                               throws Exception

Expire a region server's session

Parameters:

index - which RS

Throws:

Exception

decrementMinRegionServerCount

private void decrementMinRegionServerCount()

decrementMinRegionServerCount

private void decrementMinRegionServerCount(org.apache.hadoop.conf.Configuration conf)

expireSession

public void expireSession(org.apache.hadoop.hbase.zookeeper.ZKWatcher nodeZK)
                   throws Exception

Throws:

Exception

expireSession

public void expireSession(org.apache.hadoop.hbase.zookeeper.ZKWatcher nodeZK,
 boolean checkStatus)
                   throws Exception

Expire a ZooKeeper session as recommended in ZooKeeper documentation http://hbase.apache.org/book.html#trouble.zookeeper

There are issues when doing this:

1. http://www.mail-archive.com/dev@zookeeper.apache.org/msg01942.html
2. https://issues.apache.org/jira/browse/ZOOKEEPER-1105

Parameters:

nodeZK - - the ZK watcher to expire

checkStatus - - true to check if we can create a Table with the current configuration.

Throws:

Exception

getHBaseCluster

public SingleProcessHBaseCluster getHBaseCluster()

Get the Mini HBase cluster.

Returns:

hbase cluster

See Also:

• getHBaseClusterInterface()

getHBaseClusterInterface

public HBaseClusterInterface getHBaseClusterInterface()

Returns the HBaseCluster instance.

Returned object can be any of the subclasses of HBaseCluster, and the tests referring this should not assume that the cluster is a mini cluster or a distributed one. If the test only works on a mini cluster, then specific method getMiniHBaseCluster() can be used instead w/o the need to type-cast.

invalidateConnection

public void invalidateConnection()
                          throws IOException

Resets the connections so that the next time getConnection() is called, a new connection is created. This is needed in cases where the entire cluster / all the masters are shutdown and the connection is not valid anymore.

TODO: There should be a more coherent way of doing this. Unfortunately the way tests are written, not all start() stop() calls go through this class. Most tests directly operate on the underlying mini/local hbase cluster. That makes it difficult for this wrapper class to maintain the connection state automatically. Cleaning this is a much bigger refactor.

Throws:

IOException

getConnection

public org.apache.hadoop.hbase.client.Connection getConnection()
                                                        throws IOException

Get a shared Connection to the cluster. this method is thread safe.

Returns:

A Connection that can be shared. Don't close. Will be closed on shutdown of cluster.

Throws:

IOException

getConnection

public org.apache.hadoop.hbase.client.Connection getConnection(org.apache.hadoop.hbase.security.User user)
                                                        throws IOException

Get a assigned Connection to the cluster. this method is thread safe.

Parameters:

user - assigned user

Returns:

A Connection with assigned user.

Throws:

IOException

getAsyncConnection

public org.apache.hadoop.hbase.client.AsyncClusterConnection getAsyncConnection()
                                                                         throws IOException

Get a shared AsyncClusterConnection to the cluster. this method is thread safe.

Returns:

An AsyncClusterConnection that can be shared. Don't close. Will be closed on shutdown of cluster.

Throws:

IOException

getAsyncConnection

public org.apache.hadoop.hbase.client.AsyncClusterConnection getAsyncConnection(org.apache.hadoop.hbase.security.User user)
                                                                         throws IOException

Get a assigned AsyncClusterConnection to the cluster. this method is thread safe.

Parameters:

user - assigned user

Returns:

An AsyncClusterConnection with assigned user.

Throws:

IOException

closeConnection

public void closeConnection()
                     throws IOException

Throws:

IOException

getAdmin

public org.apache.hadoop.hbase.client.Admin getAdmin()
                                              throws IOException

Returns an Admin instance which is shared between HBaseTestingUtility instance users. Closing it has no effect, it will be closed automatically when the cluster shutdowns

Throws:

IOException

getHbck

public org.apache.hadoop.hbase.client.Hbck getHbck()
                                            throws IOException

Returns an Hbck instance. Needs be closed when done.

Throws:

IOException

unassignRegion

public void unassignRegion(String regionName)
                    throws IOException

Unassign the named region.

Parameters:

regionName - The region to unassign.

Throws:

IOException

unassignRegion

public void unassignRegion(byte[] regionName)
                    throws IOException

Unassign the named region.

Parameters:

regionName - The region to unassign.

Throws:

IOException

unassignRegionByRow

public void unassignRegionByRow(String row,
 org.apache.hadoop.hbase.client.RegionLocator table)
                         throws IOException

Closes the region containing the given row.

Parameters:

row - The row to find the containing region.

table - The table to find the region.

Throws:

IOException

unassignRegionByRow

public void unassignRegionByRow(byte[] row,
 org.apache.hadoop.hbase.client.RegionLocator table)
                         throws IOException

Closes the region containing the given row.

Parameters:

row - The row to find the containing region.

table - The table to find the region.

Throws:

IOException

getSplittableRegion

public org.apache.hadoop.hbase.regionserver.HRegion getSplittableRegion(org.apache.hadoop.hbase.TableName tableName,
 int maxAttempts)

Retrieves a splittable region randomly from tableName

Parameters:

tableName - name of table

maxAttempts - maximum number of attempts, unlimited for value of -1

Returns:

the HRegion chosen, null if none was found within limit of maxAttempts

getDFSCluster

public org.apache.hadoop.hdfs.MiniDFSCluster getDFSCluster()

setDFSCluster

public void setDFSCluster(org.apache.hadoop.hdfs.MiniDFSCluster cluster)
                   throws IllegalStateException,
IOException

Throws:

IllegalStateException

IOException

setDFSCluster

public void setDFSCluster(org.apache.hadoop.hdfs.MiniDFSCluster cluster,
 boolean requireDown)
                   throws IllegalStateException,
IOException

Set the MiniDFSCluster

Parameters:

cluster - cluster to use

requireDown - require the that cluster not be "up" (MiniDFSCluster#isClusterUp) before it is set.

Throws:

IllegalStateException - if the passed cluster is up when it is required to be down

IOException - if the FileSystem could not be set from the passed dfs cluster

getTestFileSystem

public org.apache.hadoop.fs.FileSystem getTestFileSystem()
                                                  throws IOException

Throws:

IOException

waitTableAvailable

public void waitTableAvailable(org.apache.hadoop.hbase.TableName table)
                        throws InterruptedException,
IOException

Wait until all regions in a table have been assigned. Waits default timeout before giving up (30 seconds).

Parameters:

table - Table to wait on.

Throws:

InterruptedException

IOException

waitTableAvailable

public void waitTableAvailable(org.apache.hadoop.hbase.TableName table,
 long timeoutMillis)
                        throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

waitTableAvailable

public void waitTableAvailable(byte[] table,
 long timeoutMillis)
                        throws InterruptedException,
IOException

Wait until all regions in a table have been assigned

Parameters:

table - Table to wait on.

timeoutMillis - Timeout.

Throws:

InterruptedException

IOException

explainTableAvailability

public String explainTableAvailability(org.apache.hadoop.hbase.TableName tableName)
                                throws IOException

Throws:

IOException

explainTableState

public String explainTableState(org.apache.hadoop.hbase.TableName table,
 org.apache.hadoop.hbase.client.TableState.State state)
                         throws IOException

Throws:

IOException

findLastTableState

@Nullable
public org.apache.hadoop.hbase.client.TableState findLastTableState(org.apache.hadoop.hbase.TableName table)
                                                             throws IOException

Throws:

IOException

waitTableEnabled

public void waitTableEnabled(org.apache.hadoop.hbase.TableName table)
                      throws InterruptedException,
IOException

Waits for a table to be 'enabled'. Enabled means that table is set as 'enabled' and the regions have been all assigned. Will timeout after default period (30 seconds) Tolerates nonexistent table.

Parameters:

table - the table to wait on.

Throws:

InterruptedException - if interrupted while waiting

IOException - if an IO problem is encountered

waitTableEnabled

public void waitTableEnabled(byte[] table,
 long timeoutMillis)
                      throws InterruptedException,
IOException

Waits for a table to be 'enabled'. Enabled means that table is set as 'enabled' and the regions have been all assigned.

Parameters:

table - Table to wait on.

timeoutMillis - Time to wait on it being marked enabled.

Throws:

InterruptedException

IOException

See Also:

• waitTableEnabled(TableName, long)

waitTableEnabled

public void waitTableEnabled(org.apache.hadoop.hbase.TableName table,
 long timeoutMillis)
                      throws IOException

Throws:

IOException

waitTableDisabled

public void waitTableDisabled(byte[] table)
                       throws InterruptedException,
IOException

Waits for a table to be 'disabled'. Disabled means that table is set as 'disabled' Will timeout after default period (30 seconds)

Parameters:

table - Table to wait on.

Throws:

InterruptedException

IOException

waitTableDisabled

public void waitTableDisabled(org.apache.hadoop.hbase.TableName table,
 long millisTimeout)
                       throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

waitTableDisabled

public void waitTableDisabled(byte[] table,
 long timeoutMillis)
                       throws InterruptedException,
IOException

Waits for a table to be 'disabled'. Disabled means that table is set as 'disabled'

Parameters:

table - Table to wait on.

timeoutMillis - Time to wait on it being marked disabled.

Throws:

InterruptedException

IOException

ensureSomeRegionServersAvailable

public boolean ensureSomeRegionServersAvailable(int num)
                                         throws IOException

Make sure that at least the specified number of region servers are running

Parameters:

num - minimum number of region servers that should be running

Returns:

true if we started some servers

Throws:

IOException

ensureSomeNonStoppedRegionServersAvailable

public boolean ensureSomeNonStoppedRegionServersAvailable(int num)
                                                   throws IOException

Make sure that at least the specified number of region servers are running. We don't count the ones that are currently stopping or are stopped.

Parameters:

num - minimum number of region servers that should be running

Returns:

true if we started some servers

Throws:

IOException

getDifferentUser

public static org.apache.hadoop.hbase.security.User getDifferentUser(org.apache.hadoop.conf.Configuration c,
 String differentiatingSuffix)
                                                              throws IOException

This method clones the passed c configuration setting a new user into the clone. Use it getting new instances of FileSystem. Only works for DistributedFileSystem w/o Kerberos.

Parameters:

c - Initial configuration

differentiatingSuffix - Suffix to differentiate this user from others.

Returns:

A new configuration instance with a different user set into it.

Throws:

IOException

getAllOnlineRegions

public static NavigableSet<String> getAllOnlineRegions(SingleProcessHBaseCluster cluster)
                                                throws IOException

Throws:

IOException

setMaxRecoveryErrorCount

public static void setMaxRecoveryErrorCount(OutputStream stream,
 int max)

Set maxRecoveryErrorCount in DFSClient. In 0.20 pre-append its hard-coded to 5 and makes tests linger. Here is the exception you'll see:

 2010-06-15 11:52:28,511 WARN  [DataStreamer for file /hbase/.logs/wal.1276627923013 block
 blk_928005470262850423_1021] hdfs.DFSClient$DFSOutputStream(2657): Error Recovery for block
 blk_928005470262850423_1021 failed  because recovery from primary datanode 127.0.0.1:53683
 failed 4 times.  Pipeline was 127.0.0.1:53687, 127.0.0.1:53683. Will retry...
 

Parameters:

stream - A DFSClient.DFSOutputStream.

assignRegion

public boolean assignRegion(org.apache.hadoop.hbase.client.RegionInfo regionInfo)
                     throws IOException,
InterruptedException

Uses directly the assignment manager to assign the region. and waits until the specified region has completed assignment.

Returns:

true if the region is assigned false otherwise.

Throws:

IOException

InterruptedException

moveRegionAndWait

public void moveRegionAndWait(org.apache.hadoop.hbase.client.RegionInfo destRegion,
 org.apache.hadoop.hbase.ServerName destServer)
                       throws InterruptedException,
IOException

Move region to destination server and wait till region is completely moved and online

Parameters:

destRegion - region to move

destServer - destination server of the region

Throws:

InterruptedException

IOException

waitUntilAllRegionsAssigned

public void waitUntilAllRegionsAssigned(org.apache.hadoop.hbase.TableName tableName)
                                 throws IOException

Wait until all regions for a table in hbase:meta have a non-empty info:server, up to a configuable timeout value (default is 60 seconds) This means all regions have been deployed, master has been informed and updated hbase:meta with the regions deployed server.

Parameters:

tableName - the table name

Throws:

IOException

waitUntilAllSystemRegionsAssigned

public void waitUntilAllSystemRegionsAssigned()
                                       throws IOException

Waith until all system table's regions get assigned

Throws:

IOException

waitUntilAllRegionsAssigned

public void waitUntilAllRegionsAssigned(org.apache.hadoop.hbase.TableName tableName,
 long timeout)
                                 throws IOException

Wait until all regions for a table in hbase:meta have a non-empty info:server, or until timeout. This means all regions have been deployed, master has been informed and updated hbase:meta with the regions deployed server.

Parameters:

tableName - the table name

timeout - timeout, in milliseconds

Throws:

IOException

getFromStoreFile

public static List<org.apache.hadoop.hbase.Cell> getFromStoreFile(org.apache.hadoop.hbase.regionserver.HStore store,
 org.apache.hadoop.hbase.client.Get get)
                                                           throws IOException

Do a small get/scan against one store. This is required because store has no actual methods of querying itself, and relies on StoreScanner.

Throws:

IOException

getRegionSplitStartKeys

public byte[][] getRegionSplitStartKeys(byte[] startKey,
 byte[] endKey,
 int numRegions)

Create region split keys between startkey and endKey

Parameters:

numRegions - the number of regions to be created. it has to be greater than 3.

Returns:

resulting split keys

getFromStoreFile

public static List<org.apache.hadoop.hbase.Cell> getFromStoreFile(org.apache.hadoop.hbase.regionserver.HStore store,
 byte[] row,
 NavigableSet<byte[]> columns)
                                                           throws IOException

Do a small get/scan against one store. This is required because store has no actual methods of querying itself, and relies on StoreScanner.

Throws:

IOException

assertKVListsEqual

public static void assertKVListsEqual(String additionalMsg,
 List<? extends org.apache.hadoop.hbase.Cell> expected,
 List<? extends org.apache.hadoop.hbase.Cell> actual)

safeGetAsStr

public static <T> String safeGetAsStr(List<T> lst,
 int i)

getRpcConnnectionURI

public String getRpcConnnectionURI()
                            throws UnknownHostException

Throws:

UnknownHostException

getZkConnectionURI

public String getZkConnectionURI()

getClusterKey

@Deprecated
public String getClusterKey()

Deprecated.

since 2.7.0, will be removed in 4.0.0. Now we use connection uri to specify the connection info of a cluster. Keep here only for compatibility.

Get the zk based cluster key for this cluster.

See Also:

• getRpcConnnectionURI()
• getZkConnectionURI()

createRandomTable

public org.apache.hadoop.hbase.client.Table createRandomTable(org.apache.hadoop.hbase.TableName tableName,
 Collection<String> families,
 int maxVersions,
 int numColsPerRow,
 int numFlushes,
 int numRegions,
 int numRowsPerFlush)
                                                       throws IOException,
InterruptedException

Creates a random table with the given parameters

Throws:

IOException

InterruptedException

randomFreePort

public static int randomFreePort()

randomMultiCastAddress

public static String randomMultiCastAddress()

waitForHostPort

public static void waitForHostPort(String host,
 int port)
                            throws IOException

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.TableName tableName,
 byte[] columnFamily,
 org.apache.hadoop.hbase.io.compress.Compression.Algorithm compression,
 org.apache.hadoop.hbase.io.encoding.DataBlockEncoding dataBlockEncoding)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.TableName tableName,
 byte[] columnFamily,
 org.apache.hadoop.hbase.io.compress.Compression.Algorithm compression,
 org.apache.hadoop.hbase.io.encoding.DataBlockEncoding dataBlockEncoding,
 int numRegionsPerServer,
 int regionReplication,
 org.apache.hadoop.hbase.client.Durability durability)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.TableName tableName,
 byte[][] columnFamilies,
 org.apache.hadoop.hbase.io.compress.Compression.Algorithm compression,
 org.apache.hadoop.hbase.io.encoding.DataBlockEncoding dataBlockEncoding,
 int numRegionsPerServer,
 int regionReplication,
 org.apache.hadoop.hbase.client.Durability durability)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor desc,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor hcd)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor desc,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor hcd,
 int numRegionsPerServer)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor desc,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor[] hcds,
 int numRegionsPerServer)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException

createPreSplitLoadTestTable

public static int createPreSplitLoadTestTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor td,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor[] cds,
 org.apache.hadoop.hbase.util.RegionSplitter.SplitAlgorithm splitter,
 int numRegionsPerServer)
                                       throws IOException

Creates a pre-split table for load testing. If the table already exists, logs a warning and continues.

Returns:

the number of regions the table was split into

Throws:

IOException

getMetaRSPort

public static int getMetaRSPort(org.apache.hadoop.hbase.client.Connection connection)
                         throws IOException

Throws:

IOException

assertRegionOnServer

public void assertRegionOnServer(org.apache.hadoop.hbase.client.RegionInfo hri,
 org.apache.hadoop.hbase.ServerName server,
 long timeout)
                          throws IOException,
InterruptedException

Due to async racing issue, a region may not be in the online region list of a region server yet, after the assignment znode is deleted and the new assignment is recorded in master.

Throws:

IOException

InterruptedException

assertRegionOnlyOnServer

public void assertRegionOnlyOnServer(org.apache.hadoop.hbase.client.RegionInfo hri,
 org.apache.hadoop.hbase.ServerName server,
 long timeout)
                              throws IOException,
InterruptedException

Check to make sure the region is open on the specified region server, but not on any other one.

Throws:

IOException

InterruptedException

createTestRegion

public org.apache.hadoop.hbase.regionserver.HRegion createTestRegion(String tableName,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor cd)
                                                              throws IOException

Throws:

IOException

createTestRegion

public org.apache.hadoop.hbase.regionserver.HRegion createTestRegion(String tableName,
 org.apache.hadoop.hbase.client.ColumnFamilyDescriptor cd,
 org.apache.hadoop.hbase.io.hfile.BlockCache blockCache)
                                                              throws IOException

Throws:

IOException

setFileSystemURI

public static void setFileSystemURI(String fsURI)

predicateNoRegionsInTransition

public Waiter.ExplainingPredicate<IOException> predicateNoRegionsInTransition()

Returns a Waiter.Predicate for checking that there are no regions in transition in master

predicateTableEnabled

public Waiter.Predicate<IOException> predicateTableEnabled(org.apache.hadoop.hbase.TableName tableName)

Returns a Waiter.Predicate for checking that table is enabled

predicateTableDisabled

public Waiter.Predicate<IOException> predicateTableDisabled(org.apache.hadoop.hbase.TableName tableName)

Returns a Waiter.Predicate for checking that table is enabled

predicateTableAvailable

public Waiter.Predicate<IOException> predicateTableAvailable(org.apache.hadoop.hbase.TableName tableName)

Returns a Waiter.Predicate for checking that table is enabled

waitUntilNoRegionsInTransition

public void waitUntilNoRegionsInTransition(long timeout)
                                    throws IOException

Wait until no regions in transition.

Parameters:

timeout - How long to wait.

Throws:

IOException

waitUntilNoRegionsInTransition

public void waitUntilNoRegionsInTransition()
                                    throws IOException

Wait until no regions in transition. (time limit 15min)

Throws:

IOException

waitLabelAvailable

public void waitLabelAvailable(long timeoutMillis,
 String... labels)

Wait until labels is ready in VisibilityLabelsCache.

generateColumnDescriptors

public static List<org.apache.hadoop.hbase.client.ColumnFamilyDescriptor> generateColumnDescriptors()

Create a set of column descriptors with the combination of compression, encoding, bloom codecs available.

Returns:

the list of column descriptors

generateColumnDescriptors

public static List<org.apache.hadoop.hbase.client.ColumnFamilyDescriptor> generateColumnDescriptors(String prefix)

Create a set of column descriptors with the combination of compression, encoding, bloom codecs available.

Parameters:

prefix - family names prefix

Returns:

the list of column descriptors

getSupportedCompressionAlgorithms

public static org.apache.hadoop.hbase.io.compress.Compression.Algorithm[] getSupportedCompressionAlgorithms()

Get supported compression algorithms.

Returns:

supported compression algorithms.

getClosestRowBefore

public org.apache.hadoop.hbase.client.Result getClosestRowBefore(org.apache.hadoop.hbase.regionserver.Region r,
 byte[] row,
 byte[] family)
                                                          throws IOException

Throws:

IOException

isTargetTable

private boolean isTargetTable(byte[] inRow,
 org.apache.hadoop.hbase.Cell c)

setupMiniKdc

public org.apache.hadoop.minikdc.MiniKdc setupMiniKdc(File keytabFile)
                                               throws Exception

Sets up MiniKdc for testing security. Uses HBaseKerberosUtils to set the given keytab file as HBaseKerberosUtils.KRB_KEYTAB_FILE. FYI, there is also the easier-to-use kerby KDC server and utility for using it, SimpleKdcServerUtil. The kerby KDC server is preferred; less baggage. It came in in HBASE-5291.

Throws:

Exception

getNumHFiles

public int getNumHFiles(org.apache.hadoop.hbase.TableName tableName,
 byte[] family)

getNumHFilesForRS

public int getNumHFilesForRS(org.apache.hadoop.hbase.regionserver.HRegionServer rs,
 org.apache.hadoop.hbase.TableName tableName,
 byte[] family)

verifyTableDescriptorIgnoreTableName

public void verifyTableDescriptorIgnoreTableName(org.apache.hadoop.hbase.client.TableDescriptor ltd,
 org.apache.hadoop.hbase.client.TableDescriptor rtd)

await

public static void await(long sleepMillis,
 BooleanSupplier condition)
                  throws InterruptedException

Await the successful return of condition, sleeping sleepMillis between invocations.

Throws:

InterruptedException