org.apache.hadoop.hbase.util

Class BaseTestHBaseFsck

java.lang.Object

org.apache.hadoop.hbase.util.BaseTestHBaseFsck

Direct Known Subclasses:

TestHBaseFsckMOB

public class BaseTestHBaseFsck
extends Object

This is the base class for HBaseFsck's ability to detect reasons for inconsistent tables. Actual tests are in : TestHBaseFsckTwoRS TestHBaseFsckOneRS TestHBaseFsckMOB TestHBaseFsckReplicas

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BaseTestHBaseFsck.MasterSyncCoprocessor
(package private) static class	BaseTestHBaseFsck.MockErrorReporter

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.hadoop.hbase.client.Admin	admin
protected static org.apache.hadoop.hbase.master.assignment.AssignmentManager	assignmentManager
protected static org.apache.hadoop.conf.Configuration	conf
protected static org.apache.hadoop.hbase.client.ClusterConnection	connection
protected static byte[]	FAM
protected static String	FAM_STR
protected static ScheduledThreadPoolExecutor	hbfsckExecutorService
protected static org.slf4j.Logger	LOG
org.junit.rules.TestName	name
(package private) static int	POOL_SIZE
protected static int	REGION_ONLINE_TIMEOUT
protected static org.apache.hadoop.hbase.master.assignment.RegionStates	regionStates
protected static byte[][]	ROWKEYS
protected static byte[][]	SPLITS
protected static ExecutorService	tableExecutorService
protected org.apache.hadoop.hbase.client.Table	tbl
protected static HBaseTestingUtility	TEST_UTIL

Constructor Summary

Constructors

Constructor and Description
BaseTestHBaseFsck()

Method Summary

Modifier and Type	Method and Description
(package private) void	cleanupTable(org.apache.hadoop.hbase.TableName tablename) delete table in preparation for next test
(package private) int	countRows() Counts the number of rows to verify data loss or non-dataloss.
(package private) int	countRows(byte[] start, byte[] end) Counts the number of rows to verify data loss or non-dataloss.
(package private) String	createMobFileName(String oldFileName) Creates a new mob file name by the old one.
static void	createTable(HBaseTestingUtility testUtil, org.apache.hadoop.hbase.HTableDescriptor htd, byte[][] splitKeys)
protected void	deleteMetaRegion(org.apache.hadoop.conf.Configuration conf, boolean unassign, boolean hdfs, boolean regionInfoOnly)
protected void	deleteRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.HTableDescriptor htd, byte[] startKey, byte[] endKey, boolean unassign, boolean metaRow, boolean hdfs) Delete a region from assignments, meta, or completely from hdfs.
protected void	deleteRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.HTableDescriptor htd, byte[] startKey, byte[] endKey, boolean unassign, boolean metaRow, boolean hdfs, boolean regionInfoOnly, int replicaId) Delete a region from assignments, meta, or completely from hdfs.
static void	deleteTable(HBaseTestingUtility testUtil, org.apache.hadoop.hbase.TableName tableName)
void	deleteTableDir(org.apache.hadoop.hbase.TableName table)
protected void	doQuarantineTest(org.apache.hadoop.hbase.TableName table, org.apache.hadoop.hbase.util.HBaseFsck hbck, int check, int corrupt, int fail, int quar, int missing) Test that use this should have a timeout, because this method could potentially wait forever.
protected void	dumpMeta(org.apache.hadoop.hbase.TableName tableName) Debugging method to dump the contents of meta.
(package private) org.apache.hadoop.hbase.ServerName	findDeployedHSI(Map<org.apache.hadoop.hbase.ServerName,List<String>> mm, org.apache.hadoop.hbase.client.RegionInfo hri) Returns the HSI a region info is on.
(package private) Map<org.apache.hadoop.hbase.ServerName,List<String>>	getDeployedHRIs(org.apache.hadoop.hbase.client.Admin admin) Get region info from local cluster.
(package private) org.apache.hadoop.fs.Path	getFlushedHFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.hbase.TableName table) We don't have an easy way to verify that a flush completed, so we loop until we find a legitimate hfile and return it.
(package private) org.apache.hadoop.fs.Path	getFlushedMobFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.hbase.TableName table) Gets flushed mob files.
(package private) void	setupMobTable(org.apache.hadoop.hbase.TableName tablename) Setup a clean table with a mob-enabled column.
(package private) void	setupTable(org.apache.hadoop.hbase.TableName tablename) Setup a clean table before we start mucking with it.
(package private) void	setupTableWithRegionReplica(org.apache.hadoop.hbase.TableName tablename, int replicaCount) Setup a clean table with a certain region_replica count It will set tbl which needs to be closed after test
protected void	undeployRegion(org.apache.hadoop.hbase.client.Connection conn, org.apache.hadoop.hbase.ServerName sn, org.apache.hadoop.hbase.client.RegionInfo hri) This method is used to undeploy a region -- close it and attempt to remove its state from the Master.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

POOL_SIZE

static final int POOL_SIZE

See Also:

Constant Field Values

LOG

protected static final org.slf4j.Logger LOG

TEST_UTIL

protected static final HBaseTestingUtility TEST_UTIL

conf

protected static final org.apache.hadoop.conf.Configuration conf

FAM_STR

protected static final String FAM_STR

See Also:

Constant Field Values

FAM

protected static final byte[] FAM

REGION_ONLINE_TIMEOUT

protected static final int REGION_ONLINE_TIMEOUT

See Also:

Constant Field Values

assignmentManager

protected static org.apache.hadoop.hbase.master.assignment.AssignmentManager assignmentManager

regionStates

protected static org.apache.hadoop.hbase.master.assignment.RegionStates regionStates

tableExecutorService

protected static ExecutorService tableExecutorService

hbfsckExecutorService

protected static ScheduledThreadPoolExecutor hbfsckExecutorService

connection

protected static org.apache.hadoop.hbase.client.ClusterConnection connection

admin

protected static org.apache.hadoop.hbase.client.Admin admin

tbl

protected org.apache.hadoop.hbase.client.Table tbl

SPLITS

protected static final byte[][] SPLITS

ROWKEYS

protected static final byte[][] ROWKEYS

name

public org.junit.rules.TestName name

Constructor Detail

BaseTestHBaseFsck

public BaseTestHBaseFsck()

Method Detail

dumpMeta

protected void dumpMeta(org.apache.hadoop.hbase.TableName tableName)
                 throws IOException

Debugging method to dump the contents of meta.

Throws:

IOException

undeployRegion

protected void undeployRegion(org.apache.hadoop.hbase.client.Connection conn,
                              org.apache.hadoop.hbase.ServerName sn,
                              org.apache.hadoop.hbase.client.RegionInfo hri)
                       throws IOException,
                              InterruptedException

This method is used to undeploy a region -- close it and attempt to remove its state from the Master.

Throws:

IOException

InterruptedException

deleteRegion

protected void deleteRegion(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.hbase.HTableDescriptor htd,
                            byte[] startKey,
                            byte[] endKey,
                            boolean unassign,
                            boolean metaRow,
                            boolean hdfs)
                     throws IOException,
                            InterruptedException

Delete a region from assignments, meta, or completely from hdfs.

Parameters:

unassign - if true unassign region if assigned

metaRow - if true remove region's row from META

hdfs - if true remove region's dir in HDFS

Throws:

IOException

InterruptedException

deleteRegion

protected void deleteRegion(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.hbase.HTableDescriptor htd,
                            byte[] startKey,
                            byte[] endKey,
                            boolean unassign,
                            boolean metaRow,
                            boolean hdfs,
                            boolean regionInfoOnly,
                            int replicaId)
                     throws IOException,
                            InterruptedException

Delete a region from assignments, meta, or completely from hdfs.

Parameters:

unassign - if true unassign region if assigned

metaRow - if true remove region's row from META

hdfs - if true remove region's dir in HDFS

regionInfoOnly - if true remove a region dir's .regioninfo file

replicaId - replica id

Throws:

IOException

InterruptedException

setupTable

void setupTable(org.apache.hadoop.hbase.TableName tablename)
         throws Exception

Setup a clean table before we start mucking with it. It will set tbl which needs to be closed after test

Throws:

IOException

InterruptedException

org.apache.zookeeper.KeeperException

Exception

setupTableWithRegionReplica

void setupTableWithRegionReplica(org.apache.hadoop.hbase.TableName tablename,
                                 int replicaCount)
                          throws Exception

Setup a clean table with a certain region_replica count It will set tbl which needs to be closed after test

Throws:

Exception

setupMobTable

void setupMobTable(org.apache.hadoop.hbase.TableName tablename)
            throws Exception

Setup a clean table with a mob-enabled column.

Parameters:

tablename - The name of a table to be created.

Throws:

Exception

countRows

int countRows()
       throws IOException

Counts the number of rows to verify data loss or non-dataloss.

Throws:

IOException

countRows

int countRows(byte[] start,
              byte[] end)
       throws IOException

Counts the number of rows to verify data loss or non-dataloss.

Throws:

IOException

cleanupTable

void cleanupTable(org.apache.hadoop.hbase.TableName tablename)
           throws Exception

delete table in preparation for next test

Parameters:

tablename -

Throws:

IOException

Exception

getDeployedHRIs

Map<org.apache.hadoop.hbase.ServerName,List<String>> getDeployedHRIs(org.apache.hadoop.hbase.client.Admin admin)
                                                              throws IOException

Get region info from local cluster.

Throws:

IOException

findDeployedHSI

org.apache.hadoop.hbase.ServerName findDeployedHSI(Map<org.apache.hadoop.hbase.ServerName,List<String>> mm,
                                                   org.apache.hadoop.hbase.client.RegionInfo hri)

Returns the HSI a region info is on.

deleteTableDir

public void deleteTableDir(org.apache.hadoop.hbase.TableName table)
                    throws IOException

Throws:

IOException

getFlushedHFile

org.apache.hadoop.fs.Path getFlushedHFile(org.apache.hadoop.fs.FileSystem fs,
                                          org.apache.hadoop.hbase.TableName table)
                                   throws IOException

We don't have an easy way to verify that a flush completed, so we loop until we find a legitimate hfile and return it.

Parameters:

fs -

table -

Returns:

Path of a flushed hfile.

Throws:

IOException

getFlushedMobFile

org.apache.hadoop.fs.Path getFlushedMobFile(org.apache.hadoop.fs.FileSystem fs,
                                            org.apache.hadoop.hbase.TableName table)
                                     throws IOException

Gets flushed mob files.

Parameters:

fs - The current file system.

table - The current table name.

Returns:

Path of a flushed hfile.

Throws:

IOException

createMobFileName

String createMobFileName(String oldFileName)

Creates a new mob file name by the old one.

Parameters:

oldFileName - The old mob file name.

Returns:

The new mob file name.

doQuarantineTest

protected void doQuarantineTest(org.apache.hadoop.hbase.TableName table,
                                org.apache.hadoop.hbase.util.HBaseFsck hbck,
                                int check,
                                int corrupt,
                                int fail,
                                int quar,
                                int missing)
                         throws Exception

Test that use this should have a timeout, because this method could potentially wait forever.

Throws:

Exception

deleteMetaRegion

protected void deleteMetaRegion(org.apache.hadoop.conf.Configuration conf,
                                boolean unassign,
                                boolean hdfs,
                                boolean regionInfoOnly)
                         throws IOException,
                                InterruptedException

Throws:

IOException

InterruptedException

createTable

public static void createTable(HBaseTestingUtility testUtil,
                               org.apache.hadoop.hbase.HTableDescriptor htd,
                               byte[][] splitKeys)
                        throws Exception

Throws:

Exception

deleteTable

public static void deleteTable(HBaseTestingUtility testUtil,
                               org.apache.hadoop.hbase.TableName tableName)
                        throws Exception

Throws:

Exception