Package org.apache.hadoop.hbase.util

Class BaseTestHBaseFsck

java.lang.Object
org.apache.hadoop.hbase.util.BaseTestHBaseFsck
Direct Known Subclasses:
TestHBaseFsckMOB
public class BaseTestHBaseFsck extends Object
This is the base class for HBaseFsck's ability to detect reasons for inconsistent tables. Actual tests are in : TestHBaseFsckTwoRS TestHBaseFsckOneRS TestHBaseFsckMOB TestHBaseFsckReplicas

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    BaseTestHBaseFsck.MasterSyncCoprocessor
     
    (package private) static class 
    BaseTestHBaseFsck.MockErrorReporter
     

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected static org.apache.hadoop.hbase.client.Admin
    admin
     
    protected static org.apache.hadoop.hbase.master.assignment.AssignmentManager
    assignmentManager
     
    protected static final org.apache.hadoop.conf.Configuration
    conf
     
    protected static org.apache.hadoop.hbase.client.ClusterConnection
    connection
     
    protected static final byte[]
    FAM
     
    protected static final String
    FAM_STR
     
    protected static ScheduledThreadPoolExecutor
    hbfsckExecutorService
     
    protected static final org.slf4j.Logger
    LOG
     
    org.junit.rules.TestName
    name
     
    (package private) static final int
    POOL_SIZE
     
    protected static final int
    REGION_ONLINE_TIMEOUT
     
    protected static org.apache.hadoop.hbase.master.assignment.RegionStates
    regionStates
     
    protected static final byte[][]
    ROWKEYS
     
    protected static final byte[][]
    SPLITS
     
    protected static ExecutorService
    tableExecutorService
     
    protected org.apache.hadoop.hbase.client.Table
    tbl
     
    protected static final HBaseTestingUtility
    TEST_UTIL
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    BaseTestHBaseFsck()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    (package private) void
    cleanupTable(org.apache.hadoop.hbase.TableName tablename)
    delete table in preparation for next test
    (package private) int
    countRows()
    Counts the number of rows to verify data loss or non-dataloss.
    (package private) int
    countRows(byte[] start, byte[] end)
    Counts the number of rows to verify data loss or non-dataloss.
    (package private) String
    createMobFileName(String oldFileName)
    Creates a new mob file name by the old one.
    static void
    createTable(HBaseTestingUtility testUtil, org.apache.hadoop.hbase.HTableDescriptor htd, byte[][] splitKeys)
     
    protected void
    deleteMetaRegion(org.apache.hadoop.conf.Configuration conf, boolean unassign, boolean hdfs, boolean regionInfoOnly)
     
    protected void
    deleteRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.HTableDescriptor htd, byte[] startKey, byte[] endKey, boolean unassign, boolean metaRow, boolean hdfs)
    Delete a region from assignments, meta, or completely from hdfs.
    protected void
    deleteRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.HTableDescriptor htd, byte[] startKey, byte[] endKey, boolean unassign, boolean metaRow, boolean hdfs, boolean regionInfoOnly, int replicaId)
    Delete a region from assignments, meta, or completely from hdfs.
    static void
    deleteTable(HBaseTestingUtility testUtil, org.apache.hadoop.hbase.TableName tableName)
     
    void
    deleteTableDir(org.apache.hadoop.hbase.TableName table)
     
    protected void
    doQuarantineTest(org.apache.hadoop.hbase.TableName table, org.apache.hadoop.hbase.util.HBaseFsck hbck, int check, int corrupt, int fail, int quar, int missing)
    Test that use this should have a timeout, because this method could potentially wait forever.
    protected void
    dumpMeta(org.apache.hadoop.hbase.TableName tableName)
    Debugging method to dump the contents of meta.
    (package private) org.apache.hadoop.hbase.ServerName
    findDeployedHSI(Map<org.apache.hadoop.hbase.ServerName,List<String>> mm, org.apache.hadoop.hbase.client.RegionInfo hri)
    Returns the HSI a region info is on.
    (package private) Map<org.apache.hadoop.hbase.ServerName,List<String>>
    getDeployedHRIs(org.apache.hadoop.hbase.client.Admin admin)
    Get region info from local cluster.
    (package private) org.apache.hadoop.fs.Path
    getFlushedHFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.hbase.TableName table)
    We don't have an easy way to verify that a flush completed, so we loop until we find a legitimate hfile and return it.
    (package private) org.apache.hadoop.fs.Path
    getFlushedMobFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.hbase.TableName table)
    Gets flushed mob files.
    (package private) void
    setupMobTable(org.apache.hadoop.hbase.TableName tablename)
    Setup a clean table with a mob-enabled column.
    (package private) void
    setupTable(org.apache.hadoop.hbase.TableName tablename)
    Setup a clean table before we start mucking with it.
    (package private) void
    setupTableWithRegionReplica(org.apache.hadoop.hbase.TableName tablename, int replicaCount)
    Setup a clean table with a certain region_replica count It will set tbl which needs to be closed after test
    protected void
    undeployRegion(org.apache.hadoop.hbase.client.Connection conn, org.apache.hadoop.hbase.ServerName sn, org.apache.hadoop.hbase.client.RegionInfo hri)
    This method is used to undeploy a region -- close it and attempt to remove its state from the Master.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

  • Method Details

    • dumpMeta

      protected void dumpMeta(org.apache.hadoop.hbase.TableName tableName) throws IOException
      Debugging method to dump the contents of meta.
      Throws:
      IOException

    • undeployRegion

      protected void undeployRegion(org.apache.hadoop.hbase.client.Connection conn, org.apache.hadoop.hbase.ServerName sn, org.apache.hadoop.hbase.client.RegionInfo hri) throws IOException, InterruptedException
      This method is used to undeploy a region -- close it and attempt to remove its state from the Master.
      Throws:
      IOException
      InterruptedException

    • deleteRegion

      protected void deleteRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.HTableDescriptor htd, byte[] startKey, byte[] endKey, boolean unassign, boolean metaRow, boolean hdfs) throws IOException, InterruptedException
      Delete a region from assignments, meta, or completely from hdfs.
      Parameters:
      unassign - if true unassign region if assigned
      metaRow - if true remove region's row from META
      hdfs - if true remove region's dir in HDFS
      Throws:
      IOException
      InterruptedException

    • deleteRegion

      protected void deleteRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.HTableDescriptor htd, byte[] startKey, byte[] endKey, boolean unassign, boolean metaRow, boolean hdfs, boolean regionInfoOnly, int replicaId) throws IOException, InterruptedException
      Delete a region from assignments, meta, or completely from hdfs.
      Parameters:
      unassign - if true unassign region if assigned
      metaRow - if true remove region's row from META
      hdfs - if true remove region's dir in HDFS
      regionInfoOnly - if true remove a region dir's .regioninfo file
      replicaId - replica id
      Throws:
      IOException
      InterruptedException

    • setupTable

      void setupTable(org.apache.hadoop.hbase.TableName tablename) throws Exception
      Setup a clean table before we start mucking with it. It will set tbl which needs to be closed after test
      Throws:
      Exception

    • setupTableWithRegionReplica

      void setupTableWithRegionReplica(org.apache.hadoop.hbase.TableName tablename, int replicaCount) throws Exception
      Setup a clean table with a certain region_replica count It will set tbl which needs to be closed after test
      Throws:
      Exception

    • setupMobTable

      void setupMobTable(org.apache.hadoop.hbase.TableName tablename) throws Exception
      Setup a clean table with a mob-enabled column.
      Parameters:
      tablename - The name of a table to be created.
      Throws:
      Exception

    • countRows

      int countRows() throws IOException
      Counts the number of rows to verify data loss or non-dataloss.
      Throws:
      IOException

    • countRows

      int countRows(byte[] start, byte[] end) throws IOException
      Counts the number of rows to verify data loss or non-dataloss.
      Throws:
      IOException

    • cleanupTable

      void cleanupTable(org.apache.hadoop.hbase.TableName tablename) throws Exception
      delete table in preparation for next test
      Throws:
      Exception

    • getDeployedHRIs

      Map<org.apache.hadoop.hbase.ServerName,List<String>> getDeployedHRIs(org.apache.hadoop.hbase.client.Admin admin) throws IOException
      Get region info from local cluster.
      Throws:
      IOException

    • findDeployedHSI

      org.apache.hadoop.hbase.ServerName findDeployedHSI(Map<org.apache.hadoop.hbase.ServerName,List<String>> mm, org.apache.hadoop.hbase.client.RegionInfo hri)
      Returns the HSI a region info is on.

    • deleteTableDir

      public void deleteTableDir(org.apache.hadoop.hbase.TableName table) throws IOException
      Throws:
      IOException

    • getFlushedHFile

      org.apache.hadoop.fs.Path getFlushedHFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.hbase.TableName table) throws IOException
      We don't have an easy way to verify that a flush completed, so we loop until we find a legitimate hfile and return it.
      Returns:
      Path of a flushed hfile.
      Throws:
      IOException

    • getFlushedMobFile

      org.apache.hadoop.fs.Path getFlushedMobFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.hbase.TableName table) throws IOException
      Gets flushed mob files.
      Parameters:
      fs - The current file system.
      table - The current table name.
      Returns:
      Path of a flushed hfile.
      Throws:
      IOException

    • createMobFileName

      String createMobFileName(String oldFileName)
      Creates a new mob file name by the old one.
      Parameters:
      oldFileName - The old mob file name.
      Returns:
      The new mob file name.

    • doQuarantineTest

      protected void doQuarantineTest(org.apache.hadoop.hbase.TableName table, org.apache.hadoop.hbase.util.HBaseFsck hbck, int check, int corrupt, int fail, int quar, int missing) throws Exception
      Test that use this should have a timeout, because this method could potentially wait forever.
      Throws:
      Exception

    • deleteMetaRegion

      protected void deleteMetaRegion(org.apache.hadoop.conf.Configuration conf, boolean unassign, boolean hdfs, boolean regionInfoOnly) throws IOException, InterruptedException
      Throws:
      IOException
      InterruptedException

    • createTable

      public static void createTable(HBaseTestingUtility testUtil, org.apache.hadoop.hbase.HTableDescriptor htd, byte[][] splitKeys) throws Exception
      Throws:
      Exception

    • deleteTable

      public static void deleteTable(HBaseTestingUtility testUtil, org.apache.hadoop.hbase.TableName tableName) throws Exception
      Throws:
      Exception