Package org.apache.hadoop.hbase.client

Class TestClientNoCluster

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.hbase.client.TestClientNoCluster
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
public class TestClientNoCluster extends org.apache.hadoop.conf.Configured implements org.apache.hadoop.util.Tool
Test client behavior w/o setting up a cluster. Mock up cluster emissions. See below for a method that tests retries/timeouts currently commented out.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    (package private) static class 
    TestClientNoCluster.FakeServer
    Fake 'server'.
    (package private) static class 
    TestClientNoCluster.ManyServersManyRegionsConnection
    Fake many regionservers and many regions on a connection implementation.
    private static class 
    TestClientNoCluster.MetaRowsComparator
    Comparator for meta row keys.
    (package private) static class 
    TestClientNoCluster.RegionServerStoppedOnScannerOpenConnection
    Override to shutdown going to zookeeper for cluster id and meta location.
    (package private) static class 
    TestClientNoCluster.RpcTimeoutAsyncConnection
    Override to check we are setting rpc timeout right.
    (package private) static class 
    TestClientNoCluster.RpcTimeoutConnection
    Override to check we are setting rpc timeout right.
    (package private) static class 
    TestClientNoCluster.SimpleRegistry
    Simple cluster registry inserted in place of our usual zookeeper based one.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private static final byte[]
    BIG_USER_TABLE
     
    private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString
    CATALOG_FAMILY_BYTESTRING
     
    static final HBaseClassTestRule
    CLASS_RULE
     
    private org.apache.hadoop.conf.Configuration
    conf
     
    private static final org.slf4j.Logger
    LOG
     
    static final org.apache.hadoop.hbase.ServerName
    META_SERVERNAME
    A server that does not exist.
    private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString
    REGIONINFO_QUALIFIER_BYTESTRING
     
    private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString
    SERVER_QUALIFIER_BYTESTRING
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    TestClientNoCluster()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    (package private) static void
    cycle(int id, org.apache.hadoop.conf.Configuration c, org.apache.hadoop.hbase.client.Connection sharedConnection)
    Code for each 'client' to run.
    (package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetResponse
    doMetaGetResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetRequest request)
     
    (package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanResponse
    doMetaScanResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, AtomicLong sequenceids, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest request)
     
    (package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiResponse
    doMultiResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, AtomicLong sequenceids, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiRequest request)
     
    private static byte[]
    format(long number)
    Format passed integer.
    (package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell.Builder
    getBaseCellBuilder(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)
     
    (package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell
    getRegionInfo(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row, org.apache.hadoop.hbase.HRegionInfo hri)
     
    (package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell
    getServer(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row, org.apache.hadoop.hbase.ServerName sn)
     
    (package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell
    getStartCode(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)
     
    (package private) static boolean
    isMetaRegion(byte[] name, org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier.RegionSpecifierType type)
     
    static void
    main(String[] args)
    Run a client instance against a faked up server.
    private static org.apache.hadoop.hbase.HRegionInfo[]
    makeHRegionInfos(byte[] tableName, int count, long namespaceSpan)
    Returns count regions
    (package private) static SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>>
    makeMeta(byte[] tableName, int regionCount, long namespaceSpan, int serverCount)
    Create up a map that is keyed by meta row name and whose value is the HRegionInfo and ServerName to return for this row.
    private static org.apache.hadoop.hbase.ServerName[]
    makeServerNames(int count)
    Returns Return count servernames.
    int
    run(String[] arg0)
     
    void
    setUp()
     
    void
    testAsyncTimeoutAndRetries()
    Remove the @Ignore to try out timeout and retry settings
    void
    testConnectionClosedOnRegionLocate()
     
    void
    testDoNotRetryMetaTableAccessor()
     
    void
    testDoNotRetryOnScanNext()
     
    void
    testRegionServerStoppedOnScannerOpen()
     
    void
    testRpcTimeout()
    Test that operation timeout prevails over rpc default timeout and retries, etc.
    void
    testTimeoutAndRetries()
    Remove the @Ignore to try out timeout and retry settings

    Methods inherited from class org.apache.hadoop.conf.Configured

    getConf, setConf

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.conf.Configurable

    getConf, setConf

  • Field Details

    • CLASS_RULE

      public static final HBaseClassTestRule CLASS_RULE

    • LOG

      private static final org.slf4j.Logger LOG

    • conf

      private org.apache.hadoop.conf.Configuration conf

    • META_SERVERNAME

      public static final org.apache.hadoop.hbase.ServerName META_SERVERNAME
      A server that does not exist. I've changed the server in the below to 'localhost' so we have a servername that resolves -- otherwise, we just fail on server name lookup with UnknownHost... With localhost, was able to reproduce stack traces that looked like production stack traces. Was useful figuring out how retry/timeouts are functioning.

    • CATALOG_FAMILY_BYTESTRING

      private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString CATALOG_FAMILY_BYTESTRING

    • REGIONINFO_QUALIFIER_BYTESTRING

      private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString REGIONINFO_QUALIFIER_BYTESTRING

    • SERVER_QUALIFIER_BYTESTRING

      private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString SERVER_QUALIFIER_BYTESTRING

    • BIG_USER_TABLE

      private static final byte[] BIG_USER_TABLE

  • Constructor Details

  • Method Details

    • setUp

      public void setUp() throws Exception
      Throws:
      Exception

    • testTimeoutAndRetries

      public void testTimeoutAndRetries() throws IOException
      Remove the @Ignore to try out timeout and retry settings
      Throws:
      IOException

    • testAsyncTimeoutAndRetries

      public void testAsyncTimeoutAndRetries() throws IOException, ExecutionException, InterruptedException
      Remove the @Ignore to try out timeout and retry settings
      Throws:
      IOException
      ExecutionException
      InterruptedException

    • testRpcTimeout

      public void testRpcTimeout() throws IOException
      Test that operation timeout prevails over rpc default timeout and retries, etc.
      Throws:
      IOException

    • testDoNotRetryMetaTableAccessor

      public void testDoNotRetryMetaTableAccessor() throws IOException
      Throws:
      IOException

    • testDoNotRetryOnScanNext

      public void testDoNotRetryOnScanNext() throws IOException
      Throws:
      IOException

    • testRegionServerStoppedOnScannerOpen

      public void testRegionServerStoppedOnScannerOpen() throws IOException
      Throws:
      IOException

    • testConnectionClosedOnRegionLocate

      public void testConnectionClosedOnRegionLocate() throws IOException
      Throws:
      IOException

    • doMultiResponse

      static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiResponse doMultiResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, AtomicLong sequenceids, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiRequest request)

    • doMetaScanResponse

      static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanResponse doMetaScanResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, AtomicLong sequenceids, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest request)

    • doMetaGetResponse

      static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetResponse doMetaGetResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetRequest request)

    • isMetaRegion

      static boolean isMetaRegion(byte[] name, org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier.RegionSpecifierType type)
      Parameters:
      name - region name or encoded region name.
      Returns:
      True if we are dealing with a hbase:meta region.

    • getBaseCellBuilder

      static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell.Builder getBaseCellBuilder(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)

    • getRegionInfo

      static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell getRegionInfo(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row, org.apache.hadoop.hbase.HRegionInfo hri)

    • getServer

      static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell getServer(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row, org.apache.hadoop.hbase.ServerName sn)

    • getStartCode

      static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell getStartCode(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)

    • format

      private static byte[] format(long number)
      Format passed integer. Zero-pad. Copied from hbase-server PE class and small amendment. Make them share.
      Returns:
      Returns zero-prefixed 10-byte wide decimal version of passed number (Does absolute in case number is negative).

    • makeHRegionInfos

      private static org.apache.hadoop.hbase.HRegionInfo[] makeHRegionInfos(byte[] tableName, int count, long namespaceSpan)
      Returns count regions

    • makeServerNames

      private static org.apache.hadoop.hbase.ServerName[] makeServerNames(int count)
      Returns Return count servernames.

    • makeMeta

      static SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> makeMeta(byte[] tableName, int regionCount, long namespaceSpan, int serverCount)
      Create up a map that is keyed by meta row name and whose value is the HRegionInfo and ServerName to return for this row.
      Returns:
      Map with faked hbase:meta content in it.

    • cycle

      static void cycle(int id, org.apache.hadoop.conf.Configuration c, org.apache.hadoop.hbase.client.Connection sharedConnection) throws IOException
      Code for each 'client' to run.
      Throws:
      IOException

    • run

      public int run(String[] arg0) throws Exception
      Specified by:
      run in interface org.apache.hadoop.util.Tool
      Throws:
      Exception

    • main

      public static void main(String[] args) throws Exception
      Run a client instance against a faked up server.
      Parameters:
      args - TODO
      Throws:
      Exception