org.apache.hadoop.hbase.client

Class TestClientNoCluster

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.client.TestClientNoCluster

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class TestClientNoCluster
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Test client behavior w/o setting up a cluster. Mock up cluster emissions. See below for a method that tests retries/timeouts currently commented out.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	TestClientNoCluster.FakeServer Fake 'server'.
(package private) static class	TestClientNoCluster.ManyServersManyRegionsConnection Fake many regionservers and many regions on a connection implementation.
private static class	TestClientNoCluster.MetaRowsComparator Comparator for meta row keys.
(package private) static class	TestClientNoCluster.RegionServerStoppedOnScannerOpenConnection Override to shutdown going to zookeeper for cluster id and meta location.
(package private) static class	TestClientNoCluster.RpcTimeoutAsyncConnection Override to check we are setting rpc timeout right.
(package private) static class	TestClientNoCluster.RpcTimeoutConnection Override to check we are setting rpc timeout right.
(package private) static class	TestClientNoCluster.SimpleRegistry Simple cluster registry inserted in place of our usual zookeeper based one.

Field Summary

Fields

Modifier and Type	Field and Description
private static byte[]	BIG_USER_TABLE
private static org.apache.hbase.thirdparty.com.google.protobuf.ByteString	CATALOG_FAMILY_BYTESTRING
static HBaseClassTestRule	CLASS_RULE
private org.apache.hadoop.conf.Configuration	conf
private static org.slf4j.Logger	LOG
static org.apache.hadoop.hbase.ServerName	META_SERVERNAME A server that does not exist.
private static org.apache.hbase.thirdparty.com.google.protobuf.ByteString	REGIONINFO_QUALIFIER_BYTESTRING
private static org.apache.hbase.thirdparty.com.google.protobuf.ByteString	SERVER_QUALIFIER_BYTESTRING

Constructor Summary

Constructors

Constructor and Description
TestClientNoCluster()

Method Summary

Modifier and Type	Method and Description
(package private) static void	cycle(int id, org.apache.hadoop.conf.Configuration c, org.apache.hadoop.hbase.client.Connection sharedConnection) Code for each 'client' to run.
(package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetResponse	doMetaGetResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetRequest request)
(package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanResponse	doMetaScanResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, AtomicLong sequenceids, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest request)
(package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiResponse	doMultiResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta, AtomicLong sequenceids, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiRequest request)
private static byte[]	format(long number) Format passed integer.
(package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell.Builder	getBaseCellBuilder(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)
(package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell	getRegionInfo(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row, org.apache.hadoop.hbase.HRegionInfo hri)
(package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell	getServer(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row, org.apache.hadoop.hbase.ServerName sn)
(package private) static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell	getStartCode(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)
(package private) static boolean	isMetaRegion(byte[] name, org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier.RegionSpecifierType type)
static void	main(String[] args) Run a client instance against a faked up server.
private static org.apache.hadoop.hbase.HRegionInfo[]	makeHRegionInfos(byte[] tableName, int count, long namespaceSpan) nn * @return count regions
(package private) static SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>>	makeMeta(byte[] tableName, int regionCount, long namespaceSpan, int serverCount) Create up a map that is keyed by meta row name and whose value is the HRegionInfo and ServerName to return for this row.
private static org.apache.hadoop.hbase.ServerName[]	makeServerNames(int count) n * @return Return count servernames.
int	run(String[] arg0)
void	setUp()
void	testAsyncTimeoutAndRetries() Remove the @Ignore to try out timeout and retry settings
void	testConnectionClosedOnRegionLocate()
void	testDoNotRetryMetaTableAccessor()
void	testDoNotRetryOnScanNext()
void	testRegionServerStoppedOnScannerOpen()
void	testRpcTimeout() Test that operation timeout prevails over rpc default timeout and retries, etc.
void	testTimeoutAndRetries() Remove the @Ignore to try out timeout and retry settings

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

LOG

private static final org.slf4j.Logger LOG

conf

private org.apache.hadoop.conf.Configuration conf

META_SERVERNAME

public static final org.apache.hadoop.hbase.ServerName META_SERVERNAME

A server that does not exist. I've changed the server in the below to 'localhost' so we have a servername that resolves -- otherwise, we just fail on server name lookup with UnknownHost... With localhost, was able to reproduce stack traces that looked like production stack traces. Was useful figuring out how retry/timeouts are functioning.

CATALOG_FAMILY_BYTESTRING

private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString CATALOG_FAMILY_BYTESTRING

REGIONINFO_QUALIFIER_BYTESTRING

private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString REGIONINFO_QUALIFIER_BYTESTRING

SERVER_QUALIFIER_BYTESTRING

private static final org.apache.hbase.thirdparty.com.google.protobuf.ByteString SERVER_QUALIFIER_BYTESTRING

BIG_USER_TABLE

private static final byte[] BIG_USER_TABLE

Constructor Detail

TestClientNoCluster

public TestClientNoCluster()

Method Detail

setUp

public void setUp()
           throws Exception

Throws:

Exception

testTimeoutAndRetries

public void testTimeoutAndRetries()
                           throws IOException

Remove the @Ignore to try out timeout and retry settings

Throws:

IOException

testAsyncTimeoutAndRetries

public void testAsyncTimeoutAndRetries()
                                throws IOException,
                                       ExecutionException,
                                       InterruptedException

Remove the @Ignore to try out timeout and retry settings

Throws:

IOException

ExecutionException

InterruptedException

testRpcTimeout

public void testRpcTimeout()
                    throws IOException

Test that operation timeout prevails over rpc default timeout and retries, etc. n

Throws:

IOException

testDoNotRetryMetaTableAccessor

public void testDoNotRetryMetaTableAccessor()
                                     throws IOException

Throws:

IOException

testDoNotRetryOnScanNext

public void testDoNotRetryOnScanNext()
                              throws IOException

Throws:

IOException

testRegionServerStoppedOnScannerOpen

public void testRegionServerStoppedOnScannerOpen()
                                          throws IOException

Throws:

IOException

testConnectionClosedOnRegionLocate

public void testConnectionClosedOnRegionLocate()
                                        throws IOException

Throws:

IOException

doMultiResponse

static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiResponse doMultiResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta,
                                                                                                    AtomicLong sequenceids,
                                                                                                    org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.MultiRequest request)

doMetaScanResponse

static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanResponse doMetaScanResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta,
                                                                                                      AtomicLong sequenceids,
                                                                                                      org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest request)

doMetaGetResponse

static org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetResponse doMetaGetResponse(SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> meta,
                                                                                                    org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.GetRequest request)

isMetaRegion

static boolean isMetaRegion(byte[] name,
                            org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier.RegionSpecifierType type)

Parameters:

name - region name or encoded region name. n * @return True if we are dealing with a hbase:meta region.

getBaseCellBuilder

static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell.Builder getBaseCellBuilder(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)

getRegionInfo

static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell getRegionInfo(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row,
                                                                                       org.apache.hadoop.hbase.HRegionInfo hri)

getServer

static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell getServer(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row,
                                                                                   org.apache.hadoop.hbase.ServerName sn)

getStartCode

static org.apache.hadoop.hbase.shaded.protobuf.generated.CellProtos.Cell getStartCode(org.apache.hbase.thirdparty.com.google.protobuf.ByteString row)

format

private static byte[] format(long number)

Format passed integer. Zero-pad. Copied from hbase-server PE class and small amendment. Make them share. n * @return Returns zero-prefixed 10-byte wide decimal version of passed number (Does absolute in case number is negative).

makeHRegionInfos

private static org.apache.hadoop.hbase.HRegionInfo[] makeHRegionInfos(byte[] tableName,
                                                                      int count,
                                                                      long namespaceSpan)

nn * @return count regions

makeServerNames

private static org.apache.hadoop.hbase.ServerName[] makeServerNames(int count)

n * @return Return count servernames.

makeMeta

static SortedMap<byte[],org.apache.hadoop.hbase.util.Pair<org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.ServerName>> makeMeta(byte[] tableName,
                                                                                                                                            int regionCount,
                                                                                                                                            long namespaceSpan,
                                                                                                                                            int serverCount)

Create up a map that is keyed by meta row name and whose value is the HRegionInfo and ServerName to return for this row.

Returns:

Map with faked hbase:meta content in it.

cycle

static void cycle(int id,
                  org.apache.hadoop.conf.Configuration c,
                  org.apache.hadoop.hbase.client.Connection sharedConnection)
           throws IOException

Code for each 'client' to run. nnnn

Throws:

IOException

run

public int run(String[] arg0)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Run a client instance against a faked up server.

Parameters:

args - TODO n

Throws:

Exception