org.apache.hadoop.hbase.client

Class ConnectionUtils

java.lang.Object

org.apache.hadoop.hbase.client.ConnectionUtils

@InterfaceAudience.Private
public final class ConnectionUtils
extends Object

Utility used by client connections.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	ConnectionUtils.MasterlessConnection Some tests shut down the master.
static class	ConnectionUtils.ShortCircuitingClusterConnection A ClusterConnection that will short-circuit RPC making direct invocations against the localhost if the invocation target is 'this' server; save on network and protobuf invocations.

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOG
(package private) static byte[]	MAX_BYTE_ARRAY
private static String	MY_ADDRESS
(package private) static NonceGenerator	NO_NONCE_GENERATOR Dummy nonce generator for disabled nonces.
(package private) static long	SLEEP_DELTA_NS

Constructor Summary

Constructors

Modifier	Constructor and Description
private	ConnectionUtils()

Method Summary

Modifier and Type	Method and Description
(package private) static <T> CompletableFuture<List<T>>	allOf(List<CompletableFuture<T>> futures)
(package private) static long	calcEstimatedSize(Result rs)
(package private) static int	calcPriority(int priority, TableName tableName) Select the priority for the rpc call.
(package private) static void	checkHasFamilies(Mutation mutation)
private static <T> void	connect(CompletableFuture<T> srcFuture, CompletableFuture<T> dstFuture, Optional<MetricsConnection> metrics) Connect the two futures, if the src future is done, then mark the dst future as done.
(package private) static byte[]	createCloseRowBefore(byte[] row) Create a row before the specified row and very close to the specified row.
(package private) static byte[]	createClosestRowAfter(byte[] row) Create the closest row after the specified row
static ScanResultCache	createScanResultCache(Scan scan)
static ClusterConnection	createShortCircuitConnection(org.apache.hadoop.conf.Configuration conf, ExecutorService pool, User user, ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface admin, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService.BlockingInterface client) Creates a short-circuit connection that can bypass the RPC layer (serialization, deserialization, networking, etc..) when talking to a local server.
(package private) static Result	filterCells(Result result, Cell keepCellsAfter)
(package private) static RegionLocateType	getLocateType(Scan scan)
private static String	getMyAddress()
(package private) static <T> CompletableFuture<T>	getOrFetch(AtomicReference<T> cacheRef, AtomicReference<CompletableFuture<T>> futureRef, boolean reload, Supplier<CompletableFuture<T>> fetch, Predicate<T> validator, String type)
static long	getPauseTime(long pause, int tries) Calculate pause time.
(package private) static int	getPriority(TableName tableName)
(package private) static String	getStubKey(String serviceName, ServerName serverName, boolean hostnameCanChange) Get a unique key for the rpc stub to the given server.
(package private) static void	incRegionCountMetrics(ScanMetrics scanMetrics)
(package private) static void	incRPCCallsMetrics(ScanMetrics scanMetrics, boolean isRegionServerRemote)
(package private) static void	incRPCRetriesMetrics(ScanMetrics scanMetrics, boolean isRegionServerRemote)
static NonceGenerator	injectNonceGeneratorForTesting(ClusterConnection conn, NonceGenerator cnm)
(package private) static boolean	isEmptyStartRow(byte[] row)
(package private) static boolean	isEmptyStopRow(byte[] row)
(package private) static boolean	isRemote(String host)
(package private) static boolean	noMoreResultsForReverseScan(Scan scan, RegionInfo info)
(package private) static boolean	noMoreResultsForScan(Scan scan, RegionInfo info)
(package private) static void	resetController(HBaseRpcController controller, long timeoutNs, int priority)
(package private) static int	retries2Attempts(int retries) Return retires + 1.
private static <T> void	sendRequestsToSecondaryReplicas(Function<Integer,CompletableFuture<T>> requestReplica, RegionLocations locs, CompletableFuture<T> future, Optional<MetricsConnection> metrics)
static void	setServerSideHConnectionRetriesConfig(org.apache.hadoop.conf.Configuration c, String sn, org.slf4j.Logger log) Changes the configuration to set the number of retries needed when using Connection internally, e.g.
static void	setupMasterlessConnection(org.apache.hadoop.conf.Configuration conf) Setup the connection class, so that it will not depend on master being online.
(package private) static <T> CompletableFuture<T>	timelineConsistentRead(AsyncRegionLocator locator, TableName tableName, Query query, byte[] row, RegionLocateType locateType, Function<Integer,CompletableFuture<T>> requestReplica, long rpcTimeoutNs, long primaryCallTimeoutNs, org.apache.hbase.thirdparty.io.netty.util.Timer retryTimer, Optional<MetricsConnection> metrics)
(package private) static Get	toCheckExistenceOnly(Get get)
(package private) static List<Get>	toCheckExistenceOnly(List<Get> gets)
(package private) static Throwable	translateException(Throwable t)
(package private) static void	updateResultsMetrics(ScanMetrics scanMetrics, Result[] rrs, boolean isRegionServerRemote)
(package private) static void	updateServerSideMetrics(ScanMetrics scanMetrics, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanResponse response) Use the scan metrics returned by the server to add to the identically named counters in the client side metrics.
(package private) static void	updateStats(Optional<ServerStatisticTracker> optStats, Optional<MetricsConnection> optMetrics, ServerName serverName, MultiResponse resp)
(package private) static void	validatePut(Put put, int maxKeyValueSize)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

NO_NONCE_GENERATOR

static final NonceGenerator NO_NONCE_GENERATOR

Dummy nonce generator for disabled nonces.

MAX_BYTE_ARRAY

static final byte[] MAX_BYTE_ARRAY

SLEEP_DELTA_NS

static final long SLEEP_DELTA_NS

MY_ADDRESS

private static final String MY_ADDRESS

Constructor Detail

ConnectionUtils

private ConnectionUtils()

Method Detail

getPauseTime

public static long getPauseTime(long pause,
                                int tries)

Calculate pause time. Built on HConstants.RETRY_BACKOFF.

Parameters:

pause - time to pause

tries - amount of tries

Returns:

How long to wait after tries retries

injectNonceGeneratorForTesting

public static NonceGenerator injectNonceGeneratorForTesting(ClusterConnection conn,
                                                            NonceGenerator cnm)

Parameters:

conn - The connection for which to replace the generator.

cnm - Replaces the nonce generator used, for testing.

Returns:

old nonce generator.

setServerSideHConnectionRetriesConfig

public static void setServerSideHConnectionRetriesConfig(org.apache.hadoop.conf.Configuration c,
                                                         String sn,
                                                         org.slf4j.Logger log)

Changes the configuration to set the number of retries needed when using Connection internally, e.g. for updating catalog tables, etc. Call this method before we create any Connections.

Parameters:

c - The Configuration instance to set the retries into.

log - Used to log what we set in here.

createShortCircuitConnection

public static ClusterConnection createShortCircuitConnection(org.apache.hadoop.conf.Configuration conf,
                                                             ExecutorService pool,
                                                             User user,
                                                             ServerName serverName,
                                                             org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface admin,
                                                             org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService.BlockingInterface client)
                                                      throws IOException

Creates a short-circuit connection that can bypass the RPC layer (serialization, deserialization, networking, etc..) when talking to a local server.

Parameters:

conf - the current configuration

pool - the thread pool to use for batch operations

user - the user the connection is for

serverName - the local server name

admin - the admin interface of the local server

client - the client interface of the local server

Returns:

an short-circuit connection.

Throws:

IOException - if IO failure occurred

setupMasterlessConnection

public static void setupMasterlessConnection(org.apache.hadoop.conf.Configuration conf)

Setup the connection class, so that it will not depend on master being online. Used for testing

Parameters:

conf - configuration to set

retries2Attempts

static int retries2Attempts(int retries)

Return retires + 1. The returned value will be in range [1, Integer.MAX_VALUE].

getStubKey

static String getStubKey(String serviceName,
                         ServerName serverName,
                         boolean hostnameCanChange)

Get a unique key for the rpc stub to the given server.

checkHasFamilies

static void checkHasFamilies(Mutation mutation)

createClosestRowAfter

static byte[] createClosestRowAfter(byte[] row)

Create the closest row after the specified row

createCloseRowBefore

static byte[] createCloseRowBefore(byte[] row)

Create a row before the specified row and very close to the specified row.

isEmptyStartRow

static boolean isEmptyStartRow(byte[] row)

isEmptyStopRow

static boolean isEmptyStopRow(byte[] row)

resetController

static void resetController(HBaseRpcController controller,
                            long timeoutNs,
                            int priority)

translateException

static Throwable translateException(Throwable t)

calcEstimatedSize

static long calcEstimatedSize(Result rs)

filterCells

static Result filterCells(Result result,
                          Cell keepCellsAfter)

toCheckExistenceOnly

static Get toCheckExistenceOnly(Get get)

toCheckExistenceOnly

static List<Get> toCheckExistenceOnly(List<Get> gets)

getLocateType

static RegionLocateType getLocateType(Scan scan)

noMoreResultsForScan

static boolean noMoreResultsForScan(Scan scan,
                                    RegionInfo info)

noMoreResultsForReverseScan

static boolean noMoreResultsForReverseScan(Scan scan,
                                           RegionInfo info)

allOf

static <T> CompletableFuture<List<T>> allOf(List<CompletableFuture<T>> futures)

createScanResultCache

public static ScanResultCache createScanResultCache(Scan scan)

getMyAddress

private static String getMyAddress()

isRemote

static boolean isRemote(String host)

incRPCCallsMetrics

static void incRPCCallsMetrics(ScanMetrics scanMetrics,
                               boolean isRegionServerRemote)

incRPCRetriesMetrics

static void incRPCRetriesMetrics(ScanMetrics scanMetrics,
                                 boolean isRegionServerRemote)

updateResultsMetrics

static void updateResultsMetrics(ScanMetrics scanMetrics,
                                 Result[] rrs,
                                 boolean isRegionServerRemote)

updateServerSideMetrics

static void updateServerSideMetrics(ScanMetrics scanMetrics,
                                    org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanResponse response)

Use the scan metrics returned by the server to add to the identically named counters in the client side metrics. If a counter does not exist with the same name as the server side metric, the attempt to increase the counter will fail.

incRegionCountMetrics

static void incRegionCountMetrics(ScanMetrics scanMetrics)

connect

private static <T> void connect(CompletableFuture<T> srcFuture,
                                CompletableFuture<T> dstFuture,
                                Optional<MetricsConnection> metrics)

Connect the two futures, if the src future is done, then mark the dst future as done. And if the dst future is done, then cancel the src future. This is used for timeline consistent read.

Pass empty metrics if you want to link the primary future and the dst future so we will not increase the hedge read related metrics.

sendRequestsToSecondaryReplicas

private static <T> void sendRequestsToSecondaryReplicas(Function<Integer,CompletableFuture<T>> requestReplica,
                                                        RegionLocations locs,
                                                        CompletableFuture<T> future,
                                                        Optional<MetricsConnection> metrics)

timelineConsistentRead

static <T> CompletableFuture<T> timelineConsistentRead(AsyncRegionLocator locator,
                                                       TableName tableName,
                                                       Query query,
                                                       byte[] row,
                                                       RegionLocateType locateType,
                                                       Function<Integer,CompletableFuture<T>> requestReplica,
                                                       long rpcTimeoutNs,
                                                       long primaryCallTimeoutNs,
                                                       org.apache.hbase.thirdparty.io.netty.util.Timer retryTimer,
                                                       Optional<MetricsConnection> metrics)

validatePut

static void validatePut(Put put,
                        int maxKeyValueSize)
                 throws IllegalArgumentException

Throws:

IllegalArgumentException

calcPriority

static int calcPriority(int priority,
                        TableName tableName)

Select the priority for the rpc call.

The rules are:

1. If user set a priority explicitly, then just use it.
2. For system table, use HConstants.SYSTEMTABLE_QOS.
3. For other tables, use HConstants.NORMAL_QOS.

Parameters:

priority - the priority set by user, can be HConstants.PRIORITY_UNSET.

tableName - the table we operate on

getPriority

static int getPriority(TableName tableName)

getOrFetch

static <T> CompletableFuture<T> getOrFetch(AtomicReference<T> cacheRef,
                                           AtomicReference<CompletableFuture<T>> futureRef,
                                           boolean reload,
                                           Supplier<CompletableFuture<T>> fetch,
                                           Predicate<T> validator,
                                           String type)

updateStats

static void updateStats(Optional<ServerStatisticTracker> optStats,
                        Optional<MetricsConnection> optMetrics,
                        ServerName serverName,
                        MultiResponse resp)