org.apache.hadoop.hbase.client

Class MetricsConnection

java.lang.Object

org.apache.hadoop.hbase.client.MetricsConnection

All Implemented Interfaces:

StatisticTrackable

@InterfaceAudience.Private
public class MetricsConnection
extends Object
implements StatisticTrackable

This class is for maintaining the various connection statistics and publishing them through the metrics interfaces. This class manages its own MetricRegistry and JmxReporter so as to not conflict with other uses of Yammer Metrics within the client application. Instantiating this class implicitly creates and "starts" instances of these classes; be sure to call shutdown() to terminate the thread pools they allocate.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MetricsConnection.CallStats A container class for collecting details about the RPC call as it percolates.
protected static class	MetricsConnection.CallTracker
private static interface	MetricsConnection.NewMetric<T> A lambda for dispatching to the appropriate metric factory method
protected static class	MetricsConnection.RegionStats
protected static class	MetricsConnection.RunnerStats

Field Summary

Fields

Modifier and Type	Field and Description
protected MetricsConnection.CallTracker	appendTracker
private static String	CACHE_BASE
private ConcurrentMap<String,com.codahale.metrics.Counter>	cacheDroppingExceptions
private static int	CAPACITY Anticipated number of metric entries
static String	CLIENT_SIDE_METRICS_ENABLED_KEY Set this key to true to enable metrics collection of client requests.
private static String	CLIENT_SVC
private static String	CNT_BASE
private static int	CONCURRENCY_LEVEL Anticipated number of concurrent accessor threads
protected com.codahale.metrics.Histogram	concurrentCallsPerServerHist
private MetricsConnection.NewMetric<com.codahale.metrics.Counter>	counterFactory
protected MetricsConnection.CallTracker	deleteTracker
private static String	DRTN_BASE
protected MetricsConnection.CallTracker	getTracker
private static String	HEAP_BASE
protected com.codahale.metrics.Counter	hedgedReadOps
protected com.codahale.metrics.Counter	hedgedReadWin
private MetricsConnection.NewMetric<com.codahale.metrics.Histogram>	histogramFactory
protected MetricsConnection.CallTracker	incrementTracker
private static float	LOAD_FACTOR Default load factor from HashMap.DEFAULT_LOAD_FACTOR
private static String	MEMLOAD_BASE
protected com.codahale.metrics.Counter	metaCacheHits
protected com.codahale.metrics.Counter	metaCacheMisses
protected com.codahale.metrics.Counter	metaCacheNumClearRegion
protected com.codahale.metrics.Counter	metaCacheNumClearServer
static String	METRICS_SCOPE_KEY Set to specify a custom scope for the metrics published through MetricsConnection.
protected MetricsConnection.CallTracker	multiTracker
private static String	NS_LOOKUPS
private static String	NS_LOOKUPS_FAILED
protected com.codahale.metrics.Counter	nsLookups
protected com.codahale.metrics.Counter	nsLookupsFailed
protected com.codahale.metrics.Histogram	numActionsPerServerHist
protected MetricsConnection.CallTracker	putTracker
private com.codahale.metrics.MetricRegistry	registry
private com.codahale.metrics.JmxReporter	reporter
private static String	REQ_BASE
private static String	RESP_BASE
protected ConcurrentMap<String,com.codahale.metrics.Counter>	rpcCounters
protected ConcurrentMap<String,com.codahale.metrics.Histogram>	rpcHistograms
protected ConcurrentMap<String,com.codahale.metrics.Timer>	rpcTimers
protected MetricsConnection.RunnerStats	runnerStats
protected MetricsConnection.CallTracker	scanTracker
protected String	scope
protected ConcurrentHashMap<ServerName,ConcurrentMap<byte[],MetricsConnection.RegionStats>>	serverStats
private MetricsConnection.NewMetric<com.codahale.metrics.Timer>	timerFactory
private static String	UNKNOWN_EXCEPTION

Constructor Summary

Constructors

Constructor and Description
MetricsConnection(String scope, Supplier<ThreadPoolExecutor> batchPool, Supplier<ThreadPoolExecutor> metaPool)

Method Summary

Modifier and Type	Method and Description
(package private) String	getExecutorPoolName()
(package private) String	getMetaPoolName()
private <T> T	getMetric(String key, ConcurrentMap<String,T> map, MetricsConnection.NewMetric<T> factory) Get a metric for key from map, or create it with factory.
(package private) com.codahale.metrics.MetricRegistry	getMetricRegistry()
(package private) static String	getScope(org.apache.hadoop.conf.Configuration conf, String clusterId, Object connectionObj) Returns the scope for a MetricsConnection based on the configured METRICS_SCOPE_KEY or by generating a default from the passed clusterId and connectionObj's hashCode.
void	incrCacheDroppingExceptions(Object exception)
void	incrDelayRunnersAndUpdateDelayInterval(long interval) Increment the number of delay runner counts and update delay interval of delay runner.
void	incrHedgedReadOps() Increment the number of hedged read that have occurred.
void	incrHedgedReadWin() Increment the number of hedged read returned faster than the original read.
void	incrMetaCacheHit() Increment the number of meta cache hits.
void	incrMetaCacheMiss() Increment the number of meta cache misses.
void	incrMetaCacheNumClearRegion() Increment the number of meta cache drops requested for individual region.
void	incrMetaCacheNumClearRegion(int count) Increment the number of meta cache drops requested for individual region.
void	incrMetaCacheNumClearServer() Increment the number of meta cache drops requested for entire RegionServer.
void	incrNormalRunners() Increment the number of normal runner counts.
void	incrNsLookups()
void	incrNsLookupsFailed()
static MetricsConnection.CallStats	newCallStats() Produce an instance of MetricsConnection.CallStats for clients to attach to RPCs.
void	shutdown()
void	updateRegionStats(ServerName serverName, byte[] regionName, RegionLoadStats stats) Update stats per region.
void	updateRpc(org.apache.hbase.thirdparty.com.google.protobuf.Descriptors.MethodDescriptor method, org.apache.hbase.thirdparty.com.google.protobuf.Message param, MetricsConnection.CallStats stats) Report RPC context to metrics system.
private void	updateRpcGeneric(String methodName, MetricsConnection.CallStats stats) Update call stats for non-critical-path methods
void	updateServerStats(ServerName serverName, byte[] regionName, Object r)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLIENT_SIDE_METRICS_ENABLED_KEY

public static final String CLIENT_SIDE_METRICS_ENABLED_KEY

Set this key to true to enable metrics collection of client requests.

See Also:

Constant Field Values

METRICS_SCOPE_KEY

public static final String METRICS_SCOPE_KEY

Set to specify a custom scope for the metrics published through MetricsConnection. The scope is added to JMX MBean objectName, and defaults to a combination of the Connection's clusterId and hashCode. For example, a default value for a connection to cluster "foo" might be "foo-7d9d0818", where "7d9d0818" is the hashCode of the underlying AsyncConnectionImpl. Users may set this key to give a more contextual name for this scope. For example, one might want to differentiate a read connection from a write connection by setting the scopes to "foo-read" and "foo-write" respectively. Scope is the only thing that lends any uniqueness to the metrics. Care should be taken to avoid using the same scope for multiple Connections, otherwise the metrics may aggregate in unforeseen ways.

See Also:

Constant Field Values

CNT_BASE

private static final String CNT_BASE

See Also:

Constant Field Values

DRTN_BASE

private static final String DRTN_BASE

See Also:

Constant Field Values

REQ_BASE

private static final String REQ_BASE

See Also:

Constant Field Values

RESP_BASE

private static final String RESP_BASE

See Also:

Constant Field Values

MEMLOAD_BASE

private static final String MEMLOAD_BASE

See Also:

Constant Field Values

HEAP_BASE

private static final String HEAP_BASE

See Also:

Constant Field Values

CACHE_BASE

private static final String CACHE_BASE

See Also:

Constant Field Values

UNKNOWN_EXCEPTION

private static final String UNKNOWN_EXCEPTION

See Also:

Constant Field Values

NS_LOOKUPS

private static final String NS_LOOKUPS

See Also:

Constant Field Values

NS_LOOKUPS_FAILED

private static final String NS_LOOKUPS_FAILED

See Also:

Constant Field Values

CLIENT_SVC

private static final String CLIENT_SVC

serverStats

protected ConcurrentHashMap<ServerName,ConcurrentMap<byte[],MetricsConnection.RegionStats>> serverStats

CAPACITY

private static final int CAPACITY

Anticipated number of metric entries

See Also:

Constant Field Values

LOAD_FACTOR

private static final float LOAD_FACTOR

Default load factor from HashMap.DEFAULT_LOAD_FACTOR

See Also:

Constant Field Values

CONCURRENCY_LEVEL

private static final int CONCURRENCY_LEVEL

Anticipated number of concurrent accessor threads

See Also:

Constant Field Values

registry

private final com.codahale.metrics.MetricRegistry registry

reporter

private final com.codahale.metrics.JmxReporter reporter

scope

protected final String scope

timerFactory

private final MetricsConnection.NewMetric<com.codahale.metrics.Timer> timerFactory

histogramFactory

private final MetricsConnection.NewMetric<com.codahale.metrics.Histogram> histogramFactory

counterFactory

private final MetricsConnection.NewMetric<com.codahale.metrics.Counter> counterFactory

metaCacheHits

protected final com.codahale.metrics.Counter metaCacheHits

metaCacheMisses

protected final com.codahale.metrics.Counter metaCacheMisses

getTracker

protected final MetricsConnection.CallTracker getTracker

scanTracker

protected final MetricsConnection.CallTracker scanTracker

appendTracker

protected final MetricsConnection.CallTracker appendTracker

deleteTracker

protected final MetricsConnection.CallTracker deleteTracker

incrementTracker

protected final MetricsConnection.CallTracker incrementTracker

putTracker

protected final MetricsConnection.CallTracker putTracker

multiTracker

protected final MetricsConnection.CallTracker multiTracker

runnerStats

protected final MetricsConnection.RunnerStats runnerStats

metaCacheNumClearServer

protected final com.codahale.metrics.Counter metaCacheNumClearServer

metaCacheNumClearRegion

protected final com.codahale.metrics.Counter metaCacheNumClearRegion

hedgedReadOps

protected final com.codahale.metrics.Counter hedgedReadOps

hedgedReadWin

protected final com.codahale.metrics.Counter hedgedReadWin

concurrentCallsPerServerHist

protected final com.codahale.metrics.Histogram concurrentCallsPerServerHist

numActionsPerServerHist

protected final com.codahale.metrics.Histogram numActionsPerServerHist

nsLookups

protected final com.codahale.metrics.Counter nsLookups

nsLookupsFailed

protected final com.codahale.metrics.Counter nsLookupsFailed

rpcTimers

protected final ConcurrentMap<String,com.codahale.metrics.Timer> rpcTimers

rpcHistograms

protected final ConcurrentMap<String,com.codahale.metrics.Histogram> rpcHistograms

cacheDroppingExceptions

private final ConcurrentMap<String,com.codahale.metrics.Counter> cacheDroppingExceptions

rpcCounters

protected final ConcurrentMap<String,com.codahale.metrics.Counter> rpcCounters

Constructor Detail

MetricsConnection

MetricsConnection(String scope,
                  Supplier<ThreadPoolExecutor> batchPool,
                  Supplier<ThreadPoolExecutor> metaPool)

Method Detail

getScope

static String getScope(org.apache.hadoop.conf.Configuration conf,
                       String clusterId,
                       Object connectionObj)

Returns the scope for a MetricsConnection based on the configured METRICS_SCOPE_KEY or by generating a default from the passed clusterId and connectionObj's hashCode.

Parameters:

conf - configuration for the connection

clusterId - clusterId for the connection

connectionObj - either a Connection or AsyncConnectionImpl, the instance creating this MetricsConnection.

updateServerStats

public void updateServerStats(ServerName serverName,
                              byte[] regionName,
                              Object r)

updateRegionStats

public void updateRegionStats(ServerName serverName,
                              byte[] regionName,
                              RegionLoadStats stats)

Description copied from interface: StatisticTrackable

Update stats per region.

Specified by:

updateRegionStats in interface StatisticTrackable

getExecutorPoolName

final String getExecutorPoolName()

getMetaPoolName

final String getMetaPoolName()

getMetricRegistry

com.codahale.metrics.MetricRegistry getMetricRegistry()

shutdown

public void shutdown()

newCallStats

public static MetricsConnection.CallStats newCallStats()

Produce an instance of MetricsConnection.CallStats for clients to attach to RPCs.

incrMetaCacheHit

public void incrMetaCacheHit()

Increment the number of meta cache hits.

incrMetaCacheMiss

public void incrMetaCacheMiss()

Increment the number of meta cache misses.

incrMetaCacheNumClearServer

public void incrMetaCacheNumClearServer()

Increment the number of meta cache drops requested for entire RegionServer.

incrMetaCacheNumClearRegion

public void incrMetaCacheNumClearRegion()

Increment the number of meta cache drops requested for individual region.

incrMetaCacheNumClearRegion

public void incrMetaCacheNumClearRegion(int count)

Increment the number of meta cache drops requested for individual region.

incrHedgedReadOps

public void incrHedgedReadOps()

Increment the number of hedged read that have occurred.

incrHedgedReadWin

public void incrHedgedReadWin()

Increment the number of hedged read returned faster than the original read.

incrNormalRunners

public void incrNormalRunners()

Increment the number of normal runner counts.

incrDelayRunnersAndUpdateDelayInterval

public void incrDelayRunnersAndUpdateDelayInterval(long interval)

Increment the number of delay runner counts and update delay interval of delay runner.

getMetric

private <T> T getMetric(String key,
                        ConcurrentMap<String,T> map,
                        MetricsConnection.NewMetric<T> factory)

Get a metric for key from map, or create it with factory.

updateRpcGeneric

private void updateRpcGeneric(String methodName,
                              MetricsConnection.CallStats stats)

Update call stats for non-critical-path methods

updateRpc

public void updateRpc(org.apache.hbase.thirdparty.com.google.protobuf.Descriptors.MethodDescriptor method,
                      org.apache.hbase.thirdparty.com.google.protobuf.Message param,
                      MetricsConnection.CallStats stats)

Report RPC context to metrics system.

incrCacheDroppingExceptions

public void incrCacheDroppingExceptions(Object exception)

incrNsLookups

public void incrNsLookups()

incrNsLookupsFailed

public void incrNsLookupsFailed()