Package org.apache.hadoop.hbase.client
Class MetricsConnection
java.lang.Object
org.apache.hadoop.hbase.client.MetricsConnection
- All Implemented Interfaces:
StatisticTrackable
This class is for maintaining the various connection statistics and publishing them through the
metrics interfaces. This class manages its own
MetricRegistry
and JmxReporter
so
as to not conflict with other uses of Yammer Metrics within the client application. Calling
getMetricsConnection(Configuration, String, Supplier, Supplier)
implicitly creates and
"starts" instances of these classes; be sure to call deleteMetricsConnection(String)
to
terminate the thread pools they allocate. The metrics reporter will be shutdown
shutdown()
when all connections within this metrics instances are closed.-
Nested Class Summary
Modifier and TypeClassDescriptionstatic class
A container class for collecting details about the RPC call as it percolates.protected static final class
private static interface
A lambda for dispatching to the appropriate metric factory methodprotected static class
protected static class
-
Field Summary
Modifier and TypeFieldDescriptionprivate final MetricsConnection.CallTracker
private final List<Supplier<ThreadPoolExecutor>>
private static final String
private final ConcurrentMap<String,
com.codahale.metrics.Counter> private static final int
Anticipated number of metric entriesstatic final String
Set this key totrue
to enable metrics collection of client requests.static final String
Set this key totrue
to enable table metrics collection of client requests.private static final String
private static final String
private static final int
Anticipated number of concurrent accessor threadsprivate final com.codahale.metrics.Histogram
private final com.codahale.metrics.Counter
private final MetricsConnection.NewMetric<com.codahale.metrics.Counter>
private final MetricsConnection.CallTracker
private static final String
private static final String
private final MetricsConnection.CallTracker
private static final String
private final com.codahale.metrics.Counter
private final com.codahale.metrics.Counter
private final MetricsConnection.NewMetric<com.codahale.metrics.Histogram>
private final MetricsConnection.CallTracker
private static final float
Default load factor fromHashMap.DEFAULT_LOAD_FACTOR
private static final String
private static final String
private final com.codahale.metrics.Counter
private final com.codahale.metrics.Counter
private final com.codahale.metrics.Counter
private final com.codahale.metrics.Counter
private final List<Supplier<ThreadPoolExecutor>>
private static final ConcurrentMap<String,
MetricsConnection> static final String
Set to specify a custom scope for the metrics published throughMetricsConnection
.private final MetricsConnection.CallTracker
private static final String
private static final String
private final com.codahale.metrics.Counter
private final com.codahale.metrics.Counter
private final com.codahale.metrics.Histogram
private final com.codahale.metrics.Timer
private final MetricsConnection.CallTracker
private final com.codahale.metrics.MetricRegistry
private static final String
private final com.codahale.metrics.JmxReporter
private static final String
private static final String
private final ConcurrentMap<String,
com.codahale.metrics.Counter> private final ConcurrentMap<String,
com.codahale.metrics.Histogram> private final ConcurrentMap<String,
com.codahale.metrics.Timer> private final MetricsConnection.RunnerStats
private final MetricsConnection.CallTracker
private final String
private ConcurrentHashMap<ServerName,
ConcurrentMap<byte[], MetricsConnection.RegionStats>> private final boolean
private final MetricsConnection.NewMetric<com.codahale.metrics.Timer>
private static final String
private static final String
-
Constructor Summary
ModifierConstructorDescriptionprivate
MetricsConnection
(org.apache.hadoop.conf.Configuration conf, String scope, Supplier<ThreadPoolExecutor> batchPool, Supplier<ThreadPoolExecutor> metaPool) -
Method Summary
Modifier and TypeMethodDescriptionprivate void
addThreadPools
(Supplier<ThreadPoolExecutor> batchPool, Supplier<ThreadPoolExecutor> metaPool) Add thread pools of additional connections to the metricsprivate void
Decrement the connection count of the metrics within a scope(package private) static void
deleteMetricsConnection
(String scope) appendTracker metriclong
Return the connection count of the metrics within a scopedeleteTracker metric(package private) final String
getTracker metriccom.codahale.metrics.Counter
hedgedReadOps metriccom.codahale.metrics.Counter
hedgedReadWin metricincrementTracker metriclong
com.codahale.metrics.Counter
metaCacheNumClearRegion metriccom.codahale.metrics.Counter
metaCacheNumClearServer metric(package private) final String
private <T> T
getMetric
(String key, ConcurrentMap<String, T> map, MetricsConnection.NewMetric<T> factory) Get a metric forkey
frommap
, or create it withfactory
.(package private) com.codahale.metrics.MetricRegistry
(package private) static MetricsConnection
getMetricsConnection
(org.apache.hadoop.conf.Configuration conf, String scope, Supplier<ThreadPoolExecutor> batchPool, Supplier<ThreadPoolExecutor> metaPool) scope of the metrics objectmultiTracker metriccom.codahale.metrics.Histogram
numActionsPerServerHist metricputTracker metricConcurrentMap<String,
com.codahale.metrics.Counter> rpcCounters metricConcurrentMap<String,
com.codahale.metrics.Histogram> rpcHistograms metricConcurrentMap<String,
com.codahale.metrics.Timer> rpcTimers metricrunnerStats metricscanTracker metric(package private) static String
Returns the scope for a MetricsConnection based on the configuredMETRICS_SCOPE_KEY
or by generating a default from the passed clusterId and connectionObj's hashCode.serverStats metricvoid
incrCacheDroppingExceptions
(Object exception) private void
Increment the connection count of the metrics within a scopevoid
incrDelayRunnersAndUpdateDelayInterval
(long interval) Increment the number of delay runner counts and update delay interval of delay runner.void
incrementServerOverloadedBackoffTime
(long time, TimeUnit timeUnit) Update the overloaded backoff timevoid
Increment the number of hedged read that have occurred.void
Increment the number of hedged read returned faster than the original read.void
Increment the number of meta cache hits.void
Increment the number of meta cache misses.void
Increment the number of meta cache drops requested for individual region.void
incrMetaCacheNumClearRegion
(int count) Increment the number of meta cache drops requested for individual region.void
Increment the number of meta cache drops requested for entire RegionServer.void
Increment the number of normal runner counts.void
void
static MetricsConnection.CallStats
Produce an instance ofMetricsConnection.CallStats
for clients to attach to RPCs.private void
shutdown()
void
updateRegionStats
(ServerName serverName, byte[] regionName, RegionLoadStats stats) Update stats per region.void
updateRpc
(org.apache.hbase.thirdparty.com.google.protobuf.Descriptors.MethodDescriptor method, TableName tableName, org.apache.hbase.thirdparty.com.google.protobuf.Message param, MetricsConnection.CallStats stats, Throwable e) Report RPC context to metrics system.private void
updateRpcGeneric
(String methodName, MetricsConnection.CallStats stats) Update call stats for non-critical-path methodsvoid
updateServerStats
(ServerName serverName, byte[] regionName, Object r) private void
updateTableMetric
(String methodName, TableName tableName, MetricsConnection.CallStats stats, Throwable e) Report table rpc context to metrics system.
-
Field Details
-
METRICS_INSTANCES
-
CLIENT_SIDE_METRICS_ENABLED_KEY
Set this key totrue
to enable metrics collection of client requests.- See Also:
-
CLIENT_SIDE_TABLE_METRICS_ENABLED_KEY
Set this key totrue
to enable table metrics collection of client requests.- See Also:
-
METRICS_SCOPE_KEY
Set to specify a custom scope for the metrics published throughMetricsConnection
. The scope is added to JMX MBean objectName, and defaults to a combination of the Connection's clusterId and hashCode. For example, a default value for a connection to cluster "foo" might be "foo-7d9d0818", where "7d9d0818" is the hashCode of the underlying AsyncConnectionImpl. Users may set this key to give a more contextual name for this scope. For example, one might want to differentiate a read connection from a write connection by setting the scopes to "foo-read" and "foo-write" respectively. Scope is the only thing that lends any uniqueness to the metrics. Care should be taken to avoid using the same scope for multiple Connections, otherwise the metrics may aggregate in unforeseen ways.- See Also:
-
CNT_BASE
- See Also:
-
FAILURE_CNT_BASE
- See Also:
-
TOTAL_EXCEPTION_CNT
- See Also:
-
LOCAL_EXCEPTION_CNT_BASE
- See Also:
-
REMOTE_EXCEPTION_CNT_BASE
- See Also:
-
DRTN_BASE
- See Also:
-
REQ_BASE
- See Also:
-
RESP_BASE
- See Also:
-
MEMLOAD_BASE
- See Also:
-
HEAP_BASE
- See Also:
-
CACHE_BASE
- See Also:
-
UNKNOWN_EXCEPTION
- See Also:
-
NS_LOOKUPS
- See Also:
-
NS_LOOKUPS_FAILED
- See Also:
-
CLIENT_SVC
-
serverStats
private ConcurrentHashMap<ServerName,ConcurrentMap<byte[], serverStatsMetricsConnection.RegionStats>> -
CAPACITY
Anticipated number of metric entries- See Also:
-
LOAD_FACTOR
Default load factor fromHashMap.DEFAULT_LOAD_FACTOR
- See Also:
-
CONCURRENCY_LEVEL
Anticipated number of concurrent accessor threads- See Also:
-
registry
-
reporter
-
scope
-
tableMetricsEnabled
-
timerFactory
-
histogramFactory
-
counterFactory
-
batchPools
-
metaPools
-
connectionCount
-
metaCacheHits
-
metaCacheMisses
-
getTracker
-
scanTracker
-
appendTracker
-
deleteTracker
-
incrementTracker
-
putTracker
-
multiTracker
-
runnerStats
-
metaCacheNumClearServer
-
metaCacheNumClearRegion
-
hedgedReadOps
-
hedgedReadWin
-
concurrentCallsPerServerHist
-
numActionsPerServerHist
-
nsLookups
-
nsLookupsFailed
-
overloadedBackoffTimer
-
rpcTimers
-
rpcHistograms
-
cacheDroppingExceptions
-
rpcCounters
-
-
Constructor Details
-
MetricsConnection
private MetricsConnection(org.apache.hadoop.conf.Configuration conf, String scope, Supplier<ThreadPoolExecutor> batchPool, Supplier<ThreadPoolExecutor> metaPool)
-
-
Method Details
-
getMetricsConnection
static MetricsConnection getMetricsConnection(org.apache.hadoop.conf.Configuration conf, String scope, Supplier<ThreadPoolExecutor> batchPool, Supplier<ThreadPoolExecutor> metaPool) -
deleteMetricsConnection
-
getScope
static String getScope(org.apache.hadoop.conf.Configuration conf, String clusterId, Object connectionObj) Returns the scope for a MetricsConnection based on the configuredMETRICS_SCOPE_KEY
or by generating a default from the passed clusterId and connectionObj's hashCode.- Parameters:
conf
- configuration for the connectionclusterId
- clusterId for the connectionconnectionObj
- either a Connection or AsyncConnectionImpl, the instance creating this MetricsConnection.
-
updateServerStats
-
updateRegionStats
Description copied from interface:StatisticTrackable
Update stats per region.- Specified by:
updateRegionStats
in interfaceStatisticTrackable
-
getExecutorPoolName
-
getMetaPoolName
-
getMetricRegistry
com.codahale.metrics.MetricRegistry getMetricRegistry() -
getMetricScope
scope of the metrics object -
getServerStats
public ConcurrentHashMap<ServerName,ConcurrentMap<byte[], getServerStats()MetricsConnection.RegionStats>> serverStats metric -
getRunnerStats
runnerStats metric -
getMetaCacheNumClearServer
metaCacheNumClearServer metric -
getMetaCacheNumClearRegion
metaCacheNumClearRegion metric -
getHedgedReadOps
hedgedReadOps metric -
getHedgedReadWin
hedgedReadWin metric -
getNumActionsPerServerHist
numActionsPerServerHist metric -
getRpcCounters
rpcCounters metric -
getRpcTimers
rpcTimers metric -
getRpcHistograms
rpcHistograms metric -
getGetTracker
getTracker metric -
getScanTracker
scanTracker metric -
getMultiTracker
multiTracker metric -
getAppendTracker
appendTracker metric -
getDeleteTracker
deleteTracker metric -
getIncrementTracker
incrementTracker metric -
getPutTracker
putTracker metric -
newCallStats
Produce an instance ofMetricsConnection.CallStats
for clients to attach to RPCs. -
incrMetaCacheHit
Increment the number of meta cache hits. -
incrMetaCacheMiss
Increment the number of meta cache misses. -
getMetaCacheMisses
-
incrMetaCacheNumClearServer
Increment the number of meta cache drops requested for entire RegionServer. -
incrMetaCacheNumClearRegion
Increment the number of meta cache drops requested for individual region. -
incrMetaCacheNumClearRegion
Increment the number of meta cache drops requested for individual region. -
incrHedgedReadOps
Increment the number of hedged read that have occurred. -
incrHedgedReadWin
Increment the number of hedged read returned faster than the original read. -
incrNormalRunners
Increment the number of normal runner counts. -
incrDelayRunnersAndUpdateDelayInterval
Increment the number of delay runner counts and update delay interval of delay runner. -
incrementServerOverloadedBackoffTime
Update the overloaded backoff time -
getConnectionCount
Return the connection count of the metrics within a scope -
incrConnectionCount
Increment the connection count of the metrics within a scope -
decrConnectionCount
Decrement the connection count of the metrics within a scope -
addThreadPools
private void addThreadPools(Supplier<ThreadPoolExecutor> batchPool, Supplier<ThreadPoolExecutor> metaPool) Add thread pools of additional connections to the metrics -
getMetric
private <T> T getMetric(String key, ConcurrentMap<String, T> map, MetricsConnection.NewMetric<T> factory) Get a metric forkey
frommap
, or create it withfactory
. -
updateRpcGeneric
Update call stats for non-critical-path methods -
shutdown
-
updateRpc
public void updateRpc(org.apache.hbase.thirdparty.com.google.protobuf.Descriptors.MethodDescriptor method, TableName tableName, org.apache.hbase.thirdparty.com.google.protobuf.Message param, MetricsConnection.CallStats stats, Throwable e) Report RPC context to metrics system. -
updateTableMetric
private void updateTableMetric(String methodName, TableName tableName, MetricsConnection.CallStats stats, Throwable e) Report table rpc context to metrics system. -
incrCacheDroppingExceptions
-
incrNsLookups
-
incrNsLookupsFailed
-