Class MetricsSource
java.lang.Object
org.apache.hadoop.hbase.replication.regionserver.MetricsSource
- All Implemented Interfaces:
BaseSource
This class is for maintaining the various replication statistics for a source and publishing them
through the metrics interfaces.
-
Field Summary
Modifier and TypeFieldDescriptionprivate final MetricsReplicationGlobalSourceSource
private String
private long
private static final org.slf4j.Logger
private final MetricsReplicationSourceSource
private Map<String,
MetricsReplicationTableSource> private long
Fields inherited from interface org.apache.hadoop.hbase.metrics.BaseSource
HBASE_METRICS_SYSTEM_NAME
-
Constructor Summary
ConstructorDescriptionMetricsSource
(String id) Constructor used to register the metricsMetricsSource
(String id, MetricsReplicationSourceSource singleSourceSource, MetricsReplicationGlobalSourceSource globalSourceSource, Map<String, MetricsReplicationTableSource> singleSourceSourceByTable) Constructor for injecting custom (or test) MetricsReplicationSourceSources -
Method Summary
Modifier and TypeMethodDescriptionvoid
clear()
Removes all metrics about this Source.void
Subtract some amount from a gauge.void
decrSizeOfHFileRefsQueue
(int size) void
void
Decrement the count for initializing sourcesGet AgeOfLastShippedOplong
getAgeOfLastShippedOp
(String walGroup) get age of last shipped op of given wal group.long
Gets the number of edits not eligible for replication this source queue logs so far.Get the metrics context.Get the description of what this source exposes.Get the name of the context in JMX that this source will be exposed through.Get the name of the metrics that are being exported by this source.long
long
Gets the number of OPs shipped by this source queue to target cluster.Get the slave peer IDlong
Gets the number of edits eligible for replication read from this source queue logs so far.long
int
Get the sizeOfLogQueueint
Get the source initializing countslong
TimeStamp of next edit to be replicated.long
Get the timestampsOfLastShippedOp, if there are multiple groups, return the latest onelong
Get the value of uncleanlyClosedWAL counterlong
Returns the amount of memory in bytes used in this RegionServer by edits pending replication.void
incCounters
(String counterName, long delta) Add some amount to a counter.void
Add some amount to a gauge.void
incrBytesSkippedInUncleanlyClosedWALs
(long bytes) void
void
void
Convenience method to update metrics when batch of operations has failed.void
void
The number of log edits filtered out.void
incrLogEditsFiltered
(long delta) Add on the number of log edits filteredvoid
Increment the number of log edits read by one.private void
incrLogEditsRead
(long delta) Add on the the number of log edits readvoid
incrLogReadInBytes
(long readInBytes) increase the byte number read by source from log filevoid
incrRepeatedFileBytes
(long bytes) void
void
incrSizeOfHFileRefsQueue
(long size) void
Increment size of the log queue.void
Increment the count for initializing sourcesvoid
void
void
init()
Clear out the metrics and re-prepare the source.void
refreshAgeOfLastShippedOp
(String walGroupId) Convenience method to use the last given timestamp to refresh the age of the last edit.void
removeMetric
(String key) Remove a metric and no longer announce it.void
setAgeOfLastShippedOp
(long timestamp, String walGroup) Set the age of the last edit that was shippedvoid
setAgeOfLastShippedOpByTable
(long timestamp, String tableName) Set the age of the last edit that was shipped group by tablevoid
Set a gauge to a specific value.void
setOldestWalAge
(long age) void
setTimeStampNextToReplicate
(long timeStampNextToReplicate) TimeStamp of next edit targeted for replication.void
setWALReaderEditsBufferUsage
(long usageInBytes) Sets the amount of memory in bytes used in this RegionServer by edits pending replication.void
shipBatch
(long batchSize, int sizeInBytes) Convience method to apply changes to metrics do to shipping a batch of logs.void
shipBatch
(long batchSize, int sizeInBytes, long hfiles) Convience method to apply changes to metrics do to shipping a batch of logs.void
void
updateHistogram
(String name, long value) Add some value to a histogram.void
updateTableLevelMetrics
(List<Pair<WAL.Entry, Long>> walEntries) Update the table level replication metrics per tableMethods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.hadoop.hbase.metrics.BaseSource
getMetricRegistryInfo
-
Field Details
-
LOG
-
lastShippedTimeStamps
-
ageOfLastShippedOp
-
lastHFileRefsQueueSize
-
id
-
timeStampNextToReplicate
-
singleSourceSource
-
globalSourceSource
-
singleSourceSourceByTable
-
-
Constructor Details
-
MetricsSource
Constructor used to register the metrics- Parameters:
id
- Name of the source this class is monitoring
-
MetricsSource
public MetricsSource(String id, MetricsReplicationSourceSource singleSourceSource, MetricsReplicationGlobalSourceSource globalSourceSource, Map<String, MetricsReplicationTableSource> singleSourceSourceByTable) Constructor for injecting custom (or test) MetricsReplicationSourceSources- Parameters:
id
- Name of the source this class is monitoringsingleSourceSource
- Class to monitor id-scoped metricsglobalSourceSource
- Class to monitor global-scoped metrics
-
-
Method Details
-
setAgeOfLastShippedOp
Set the age of the last edit that was shipped- Parameters:
timestamp
- target write time of the editwalGroup
- which group we are setting
-
updateTableLevelMetrics
Update the table level replication metrics per table- Parameters:
walEntries
- List of pairs of WAL entry and it's size
-
setAgeOfLastShippedOpByTable
Set the age of the last edit that was shipped group by table- Parameters:
timestamp
- write time of the edittableName
- String as group and tableName
-
getAgeOfLastShippedOp
get age of last shipped op of given wal group. If the walGroup is null, return 0- Parameters:
walGroup
- which group we are getting
-
refreshAgeOfLastShippedOp
Convenience method to use the last given timestamp to refresh the age of the last edit. Used when replication fails and need to keep that metric accurate.- Parameters:
walGroupId
- id of the group to update
-
incrSizeOfLogQueue
Increment size of the log queue. -
decrSizeOfLogQueue
-
incrSourceInitializing
Increment the count for initializing sources -
decrSourceInitializing
Decrement the count for initializing sources -
incrLogEditsRead
Add on the the number of log edits read- Parameters:
delta
- the number of log edits read.
-
incrLogEditsRead
Increment the number of log edits read by one. -
incrLogEditsFiltered
Add on the number of log edits filtered- Parameters:
delta
- the number filtered.
-
incrLogEditsFiltered
The number of log edits filtered out. -
shipBatch
Convience method to apply changes to metrics do to shipping a batch of logs.- Parameters:
batchSize
- the size of the batch that was shipped to sinks.
-
incrementFailedBatches
Convenience method to update metrics when batch of operations has failed. -
getEditsFiltered
Gets the number of edits not eligible for replication this source queue logs so far.- Returns:
- logEditsFiltered non-replicable edits filtered from this queue logs.
-
getReplicableEdits
Gets the number of edits eligible for replication read from this source queue logs so far.- Returns:
- replicableEdits total number of replicable edits read from this queue logs.
-
getOpsShipped
Gets the number of OPs shipped by this source queue to target cluster.- Returns:
- oPsShipped total number of OPs shipped by this source.
-
shipBatch
Convience method to apply changes to metrics do to shipping a batch of logs.- Parameters:
batchSize
- the size of the batch that was shipped to sinks.hfiles
- total number of hfiles shipped to sinks.
-
incrLogReadInBytes
increase the byte number read by source from log file -
clear
Removes all metrics about this Source. -
terminate
-
getAgeOfLastShippedOp
Get AgeOfLastShippedOp -
getSizeOfLogQueue
Get the sizeOfLogQueue -
getUncleanlyClosedWALs
Get the value of uncleanlyClosedWAL counter -
getTimestampOfLastShippedOp
Get the timestampsOfLastShippedOp, if there are multiple groups, return the latest one -
getTimeStampNextToReplicate
TimeStamp of next edit to be replicated.- Returns:
- timeStampNextToReplicate - TimeStamp of next edit to be replicated.
-
setTimeStampNextToReplicate
TimeStamp of next edit targeted for replication. Used for calculating lag, as if this timestamp is greater than timestamp of last shipped, it means there's at least one edit pending replication.- Parameters:
timeStampNextToReplicate
- timestamp of next edit in the queue that should be replicated.
-
getReplicationDelay
-
getSourceInitializing
Get the source initializing counts- Returns:
- number of replication sources getting initialized
-
getPeerID
Get the slave peer ID -
incrSizeOfHFileRefsQueue
-
decrSizeOfHFileRefsQueue
-
incrUnknownFileLengthForClosedWAL
-
incrUncleanlyClosedWALs
-
incrBytesSkippedInUncleanlyClosedWALs
-
incrRestartedWALReading
-
incrRepeatedFileBytes
-
incrCompletedWAL
-
incrCompletedRecoveryQueue
-
incrFailedRecoveryQueue
-
setOldestWalAge
-
getOldestWalAge
-
init
Description copied from interface:BaseSource
Clear out the metrics and re-prepare the source.- Specified by:
init
in interfaceBaseSource
-
setGauge
Description copied from interface:BaseSource
Set a gauge to a specific value.- Specified by:
setGauge
in interfaceBaseSource
- Parameters:
gaugeName
- the name of the gaugevalue
- the value
-
incGauge
Description copied from interface:BaseSource
Add some amount to a gauge.- Specified by:
incGauge
in interfaceBaseSource
- Parameters:
gaugeName
- the name of the gaugedelta
- the amount to change the gauge by.
-
decGauge
Description copied from interface:BaseSource
Subtract some amount from a gauge.- Specified by:
decGauge
in interfaceBaseSource
- Parameters:
gaugeName
- the name of the gaugedelta
- the amount to change the gauge by.
-
removeMetric
Description copied from interface:BaseSource
Remove a metric and no longer announce it.- Specified by:
removeMetric
in interfaceBaseSource
- Parameters:
key
- Name of the gauge to remove.
-
incCounters
Description copied from interface:BaseSource
Add some amount to a counter.- Specified by:
incCounters
in interfaceBaseSource
- Parameters:
counterName
- the name of the counterdelta
- the amount to change the counter by.
-
updateHistogram
Description copied from interface:BaseSource
Add some value to a histogram.- Specified by:
updateHistogram
in interfaceBaseSource
- Parameters:
name
- the name of the histogramvalue
- the value to add to the histogram
-
getMetricsContext
Description copied from interface:BaseSource
Get the metrics context. For hadoop metrics2 system this is usually an all lowercased string. eg. regionserver, master, thriftserver- Specified by:
getMetricsContext
in interfaceBaseSource
- Returns:
- The string context used to register this source to hadoop's metrics2 system.
-
getMetricsDescription
Description copied from interface:BaseSource
Get the description of what this source exposes.- Specified by:
getMetricsDescription
in interfaceBaseSource
-
getMetricsJmxContext
Description copied from interface:BaseSource
Get the name of the context in JMX that this source will be exposed through. This is in ObjectName format. With the default context being Hadoop -> HBase- Specified by:
getMetricsJmxContext
in interfaceBaseSource
-
getMetricsName
Description copied from interface:BaseSource
Get the name of the metrics that are being exported by this source. Eg. IPC, GC, WAL- Specified by:
getMetricsName
in interfaceBaseSource
-
getSingleSourceSourceByTable
-
setWALReaderEditsBufferUsage
Sets the amount of memory in bytes used in this RegionServer by edits pending replication. -
getWALReaderEditsBufferUsage
Returns the amount of memory in bytes used in this RegionServer by edits pending replication.
-