org.apache.hadoop.hbase.replication.regionserver

Class MetricsSource

java.lang.Object

org.apache.hadoop.hbase.replication.regionserver.MetricsSource

All Implemented Interfaces:

BaseSource

@InterfaceAudience.LimitedPrivate(value="Replication")
public class MetricsSource
extends Object
implements BaseSource

This class is for maintaining the various replication statistics for a source and publishing them through the metrics interfaces.

Field Summary

Fields

Modifier and Type	Field and Description
private Map<String,Long>	ageOfLastShippedOp
private MetricsReplicationGlobalSourceSource	globalSourceSource
private String	id
private long	lastHFileRefsQueueSize
private Map<String,Long>	lastShippedTimeStamps
private static org.slf4j.Logger	LOG
private MetricsReplicationSourceSource	singleSourceSource
private Map<String,MetricsReplicationTableSource>	singleSourceSourceByTable
private long	timeStampNextToReplicate

Fields inherited from interface org.apache.hadoop.hbase.metrics.BaseSource

HBASE_METRICS_SYSTEM_NAME

Constructor Summary

Constructors

Constructor and Description
MetricsSource(String id) Constructor used to register the metrics
MetricsSource(String id, MetricsReplicationSourceSource singleSourceSource, MetricsReplicationGlobalSourceSource globalSourceSource, Map<String,MetricsReplicationTableSource> singleSourceSourceByTable) Constructor for injecting custom (or test) MetricsReplicationSourceSources

Method Summary

Modifier and Type	Method and Description
void	clear() Removes all metrics about this Source.
void	decGauge(String gaugeName, long delta) Subtract some amount from a gauge.
void	decrSizeOfHFileRefsQueue(int size)
void	decrSizeOfLogQueue()
void	decrSourceInitializing() Decrement the count for initializing sources
Long	getAgeOfLastShippedOp() Get AgeOfLastShippedOp n
long	getAgeOfLastShippedOp(String walGroup) get age of last shipped op of given wal group.
long	getEditsFiltered() Gets the number of edits not eligible for replication this source queue logs so far.
String	getMetricsContext() Get the metrics context.
String	getMetricsDescription() Get the description of what this source exposes.
String	getMetricsJmxContext() Get the name of the context in JMX that this source will be exposed through.
String	getMetricsName() Get the name of the metrics that are being exported by this source.
long	getOldestWalAge()
long	getOpsShipped() Gets the number of OPs shipped by this source queue to target cluster.
String	getPeerID() Get the slave peer ID n
long	getReplicableEdits() Gets the number of edits eligible for replication read from this source queue logs so far.
long	getReplicationDelay()
Map<String,MetricsReplicationTableSource>	getSingleSourceSourceByTable()
int	getSizeOfLogQueue() Get the sizeOfLogQueue n
int	getSourceInitializing() Get the source initializing counts
long	getTimeStampNextToReplicate() TimeStamp of next edit to be replicated.
long	getTimestampOfLastShippedOp() Get the timestampsOfLastShippedOp, if there are multiple groups, return the latest one n
long	getTimeStampOfLastShippedOp() Deprecated.
long	getUncleanlyClosedWALs() Get the value of uncleanlyClosedWAL counter n
long	getWALReaderEditsBufferUsage() Returns the amount of memory in bytes used in this RegionServer by edits pending replication.
void	incCounters(String counterName, long delta) Add some amount to a counter.
void	incGauge(String gaugeName, long delta) Add some amount to a gauge.
void	incrBytesSkippedInUncleanlyClosedWALs(long bytes)
void	incrCompletedRecoveryQueue()
void	incrCompletedWAL()
void	incrementFailedBatches() Convenience method to update metrics when batch of operations has failed.
void	incrFailedRecoveryQueue()
void	incrLogEditsFiltered() The number of log edits filtered out.
void	incrLogEditsFiltered(long delta) Add on the number of log edits filtered
void	incrLogEditsRead() Increment the number of log edits read by one.
private void	incrLogEditsRead(long delta) Add on the the number of log edits read
void	incrLogReadInBytes(long readInBytes) increase the byte number read by source from log file
void	incrRepeatedFileBytes(long bytes)
void	incrRestartedWALReading()
void	incrSizeOfHFileRefsQueue(long size)
void	incrSizeOfLogQueue() Increment size of the log queue.
void	incrSourceInitializing() Increment the count for initializing sources
void	incrUncleanlyClosedWALs()
void	incrUnknownFileLengthForClosedWAL()
void	init() Clear out the metrics and re-prepare the source.
void	refreshAgeOfLastShippedOp(String walGroupId) Convenience method to use the last given timestamp to refresh the age of the last edit.
void	removeMetric(String key) Remove a metric and no longer announce it.
void	setAgeOfLastShippedOp(long timestamp, String walGroup) Set the age of the last edit that was shipped
void	setAgeOfLastShippedOpByTable(long timestamp, String tableName) Set the age of the last edit that was shipped group by table
void	setGauge(String gaugeName, long value) Set a gauge to a specific value.
void	setOldestWalAge(long age)
void	setTimeStampNextToReplicate(long timeStampNextToReplicate) TimeStamp of next edit targeted for replication.
void	setWALReaderEditsBufferUsage(long usageInBytes) Sets the amount of memory in bytes used in this RegionServer by edits pending replication.
void	shipBatch(long batchSize, int sizeInBytes) Convience method to apply changes to metrics do to shipping a batch of logs.
void	shipBatch(long batchSize, int sizeInBytes, long hfiles) Convience method to apply changes to metrics do to shipping a batch of logs.
void	updateHistogram(String name, long value) Add some value to a histogram.
void	updateTableLevelMetrics(List<Pair<WAL.Entry,Long>> walEntries) Update the table level replication metrics per table

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.metrics.BaseSource

getMetricRegistryInfo

Field Detail

LOG

private static final org.slf4j.Logger LOG

lastShippedTimeStamps

private Map<String,Long> lastShippedTimeStamps

ageOfLastShippedOp

private Map<String,Long> ageOfLastShippedOp

lastHFileRefsQueueSize

private long lastHFileRefsQueueSize

id

private String id

timeStampNextToReplicate

private long timeStampNextToReplicate

singleSourceSource

private final MetricsReplicationSourceSource singleSourceSource

globalSourceSource

private final MetricsReplicationGlobalSourceSource globalSourceSource

singleSourceSourceByTable

private Map<String,MetricsReplicationTableSource> singleSourceSourceByTable

Constructor Detail

MetricsSource

public MetricsSource(String id)

Constructor used to register the metrics

Parameters:

id - Name of the source this class is monitoring

MetricsSource

public MetricsSource(String id,
                     MetricsReplicationSourceSource singleSourceSource,
                     MetricsReplicationGlobalSourceSource globalSourceSource,
                     Map<String,MetricsReplicationTableSource> singleSourceSourceByTable)

Constructor for injecting custom (or test) MetricsReplicationSourceSources

Parameters:

id - Name of the source this class is monitoring

singleSourceSource - Class to monitor id-scoped metrics

globalSourceSource - Class to monitor global-scoped metrics

Method Detail

setAgeOfLastShippedOp

public void setAgeOfLastShippedOp(long timestamp,
                                  String walGroup)

Set the age of the last edit that was shipped

Parameters:

timestamp - target write time of the edit

walGroup - which group we are setting

updateTableLevelMetrics

public void updateTableLevelMetrics(List<Pair<WAL.Entry,Long>> walEntries)

Update the table level replication metrics per table

Parameters:

walEntries - List of pairs of WAL entry and it's size

setAgeOfLastShippedOpByTable

public void setAgeOfLastShippedOpByTable(long timestamp,
                                         String tableName)

Set the age of the last edit that was shipped group by table

Parameters:

timestamp - write time of the edit

tableName - String as group and tableName

getAgeOfLastShippedOp

public long getAgeOfLastShippedOp(String walGroup)

get age of last shipped op of given wal group. If the walGroup is null, return 0

Parameters:

walGroup - which group we are getting n

refreshAgeOfLastShippedOp

public void refreshAgeOfLastShippedOp(String walGroupId)

Convenience method to use the last given timestamp to refresh the age of the last edit. Used when replication fails and need to keep that metric accurate.

Parameters:

walGroupId - id of the group to update

incrSizeOfLogQueue

public void incrSizeOfLogQueue()

Increment size of the log queue.

decrSizeOfLogQueue

public void decrSizeOfLogQueue()

incrSourceInitializing

public void incrSourceInitializing()

Increment the count for initializing sources

decrSourceInitializing

public void decrSourceInitializing()

Decrement the count for initializing sources

incrLogEditsRead

private void incrLogEditsRead(long delta)

Add on the the number of log edits read

Parameters:

delta - the number of log edits read.

incrLogEditsRead

public void incrLogEditsRead()

Increment the number of log edits read by one.

incrLogEditsFiltered

public void incrLogEditsFiltered(long delta)

Add on the number of log edits filtered

Parameters:

delta - the number filtered.

incrLogEditsFiltered

public void incrLogEditsFiltered()

The number of log edits filtered out.

shipBatch

public void shipBatch(long batchSize,
                      int sizeInBytes)

Convience method to apply changes to metrics do to shipping a batch of logs.

Parameters:

batchSize - the size of the batch that was shipped to sinks.

incrementFailedBatches

public void incrementFailedBatches()

Convenience method to update metrics when batch of operations has failed.

getEditsFiltered

public long getEditsFiltered()

Gets the number of edits not eligible for replication this source queue logs so far.

Returns:

logEditsFiltered non-replicable edits filtered from this queue logs.

getReplicableEdits

public long getReplicableEdits()

Gets the number of edits eligible for replication read from this source queue logs so far.

Returns:

replicableEdits total number of replicable edits read from this queue logs.

getOpsShipped

public long getOpsShipped()

Gets the number of OPs shipped by this source queue to target cluster.

Returns:

oPsShipped total number of OPs shipped by this source.

shipBatch

public void shipBatch(long batchSize,
                      int sizeInBytes,
                      long hfiles)

Convience method to apply changes to metrics do to shipping a batch of logs.

Parameters:

batchSize - the size of the batch that was shipped to sinks.

hfiles - total number of hfiles shipped to sinks.

incrLogReadInBytes

public void incrLogReadInBytes(long readInBytes)

increase the byte number read by source from log file

clear

public void clear()

Removes all metrics about this Source.

getAgeOfLastShippedOp

public Long getAgeOfLastShippedOp()

Get AgeOfLastShippedOp n

getSizeOfLogQueue

public int getSizeOfLogQueue()

Get the sizeOfLogQueue n

getTimeStampOfLastShippedOp

@Deprecated
public long getTimeStampOfLastShippedOp()

Deprecated.

Get the timeStampsOfLastShippedOp, if there are multiple groups, return the latest one n * @deprecated Since 2.0.0. Removed in 3.0.0.

See Also:

getTimestampOfLastShippedOp()

getUncleanlyClosedWALs

public long getUncleanlyClosedWALs()

Get the value of uncleanlyClosedWAL counter n

getTimestampOfLastShippedOp

public long getTimestampOfLastShippedOp()

Get the timestampsOfLastShippedOp, if there are multiple groups, return the latest one n

getTimeStampNextToReplicate

public long getTimeStampNextToReplicate()

TimeStamp of next edit to be replicated.

Returns:

timeStampNextToReplicate - TimeStamp of next edit to be replicated.

setTimeStampNextToReplicate

public void setTimeStampNextToReplicate(long timeStampNextToReplicate)

TimeStamp of next edit targeted for replication. Used for calculating lag, as if this timestamp is greater than timestamp of last shipped, it means there's at least one edit pending replication.

Parameters:

timeStampNextToReplicate - timestamp of next edit in the queue that should be replicated.

getReplicationDelay

public long getReplicationDelay()

getSourceInitializing

public int getSourceInitializing()

Get the source initializing counts

Returns:

number of replication sources getting initialized

getPeerID

public String getPeerID()

Get the slave peer ID n

incrSizeOfHFileRefsQueue

public void incrSizeOfHFileRefsQueue(long size)

decrSizeOfHFileRefsQueue

public void decrSizeOfHFileRefsQueue(int size)

incrUnknownFileLengthForClosedWAL

public void incrUnknownFileLengthForClosedWAL()

incrUncleanlyClosedWALs

public void incrUncleanlyClosedWALs()

incrBytesSkippedInUncleanlyClosedWALs

public void incrBytesSkippedInUncleanlyClosedWALs(long bytes)

incrRestartedWALReading

public void incrRestartedWALReading()

incrRepeatedFileBytes

public void incrRepeatedFileBytes(long bytes)

incrCompletedWAL

public void incrCompletedWAL()

incrCompletedRecoveryQueue

public void incrCompletedRecoveryQueue()

incrFailedRecoveryQueue

public void incrFailedRecoveryQueue()

setOldestWalAge

public void setOldestWalAge(long age)

getOldestWalAge

public long getOldestWalAge()

init

public void init()

Description copied from interface: BaseSource

Clear out the metrics and re-prepare the source.

Specified by:

init in interface BaseSource

setGauge

public void setGauge(String gaugeName,
                     long value)

Description copied from interface: BaseSource

Set a gauge to a specific value.

Specified by:

setGauge in interface BaseSource

Parameters:

gaugeName - the name of the gauge

value - the value

incGauge

public void incGauge(String gaugeName,
                     long delta)

Description copied from interface: BaseSource

Add some amount to a gauge.

Specified by:

incGauge in interface BaseSource

Parameters:

gaugeName - the name of the gauge

delta - the amount to change the gauge by.

decGauge

public void decGauge(String gaugeName,
                     long delta)

Description copied from interface: BaseSource

Subtract some amount from a gauge.

Specified by:

decGauge in interface BaseSource

Parameters:

gaugeName - the name of the gauge

delta - the amount to change the gauge by.

removeMetric

public void removeMetric(String key)

Description copied from interface: BaseSource

Remove a metric and no longer announce it.

Specified by:

removeMetric in interface BaseSource

Parameters:

key - Name of the gauge to remove.

incCounters

public void incCounters(String counterName,
                        long delta)

Description copied from interface: BaseSource

Add some amount to a counter.

Specified by:

incCounters in interface BaseSource

Parameters:

counterName - the name of the counter

delta - the amount to change the counter by.

updateHistogram

public void updateHistogram(String name,
                            long value)

Description copied from interface: BaseSource

Add some value to a histogram.

Specified by:

updateHistogram in interface BaseSource

Parameters:

name - the name of the histogram

value - the value to add to the histogram

getMetricsContext

public String getMetricsContext()

Description copied from interface: BaseSource

Get the metrics context. For hadoop metrics2 system this is usually an all lowercased string. eg. regionserver, master, thriftserver

Specified by:

getMetricsContext in interface BaseSource

Returns:

The string context used to register this source to hadoop's metrics2 system.

getMetricsDescription

public String getMetricsDescription()

Description copied from interface: BaseSource

Get the description of what this source exposes.

Specified by:

getMetricsDescription in interface BaseSource

getMetricsJmxContext

public String getMetricsJmxContext()

Description copied from interface: BaseSource

Get the name of the context in JMX that this source will be exposed through. This is in ObjectName format. With the default context being Hadoop -> HBase

Specified by:

getMetricsJmxContext in interface BaseSource

getMetricsName

public String getMetricsName()

Description copied from interface: BaseSource

Get the name of the metrics that are being exported by this source. Eg. IPC, GC, WAL

Specified by:

getMetricsName in interface BaseSource

getSingleSourceSourceByTable

@InterfaceAudience.Private
public Map<String,MetricsReplicationTableSource> getSingleSourceSourceByTable()

setWALReaderEditsBufferUsage

public void setWALReaderEditsBufferUsage(long usageInBytes)

Sets the amount of memory in bytes used in this RegionServer by edits pending replication.

getWALReaderEditsBufferUsage

public long getWALReaderEditsBufferUsage()

Returns the amount of memory in bytes used in this RegionServer by edits pending replication. n