ReplicationSource (Apache HBase 1.2.12 API)

java.lang.Object
- java.lang.Thread
- - org.apache.hadoop.hbase.replication.regionserver.ReplicationSource

All Implemented Interfaces:

Runnable, ReplicationSourceInterface
```
@InterfaceAudience.Private
public class ReplicationSource
extends Thread
implements ReplicationSourceInterface
```
Class that handles the source of a replication stream. Currently does not handle more than 1 slave For each slave cluster it selects a random number of peers using a replication ratio. For example, if replication ration = 0.1 and slave cluster has 100 region servers, 10 will be selected.
A stream is considered down when we cannot contact a region server on the peer cluster for more than 55 seconds by default.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`ReplicationSource.LogsComparator` Comparator used to compare logs together based on their start time
`class`	`ReplicationSource.ReplicationSourceWorkerThread`
`static class`	`ReplicationSource.WorkerState`

Nested classes/interfaces inherited from class java.lang.Thread
Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields
Modifier and Type	Field and Description
`private UUID`	`clusterId`
`private org.apache.hadoop.conf.Configuration`	`conf`
`private org.apache.hadoop.fs.FileSystem`	`fs`
`private static org.apache.commons.logging.Log`	`LOG`
`private AtomicInteger`	`logQueueSize`
`private int`	`logQueueWarnThreshold`
`private ReplicationSourceManager`	`manager`
`private int`	`maxRetriesMultiplier`
`private MetricsSource`	`metrics`
`private UUID`	`peerClusterId`
`private String`	`peerClusterZnode`
`private String`	`peerId`
`private Map<String,PriorityBlockingQueue<org.apache.hadoop.fs.Path>>`	`queues`
`private int`	`queueSizePerGroup`
`private ReplicationEndpoint`	`replicationEndpoint`
`private ReplicationPeers`	`replicationPeers`
`private ReplicationQueueInfo`	`replicationQueueInfo`
`private int`	`replicationQueueNbCapacity`
`private ReplicationQueues`	`replicationQueues`
`private long`	`replicationQueueSizeCapacity`
`private long`	`sleepForRetries`
`private boolean`	`sourceRunning`
`private Stoppable`	`stopper`
`private ReplicationThrottler`	`throttler`
`private AtomicLong`	`totalReplicatedEdits`
`private AtomicLong`	`totalReplicatedOperations`
`private WALEntryFilter`	`walEntryFilter`
`private ConcurrentHashMap<String,ReplicationSource.ReplicationSourceWorkerThread>`	`workerThreads`

Fields inherited from class java.lang.Thread
MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors
Constructor and Description

ReplicationSource()

Constructors
Constructor and Description
`ReplicationSource()`

Method Summary

Methods
Modifier and Type	Method and Description
`private void`	`decorateConf()`
`void`	`enqueueLog(org.apache.hadoop.fs.Path log)` Add a log to the list of logs to replicate
`org.apache.hadoop.fs.Path`	`getCurrentPath()` Get the current log that's replicated
`String`	`getPeerClusterId()` Get the id that the source is replicating to.
`String`	`getPeerClusterZnode()` Get the id that the source is replicating to
`MetricsSource`	`getSourceMetrics()` Get Replication Source Metrics
`String`	`getStats()` Get a string representation of the current statistics for this source
`void`	`init(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, ReplicationSourceManager manager, ReplicationQueues replicationQueues, ReplicationPeers replicationPeers, Stoppable stopper, String peerClusterZnode, UUID clusterId, ReplicationEndpoint replicationEndpoint, MetricsSource metrics)` Instantiation method used by region servers
`protected boolean`	`isPeerEnabled()` check whether the peer is enabled or not
`private boolean`	`isSourceActive()`
`void`	`run()`
`protected boolean`	`sleepForRetries(String msg, int sleepMultiplier)` Do the sleeping logic
`void`	`startup()` Start the replication
`void`	`terminate(String reason)` End the replication
`void`	`terminate(String reason, Exception cause)` End the replication
`void`	`terminate(String reason, Exception cause, boolean join)`
`private void`	`uninitialize()`

Methods inherited from class java.lang.Thread
activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.apache.commons.logging.Log LOG
```
  - queues
```
private Map<String,PriorityBlockingQueue<org.apache.hadoop.fs.Path>> queues
```
  - queueSizePerGroup
```
private int queueSizePerGroup
```
  - replicationQueues
```
private ReplicationQueues replicationQueues
```
  - replicationPeers
```
private ReplicationPeers replicationPeers
```
  - conf
```
private org.apache.hadoop.conf.Configuration conf
```
  - replicationQueueInfo
```
private ReplicationQueueInfo replicationQueueInfo
```
  - peerId
```
private String peerId
```
  - manager
```
private ReplicationSourceManager manager
```
  - stopper
```
private Stoppable stopper
```
  - sleepForRetries
```
private long sleepForRetries
```
  - replicationQueueSizeCapacity
```
private long replicationQueueSizeCapacity
```
  - replicationQueueNbCapacity
```
private int replicationQueueNbCapacity
```
  - fs
```
private org.apache.hadoop.fs.FileSystem fs
```
  - clusterId
```
private UUID clusterId
```
  - peerClusterId
```
private UUID peerClusterId
```
  - totalReplicatedEdits
```
private AtomicLong totalReplicatedEdits
```
  - totalReplicatedOperations
```
private AtomicLong totalReplicatedOperations
```
  - peerClusterZnode
```
private String peerClusterZnode
```
  - maxRetriesMultiplier
```
private int maxRetriesMultiplier
```
  - sourceRunning
```
private volatile boolean sourceRunning
```
  - metrics
```
private MetricsSource metrics
```
  - logQueueWarnThreshold
```
private int logQueueWarnThreshold
```
  - replicationEndpoint
```
private ReplicationEndpoint replicationEndpoint
```
  - walEntryFilter
```
private WALEntryFilter walEntryFilter
```
  - throttler
```
private ReplicationThrottler throttler
```
  - logQueueSize
```
private AtomicInteger logQueueSize
```
  - workerThreads
```
private ConcurrentHashMap<String,ReplicationSource.ReplicationSourceWorkerThread> workerThreads
```
- Constructor Detail
  - ReplicationSource
```
public ReplicationSource()
```
- Method Detail
  - init
```
public void init(org.apache.hadoop.conf.Configuration conf,
        org.apache.hadoop.fs.FileSystem fs,
        ReplicationSourceManager manager,
        ReplicationQueues replicationQueues,
        ReplicationPeers replicationPeers,
        Stoppable stopper,
        String peerClusterZnode,
        UUID clusterId,
        ReplicationEndpoint replicationEndpoint,
        MetricsSource metrics)
          throws IOException
```
    Instantiation method used by region servers
    
    Specified by:
    
    init in interface ReplicationSourceInterface
    
    Parameters:
    conf - configuration to use
    fs - file system to use
    manager - replication manager to ping to
    stopper - the atomic boolean to use to stop the regionserver
    peerClusterZnode - the name of our znode
    clusterId - unique UUID for the cluster
    replicationEndpoint - the replication endpoint implementation
    metrics - metrics for replication source
    
    Throws:
    
    IOException
  - decorateConf
```
private void decorateConf()
```
  - enqueueLog
```
public void enqueueLog(org.apache.hadoop.fs.Path log)
```
    Description copied from interface: ReplicationSourceInterface
    
    Add a log to the list of logs to replicate
    
    Specified by:
    
    enqueueLog in interface ReplicationSourceInterface
    
    Parameters:
    log - path to the log to replicate
  - uninitialize
```
private void uninitialize()
```
  - run
```
public void run()
```
    Specified by:
    
    run in interface Runnable
    
    Overrides:
    
    run in class Thread
  - sleepForRetries
```
protected boolean sleepForRetries(String msg,
                      int sleepMultiplier)
```
    Do the sleeping logic
    
    Parameters:
    msg - Why we sleep
    sleepMultiplier - by how many times the default sleeping time is augmented
    
    Returns:
    True if sleepMultiplier is < maxRetriesMultiplier
  - isPeerEnabled
```
protected boolean isPeerEnabled()
```
    check whether the peer is enabled or not
    
    Returns:
    true if the peer is enabled, otherwise false
  - startup
```
public void startup()
```
    Description copied from interface: ReplicationSourceInterface
    
    Start the replication
    
    Specified by:
    
    startup in interface ReplicationSourceInterface
  - terminate
```
public void terminate(String reason)
```
    Description copied from interface: ReplicationSourceInterface
    
    End the replication
    
    Specified by:
    
    terminate in interface ReplicationSourceInterface
    
    Parameters:
    reason - why it's terminating
  - terminate
```
public void terminate(String reason,
             Exception cause)
```
    Description copied from interface: ReplicationSourceInterface
    
    End the replication
    
    Specified by:
    
    terminate in interface ReplicationSourceInterface
    
    Parameters:
    reason - why it's terminating
    cause - the error that's causing it
  - terminate
```
public void terminate(String reason,
             Exception cause,
             boolean join)
```
  - getPeerClusterZnode
```
public String getPeerClusterZnode()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get the id that the source is replicating to
    
    Specified by:
    
    getPeerClusterZnode in interface ReplicationSourceInterface
    
    Returns:
    peer cluster id
  - getPeerClusterId
```
public String getPeerClusterId()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get the id that the source is replicating to.
    
    Specified by:
    
    getPeerClusterId in interface ReplicationSourceInterface
    
    Returns:
    peer cluster id
  - getCurrentPath
```
public org.apache.hadoop.fs.Path getCurrentPath()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get the current log that's replicated
    
    Specified by:
    
    getCurrentPath in interface ReplicationSourceInterface
    
    Returns:
    the current log
  - isSourceActive
```
private boolean isSourceActive()
```
  - getStats
```
public String getStats()
```
    Description copied from interface: ReplicationSourceInterface
    
    Get a string representation of the current statistics for this source
    
    Specified by:
    
    getStats in interface ReplicationSourceInterface
    
    Returns:
    printable stats
  - getSourceMetrics
```
public MetricsSource getSourceMetrics()
```
    Get Replication Source Metrics
    
    Returns:
    sourceMetrics

Class ReplicationSource

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

Field Summary

Fields inherited from class java.lang.Thread

Constructor Summary

Method Summary

Methods inherited from class java.lang.Thread

Methods inherited from class java.lang.Object

Field Detail

LOG

queues

queueSizePerGroup

replicationQueues

replicationPeers

conf

replicationQueueInfo

peerId

manager

stopper

sleepForRetries

replicationQueueSizeCapacity

replicationQueueNbCapacity

fs

clusterId

peerClusterId

totalReplicatedEdits

totalReplicatedOperations

peerClusterZnode

maxRetriesMultiplier

sourceRunning

metrics

logQueueWarnThreshold

replicationEndpoint

walEntryFilter

throttler

logQueueSize

workerThreads

Constructor Detail

ReplicationSource

Method Detail

init

decorateConf

enqueueLog

uninitialize

run

sleepForRetries

isPeerEnabled

startup

terminate

terminate

terminate

getPeerClusterZnode

getPeerClusterId

getCurrentPath

isSourceActive

getStats

getSourceMetrics