@InterfaceAudience.Private public class ReplicationSink extends Object
This class is responsible for replicating the edits coming from another cluster.
This replication process is currently waiting for the edits to be applied before the method can return. This means that the replication of edits is synchronized (after reading from WALs in ReplicationSource) and that a single region server cannot receive edits from two sources at the same time
This class uses the native HBase client in order to replicate entries.
TODO make this class more like ReplicationSource wrt log handlingModifier and Type | Field and Description |
---|---|
private org.apache.hadoop.conf.Configuration |
conf |
private static org.apache.commons.logging.Log |
LOG |
private MetricsSink |
metrics |
private Connection |
sharedHtableCon |
private Object |
sharedHtableConLock |
private AtomicLong |
totalReplicatedEdits |
Constructor and Description |
---|
ReplicationSink(org.apache.hadoop.conf.Configuration conf,
Stoppable stopper)
Create a sink for replication
|
Modifier and Type | Method and Description |
---|---|
private <K1,K2,V> List<V> |
addToHashMultiMap(Map<K1,Map<K2,List<V>>> map,
K1 key1,
K2 key2,
V value)
Simple helper to a map from key to (a list of) values
TODO: Make a general utility method
|
protected void |
batch(TableName tableName,
Collection<List<Row>> allRows)
Do the changes and handle the pool
|
private void |
decorateConf()
decorate the Configuration object to make replication more receptive to delays:
lessen the timeout and numTries.
|
MetricsSink |
getSinkMetrics()
Get replication Sink Metrics
|
String |
getStats()
Get a string representation of this sink's metrics
|
private boolean |
isNewRowOrType(Cell previousCell,
Cell cell) |
void |
replicateEntries(List<org.apache.hadoop.hbase.protobuf.generated.AdminProtos.WALEntry> entries,
CellScanner cells)
Replicate this array of entries directly into the local cluster using the native client.
|
void |
stopReplicationSinkServices()
stop the thread pool executor.
|
private UUID |
toUUID(org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.UUID uuid) |
private static final org.apache.commons.logging.Log LOG
private final org.apache.hadoop.conf.Configuration conf
private volatile Connection sharedHtableCon
private final MetricsSink metrics
private final AtomicLong totalReplicatedEdits
private final Object sharedHtableConLock
public ReplicationSink(org.apache.hadoop.conf.Configuration conf, Stoppable stopper) throws IOException
conf
- conf objectstopper
- boolean to tell this thread to stopIOException
- thrown when HDFS goes bad or bad file nameprivate void decorateConf()
public void replicateEntries(List<org.apache.hadoop.hbase.protobuf.generated.AdminProtos.WALEntry> entries, CellScanner cells) throws IOException
entries
- cells
- IOException
private boolean isNewRowOrType(Cell previousCell, Cell cell)
previousCell
- cell
- private UUID toUUID(org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.UUID uuid)
private <K1,K2,V> List<V> addToHashMultiMap(Map<K1,Map<K2,List<V>>> map, K1 key1, K2 key2, V value)
map
- key1
- key2
- value
- public void stopReplicationSinkServices()
protected void batch(TableName tableName, Collection<List<Row>> allRows) throws IOException
tableName
- table to insert intoallRows
- list of actionsIOException
public String getStats()
public MetricsSink getSinkMetrics()
Copyright © 2007–2019 The Apache Software Foundation. All rights reserved.