ReplicationQueuesZKImpl (Apache HBase 1.4.11 API)

java.lang.Object
- org.apache.hadoop.hbase.replication.ReplicationStateZKBase
- - org.apache.hadoop.hbase.replication.ReplicationQueuesZKImpl

All Implemented Interfaces:

ReplicationQueues
```
@InterfaceAudience.Private
public class ReplicationQueuesZKImpl
extends ReplicationStateZKBase
implements ReplicationQueues
```
This class provides an implementation of the ReplicationQueues interface using Zookeeper. The base znode that this class works at is the myQueuesZnode. The myQueuesZnode contains a list of all outstanding WAL files on this region server that need to be replicated. The myQueuesZnode is the regionserver name (a concatenation of the region server’s hostname, client port and start code). For example: /hbase/replication/rs/hostname.example.org,6020,1234 Within this znode, the region server maintains a set of WAL replication queues. These queues are represented by child znodes named using there give queue id. For example: /hbase/replication/rs/hostname.example.org,6020,1234/1 /hbase/replication/rs/hostname.example.org,6020,1234/2 Each queue has one child znode for every WAL that still needs to be replicated. The value of these WAL child znodes is the latest position that has been replicated. This position is updated every time a WAL entry is replicated. For example: /hbase/replication/rs/hostname.example.org,6020,1234/1/23522342.23422 [VALUE: 254]

Field Summary

Fields
Modifier and Type Field and Description

static String RS_LOCK_ZNODE
Name of znode we use to lock during failover
- Fields inherited from class org.apache.hadoop.hbase.replication.ReplicationStateZKBase
  abortable, conf, DISABLED_ZNODE_BYTES, ENABLED_ZNODE_BYTES, hfileRefsZNode, ourClusterKey, peerStateNodeName, peersZNode, queuesZNode, replicationZNode, tableCFsNodeName, zookeeper, ZOOKEEPER_ZNODE_REPLICATION_HFILE_REFS_DEFAULT, ZOOKEEPER_ZNODE_REPLICATION_HFILE_REFS_KEY

Fields
Modifier and Type	Field and Description
`static String`	`RS_LOCK_ZNODE` Name of znode we use to lock during failover

Constructor Summary

Constructors
Constructor and Description
`ReplicationQueuesZKImpl(ZooKeeperWatcher zk, org.apache.hadoop.conf.Configuration conf, Abortable abortable)`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`addHFileRefs(String peerId, List<Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>> pairs)` Add new hfile references to the queue.
`void`	`addLog(String queueId, String filename)` Add a new WAL file to the given queue.
`void`	`addPeerToHFileRefs(String peerId)` Add a peer to hfile reference queue if peer does not exist.
`boolean`	`checkLockExists(String znode)`
`Pair<String,SortedSet<String>>`	`claimQueue(String regionserver, String queueId)` Take ownership for the queue identified by queueId and belongs to a dead region server.
`List<String>`	`getAllQueues()` Get a list of all queues for this region server.
`List<String>`	`getListOfReplicators()` Get a list of all region servers that have outstanding replication queues.
`String`	`getLockZNode(String znode)`
`long`	`getLogPosition(String queueId, String filename)` Get the current position for a specific WAL in a given queue.
`List<String>`	`getLogsInQueue(String queueId)` Get a list of all WALs in the given queue.
`List<String>`	`getUnClaimedQueueIds(String regionserver)` Get queueIds from a dead region server, whose queues has not been claimed by other region servers.
`void`	`init(String serverName)` Initialize the region server replication queue interface.
`boolean`	`isThisOurRegionServer(String regionserver)` Checks if the provided znode is the same as this region server's
`boolean`	`isThisOurZnode(String znode)` Checks if the provided znode is the same as this region server's
`boolean`	`lockOtherRS(String znode)` Try to set a lock in another region server's znode.
`void`	`removeAllQueues()` Remove all replication queues for this region server.
`void`	`removeHFileRefs(String peerId, List<String> files)` Remove hfile references from the queue.
`void`	`removeLog(String queueId, String filename)` Remove an WAL file from the given queue.
`void`	`removePeerFromHFileRefs(String peerId)` Remove a peer from hfile reference queue.
`void`	`removeQueue(String queueId)` Remove a replication queue.
`void`	`removeReplicatorIfQueueIsEmpty(String regionserver)` Remove the znode of region server if the queue is empty.
`void`	`setLogPosition(String queueId, String filename, long position)` Set the current position for a specific WAL in a given queue.

Methods inherited from class org.apache.hadoop.hbase.replication.ReplicationStateZKBase
getListOfReplicatorsZK, getPeerNode, getPeerStateNode, getTableCFsNode, isPeerPath, peerExists, toByteArray

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - RS_LOCK_ZNODE
```
public static final String RS_LOCK_ZNODE
```
    Name of znode we use to lock during failover
    
    See Also:
    Constant Field Values
- Constructor Detail
  - ReplicationQueuesZKImpl
```
public ReplicationQueuesZKImpl(ZooKeeperWatcher zk,
                       org.apache.hadoop.conf.Configuration conf,
                       Abortable abortable)
```
- Method Detail
  - init
```
public void init(String serverName)
          throws ReplicationException
```
    Description copied from interface: ReplicationQueues
    
    Initialize the region server replication queue interface.
    
    Specified by:
    
    init in interface ReplicationQueues
    
    Parameters:
    serverName - The server name of the region server that owns the replication queues this interface manages.
    
    Throws:
    
    ReplicationException
  - getListOfReplicators
```
public List<String> getListOfReplicators()
                                  throws ReplicationException
```
    Description copied from interface: ReplicationQueues
    
    Get a list of all region servers that have outstanding replication queues. These servers could be alive, dead or from a previous run of the cluster.
    
    Specified by:
    
    getListOfReplicators in interface ReplicationQueues
    
    Returns:
    a list of server names
    
    Throws:
    
    ReplicationException
  - removeQueue
```
public void removeQueue(String queueId)
```
    Description copied from interface: ReplicationQueues
    
    Remove a replication queue.
    
    Specified by:
    
    removeQueue in interface ReplicationQueues
    
    Parameters:
    queueId - a String that identifies the queue.
  - addLog
```
public void addLog(String queueId,
          String filename)
            throws ReplicationException
```
    Description copied from interface: ReplicationQueues
    
    Add a new WAL file to the given queue. If the queue does not exist it is created.
    
    Specified by:
    
    addLog in interface ReplicationQueues
    
    Parameters:
    queueId - a String that identifies the queue.
    filename - name of the WAL
    
    Throws:
    
    ReplicationException
  - removeLog
```
public void removeLog(String queueId,
             String filename)
```
    Description copied from interface: ReplicationQueues
    
    Remove an WAL file from the given queue.
    
    Specified by:
    
    removeLog in interface ReplicationQueues
    
    Parameters:
    queueId - a String that identifies the queue.
    filename - name of the WAL
  - setLogPosition
```
public void setLogPosition(String queueId,
                  String filename,
                  long position)
```
    Description copied from interface: ReplicationQueues
    
    Set the current position for a specific WAL in a given queue.
    
    Specified by:
    
    setLogPosition in interface ReplicationQueues
    
    Parameters:
    queueId - a String that identifies the queue
    filename - name of the WAL
    position - the current position in the file
  - getLogPosition
```
public long getLogPosition(String queueId,
                  String filename)
                    throws ReplicationException
```
    Description copied from interface: ReplicationQueues
    
    Get the current position for a specific WAL in a given queue.
    
    Specified by:
    
    getLogPosition in interface ReplicationQueues
    
    Parameters:
    queueId - a String that identifies the queue
    filename - name of the WAL
    
    Returns:
    the current position in the file
    
    Throws:
    
    ReplicationException
  - isThisOurZnode
```
public boolean isThisOurZnode(String znode)
```
    Description copied from interface: ReplicationQueues
    
    Checks if the provided znode is the same as this region server's
    
    Specified by:
    
    isThisOurZnode in interface ReplicationQueues
    
    Parameters:
    znode - to check
    
    Returns:
    if this is this rs's znode
  - removeAllQueues
```
public void removeAllQueues()
```
    Description copied from interface: ReplicationQueues
    
    Remove all replication queues for this region server.
    
    Specified by:
    
    removeAllQueues in interface ReplicationQueues
  - getLogsInQueue
```
public List<String> getLogsInQueue(String queueId)
```
    Description copied from interface: ReplicationQueues
    
    Get a list of all WALs in the given queue.
    
    Specified by:
    
    getLogsInQueue in interface ReplicationQueues
    
    Parameters:
    queueId - a String that identifies the queue
    
    Returns:
    a list of WALs, null if this region server is dead and has no outstanding queues
  - getAllQueues
```
public List<String> getAllQueues()
```
    Description copied from interface: ReplicationQueues
    
    Get a list of all queues for this region server.
    
    Specified by:
    
    getAllQueues in interface ReplicationQueues
    
    Returns:
    a list of queueIds, null if this region server is dead and has no outstanding queues
  - isThisOurRegionServer
```
public boolean isThisOurRegionServer(String regionserver)
```
    Description copied from interface: ReplicationQueues
    
    Checks if the provided znode is the same as this region server's
    
    Specified by:
    
    isThisOurRegionServer in interface ReplicationQueues
    
    Parameters:
    regionserver - the id of the region server
    
    Returns:
    if this is this rs's znode
  - getUnClaimedQueueIds
```
public List<String> getUnClaimedQueueIds(String regionserver)
```
    Description copied from interface: ReplicationQueues
    
    Get queueIds from a dead region server, whose queues has not been claimed by other region servers.
    
    Specified by:
    
    getUnClaimedQueueIds in interface ReplicationQueues
    
    Returns:
    empty if the queue exists but no children, null if the queue does not exist.
  - claimQueue
```
public Pair<String,SortedSet<String>> claimQueue(String regionserver,
                                        String queueId)
```
    Description copied from interface: ReplicationQueues
    
    Take ownership for the queue identified by queueId and belongs to a dead region server.
    
    Specified by:
    
    claimQueue in interface ReplicationQueues
    
    Parameters:
    regionserver - the id of the dead region server
    queueId - the id of the queue
    
    Returns:
    the new PeerId and A SortedSet of WALs in its queue, and null if no unclaimed queue.
  - removeReplicatorIfQueueIsEmpty
```
public void removeReplicatorIfQueueIsEmpty(String regionserver)
```
    Description copied from interface: ReplicationQueues
    
    Remove the znode of region server if the queue is empty.
    
    Specified by:
    
    removeReplicatorIfQueueIsEmpty in interface ReplicationQueues
  - lockOtherRS
```
public boolean lockOtherRS(String znode)
```
    Try to set a lock in another region server's znode.
    
    Parameters:
    znode - the server names of the other server
    
    Returns:
    true if the lock was acquired, false in every other cases
  - getLockZNode
```
public String getLockZNode(String znode)
```
  - checkLockExists
```
public boolean checkLockExists(String znode)
                        throws org.apache.zookeeper.KeeperException
```
    Throws:
    
    org.apache.zookeeper.KeeperException
  - addHFileRefs
```
public void addHFileRefs(String peerId,
                List<Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>> pairs)
                  throws ReplicationException
```
    Description copied from interface: ReplicationQueues
    
    Add new hfile references to the queue.
    
    Specified by:
    
    addHFileRefs in interface ReplicationQueues
    
    Parameters:
    peerId - peer cluster id to which the hfiles need to be replicated
    pairs - list of pairs of { HFile location in staging dir, HFile path in region dir which will be added in the queue }
    
    Throws:
    
    ReplicationException - if fails to add a hfile reference
  - removeHFileRefs
```
public void removeHFileRefs(String peerId,
                   List<String> files)
```
    Description copied from interface: ReplicationQueues
    
    Remove hfile references from the queue.
    
    Specified by:
    
    removeHFileRefs in interface ReplicationQueues
    
    Parameters:
    peerId - peer cluster id from which this hfile references needs to be removed
    files - list of hfile references to be removed
  - addPeerToHFileRefs
```
public void addPeerToHFileRefs(String peerId)
                        throws ReplicationException
```
    Description copied from interface: ReplicationQueues
    
    Add a peer to hfile reference queue if peer does not exist.
    
    Specified by:
    
    addPeerToHFileRefs in interface ReplicationQueues
    
    Parameters:
    peerId - peer cluster id to be added
    
    Throws:
    
    ReplicationException - if fails to add a peer id to hfile reference queue
  - removePeerFromHFileRefs
```
public void removePeerFromHFileRefs(String peerId)
```
    Description copied from interface: ReplicationQueues
    
    Remove a peer from hfile reference queue.
    
    Specified by:
    
    removePeerFromHFileRefs in interface ReplicationQueues
    
    Parameters:
    peerId - peer cluster id to be removed

Class ReplicationQueuesZKImpl

Field Summary

Fields inherited from class org.apache.hadoop.hbase.replication.ReplicationStateZKBase

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.replication.ReplicationStateZKBase

Methods inherited from class java.lang.Object

Field Detail

RS_LOCK_ZNODE

Constructor Detail

ReplicationQueuesZKImpl

Method Detail

init

getListOfReplicators

removeQueue

addLog

removeLog

setLogPosition

getLogPosition

isThisOurZnode

removeAllQueues

getLogsInQueue

getAllQueues

isThisOurRegionServer

getUnClaimedQueueIds

claimQueue

removeReplicatorIfQueueIsEmpty

lockOtherRS

getLockZNode

checkLockExists

addHFileRefs

removeHFileRefs

addPeerToHFileRefs

removePeerFromHFileRefs