ZKSplitLogManagerCoordination (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
- - org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination

All Implemented Interfaces:: SplitLogManagerCoordination

@InterfaceAudience.Private
public class ZKSplitLogManagerCoordination
extends ZooKeeperListener
implements SplitLogManagerCoordination

ZooKeeper based implementation of SplitLogManagerCoordination

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`class`	`ZKSplitLogManagerCoordination.CreateAsyncCallback` Asynchronous handler for zk create node results.
`class`	`ZKSplitLogManagerCoordination.CreateRescanAsyncCallback` Asynchronous handler for zk create RESCAN-node results.
`class`	`ZKSplitLogManagerCoordination.DeleteAsyncCallback` Asynchronous handler for zk delete node results.
`class`	`ZKSplitLogManagerCoordination.GetDataAsyncCallback` Asynchronous handler for zk get-data-set-watch on node results.
`static interface`	`ZKSplitLogManagerCoordination.TaskFinisher` `SplitLogManager` can use objects implementing this interface to finish off a partially done task by `SplitLogWorker`.
`static class`	`ZKSplitLogManagerCoordination.ZkSplitLogManagerDetails`

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coordination.SplitLogManagerCoordination
SplitLogManagerCoordination.SplitLogManagerDetails

Field Summary

Fields
Modifier and Type	Field and Description
`private org.apache.hadoop.conf.Configuration`	`conf`
`static int`	`DEFAULT_MAX_RESUBMIT`
`static int`	`DEFAULT_TIMEOUT`
`static int`	`DEFAULT_ZK_RETRIES`
`(package private) SplitLogManagerCoordination.SplitLogManagerDetails`	`details`
`boolean`	`ignoreZKDeleteForTesting`
`private boolean`	`isDrainingDone`
`private long`	`lastRecoveringNodeCreationTime`
`private static org.apache.commons.logging.Log`	`LOG`
`private org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode`	`recoveryMode`
`private long`	`resubmitThreshold`
`private Server`	`server`
`private ZKSplitLogManagerCoordination.TaskFinisher`	`taskFinisher`
`private long`	`timeout`
`private long`	`zkretries`

Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
watcher

Constructor Summary

Constructors
Constructor and Description

ZKSplitLogManagerCoordination(CoordinatedStateManager manager, ZooKeeperWatcher watcher)

Constructors
Constructor and Description
`ZKSplitLogManagerCoordination(CoordinatedStateManager manager, ZooKeeperWatcher watcher)`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`checkTasks()` tells Coordination that it should check for new tasks
`void`	`checkTaskStillAvailable(String path)` Check that the task is still there
`private void`	`createNode(String path, Long retry_count)`
`private void`	`createNodeFailure(String path)`
`private void`	`createNodeSuccess(String path)`
`private void`	`createRescanFailure()`
`private void`	`createRescanSuccess(String path)`
`private void`	`deleteNode(String path, Long retries)`
`private void`	`deleteNodeFailure(String path)`
`private void`	`deleteNodeSuccess(String path)`
`void`	`deleteTask(String path)`
`(package private) SplitLogManager.Task`	`findOrCreateOrphanTask(String path)`
`private void`	`getDataSetWatch(String path, Long retry_count)`
`private void`	`getDataSetWatchFailure(String path)`
`private void`	`getDataSetWatchSuccess(String path, byte[] data, int version)`
`SplitLogManagerCoordination.SplitLogManagerDetails`	`getDetails()` Returns the configuration that was provided previously
`long`	`getLastRecoveryTime()`
`org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode`	`getRecoveryMode()` Temporary function, mostly for UTs.
`private void`	`handleUnassignedTask(String path)` It is possible for a task to stay in UNASSIGNED state indefinitely - say SplitLogManager wants to resubmit a task.
`private void`	`heartbeat(String path, int new_version, ServerName workerName)`
`void`	`init()` Support method to init constants such as timeout.
`private boolean`	`isDistributedLogReplay(org.apache.hadoop.conf.Configuration conf)` Returns if distributed log replay is turned on or not
`boolean`	`isReplaying()`
`boolean`	`isSplitting()`
`private List<String>`	`listSplitLogTasks()`
`private void`	`lookForOrphans()`
`void`	`markRegionsRecovering(ServerName serverName, Set<HRegionInfo> userRegions)` Create znodes /hbase/recovering-regions/[region_ids...]/[failed region server names ...] for all regions of the passed in region servers
`private boolean`	`needAbandonRetries(int statusCode, String action)` Helper function to check whether to abandon retries in ZooKeeper AsyncCallback functions
`void`	`nodeDataChanged(String path)` Called when an existing node has changed data.
`String`	`prepareTask(String taskname)` Prepare the new task
`int`	`remainingTasksInCoordination()` Return the number of remaining tasks
`void`	`removeRecoveringRegions(Set<String> recoveredServerNameSet, Boolean isMetaRecovery)` It removes recovering regions under /hbase/recovering-regions/[encoded region name] so that the region server hosting the region can allow reads to the recovered region
`void`	`removeStaleRecoveringRegions(Set<String> knownFailedServers)` ZooKeeper implementation of `SplitLogManagerCoordination.removeStaleRecoveringRegions(Set)`
`private void`	`rescan(long retries)` signal the workers that a task was resubmitted by creating the RESCAN node.
`private boolean`	`resubmit(ServerName serverName, String path, int version)`
`private void`	`resubmitOrFail(String path, SplitLogManager.ResubmitDirective directive)`
`boolean`	`resubmitTask(String path, SplitLogManager.Task task, SplitLogManager.ResubmitDirective directive)` Resubmit the task in case if found unassigned or failed
`void`	`setDetails(SplitLogManagerCoordination.SplitLogManagerDetails details)` Provide the configuration from the SplitLogManager
`private void`	`setDone(String path, SplitLogManager.TerminationStatus status)`
`void`	`setIgnoreDeleteForTesting(boolean b)` Temporary function that is used by unit tests only
`void`	`setRecoveryMode(boolean isForInitialization)` This function is to set recovery mode from outstanding split log tasks from before or current configuration setting
`void`	`submitTask(String path)`

Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
getWatcher, nodeChildrenChanged, nodeCreated, nodeDeleted

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - DEFAULT_TIMEOUT
```
public static final int DEFAULT_TIMEOUT
```
    See Also:
    Constant Field Values
  - DEFAULT_ZK_RETRIES
```
public static final int DEFAULT_ZK_RETRIES
```
    See Also:
    Constant Field Values
  - DEFAULT_MAX_RESUBMIT
```
public static final int DEFAULT_MAX_RESUBMIT
```
    See Also:
    Constant Field Values
  - LOG
```
private static final org.apache.commons.logging.Log LOG
```
  - server
```
private Server server
```
  - zkretries
```
private long zkretries
```
  - resubmitThreshold
```
private long resubmitThreshold
```
  - timeout
```
private long timeout
```
  - taskFinisher
```
private ZKSplitLogManagerCoordination.TaskFinisher taskFinisher
```
  - details
```
SplitLogManagerCoordination.SplitLogManagerDetails details
```
  - lastRecoveringNodeCreationTime
```
private volatile long lastRecoveringNodeCreationTime
```
  - conf
```
private org.apache.hadoop.conf.Configuration conf
```
  - ignoreZKDeleteForTesting
```
public boolean ignoreZKDeleteForTesting
```
  - recoveryMode
```
private org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode recoveryMode
```
  - isDrainingDone
```
private boolean isDrainingDone
```
- Constructor Detail
  - ZKSplitLogManagerCoordination
```
public ZKSplitLogManagerCoordination(CoordinatedStateManager manager,
                             ZooKeeperWatcher watcher)
```
- Method Detail
  - init
```
public void init()
          throws IOException
```
    Description copied from interface: SplitLogManagerCoordination
    
    Support method to init constants such as timeout. Mostly required for UTs.
    
    Specified by:
    
    init in interface SplitLogManagerCoordination
    
    Throws:
    
    IOException
  - prepareTask
```
public String prepareTask(String taskname)
```
    Description copied from interface: SplitLogManagerCoordination
    
    Prepare the new task
    
    Specified by:
    
    prepareTask in interface SplitLogManagerCoordination
    
    Parameters:
    taskname - name of the task
    
    Returns:
    the task id
  - remainingTasksInCoordination
```
public int remainingTasksInCoordination()
```
    Description copied from interface: SplitLogManagerCoordination
    
    Return the number of remaining tasks
    
    Specified by:
    
    remainingTasksInCoordination in interface SplitLogManagerCoordination
  - handleUnassignedTask
```
private void handleUnassignedTask(String path)
```
    It is possible for a task to stay in UNASSIGNED state indefinitely - say SplitLogManager wants to resubmit a task. It forces the task to UNASSIGNED state but it dies before it could create the RESCAN task node to signal the SplitLogWorkers to pick up the task. To prevent this scenario the SplitLogManager resubmits all orphan and UNASSIGNED tasks at startup.
    
    Parameters:
    path -
  - deleteTask
```
public void deleteTask(String path)
```
    Specified by:
    
    deleteTask in interface SplitLogManagerCoordination
    
    Parameters:
    path - to be removed
  - resubmitTask
```
public boolean resubmitTask(String path,
                   SplitLogManager.Task task,
                   SplitLogManager.ResubmitDirective directive)
```
    Description copied from interface: SplitLogManagerCoordination
    
    Resubmit the task in case if found unassigned or failed
    
    Specified by:
    
    resubmitTask in interface SplitLogManagerCoordination
    
    Parameters:
    path - path related to task
    task - to resubmit
    directive - whether it should be forced
    
    Returns:
    whether it was successful
  - checkTasks
```
public void checkTasks()
```
    Description copied from interface: SplitLogManagerCoordination
    
    tells Coordination that it should check for new tasks
    
    Specified by:
    
    checkTasks in interface SplitLogManagerCoordination
  - rescan
```
private void rescan(long retries)
```
    signal the workers that a task was resubmitted by creating the RESCAN node.
  - submitTask
```
public void submitTask(String path)
```
    Specified by:
    
    submitTask in interface SplitLogManagerCoordination
    
    Parameters:
    path - to be submitted
  - checkTaskStillAvailable
```
public void checkTaskStillAvailable(String path)
```
    Description copied from interface: SplitLogManagerCoordination
    
    Check that the task is still there
    
    Specified by:
    
    checkTaskStillAvailable in interface SplitLogManagerCoordination
    
    Parameters:
    path - node to check
  - removeRecoveringRegions
```
public void removeRecoveringRegions(Set<String> recoveredServerNameSet,
                           Boolean isMetaRecovery)
                             throws IOException
```
    It removes recovering regions under /hbase/recovering-regions/[encoded region name] so that the region server hosting the region can allow reads to the recovered region
    
    Specified by:
    
    removeRecoveringRegions in interface SplitLogManagerCoordination
    
    Parameters:
    recoveredServerNameSet - servers which are just recovered
    isMetaRecovery - whether current recovery is for the meta region on serverNames
    
    Throws:
    
    IOException
  - deleteNode
```
private void deleteNode(String path,
              Long retries)
```
  - deleteNodeSuccess
```
private void deleteNodeSuccess(String path)
```
  - deleteNodeFailure
```
private void deleteNodeFailure(String path)
```
  - createRescanSuccess
```
private void createRescanSuccess(String path)
```
  - createRescanFailure
```
private void createRescanFailure()
```
  - needAbandonRetries
```
private boolean needAbandonRetries(int statusCode,
                         String action)
```
    Helper function to check whether to abandon retries in ZooKeeper AsyncCallback functions
    
    Parameters:
    statusCode - integer value of a ZooKeeper exception code
    action - description message about the retried action
    
    Returns:
    true when need to abandon retries otherwise false
  - createNode
```
private void createNode(String path,
              Long retry_count)
```
  - createNodeSuccess
```
private void createNodeSuccess(String path)
```
  - createNodeFailure
```
private void createNodeFailure(String path)
```
  - getDataSetWatch
```
private void getDataSetWatch(String path,
                   Long retry_count)
```
  - getDataSetWatchSuccess
```
private void getDataSetWatchSuccess(String path,
                          byte[] data,
                          int version)
                             throws DeserializationException
```
    Throws:
    
    DeserializationException
  - resubmitOrFail
```
private void resubmitOrFail(String path,
                  SplitLogManager.ResubmitDirective directive)
```
  - getDataSetWatchFailure
```
private void getDataSetWatchFailure(String path)
```
  - setDone
```
private void setDone(String path,
           SplitLogManager.TerminationStatus status)
```
  - findOrCreateOrphanTask
```
SplitLogManager.Task findOrCreateOrphanTask(String path)
```
  - heartbeat
```
private void heartbeat(String path,
             int new_version,
             ServerName workerName)
```
  - lookForOrphans
```
private void lookForOrphans()
```
  - markRegionsRecovering
```
public void markRegionsRecovering(ServerName serverName,
                         Set<HRegionInfo> userRegions)
                           throws IOException,
                                  InterruptedIOException
```
    Create znodes /hbase/recovering-regions/[region_ids...]/[failed region server names ...] for all regions of the passed in region servers
    
    Specified by:
    
    markRegionsRecovering in interface SplitLogManagerCoordination
    
    Parameters:
    serverName - the name of a region server
    userRegions - user regiones assigned on the region server
    
    Throws:
    
    IOException - in case of failure
    
    InterruptedIOException
  - nodeDataChanged
```
public void nodeDataChanged(String path)
```
    Description copied from class: ZooKeeperListener
    
    Called when an existing node has changed data.
    
    Overrides:
    
    nodeDataChanged in class ZooKeeperListener
    
    Parameters:
    path - full path of the updated node
  - removeStaleRecoveringRegions
```
public void removeStaleRecoveringRegions(Set<String> knownFailedServers)
                                  throws IOException,
                                         InterruptedIOException
```
    ZooKeeper implementation of SplitLogManagerCoordination.removeStaleRecoveringRegions(Set)
    
    Specified by:
    
    removeStaleRecoveringRegions in interface SplitLogManagerCoordination
    
    Parameters:
    knownFailedServers - set of previously failed servers
    
    Throws:
    
    IOException - in case of failure
    
    InterruptedIOException
  - isReplaying
```
public boolean isReplaying()
```
    Specified by:
    
    isReplaying in interface SplitLogManagerCoordination
    
    Returns:
    shows whether the log recovery mode is in replaying state
  - isSplitting
```
public boolean isSplitting()
```
    Specified by:
    
    isSplitting in interface SplitLogManagerCoordination
    
    Returns:
    shows whether the log recovery mode is in splitting state
  - listSplitLogTasks
```
private List<String> listSplitLogTasks()
                                throws org.apache.zookeeper.KeeperException
```
    Throws:
    
    org.apache.zookeeper.KeeperException
  - setRecoveryMode
```
public void setRecoveryMode(boolean isForInitialization)
                     throws IOException
```
    This function is to set recovery mode from outstanding split log tasks from before or current configuration setting
    
    Specified by:
    
    setRecoveryMode in interface SplitLogManagerCoordination
    
    Parameters:
    isForInitialization -
    
    Throws:
    
    IOException
    
    InterruptedIOException
  - isDistributedLogReplay
```
private boolean isDistributedLogReplay(org.apache.hadoop.conf.Configuration conf)
```
    Returns if distributed log replay is turned on or not
    
    Parameters:
    conf -
    
    Returns:
    true when distributed log replay is turned on
  - resubmit
```
private boolean resubmit(ServerName serverName,
               String path,
               int version)
```
  - setDetails
```
public void setDetails(SplitLogManagerCoordination.SplitLogManagerDetails details)
```
    Description copied from interface: SplitLogManagerCoordination
    
    Provide the configuration from the SplitLogManager
    
    Specified by:
    
    setDetails in interface SplitLogManagerCoordination
  - getDetails
```
public SplitLogManagerCoordination.SplitLogManagerDetails getDetails()
```
    Description copied from interface: SplitLogManagerCoordination
    
    Returns the configuration that was provided previously
    
    Specified by:
    
    getDetails in interface SplitLogManagerCoordination
  - getRecoveryMode
```
public org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode getRecoveryMode()
```
    Description copied from interface: SplitLogManagerCoordination
    
    Temporary function, mostly for UTs. In the regular code isReplaying or isSplitting should be used.
    
    Specified by:
    
    getRecoveryMode in interface SplitLogManagerCoordination
    
    Returns:
    the current log recovery mode.
  - getLastRecoveryTime
```
public long getLastRecoveryTime()
```
    Specified by:
    
    getLastRecoveryTime in interface SplitLogManagerCoordination
    
    Returns:
    the time of last attempt to recover
  - setIgnoreDeleteForTesting
```
public void setIgnoreDeleteForTesting(boolean b)
```
    Temporary function that is used by unit tests only

Class ZKSplitLogManagerCoordination

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coordination.SplitLogManagerCoordination

Field Summary

Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener

Methods inherited from class java.lang.Object

Field Detail

DEFAULT_TIMEOUT

DEFAULT_ZK_RETRIES

DEFAULT_MAX_RESUBMIT

LOG

server

zkretries

resubmitThreshold

timeout

taskFinisher

details

lastRecoveringNodeCreationTime

conf

ignoreZKDeleteForTesting

recoveryMode

isDrainingDone

Constructor Detail

ZKSplitLogManagerCoordination

Method Detail

init

prepareTask

remainingTasksInCoordination

handleUnassignedTask

deleteTask

resubmitTask

checkTasks

rescan

submitTask

checkTaskStillAvailable

removeRecoveringRegions

deleteNode

deleteNodeSuccess

deleteNodeFailure

createRescanSuccess

createRescanFailure

needAbandonRetries

createNode

createNodeSuccess

createNodeFailure

getDataSetWatch

getDataSetWatchSuccess

resubmitOrFail

getDataSetWatchFailure

setDone

findOrCreateOrphanTask

heartbeat

lookForOrphans

markRegionsRecovering

nodeDataChanged

removeStaleRecoveringRegions

isReplaying

isSplitting

listSplitLogTasks

setRecoveryMode

isDistributedLogReplay

resubmit

setDetails

getDetails

getRecoveryMode

getLastRecoveryTime

setIgnoreDeleteForTesting