ZkSplitLogWorkerCoordination (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
- - org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination

All Implemented Interfaces:

SplitLogWorkerCoordination
```
@InterfaceAudience.Private
public class ZkSplitLogWorkerCoordination
extends ZooKeeperListener
implements SplitLogWorkerCoordination
```
ZooKeeper based implementation of SplitLogWorkerCoordination It listen for changes in ZooKeeper and

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`(package private) class`	`ZkSplitLogWorkerCoordination.GetDataAsyncCallback` Asynchronous handler for zk get-data-set-watch on node results.
`static class`	`ZkSplitLogWorkerCoordination.ZkSplitTaskDetails` When ZK-based implementation wants to complete the task, it needs to know task znode and current znode cversion (needed for subsequent update operation).

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coordination.SplitLogWorkerCoordination
SplitLogWorkerCoordination.SplitTaskDetails

Field Summary

Fields
Modifier and Type	Field and Description
`private static int`	`checkInterval`
`private String`	`currentTask`
`private int`	`currentVersion`
`private static int`	`FAILED_TO_OWN_TASK`
`private Object`	`grabTaskLock`
`private static org.apache.commons.logging.Log`	`LOG`
`private ZkCoordinatedStateManager`	`manager`
`private int`	`maxConcurrentTasks`
`private int`	`reportPeriod`
`private RegionServerServices`	`server`
`private boolean`	`shouldStop`
`private SplitLogWorker.TaskExecutor`	`splitTaskExecutor`
`private Object`	`taskReadyLock`
`private AtomicInteger`	`taskReadySeq`
`protected AtomicInteger`	`tasksInProgress`
`private SplitLogWorker`	`worker`
`private boolean`	`workerInGrabTask`

Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
watcher

Fields inherited from interface org.apache.hadoop.hbase.coordination.SplitLogWorkerCoordination
DEFAULT_MAX_SPLITTERS

Constructor Summary

Constructors
Constructor and Description

ZkSplitLogWorkerCoordination(ZkCoordinatedStateManager zkCoordinatedStateManager, ZooKeeperWatcher watcher)

Constructors
Constructor and Description
`ZkSplitLogWorkerCoordination(ZkCoordinatedStateManager zkCoordinatedStateManager, ZooKeeperWatcher watcher)`

Method Summary

Methods
Modifier and Type	Method and Description
`protected static int`	`attemptToOwnTask(boolean isFirstTime, ZooKeeperWatcher zkw, ServerName server, String task, org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode mode, int taskZKVersion)` Try to own the task by transitioning the zk node data from UNASSIGNED to OWNED.
`private int`	`calculateAvailableSplitters(int numTasks)` This function calculates how many splitters it could create based on expected average tasks per RS and the hard limit upper bound(maxConcurrentTasks) set by configuration.
`void`	`endTask(SplitLogTask slt, AtomicLong ctr, SplitLogWorkerCoordination.SplitTaskDetails details)` endTask() can fail and the only way to recover out of it is for the `SplitLogManager` to timeout the task node.
`void`	`getDataSetWatchAsync()`
`(package private) void`	`getDataSetWatchFailure(String path)`
`(package private) void`	`getDataSetWatchSuccess(String path, byte[] data)`
`org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos.RegionStoreSequenceIds`	`getRegionFlushedSequenceId(String failedServerName, String key)`
`private List<String>`	`getTaskList()`
`int`	`getTaskReadySeq()` Used by unit tests to check how many tasks were processed
`private void`	`grabTask(String path)` try to grab a 'lock' on the task zk node to own and execute the task.
`void`	`init(RegionServerServices server, org.apache.hadoop.conf.Configuration conf, SplitLogWorker.TaskExecutor splitExecutor, SplitLogWorker worker)` Override setter from `SplitLogWorkerCoordination`
`boolean`	`isReady()` Check whether the log splitter is ready to supply tasks
`boolean`	`isStop()`
`void`	`markCorrupted(org.apache.hadoop.fs.Path rootDir, String name, org.apache.hadoop.fs.FileSystem fs)` marks log file as corrupted
`void`	`nodeChildrenChanged(String path)` Override handler from `ZooKeeperListener`
`void`	`nodeDataChanged(String path)` Override handler from `ZooKeeperListener`
`void`	`registerListener()` set the listener for task changes.
`void`	`removeListener()` remove the listener for task changes.
`void`	`stopProcessingTasks()` called when Coordination should stop processing tasks and exit
`(package private) void`	`submitTask(String curTask, org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode mode, int curTaskZKVersion, int reportPeriod)` Submit a log split task to executor service
`void`	`taskLoop()` Wait for tasks to become available at /hbase/splitlog zknode.

Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
getWatcher, nodeCreated, nodeDeleted

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.apache.commons.logging.Log LOG
```
  - checkInterval
```
private static final int checkInterval
```
    See Also:
    Constant Field Values
  - FAILED_TO_OWN_TASK
```
private static final int FAILED_TO_OWN_TASK
```
    See Also:
    Constant Field Values
  - worker
```
private SplitLogWorker worker
```
  - splitTaskExecutor
```
private SplitLogWorker.TaskExecutor splitTaskExecutor
```
  - taskReadyLock
```
private final Object taskReadyLock
```
  - taskReadySeq
```
private AtomicInteger taskReadySeq
```
  - currentTask
```
private volatile String currentTask
```
  - currentVersion
```
private int currentVersion
```
  - shouldStop
```
private volatile boolean shouldStop
```
  - grabTaskLock
```
private final Object grabTaskLock
```
  - workerInGrabTask
```
private boolean workerInGrabTask
```
  - reportPeriod
```
private int reportPeriod
```
  - server
```
private RegionServerServices server
```
  - tasksInProgress
```
protected final AtomicInteger tasksInProgress
```
  - maxConcurrentTasks
```
private int maxConcurrentTasks
```
  - manager
```
private final ZkCoordinatedStateManager manager
```
- Constructor Detail
  - ZkSplitLogWorkerCoordination
```
public ZkSplitLogWorkerCoordination(ZkCoordinatedStateManager zkCoordinatedStateManager,
                            ZooKeeperWatcher watcher)
```
- Method Detail
  - nodeChildrenChanged
```
public void nodeChildrenChanged(String path)
```
    Override handler from ZooKeeperListener
    
    Overrides:
    
    nodeChildrenChanged in class ZooKeeperListener
    
    Parameters:
    path - full path of the node whose children have changed
  - nodeDataChanged
```
public void nodeDataChanged(String path)
```
    Override handler from ZooKeeperListener
    
    Overrides:
    
    nodeDataChanged in class ZooKeeperListener
    
    Parameters:
    path - full path of the updated node
  - init
```
public void init(RegionServerServices server,
        org.apache.hadoop.conf.Configuration conf,
        SplitLogWorker.TaskExecutor splitExecutor,
        SplitLogWorker worker)
```
    Override setter from SplitLogWorkerCoordination
    
    Specified by:
    
    init in interface SplitLogWorkerCoordination
    
    Parameters:
    server - instance of RegionServerServices to work with
    conf - is current configuration.
    splitExecutor - split executor from SplitLogWorker
    worker - instance of SplitLogWorker
  - getDataSetWatchFailure
```
void getDataSetWatchFailure(String path)
```
  - getDataSetWatchAsync
```
public void getDataSetWatchAsync()
```
  - getDataSetWatchSuccess
```
void getDataSetWatchSuccess(String path,
                          byte[] data)
```
  - grabTask
```
private void grabTask(String path)
```
    try to grab a 'lock' on the task zk node to own and execute the task.
    
    Parameters:
    path - zk node for the task
  - submitTask
```
void submitTask(String curTask,
              org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode mode,
              int curTaskZKVersion,
              int reportPeriod)
```
    Submit a log split task to executor service
    
    Parameters:
    curTask - task to submit
    curTaskZKVersion - current version of task
  - calculateAvailableSplitters
```
private int calculateAvailableSplitters(int numTasks)
```
    This function calculates how many splitters it could create based on expected average tasks per RS and the hard limit upper bound(maxConcurrentTasks) set by configuration.
    At any given time, a RS allows spawn MIN(Expected Tasks/RS, Hard Upper Bound)
    
    Parameters:
    numTasks - current total number of available tasks
  - attemptToOwnTask
```
protected static int attemptToOwnTask(boolean isFirstTime,
                   ZooKeeperWatcher zkw,
                   ServerName server,
                   String task,
                   org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos.SplitLogTask.RecoveryMode mode,
                   int taskZKVersion)
```
    Try to own the task by transitioning the zk node data from UNASSIGNED to OWNED.
    This method is also used to periodically heartbeat the task progress by transitioning the node from OWNED to OWNED.
    
    Parameters:
    isFirstTime - shows whther it's the first attempt.
    zkw - zk wathcer
    server - name
    task - to own
    taskZKVersion - version of the task in zk
    
    Returns:
    non-negative integer value when task can be owned by current region server otherwise -1
  - taskLoop
```
public void taskLoop()
              throws InterruptedException
```
    Wait for tasks to become available at /hbase/splitlog zknode. Grab a task one at a time. This policy puts an upper-limit on the number of simultaneous log splitting that could be happening in a cluster.
    Synchronization using taskReadyLock ensures that it will try to grab every task that has been put up
    
    Specified by:
    
    taskLoop in interface SplitLogWorkerCoordination
    
    Throws:
    
    InterruptedException
  - getTaskList
```
private List<String> getTaskList()
                          throws InterruptedException
```
    Throws:
    
    InterruptedException
  - markCorrupted
```
public void markCorrupted(org.apache.hadoop.fs.Path rootDir,
                 String name,
                 org.apache.hadoop.fs.FileSystem fs)
```
    Description copied from interface: SplitLogWorkerCoordination
    
    marks log file as corrupted
    
    Specified by:
    
    markCorrupted in interface SplitLogWorkerCoordination
    
    Parameters:
    rootDir - where to find the log
    name - of the log
    fs - file system
  - isReady
```
public boolean isReady()
                throws InterruptedException
```
    Description copied from interface: SplitLogWorkerCoordination
    
    Check whether the log splitter is ready to supply tasks
    
    Specified by:
    
    isReady in interface SplitLogWorkerCoordination
    
    Returns:
    false if there is no tasks
    
    Throws:
    
    InterruptedException - if the SplitLogWorker was stopped
  - getTaskReadySeq
```
public int getTaskReadySeq()
```
    Description copied from interface: SplitLogWorkerCoordination
    
    Used by unit tests to check how many tasks were processed
    
    Specified by:
    
    getTaskReadySeq in interface SplitLogWorkerCoordination
    
    Returns:
    number of tasks
  - registerListener
```
public void registerListener()
```
    Description copied from interface: SplitLogWorkerCoordination
    
    set the listener for task changes. Implementation specific
    
    Specified by:
    
    registerListener in interface SplitLogWorkerCoordination
  - removeListener
```
public void removeListener()
```
    Description copied from interface: SplitLogWorkerCoordination
    
    remove the listener for task changes. Implementation specific
    
    Specified by:
    
    removeListener in interface SplitLogWorkerCoordination
  - stopProcessingTasks
```
public void stopProcessingTasks()
```
    Description copied from interface: SplitLogWorkerCoordination
    
    called when Coordination should stop processing tasks and exit
    
    Specified by:
    
    stopProcessingTasks in interface SplitLogWorkerCoordination
  - isStop
```
public boolean isStop()
```
    Specified by:
    
    isStop in interface SplitLogWorkerCoordination
    
    Returns:
    the current value of exitWorker
  - getRegionFlushedSequenceId
```
public org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos.RegionStoreSequenceIds getRegionFlushedSequenceId(String failedServerName,
                                                                                                               String key)
                                                                                                                 throws IOException
```
    Specified by:
    
    getRegionFlushedSequenceId in interface SplitLogWorkerCoordination
    
    Throws:
    
    IOException
  - endTask
```
public void endTask(SplitLogTask slt,
           AtomicLong ctr,
           SplitLogWorkerCoordination.SplitTaskDetails details)
```
    endTask() can fail and the only way to recover out of it is for the SplitLogManager to timeout the task node.
    
    Specified by:
    
    endTask in interface SplitLogWorkerCoordination
    
    Parameters:
    slt -
    ctr -
    details - details about log split task (specific to coordination engine being used).

Class ZkSplitLogWorkerCoordination

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coordination.SplitLogWorkerCoordination

Field Summary

Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener

Fields inherited from interface org.apache.hadoop.hbase.coordination.SplitLogWorkerCoordination

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener

Methods inherited from class java.lang.Object

Field Detail

LOG

checkInterval

FAILED_TO_OWN_TASK

worker

splitTaskExecutor

taskReadyLock

taskReadySeq

currentTask

currentVersion

shouldStop

grabTaskLock

workerInGrabTask

reportPeriod

server

tasksInProgress

maxConcurrentTasks

manager

Constructor Detail

ZkSplitLogWorkerCoordination

Method Detail

nodeChildrenChanged

nodeDataChanged

init

getDataSetWatchFailure

getDataSetWatchAsync

getDataSetWatchSuccess

grabTask

submitTask

calculateAvailableSplitters

attemptToOwnTask

taskLoop

getTaskList

markCorrupted

isReady

getTaskReadySeq

registerListener

removeListener

stopProcessingTasks

isStop

getRegionFlushedSequenceId

endTask