@InterfaceAudience.Private public class ZkSplitLogWorkerCoordination extends ZKListener implements SplitLogWorkerCoordination
SplitLogWorkerCoordination
It listen for changes in
ZooKeeper andModifier and Type | Class and Description |
---|---|
(package private) class |
ZkSplitLogWorkerCoordination.GetDataAsyncCallback
Asynchronous handler for zk get-data-set-watch on node results.
|
static class |
ZkSplitLogWorkerCoordination.ZkSplitTaskDetails
When ZK-based implementation wants to complete the task, it needs to know task znode and
current znode cversion (needed for subsequent update operation).
|
SplitLogWorkerCoordination.SplitTaskDetails
Modifier and Type | Field and Description |
---|---|
private static int |
checkInterval |
private String |
currentTask |
private int |
currentVersion |
private static int |
FAILED_TO_OWN_TASK |
private Object |
grabTaskLock |
private static org.slf4j.Logger |
LOG |
private int |
maxConcurrentTasks |
private int |
reportPeriod |
private RegionServerServices |
server |
private ServerName |
serverName |
private boolean |
shouldStop |
private SplitLogWorker.TaskExecutor |
splitTaskExecutor |
private AtomicInteger |
taskReadySeq |
protected AtomicInteger |
tasksInProgress |
private SplitLogWorker |
worker |
private boolean |
workerInGrabTask |
watcher
Constructor and Description |
---|
ZkSplitLogWorkerCoordination(ServerName serverName,
ZKWatcher watcher) |
Modifier and Type | Method and Description |
---|---|
private boolean |
areSplittersAvailable()
Returns true if more splitters are available, otherwise false.
|
protected static int |
attemptToOwnTask(boolean isFirstTime,
ZKWatcher zkw,
ServerName server,
String task,
int taskZKVersion)
Try to own the task by transitioning the zk node data from UNASSIGNED to OWNED.
|
void |
endTask(SplitLogTask slt,
LongAdder ctr,
SplitLogWorkerCoordination.SplitTaskDetails details)
endTask() can fail and the only way to recover out of it is for the
SplitLogManager to timeout the task node. |
void |
getDataSetWatchAsync() |
(package private) void |
getDataSetWatchFailure(String path) |
(package private) void |
getDataSetWatchSuccess(String path,
byte[] data) |
private List<String> |
getTaskList() |
int |
getTaskReadySeq()
Used by unit tests to check how many tasks were processed
|
private boolean |
grabTask(String path)
try to grab a 'lock' on the task zk node to own and execute the task.
|
void |
init(RegionServerServices server,
org.apache.hadoop.conf.Configuration conf,
SplitLogWorker.TaskExecutor splitExecutor,
SplitLogWorker worker)
Override setter from
SplitLogWorkerCoordination |
boolean |
isReady()
Check whether the log splitter is ready to supply tasks
|
boolean |
isStop()
Returns the current value of exitWorker
|
void |
markCorrupted(org.apache.hadoop.fs.Path rootDir,
String name,
org.apache.hadoop.fs.FileSystem fs)
marks log file as corrupted
|
void |
nodeChildrenChanged(String path)
Override handler from
ZKListener |
void |
nodeDataChanged(String path)
Override handler from
ZKListener |
void |
registerListener()
set the listener for task changes.
|
void |
removeListener()
remove the listener for task changes.
|
void |
stopProcessingTasks()
called when Coordination should stop processing tasks and exit
|
(package private) void |
submitTask(String curTask,
int curTaskZKVersion,
int reportPeriod)
Submit a log split task to executor service
|
void |
taskLoop()
Wait for tasks to become available at /hbase/splitlog zknode.
|
getWatcher, nodeCreated, nodeDeleted
private static final org.slf4j.Logger LOG
private static final int checkInterval
private static final int FAILED_TO_OWN_TASK
private SplitLogWorker worker
private SplitLogWorker.TaskExecutor splitTaskExecutor
private final AtomicInteger taskReadySeq
private volatile String currentTask
private int currentVersion
private volatile boolean shouldStop
private final Object grabTaskLock
private boolean workerInGrabTask
private int reportPeriod
private RegionServerServices server
protected final AtomicInteger tasksInProgress
private int maxConcurrentTasks
private final ServerName serverName
public ZkSplitLogWorkerCoordination(ServerName serverName, ZKWatcher watcher)
public void nodeChildrenChanged(String path)
ZKListener
nodeChildrenChanged
in class ZKListener
path
- full path of the node whose children have changedpublic void nodeDataChanged(String path)
ZKListener
nodeDataChanged
in class ZKListener
path
- full path of the updated nodepublic void init(RegionServerServices server, org.apache.hadoop.conf.Configuration conf, SplitLogWorker.TaskExecutor splitExecutor, SplitLogWorker worker)
SplitLogWorkerCoordination
init
in interface SplitLogWorkerCoordination
server
- instance of RegionServerServices to work withconf
- is current configuration.splitExecutor
- split executor from SplitLogWorkerworker
- instance of SplitLogWorkervoid getDataSetWatchFailure(String path)
public void getDataSetWatchAsync()
void getDataSetWatchSuccess(String path, byte[] data)
private boolean grabTask(String path)
path
- zk node for the taskvoid submitTask(String curTask, int curTaskZKVersion, int reportPeriod)
curTask
- task to submitcurTaskZKVersion
- current version of taskprivate boolean areSplittersAvailable()
protected static int attemptToOwnTask(boolean isFirstTime, ZKWatcher zkw, ServerName server, String task, int taskZKVersion)
This method is also used to periodically heartbeat the task progress by transitioning the node from OWNED to OWNED.
isFirstTime
- shows whther it's the first attempt.zkw
- zk wathcerserver
- nametask
- to owntaskZKVersion
- version of the task in zkpublic void taskLoop() throws InterruptedException
Synchronization using taskReadySeq
ensures that it will try to grab every task
that has been put up n
taskLoop
in interface SplitLogWorkerCoordination
InterruptedException
- if the SplitLogWorker was stoppedprivate List<String> getTaskList() throws InterruptedException
InterruptedException
public void markCorrupted(org.apache.hadoop.fs.Path rootDir, String name, org.apache.hadoop.fs.FileSystem fs)
SplitLogWorkerCoordination
markCorrupted
in interface SplitLogWorkerCoordination
rootDir
- where to find the logname
- of the logfs
- file systempublic boolean isReady() throws InterruptedException
SplitLogWorkerCoordination
isReady
in interface SplitLogWorkerCoordination
InterruptedException
- if the SplitLogWorker was stoppedpublic int getTaskReadySeq()
SplitLogWorkerCoordination
getTaskReadySeq
in interface SplitLogWorkerCoordination
public void registerListener()
SplitLogWorkerCoordination
registerListener
in interface SplitLogWorkerCoordination
public void removeListener()
SplitLogWorkerCoordination
removeListener
in interface SplitLogWorkerCoordination
public void stopProcessingTasks()
SplitLogWorkerCoordination
stopProcessingTasks
in interface SplitLogWorkerCoordination
public boolean isStop()
SplitLogWorkerCoordination
isStop
in interface SplitLogWorkerCoordination
public void endTask(SplitLogTask slt, LongAdder ctr, SplitLogWorkerCoordination.SplitTaskDetails details)
SplitLogManager
to timeout the task node. nnendTask
in interface SplitLogWorkerCoordination
slt
- See SplitLogTask
ctr
- counter to be updateddetails
- details about log split task (specific to coordination engine being
used).Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.