org.apache.hadoop.hbase.coordination

Class ZKSplitLogManagerCoordination

java.lang.Object

org.apache.hadoop.hbase.zookeeper.ZKListener

org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination

All Implemented Interfaces:

SplitLogManagerCoordination

@InterfaceAudience.Private
public class ZKSplitLogManagerCoordination
extends ZKListener
implements SplitLogManagerCoordination

ZooKeeper based implementation of SplitLogManagerCoordination

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ZKSplitLogManagerCoordination.CreateAsyncCallback Asynchronous handler for zk create node results.
class	ZKSplitLogManagerCoordination.CreateRescanAsyncCallback Asynchronous handler for zk create RESCAN-node results.
class	ZKSplitLogManagerCoordination.DeleteAsyncCallback Asynchronous handler for zk delete node results.
class	ZKSplitLogManagerCoordination.GetDataAsyncCallback Asynchronous handler for zk get-data-set-watch on node results.
static interface	ZKSplitLogManagerCoordination.TaskFinisher SplitLogManager can use objects implementing this interface to finish off a partially done task by SplitLogWorker.

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coordination.SplitLogManagerCoordination

SplitLogManagerCoordination.SplitLogManagerDetails

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf
static int	DEFAULT_MAX_RESUBMIT
static int	DEFAULT_TIMEOUT
static int	DEFAULT_ZK_RETRIES
(package private) SplitLogManagerCoordination.SplitLogManagerDetails	details
boolean	ignoreZKDeleteForTesting
private static org.slf4j.Logger	LOG
private long	resubmitThreshold
private ZKSplitLogManagerCoordination.TaskFinisher	taskFinisher
private long	timeout
private long	zkretries

Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZKListener

watcher

Constructor Summary

Constructors

Constructor and Description
ZKSplitLogManagerCoordination(org.apache.hadoop.conf.Configuration conf, ZKWatcher watcher)

Method Summary

Modifier and Type	Method and Description
void	checkTasks() tells Coordination that it should check for new tasks
void	checkTaskStillAvailable(String path) Check that the task is still there
private void	createNode(String path, Long retry_count)
private void	createNodeFailure(String path)
private void	createNodeSuccess(String path)
private void	createRescanFailure()
private void	createRescanSuccess(String path)
private void	deleteNode(String path, Long retries)
private void	deleteNodeFailure(String path)
private void	deleteNodeSuccess(String path)
void	deleteTask(String path)
private SplitLogManager.Task	findOrCreateOrphanTask(String path)
private void	getDataSetWatch(String path, Long retry_count)
private void	getDataSetWatchFailure(String path)
private void	getDataSetWatchSuccess(String path, byte[] data, int version)
SplitLogManagerCoordination.SplitLogManagerDetails	getDetails() Returns the configuration that was provided previously
private void	handleUnassignedTask(String path) It is possible for a task to stay in UNASSIGNED state indefinitely - say SplitLogManager wants to resubmit a task.
private void	heartbeat(String path, int new_version, ServerName workerName)
void	init() Support method to init constants such as timeout.
private void	lookForOrphans()
private boolean	needAbandonRetries(int statusCode, String action) Helper function to check whether to abandon retries in ZooKeeper AsyncCallback functions
void	nodeDataChanged(String path) Called when an existing node has changed data.
String	prepareTask(String taskname) Prepare the new task
int	remainingTasksInCoordination() Return the number of remaining tasks
private void	rescan(long retries) signal the workers that a task was resubmitted by creating the RESCAN node.
private boolean	resubmit(String path, int version)
private void	resubmitOrFail(String path, SplitLogManager.ResubmitDirective directive)
boolean	resubmitTask(String path, SplitLogManager.Task task, SplitLogManager.ResubmitDirective directive) Resubmit the task in case if found unassigned or failed
void	setDetails(SplitLogManagerCoordination.SplitLogManagerDetails details) Provide the configuration from the SplitLogManager
private void	setDone(String path, SplitLogManager.TerminationStatus status)
void	setIgnoreDeleteForTesting(boolean b) Temporary function that is used by unit tests only
void	submitTask(String path)

Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZKListener

getWatcher, nodeChildrenChanged, nodeCreated, nodeDeleted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_TIMEOUT

public static final int DEFAULT_TIMEOUT

See Also:

Constant Field Values

DEFAULT_ZK_RETRIES

public static final int DEFAULT_ZK_RETRIES

See Also:

Constant Field Values

DEFAULT_MAX_RESUBMIT

public static final int DEFAULT_MAX_RESUBMIT

See Also:

Constant Field Values

LOG

private static final org.slf4j.Logger LOG

taskFinisher

private final ZKSplitLogManagerCoordination.TaskFinisher taskFinisher

conf

private final org.apache.hadoop.conf.Configuration conf

zkretries

private long zkretries

resubmitThreshold

private long resubmitThreshold

timeout

private long timeout

details

SplitLogManagerCoordination.SplitLogManagerDetails details

ignoreZKDeleteForTesting

public boolean ignoreZKDeleteForTesting

Constructor Detail

ZKSplitLogManagerCoordination

public ZKSplitLogManagerCoordination(org.apache.hadoop.conf.Configuration conf,
                                     ZKWatcher watcher)

Method Detail

init

public void init()
          throws IOException

Description copied from interface: SplitLogManagerCoordination

Support method to init constants such as timeout. Mostly required for UTs. n

Specified by:

init in interface SplitLogManagerCoordination

Throws:

IOException

prepareTask

public String prepareTask(String taskname)

Description copied from interface: SplitLogManagerCoordination

Prepare the new task

Specified by:

prepareTask in interface SplitLogManagerCoordination

Parameters:

taskname - name of the task

Returns:

the task id

remainingTasksInCoordination

public int remainingTasksInCoordination()

Description copied from interface: SplitLogManagerCoordination

Return the number of remaining tasks

Specified by:

remainingTasksInCoordination in interface SplitLogManagerCoordination

handleUnassignedTask

private void handleUnassignedTask(String path)

It is possible for a task to stay in UNASSIGNED state indefinitely - say SplitLogManager wants to resubmit a task. It forces the task to UNASSIGNED state but it dies before it could create the RESCAN task node to signal the SplitLogWorkers to pick up the task. To prevent this scenario the SplitLogManager resubmits all orphan and UNASSIGNED tasks at startup. n

deleteTask

public void deleteTask(String path)

Specified by:

deleteTask in interface SplitLogManagerCoordination

Parameters:

path - to be removed

resubmitTask

public boolean resubmitTask(String path,
                            SplitLogManager.Task task,
                            SplitLogManager.ResubmitDirective directive)

Description copied from interface: SplitLogManagerCoordination

Resubmit the task in case if found unassigned or failed

Specified by:

resubmitTask in interface SplitLogManagerCoordination

Parameters:

path - path related to task

task - to resubmit

directive - whether it should be forced

Returns:

whether it was successful

checkTasks

public void checkTasks()

Description copied from interface: SplitLogManagerCoordination

tells Coordination that it should check for new tasks

Specified by:

checkTasks in interface SplitLogManagerCoordination

rescan

private void rescan(long retries)

signal the workers that a task was resubmitted by creating the RESCAN node.

submitTask

public void submitTask(String path)

Specified by:

submitTask in interface SplitLogManagerCoordination

Parameters:

path - to be submitted

checkTaskStillAvailable

public void checkTaskStillAvailable(String path)

Description copied from interface: SplitLogManagerCoordination

Check that the task is still there

Specified by:

checkTaskStillAvailable in interface SplitLogManagerCoordination

Parameters:

path - node to check

deleteNode

private void deleteNode(String path,
                        Long retries)

deleteNodeSuccess

private void deleteNodeSuccess(String path)

deleteNodeFailure

private void deleteNodeFailure(String path)

createRescanSuccess

private void createRescanSuccess(String path)

createRescanFailure

private void createRescanFailure()

needAbandonRetries

private boolean needAbandonRetries(int statusCode,
                                   String action)

Helper function to check whether to abandon retries in ZooKeeper AsyncCallback functions

Parameters:

statusCode - integer value of a ZooKeeper exception code

action - description message about the retried action

Returns:

true when need to abandon retries otherwise false

createNode

private void createNode(String path,
                        Long retry_count)

createNodeSuccess

private void createNodeSuccess(String path)

createNodeFailure

private void createNodeFailure(String path)

getDataSetWatch

private void getDataSetWatch(String path,
                             Long retry_count)

getDataSetWatchSuccess

private void getDataSetWatchSuccess(String path,
                                    byte[] data,
                                    int version)
                             throws DeserializationException

Throws:

DeserializationException

resubmitOrFail

private void resubmitOrFail(String path,
                            SplitLogManager.ResubmitDirective directive)

getDataSetWatchFailure

private void getDataSetWatchFailure(String path)

setDone

private void setDone(String path,
                     SplitLogManager.TerminationStatus status)

findOrCreateOrphanTask

private SplitLogManager.Task findOrCreateOrphanTask(String path)

heartbeat

private void heartbeat(String path,
                       int new_version,
                       ServerName workerName)

lookForOrphans

private void lookForOrphans()

nodeDataChanged

public void nodeDataChanged(String path)

Description copied from class: ZKListener

Called when an existing node has changed data.

Overrides:

nodeDataChanged in class ZKListener

Parameters:

path - full path of the updated node

resubmit

private boolean resubmit(String path,
                         int version)

setDetails

public void setDetails(SplitLogManagerCoordination.SplitLogManagerDetails details)

Description copied from interface: SplitLogManagerCoordination

Provide the configuration from the SplitLogManager

Specified by:

setDetails in interface SplitLogManagerCoordination

getDetails

public SplitLogManagerCoordination.SplitLogManagerDetails getDetails()

Description copied from interface: SplitLogManagerCoordination

Returns the configuration that was provided previously

Specified by:

getDetails in interface SplitLogManagerCoordination

setIgnoreDeleteForTesting

public void setIgnoreDeleteForTesting(boolean b)

Temporary function that is used by unit tests only