org.apache.hadoop.hbase.procedure

Class ProcedureCoordinator

java.lang.Object

org.apache.hadoop.hbase.procedure.ProcedureCoordinator

@InterfaceAudience.Private
public class ProcedureCoordinator
extends Object

This is the master side of a distributed complex procedure execution.

The Procedure is generic and subclassing or customization shouldn't be necessary -- any customization should happen just in Subprocedures.

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static long	KEEP_ALIVE_MILLIS_DEFAULT
private static org.slf4j.Logger	LOG
private ExecutorService	pool
private ConcurrentMap<String,Procedure>	procedures
private ProcedureCoordinatorRpcs	rpcs
(package private) static long	TIMEOUT_MILLIS_DEFAULT
private long	timeoutMillis
(package private) static long	WAKE_MILLIS_DEFAULT
private long	wakeTimeMillis

Constructor Summary

Constructors

Constructor and Description
ProcedureCoordinator(ProcedureCoordinatorRpcs rpcs, ThreadPoolExecutor pool) Create and start a ProcedureCoordinator.
ProcedureCoordinator(ProcedureCoordinatorRpcs rpcs, ThreadPoolExecutor pool, long timeoutMillis, long wakeTimeMillis) Create and start a ProcedureCoordinator.

Method Summary

Modifier and Type	Method and Description
void	abortProcedure(String procName, ForeignException reason) Abort the procedure with the given name
void	close() Shutdown the thread pools and release rpc resources
(package private) Procedure	createProcedure(ForeignExceptionDispatcher fed, String procName, byte[] procArgs, List<String> expectedMembers) Exposed for hooking with unit tests.
static ThreadPoolExecutor	defaultPool(String coordName, int opThreads) Default thread pool for the procedure
static ThreadPoolExecutor	defaultPool(String coordName, int opThreads, long keepAliveMillis) Default thread pool for the procedure
Procedure	getProcedure(String name) Returns the procedure.
Set<String>	getProcedureNames()
(package private) ProcedureCoordinatorRpcs	getRpcs()
(package private) void	memberAcquiredBarrier(String procName, String member) Notification that the procedure had the specified member acquired its part of the barrier via Subprocedure.acquireBarrier().
(package private) void	memberFinishedBarrier(String procName, String member, byte[] dataFromMember) Notification that the procedure had another member finished executing its in-barrier subproc via Subprocedure.insideBarrier().
(package private) void	rpcConnectionFailure(String message, IOException cause) The connection to the rest of the procedure group (members and coordinator) has been broken/lost/failed.
Procedure	startProcedure(ForeignExceptionDispatcher fed, String procName, byte[] procArgs, List<String> expectedMembers) Kick off the named procedure Currently only one procedure with the same type and name is allowed to run at a time.
(package private) boolean	submitProcedure(Procedure proc) Submit an procedure to kick off its dependent subprocedures.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

KEEP_ALIVE_MILLIS_DEFAULT

static final long KEEP_ALIVE_MILLIS_DEFAULT

See Also:

Constant Field Values

TIMEOUT_MILLIS_DEFAULT

static final long TIMEOUT_MILLIS_DEFAULT

See Also:

Constant Field Values

WAKE_MILLIS_DEFAULT

static final long WAKE_MILLIS_DEFAULT

See Also:

Constant Field Values

rpcs

private final ProcedureCoordinatorRpcs rpcs

pool

private final ExecutorService pool

wakeTimeMillis

private final long wakeTimeMillis

timeoutMillis

private final long timeoutMillis

procedures

private final ConcurrentMap<String,Procedure> procedures

Constructor Detail

ProcedureCoordinator

public ProcedureCoordinator(ProcedureCoordinatorRpcs rpcs,
                            ThreadPoolExecutor pool)

Create and start a ProcedureCoordinator. The rpc object registers the ProcedureCoordinator and starts any threads in this constructor.

Parameters:

pool - Used for executing procedures.

ProcedureCoordinator

public ProcedureCoordinator(ProcedureCoordinatorRpcs rpcs,
                            ThreadPoolExecutor pool,
                            long timeoutMillis,
                            long wakeTimeMillis)

Create and start a ProcedureCoordinator. The rpc object registers the ProcedureCoordinator and starts any threads in this constructor.

Parameters:

pool - Used for executing procedures.

Method Detail

defaultPool

public static ThreadPoolExecutor defaultPool(String coordName,
                                             int opThreads)

Default thread pool for the procedure

Parameters:

coordName -

opThreads - the maximum number of threads to allow in the pool

defaultPool

public static ThreadPoolExecutor defaultPool(String coordName,
                                             int opThreads,
                                             long keepAliveMillis)

Default thread pool for the procedure

Parameters:

coordName -

opThreads - the maximum number of threads to allow in the pool

keepAliveMillis - the maximum time (ms) that excess idle threads will wait for new tasks

close

public void close()
           throws IOException

Shutdown the thread pools and release rpc resources

Throws:

IOException

submitProcedure

boolean submitProcedure(Procedure proc)

Submit an procedure to kick off its dependent subprocedures.

Parameters:

proc - Procedure to execute

Returns:

true if the procedure was started correctly, false if the procedure or any subprocedures could not be started. Failure could be due to submitting a procedure multiple times (or one with the same name), or some sort of IO problem. On errors, the procedure's monitor holds a reference to the exception that caused the failure.

rpcConnectionFailure

void rpcConnectionFailure(String message,
                          IOException cause)

The connection to the rest of the procedure group (members and coordinator) has been broken/lost/failed. This should fail any interested procedures, but not attempt to notify other members since we cannot reach them anymore.

Parameters:

message - description of the error

cause - the actual cause of the failure

abortProcedure

public void abortProcedure(String procName,
                           ForeignException reason)

Abort the procedure with the given name

Parameters:

procName - name of the procedure to abort

reason - serialized information about the abort

createProcedure

Procedure createProcedure(ForeignExceptionDispatcher fed,
                          String procName,
                          byte[] procArgs,
                          List<String> expectedMembers)

Exposed for hooking with unit tests.

Parameters:

procName -

procArgs -

expectedMembers -

Returns:

the newly created procedure

startProcedure

public Procedure startProcedure(ForeignExceptionDispatcher fed,
                                String procName,
                                byte[] procArgs,
                                List<String> expectedMembers)

Kick off the named procedure Currently only one procedure with the same type and name is allowed to run at a time.

Parameters:

procName - name of the procedure to start

procArgs - arguments for the procedure

expectedMembers - expected members to start

Returns:

handle to the running procedure, if it was started correctly, null otherwise. Null could be due to submitting a procedure multiple times (or one with the same name), or runtime exception. Check the procedure's monitor that holds a reference to the exception that caused the failure.

memberAcquiredBarrier

void memberAcquiredBarrier(String procName,
                           String member)

Notification that the procedure had the specified member acquired its part of the barrier via Subprocedure.acquireBarrier().

Parameters:

procName - name of the procedure that acquired

member - name of the member that acquired

memberFinishedBarrier

void memberFinishedBarrier(String procName,
                           String member,
                           byte[] dataFromMember)

Notification that the procedure had another member finished executing its in-barrier subproc via Subprocedure.insideBarrier().

Parameters:

procName - name of the subprocedure that finished

member - name of the member that executed and released its barrier

dataFromMember - the data that the member returned along with the notification

getRpcs

ProcedureCoordinatorRpcs getRpcs()

Returns:

the rpcs implementation for all current procedures

getProcedure

public Procedure getProcedure(String name)

Returns the procedure. This Procedure is a live instance so should not be modified but can be inspected.

Parameters:

name - Name of the procedure

Returns:

Procedure or null if not present any more

getProcedureNames

public Set<String> getProcedureNames()

Returns:

Return set of all procedure names.