org.apache.hadoop.hbase.master.assignment

Class RegionTransitionProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

Direct Known Subclasses:

AssignProcedure, UnassignProcedure

@InterfaceAudience.Private
public abstract class RegionTransitionProcedure
extends Procedure<MasterProcedureEnv>
implements TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

Base class for the Assign and Unassign Procedure. Locking: Takes exclusive lock on the region being assigned/unassigned. Thus, there can only be one RegionTransitionProcedure per region running at a time (see MasterProcedureScheduler).

This procedure is asynchronous and responds to external events. The AssignmentManager will notify this procedure when the RS completes the operation and reports the transitioned state (see the Assign and Unassign class for more detail).

Procedures move from the REGION_TRANSITION_QUEUE state when they are first submitted, to the REGION_TRANSITION_DISPATCH state when the request to remote server is sent and the Procedure is suspended waiting on external event to be woken again. Once the external event is triggered, Procedure moves to the REGION_TRANSITION_FINISH state.

NOTE: AssignProcedure and UnassignProcedure should not be thought of as being asymmetric, at least currently.

• AssignProcedure moves through all the above described states and implements methods associated with each while UnassignProcedure starts at state REGION_TRANSITION_DISPATCH and state REGION_TRANSITION_QUEUE is not supported.
• When any step in AssignProcedure fails, failure handler AssignProcedure#handleFailure(MasterProcedureEnv, RegionStateNode) re-attempts the assignment by setting the procedure state to REGION_TRANSITION_QUEUE and forces assignment to a different target server by setting AssignProcedure.forceNewPlan. When the number of attempts reaches threshold configuration 'hbase.assignment.maximum.attempts', the procedure is aborted. For UnassignProcedure, similar re-attempts are intentionally not implemented. It is a 'one shot' procedure. See its class doc for how it handles failure.
• If we find a region in an 'unexpected' state, we'll complain and retry with backoff forever. The 'unexpected' state needs to be fixed either by another running Procedure or by operator intervention (Regions in 'unexpected' state indicates bug or unexpected transition type). For this to work, subclasses need to persist the 'attempt' counter kept in this class when they do serializeStateData and restore it inside their deserializeStateData, just as they do for regionInfo.

TODO: Considering it is a priority doing all we can to get make a region available as soon as possible, re-attempting with any target makes sense if specified target fails in case of AssignProcedure. For UnassignProcedure, our concern is preventing data loss on failed unassign. See class doc for explanation.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicBoolean	aborted
private int	attempt Like regionInfo, the expectation is that subclasses persist the value of this data member.
private static org.slf4j.Logger	LOG
private boolean	override this data member must also be persisted.
private RegionInfo	regionInfo This data member must be persisted.
private org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState	transitionState

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
RegionTransitionProcedure()
RegionTransitionProcedure(RegionInfo regionInfo, boolean override)

Method Summary

Modifier and Type	Method and Description
protected boolean	abort(MasterProcedureEnv env) The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable.
protected Procedure.LockState	acquireLock(MasterProcedureEnv env) The user should override this method if they need a lock on an Entity.
protected boolean	addToRemoteDispatcher(MasterProcedureEnv env, ServerName targetServer) Be careful! At the end of this method, the procedure has either succeeded and this procedure has been set into a suspended state OR, we failed and this procedure has been put back on the scheduler ready for another worker to pick it up.
protected void	bypass(MasterProcedureEnv env) Set the bypass to true.
protected Procedure[]	execute(MasterProcedureEnv env) The main code of the procedure.
protected abstract void	finishTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected int	getAttempt()
RegionInfo	getRegionInfo()
RegionStates.RegionStateNode	getRegionState(MasterProcedureEnv env)
abstract ServerName	getServer(MasterProcedureEnv env) Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.
TableName	getTableName()
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState	getTransitionState()
protected boolean	holdLock(MasterProcedureEnv env) Used to keep the procedure lock even when the procedure is yielding or suspended.
boolean	isMeta()
(package private) boolean	isOverride()
protected abstract boolean	isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)
protected boolean	isServerOnline(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected boolean	isServerOnline(MasterProcedureEnv env, ServerName serverName)
protected void	releaseLock(MasterProcedureEnv env) The user should override this method, and release lock if necessary.
abstract RemoteProcedureDispatcher.RemoteOperation	remoteCallBuild(MasterProcedureEnv env, ServerName serverName)
void	remoteCallCompleted(MasterProcedureEnv env, ServerName serverName, RemoteProcedureDispatcher.RemoteOperation response)
protected abstract boolean	remoteCallFailed(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, IOException exception)
boolean	remoteCallFailed(MasterProcedureEnv env, ServerName serverName, IOException exception)
protected abstract void	reportTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long seqId)
protected void	reportTransition(MasterProcedureEnv env, ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long seqId)
protected void	rollback(MasterProcedureEnv env) The code to undo what was done by the execute() code.
protected void	setAttempt(int attempt) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected void	setOverride(boolean override) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected void	setRegionInfo(RegionInfo regionInfo) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected boolean	setTimeoutFailure(MasterProcedureEnv env) At end of timeout, wake ourselves up so we run again.
(package private) void	setTransitionState(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)
protected boolean	shouldWaitClientAck(MasterProcedureEnv env) By default, the executor will keep the procedure result around util the eviction TTL is expired.
protected abstract boolean	startTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
void	toStringClassDetails(StringBuilder sb) Extend the toString() information with the procedure details e.g.
protected void	toStringState(StringBuilder builder) Called from Procedure.toString() when interpolating Procedure State.
protected abstract boolean	updateTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode) Called when the Procedure is in the REGION_TRANSITION_DISPATCH state.
protected boolean	waitInitialized(MasterProcedureEnv env) The Procedure.doAcquireLock(Object, ProcedureStore) will be split into two steps, first, it will call us to determine whether we need to wait for initialization, second, it will call Procedure.acquireLock(Object) to actually handle the lock for this procedure.

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, compareTo, completionCleanup, deserializeStateData, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcedureMetrics, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, isYieldAfterExecutionStep, removeStackIndex, serializeStateData, setAbortFailure, setChildrenLatch, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, skipPersistence, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

getTableOperationType

Field Detail

LOG

private static final org.slf4j.Logger LOG

aborted

protected final AtomicBoolean aborted

transitionState

private org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState transitionState

regionInfo

private RegionInfo regionInfo

This data member must be persisted. Expectation is that it is done by subclasses in their Procedure.serializeStateData(ProcedureStateSerializer) call, restoring regionInfo in their Procedure.deserializeStateData(ProcedureStateSerializer) method.

override

private boolean override

this data member must also be persisted.

See Also:

regionInfo

attempt

private int attempt

Like regionInfo, the expectation is that subclasses persist the value of this data member. It is used doing backoff when Procedure gets stuck.

Constructor Detail

RegionTransitionProcedure

public RegionTransitionProcedure()

RegionTransitionProcedure

public RegionTransitionProcedure(RegionInfo regionInfo,
                                 boolean override)

Method Detail

getRegionInfo

public RegionInfo getRegionInfo()

setRegionInfo

protected void setRegionInfo(RegionInfo regionInfo)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method. Expectation is that subclasses will persist `regioninfo` in their Procedure.serializeStateData(ProcedureStateSerializer) method and then restore `regionInfo` on deserialization by calling this.

setOverride

protected void setOverride(boolean override)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method. Expectation is that subclasses will persist `override` in their Procedure.serializeStateData(ProcedureStateSerializer) method and then restore `override` on deserialization by calling this.

setAttempt

protected void setAttempt(int attempt)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.

See Also:

setRegionInfo(RegionInfo)

getAttempt

protected int getAttempt()

getTableName

public TableName getTableName()

Specified by:

getTableName in interface TableProcedureInterface

Returns:

the name of the table the procedure is operating on

isMeta

public boolean isMeta()

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Description copied from class: Procedure

Extend the toString() information with the procedure details e.g. className and parameters

Overrides:

toStringClassDetails in class Procedure<MasterProcedureEnv>

Parameters:

sb - the string builder to use to append the proc specific information

getRegionState

public RegionStates.RegionStateNode getRegionState(MasterProcedureEnv env)

setTransitionState

void setTransitionState(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)

getTransitionState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState getTransitionState()

startTransition

protected abstract boolean startTransition(MasterProcedureEnv env,
                                           RegionStates.RegionStateNode regionNode)
                                    throws IOException,
                                           ProcedureSuspendedException

Throws:

IOException

ProcedureSuspendedException

updateTransition

protected abstract boolean updateTransition(MasterProcedureEnv env,
                                            RegionStates.RegionStateNode regionNode)
                                     throws IOException,
                                            ProcedureSuspendedException

Called when the Procedure is in the REGION_TRANSITION_DISPATCH state. In here we do the RPC call to OPEN/CLOSE the region. The suspending of the thread so it sleeps until it gets update that the OPEN/CLOSE has succeeded is complicated. Read the implementations to learn more.

Throws:

IOException

ProcedureSuspendedException

finishTransition

protected abstract void finishTransition(MasterProcedureEnv env,
                                         RegionStates.RegionStateNode regionNode)
                                  throws IOException,
                                         ProcedureSuspendedException

Throws:

IOException

ProcedureSuspendedException

reportTransition

protected abstract void reportTransition(MasterProcedureEnv env,
                                         RegionStates.RegionStateNode regionNode,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                         long seqId)
                                  throws UnexpectedStateException

Throws:

UnexpectedStateException

remoteCallBuild

public abstract RemoteProcedureDispatcher.RemoteOperation remoteCallBuild(MasterProcedureEnv env,
                                                                          ServerName serverName)

Specified by:

remoteCallBuild in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

remoteCallFailed

protected abstract boolean remoteCallFailed(MasterProcedureEnv env,
                                            RegionStates.RegionStateNode regionNode,
                                            IOException exception)

Returns:

True if processing of fail is complete; the procedure will be woken from its suspend and we'll go back to running through procedure steps: otherwise if false we leave the procedure in suspended state.

remoteCallCompleted

public void remoteCallCompleted(MasterProcedureEnv env,
                                ServerName serverName,
                                RemoteProcedureDispatcher.RemoteOperation response)

Specified by:

remoteCallCompleted in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

remoteCallFailed

public boolean remoteCallFailed(MasterProcedureEnv env,
                                ServerName serverName,
                                IOException exception)

Specified by:

remoteCallFailed in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

addToRemoteDispatcher

protected boolean addToRemoteDispatcher(MasterProcedureEnv env,
                                        ServerName targetServer)

Be careful! At the end of this method, the procedure has either succeeded and this procedure has been set into a suspended state OR, we failed and this procedure has been put back on the scheduler ready for another worker to pick it up. In both cases, we need to exit the current Worker processing immediately!

Returns:

True if we successfully dispatched the call and false if we failed; if failed, we need to roll back any setup done for the dispatch.

reportTransition

protected void reportTransition(MasterProcedureEnv env,
                                ServerName serverName,
                                org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                long seqId)
                         throws UnexpectedStateException

Throws:

UnexpectedStateException

isServerOnline

protected boolean isServerOnline(MasterProcedureEnv env,
                                 RegionStates.RegionStateNode regionNode)

isServerOnline

protected boolean isServerOnline(MasterProcedureEnv env,
                                 ServerName serverName)

toStringState

protected void toStringState(StringBuilder builder)

Description copied from class: Procedure

Called from Procedure.toString() when interpolating Procedure State. Allows decorating generic Procedure State with Procedure particulars.

Overrides:

toStringState in class Procedure<MasterProcedureEnv>

Parameters:

builder - Append current ProcedureProtos.ProcedureState

execute

protected Procedure[] execute(MasterProcedureEnv env)
                       throws ProcedureSuspendedException

Description copied from class: Procedure

The main code of the procedure. It must be idempotent since execute() may be called multiple times in case of machine failure in the middle of the execution.

Specified by:

execute in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

a set of sub-procedures to run or ourselves if there is more work to do or null if the procedure is done.

Throws:

ProcedureSuspendedException - Signal to the executor that Procedure has suspended itself and has set itself up waiting for an external event to wake it back up again.

setTimeoutFailure

protected boolean setTimeoutFailure(MasterProcedureEnv env)

At end of timeout, wake ourselves up so we run again.

Overrides:

setTimeoutFailure in class Procedure<MasterProcedureEnv>

Returns:

true to let the framework handle the timeout as abort, false in case the procedure handled the timeout itself.

rollback

protected void rollback(MasterProcedureEnv env)

Description copied from class: Procedure

The code to undo what was done by the execute() code. It is called when the procedure or one of the sub-procedures failed or an abort was requested. It should cleanup all the resources created by the execute() call. The implementation must be idempotent since rollback() may be called multiple time in case of machine failure in the middle of the execution.

Specified by:

rollback in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

isRollbackSupported

protected abstract boolean isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)

abort

protected boolean abort(MasterProcedureEnv env)

Description copied from class: Procedure

The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable. The simplest implementation is to have an AtomicBoolean set in the abort() method and then the execute() will check if the abort flag is set or not. abort() may be called multiple times from the client, so the implementation must be idempotent.

NOTE: abort() is not like Thread.interrupt(). It is just a notification that allows the procedure implementor abort.

Specified by:

abort in class Procedure<MasterProcedureEnv>

waitInitialized

protected boolean waitInitialized(MasterProcedureEnv env)

Description copied from class: Procedure

The Procedure.doAcquireLock(Object, ProcedureStore) will be split into two steps, first, it will call us to determine whether we need to wait for initialization, second, it will call Procedure.acquireLock(Object) to actually handle the lock for this procedure.

This is because that when master restarts, we need to restore the lock state for all the procedures to not break the semantic if Procedure.holdLock(Object) is true. But the ProcedureExecutor will be started before the master finish initialization(as it is part of the initialization!), so we need to split the code into two steps, and when restore, we just restore the lock part and ignore the waitInitialized part. Otherwise there will be dead lock.

Overrides:

waitInitialized in class Procedure<MasterProcedureEnv>

Returns:

true means we need to wait until the environment has been initialized, otherwise true.

acquireLock

protected Procedure.LockState acquireLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method if they need a lock on an Entity. A lock can be anything, and it is up to the implementor. The Procedure Framework will call this method just before it invokes Procedure.execute(Object). It calls Procedure.releaseLock(Object) after the call to execute.

If you need to hold the lock for the life of the Procedure -- i.e. you do not want any other Procedure interfering while this Procedure is running, see Procedure.holdLock(Object).

Example: in our Master we can execute request in parallel for different tables. We can create t1 and create t2 and these creates can be executed at the same time. Anything else on t1/t2 is queued waiting that specific table create to happen.

There are 3 LockState:

• LOCK_ACQUIRED should be returned when the proc has the lock and the proc is ready to execute.
• LOCK_YIELD_WAIT should be returned when the proc has not the lock and the framework should take care of readding the procedure back to the runnable set for retry
• LOCK_EVENT_WAIT should be returned when the proc has not the lock and someone will take care of readding the procedure back to the runnable set when the lock is available.

Overrides:

acquireLock in class Procedure<MasterProcedureEnv>

Returns:

the lock state as described above.

releaseLock

protected void releaseLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method, and release lock if necessary.

Overrides:

releaseLock in class Procedure<MasterProcedureEnv>

holdLock

protected boolean holdLock(MasterProcedureEnv env)

Description copied from class: Procedure

Used to keep the procedure lock even when the procedure is yielding or suspended.

Overrides:

holdLock in class Procedure<MasterProcedureEnv>

Returns:

true if the procedure should hold on the lock until completionCleanup()

shouldWaitClientAck

protected boolean shouldWaitClientAck(MasterProcedureEnv env)

Description copied from class: Procedure

By default, the executor will keep the procedure result around util the eviction TTL is expired. The client can cut down the waiting time by requesting that the result is removed from the executor. In case of system started procedure, we can force the executor to auto-ack.

Overrides:

shouldWaitClientAck in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

true if the executor should wait the client ack for the result. Defaults to return true.

getServer

public abstract ServerName getServer(MasterProcedureEnv env)

Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.

Returns:

ServerName the Assign or Unassign is going against.

bypass

protected void bypass(MasterProcedureEnv env)

Description copied from class: Procedure

Set the bypass to true. Only called in ProcedureExecutor.bypassProcedure(long, long, boolean, boolean) for now. DO NOT use this method alone, since we can't just bypass one single procedure. We need to bypass its ancestor too. If your Procedure has set state, it needs to undo it in here.

Overrides:

bypass in class Procedure<MasterProcedureEnv>

Parameters:

env - Current environment. May be null because of context; e.g. pretty-printing procedure WALs where there is no 'environment' (and where Procedures that require an 'environment' won't be run.

isOverride

boolean isOverride()