org.apache.hadoop.hbase.master.assignment

Class AssignProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

org.apache.hadoop.hbase.master.assignment.AssignProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

@InterfaceAudience.Private
public class AssignProcedure
extends RegionTransitionProcedure

Procedure that describe the assignment of a single region. There can only be one RegionTransitionProcedure per region running at a time since each procedure takes a lock on the region.

The Assign starts by pushing the "assign" operation to the AssignmentManager and then will go in a "waiting" state. The AM will batch the "assign" requests and ask the Balancer where to put the region (the various policies will be respected: retain, round-robin, random). Once the AM and the balancer have found a place for the region the procedure will be resumed and an "open region" request will be placed in the Remote Dispatcher queue, and the procedure once again will go in a "waiting state". The Remote Dispatcher will batch the various requests for that server and they will be sent to the RS for execution. The RS will complete the open operation by calling master.reportRegionStateTransition(). The AM will intercept the transition report, and notify the procedure. The procedure will finish the assignment by publishing to new state on meta or it will retry the assignment.

This procedure does not rollback when beyond the first REGION_TRANSITION_QUEUE step; it will press on trying to assign in the face of failure. Should we ignore rollback calls to Assign/Unassign then? Or just remove rollback here?

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AssignProcedure.CompareAssignProcedure Sort AssignProcedures such that meta and system assigns come first before user-space assigns.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field and Description
static AssignProcedure.CompareAssignProcedure	COMPARATOR Comparator that will sort AssignProcedures so meta assigns come first, then system table assigns and finally user space assigns.
private boolean	forceNewPlan Set to true when we need recalibrate -- choose a new target -- because original assign failed.
private static org.slf4j.Logger	LOG
protected ServerName	targetServer Gets set as desired target on move, merge, etc., when we want to go to a particular server.

Fields inherited from class org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

aborted

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
AssignProcedure()
AssignProcedure(RegionInfo regionInfo)
AssignProcedure(RegionInfo regionInfo, ServerName destinationServer)
AssignProcedure(RegionInfo regionInfo, ServerName destinationServer, boolean override)

Method Summary

Modifier and Type	Method and Description
static boolean	assign(MasterServices masterServices, RegionInfo ri) Used by ServerCrashProcedure too skip creating Assigns if not needed.
protected static boolean	assign(MasterServices masterServices, RegionStates.RegionStateNode regionNode)
protected void	deserializeStateData(ProcedureStateSerializer serializer) Called on store load to allow the user to decode the previously serialized state.
protected void	finishTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected ProcedureMetrics	getProcedureMetrics(MasterProcedureEnv env) Override this method to provide procedure specific counters for submitted count, failed count and time histogram.
ServerName	getServer(MasterProcedureEnv env) Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.
TableProcedureInterface.TableOperationType	getTableOperationType() Given an operation type we can take decisions about what to do with pending operations.
private void	handleFailure(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode) Called when dispatch or subsequent OPEN request fail.
private boolean	incrementAndCheckMaxAttempts(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected boolean	isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)
RemoteProcedureDispatcher.RemoteOperation	remoteCallBuild(MasterProcedureEnv env, ServerName serverName)
protected boolean	remoteCallFailed(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, IOException exception)
protected void	reportTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long openSeqNum)
protected void	serializeStateData(ProcedureStateSerializer serializer) The user-level code of the procedure may have some state to persist (e.g.
protected boolean	startTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
void	toStringClassDetails(StringBuilder sb) Extend the toString() information with the procedure details e.g.
protected boolean	updateTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode) Called when the Procedure is in the REGION_TRANSITION_DISPATCH state.

Methods inherited from class org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

abort, acquireLock, addToRemoteDispatcher, bypass, execute, getAttempt, getRegionInfo, getRegionState, getTableName, getTransitionState, holdLock, isMeta, isOverride, isServerOnline, isServerOnline, releaseLock, remoteCallCompleted, remoteCallFailed, reportTransition, rollback, setAttempt, setOverride, setRegionInfo, setTimeoutFailure, setTransitionState, shouldWaitClientAck, toStringState, waitInitialized

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, compareTo, completionCleanup, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, isYieldAfterExecutionStep, removeStackIndex, setAbortFailure, setChildrenLatch, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, skipPersistence, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

forceNewPlan

private boolean forceNewPlan

Set to true when we need recalibrate -- choose a new target -- because original assign failed.

targetServer

protected volatile ServerName targetServer

Gets set as desired target on move, merge, etc., when we want to go to a particular server. We may not be able to respect this request but will try. When it is NOT set, then we ask the balancer to assign. This value is used below in startTransition to set regionLocation if non-null. Setting regionLocation in regionServerNode is how we override balancer setting destination.

COMPARATOR

public static final AssignProcedure.CompareAssignProcedure COMPARATOR

Comparator that will sort AssignProcedures so meta assigns come first, then system table assigns and finally user space assigns.

Constructor Detail

AssignProcedure

public AssignProcedure()

AssignProcedure

public AssignProcedure(RegionInfo regionInfo)

AssignProcedure

public AssignProcedure(RegionInfo regionInfo,
                       ServerName destinationServer)

AssignProcedure

public AssignProcedure(RegionInfo regionInfo,
                       ServerName destinationServer,
                       boolean override)

Method Detail

getTableOperationType

public TableProcedureInterface.TableOperationType getTableOperationType()

Description copied from interface: TableProcedureInterface

Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a delete and we have some table operation pending (e.g. add column) we can abort those operations.

Returns:

the operation type that the procedure is executing.

isRollbackSupported

protected boolean isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)

Specified by:

isRollbackSupported in class RegionTransitionProcedure

serializeStateData

protected void serializeStateData(ProcedureStateSerializer serializer)
                           throws IOException

Description copied from class: Procedure

The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.

Specified by:

serializeStateData in class Procedure<MasterProcedureEnv>

Parameters:

serializer - stores the serializable state

Throws:

IOException

deserializeStateData

protected void deserializeStateData(ProcedureStateSerializer serializer)
                             throws IOException

Description copied from class: Procedure

Called on store load to allow the user to decode the previously serialized state.

Specified by:

deserializeStateData in class Procedure<MasterProcedureEnv>

Parameters:

serializer - contains the serialized state

Throws:

IOException

assign

public static boolean assign(MasterServices masterServices,
                             RegionInfo ri)

Used by ServerCrashProcedure too skip creating Assigns if not needed.

Returns:

Skip out on the assign; returns 'true'/assign if exception.

assign

protected static boolean assign(MasterServices masterServices,
                                RegionStates.RegionStateNode regionNode)
                         throws IOException

Throws:

IOException

startTransition

protected boolean startTransition(MasterProcedureEnv env,
                                  RegionStates.RegionStateNode regionNode)
                           throws IOException

Specified by:

startTransition in class RegionTransitionProcedure

Throws:

IOException

updateTransition

protected boolean updateTransition(MasterProcedureEnv env,
                                   RegionStates.RegionStateNode regionNode)
                            throws IOException,
                                   ProcedureSuspendedException

Description copied from class: RegionTransitionProcedure

Called when the Procedure is in the REGION_TRANSITION_DISPATCH state. In here we do the RPC call to OPEN/CLOSE the region. The suspending of the thread so it sleeps until it gets update that the OPEN/CLOSE has succeeded is complicated. Read the implementations to learn more.

Specified by:

updateTransition in class RegionTransitionProcedure

Throws:

IOException

ProcedureSuspendedException

finishTransition

protected void finishTransition(MasterProcedureEnv env,
                                RegionStates.RegionStateNode regionNode)
                         throws IOException

Specified by:

finishTransition in class RegionTransitionProcedure

Throws:

IOException

reportTransition

protected void reportTransition(MasterProcedureEnv env,
                                RegionStates.RegionStateNode regionNode,
                                org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                long openSeqNum)
                         throws UnexpectedStateException

Specified by:

reportTransition in class RegionTransitionProcedure

Throws:

UnexpectedStateException

handleFailure

private void handleFailure(MasterProcedureEnv env,
                           RegionStates.RegionStateNode regionNode)

Called when dispatch or subsequent OPEN request fail. Can be run by the inline dispatch call or later by the ServerCrashProcedure. Our state is generally OPENING. Cleanup and reset to OFFLINE and put our Procedure State back to REGION_TRANSITION_QUEUE so the Assign starts over.

incrementAndCheckMaxAttempts

private boolean incrementAndCheckMaxAttempts(MasterProcedureEnv env,
                                             RegionStates.RegionStateNode regionNode)

remoteCallBuild

public RemoteProcedureDispatcher.RemoteOperation remoteCallBuild(MasterProcedureEnv env,
                                                                 ServerName serverName)

Specified by:

remoteCallBuild in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

Specified by:

remoteCallBuild in class RegionTransitionProcedure

remoteCallFailed

protected boolean remoteCallFailed(MasterProcedureEnv env,
                                   RegionStates.RegionStateNode regionNode,
                                   IOException exception)

Specified by:

remoteCallFailed in class RegionTransitionProcedure

Returns:

True if processing of fail is complete; the procedure will be woken from its suspend and we'll go back to running through procedure steps: otherwise if false we leave the procedure in suspended state.

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Description copied from class: Procedure

Extend the toString() information with the procedure details e.g. className and parameters

Overrides:

toStringClassDetails in class RegionTransitionProcedure

Parameters:

sb - the string builder to use to append the proc specific information

getServer

public ServerName getServer(MasterProcedureEnv env)

Description copied from class: RegionTransitionProcedure

Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.

Specified by:

getServer in class RegionTransitionProcedure

Returns:

ServerName the Assign or Unassign is going against.

getProcedureMetrics

protected ProcedureMetrics getProcedureMetrics(MasterProcedureEnv env)

Description copied from class: Procedure

Override this method to provide procedure specific counters for submitted count, failed count and time histogram.

Overrides:

getProcedureMetrics in class Procedure<MasterProcedureEnv>

Parameters:

env - The environment passed to the procedure executor

Returns:

Container object for procedure related metric