org.apache.hadoop.hbase.master.assignment

Class UnassignProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

org.apache.hadoop.hbase.master.assignment.UnassignProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

@InterfaceAudience.Private
public class UnassignProcedure
extends RegionTransitionProcedure

Procedure that describes the unassignment of a single region. There can only be one RegionTransitionProcedure -- i.e. an assign or an unassign -- per region running at a time, since each procedure takes a lock on the region.

The Unassign starts by placing a "close region" request in the Remote Dispatcher queue, and the procedure will then go into a "waiting state" (suspend). The Remote Dispatcher will batch the various requests for that server and they will be sent to the RS for execution. The RS will complete the open operation by calling master.reportRegionStateTransition(). The AM will intercept the transition report, and notify this procedure. The procedure will wakeup and finish the unassign by publishing its new state on meta.

If we are unable to contact the remote regionserver whether because of ConnectException or socket timeout, we will call expire on the server we were trying to contact. We will remain in suspended state waiting for a wake up from the ServerCrashProcedure that is processing the failed server. The basic idea is that if we notice a crashed server, then we have a responsibility; i.e. we should not let go of the region until we are sure the server that was hosting has had its crash processed. If we let go of the region before then, an assign might run before the logs have been split which would make for data loss.

TODO: Rather than this tricky coordination between SCP and this Procedure, instead, work on returning a SCP as our subprocedure; probably needs work on the framework to do this, especially if the SCP already created.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field and Description
protected ServerName	destinationServer The Server we will subsequently assign the region too (can be null).
protected ServerName	hostingServer Where to send the unassign RPC.
private static org.slf4j.Logger	LOG
private boolean	removeAfterUnassigning Whether deleting the region from in-memory states after unassigning the region.

Fields inherited from class org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

aborted

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
UnassignProcedure()
UnassignProcedure(RegionInfo regionInfo, ServerName hostingServer, boolean force, boolean removeAfterUnassigning)
UnassignProcedure(RegionInfo regionInfo, ServerName hostingServer, ServerName destinationServer, boolean force)
UnassignProcedure(RegionInfo regionInfo, ServerName hostingServer, ServerName destinationServer, boolean override, boolean removeAfterUnassigning)

Method Summary

Modifier and Type	Method and Description
protected void	deserializeStateData(ProcedureStateSerializer serializer) Called on store load to allow the user to decode the previously serialized state.
protected void	finishTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected ProcedureMetrics	getProcedureMetrics(MasterProcedureEnv env) Override this method to provide procedure specific counters for submitted count, failed count and time histogram.
ServerName	getServer(MasterProcedureEnv env) Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.
TableProcedureInterface.TableOperationType	getTableOperationType() Given an operation type we can take decisions about what to do with pending operations.
protected boolean	isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)
private boolean	isSafeToProceed(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, IOException exception) Our remote call failed but there are a few states where it is safe to proceed with the unassign; e.g.
protected void	proceed(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode) Set it up so when procedure is unsuspended, we'll move to the procedure finish.
RemoteProcedureDispatcher.RemoteOperation	remoteCallBuild(MasterProcedureEnv env, ServerName serverName)
protected boolean	remoteCallFailed(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, IOException exception)
protected void	reportTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long seqId)
protected void	serializeStateData(ProcedureStateSerializer serializer) The user-level code of the procedure may have some state to persist (e.g.
protected boolean	startTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
void	toStringClassDetails(StringBuilder sb) Extend the toString() information with the procedure details e.g.
protected boolean	updateTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode) Called when the Procedure is in the REGION_TRANSITION_DISPATCH state.

Methods inherited from class org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

abort, acquireLock, addToRemoteDispatcher, bypass, execute, getAttempt, getRegionInfo, getRegionState, getTableName, getTransitionState, holdLock, isMeta, isOverride, isServerOnline, isServerOnline, releaseLock, remoteCallCompleted, remoteCallFailed, reportTransition, rollback, setAttempt, setOverride, setRegionInfo, setTimeoutFailure, setTransitionState, shouldWaitClientAck, toStringState, waitInitialized

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, compareTo, completionCleanup, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, isYieldAfterExecutionStep, removeStackIndex, setAbortFailure, setChildrenLatch, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, skipPersistence, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

hostingServer

protected volatile ServerName hostingServer

Where to send the unassign RPC. this one may not accurate since another RTP may change this location for the region. The hostingServer will be updated in updateTransition

destinationServer

protected volatile ServerName destinationServer

The Server we will subsequently assign the region too (can be null).

removeAfterUnassigning

private boolean removeAfterUnassigning

Whether deleting the region from in-memory states after unassigning the region.

Constructor Detail

UnassignProcedure

public UnassignProcedure()

UnassignProcedure

public UnassignProcedure(RegionInfo regionInfo,
                         ServerName hostingServer,
                         boolean force,
                         boolean removeAfterUnassigning)

UnassignProcedure

public UnassignProcedure(RegionInfo regionInfo,
                         ServerName hostingServer,
                         ServerName destinationServer,
                         boolean force)

UnassignProcedure

public UnassignProcedure(RegionInfo regionInfo,
                         ServerName hostingServer,
                         ServerName destinationServer,
                         boolean override,
                         boolean removeAfterUnassigning)

Method Detail

getTableOperationType

public TableProcedureInterface.TableOperationType getTableOperationType()

Description copied from interface: TableProcedureInterface

Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a delete and we have some table operation pending (e.g. add column) we can abort those operations.

Returns:

the operation type that the procedure is executing.

isRollbackSupported

protected boolean isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)

Specified by:

isRollbackSupported in class RegionTransitionProcedure

serializeStateData

protected void serializeStateData(ProcedureStateSerializer serializer)
                           throws IOException

Description copied from class: Procedure

The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.

Specified by:

serializeStateData in class Procedure<MasterProcedureEnv>

Parameters:

serializer - stores the serializable state

Throws:

IOException

deserializeStateData

protected void deserializeStateData(ProcedureStateSerializer serializer)
                             throws IOException

Description copied from class: Procedure

Called on store load to allow the user to decode the previously serialized state.

Specified by:

deserializeStateData in class Procedure<MasterProcedureEnv>

Parameters:

serializer - contains the serialized state

Throws:

IOException

startTransition

protected boolean startTransition(MasterProcedureEnv env,
                                  RegionStates.RegionStateNode regionNode)

Specified by:

startTransition in class RegionTransitionProcedure

updateTransition

protected boolean updateTransition(MasterProcedureEnv env,
                                   RegionStates.RegionStateNode regionNode)
                            throws IOException

Description copied from class: RegionTransitionProcedure

Called when the Procedure is in the REGION_TRANSITION_DISPATCH state. In here we do the RPC call to OPEN/CLOSE the region. The suspending of the thread so it sleeps until it gets update that the OPEN/CLOSE has succeeded is complicated. Read the implementations to learn more.

Specified by:

updateTransition in class RegionTransitionProcedure

Throws:

IOException

finishTransition

protected void finishTransition(MasterProcedureEnv env,
                                RegionStates.RegionStateNode regionNode)
                         throws IOException

Specified by:

finishTransition in class RegionTransitionProcedure

Throws:

IOException

remoteCallBuild

public RemoteProcedureDispatcher.RemoteOperation remoteCallBuild(MasterProcedureEnv env,
                                                                 ServerName serverName)

Specified by:

remoteCallBuild in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

Specified by:

remoteCallBuild in class RegionTransitionProcedure

reportTransition

protected void reportTransition(MasterProcedureEnv env,
                                RegionStates.RegionStateNode regionNode,
                                org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                long seqId)
                         throws UnexpectedStateException

Specified by:

reportTransition in class RegionTransitionProcedure

Throws:

UnexpectedStateException

isSafeToProceed

private boolean isSafeToProceed(MasterProcedureEnv env,
                                RegionStates.RegionStateNode regionNode,
                                IOException exception)

Our remote call failed but there are a few states where it is safe to proceed with the unassign; e.g. if a server crash and it has had all of its WALs processed, then we can allow this unassign to go to completion.

Returns:

True if it is safe to proceed with the unassign.

proceed

protected void proceed(MasterProcedureEnv env,
                       RegionStates.RegionStateNode regionNode)

Set it up so when procedure is unsuspended, we'll move to the procedure finish.

remoteCallFailed

protected boolean remoteCallFailed(MasterProcedureEnv env,
                                   RegionStates.RegionStateNode regionNode,
                                   IOException exception)

Specified by:

remoteCallFailed in class RegionTransitionProcedure

Returns:

If true, we will re-wake up this procedure; if false, the procedure stays suspended.

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Description copied from class: Procedure

Extend the toString() information with the procedure details e.g. className and parameters

Overrides:

toStringClassDetails in class RegionTransitionProcedure

Parameters:

sb - the string builder to use to append the proc specific information

getServer

public ServerName getServer(MasterProcedureEnv env)

Description copied from class: RegionTransitionProcedure

Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.

Specified by:

getServer in class RegionTransitionProcedure

Returns:

ServerName the Assign or Unassign is going against.

getProcedureMetrics

protected ProcedureMetrics getProcedureMetrics(MasterProcedureEnv env)

Description copied from class: Procedure

Override this method to provide procedure specific counters for submitted count, failed count and time histogram.

Overrides:

getProcedureMetrics in class Procedure<MasterProcedureEnv>

Parameters:

env - The environment passed to the procedure executor

Returns:

Container object for procedure related metric