Class SnapshotProcedure
java.lang.Object
org.apache.hadoop.hbase.procedure2.Procedure<TEnvironment>
org.apache.hadoop.hbase.procedure2.StateMachineProcedure<MasterProcedureEnv,TState>
org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
org.apache.hadoop.hbase.master.procedure.SnapshotProcedure
- All Implemented Interfaces:
Comparable<Procedure<MasterProcedureEnv>>
,TableProcedureInterface
@Private
public class SnapshotProcedure
extends AbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
A procedure used to take snapshot on tables.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure
StateMachineProcedure.Flow
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure
Procedure.LockState
Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface
TableProcedureInterface.TableOperationType
-
Field Summary
Modifier and TypeFieldDescriptionprivate org.apache.hadoop.conf.Configuration
private TableDescriptor
private static final org.slf4j.Logger
private final MetricsSnapshot
private RetryCounter
private org.apache.hadoop.fs.Path
private org.apache.hadoop.fs.FileSystem
private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription
private org.apache.hadoop.fs.Path
private SnapshotManifest
private TableName
private MonitoredTask
private org.apache.hadoop.fs.Path
private org.apache.hadoop.fs.FileSystem
Fields inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure
stateCount
Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure
NO_PROC_ID, NO_TIMEOUT
Fields inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface
DUMMY_NAMESPACE_TABLE_NAME
-
Constructor Summary
ConstructorDescriptionSnapshotProcedure
(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) -
Method Summary
Modifier and TypeMethodDescriptionprotected Procedure.LockState
The user should override this method if they need a lock on an Entity.protected void
Called when the procedure is ready to be added to the queue after the loading/replay operation.private void
private Procedure<MasterProcedureEnv>[]
protected void
deserializeStateData
(ProcedureStateSerializer serializer) Called on store load to allow the user to decode the previously serialized state.protected StateMachineProcedure.Flow
executeFromState
(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) called to perform a single step of the specified 'state' of the procedureprivate Stream<RegionInfo>
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState
Return the initial state object that will be used for the first call to executeFromState().org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription
org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState
getState
(int stateId) Convert an ordinal (or state id) to an Enum (or more descriptive) state object.protected int
getStateId
(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) Convert the Enum (or more descriptive) state object to an ordinal (or state id).Returns the name of the table the procedure is operating onGiven an operation type we can take decisions about what to do with pending operations.protected boolean
Used to keep the procedure lock even when the procedure is yielding or suspended.private boolean
protected boolean
isRollbackSupported
(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.boolean
private TableDescriptor
void
private void
private void
private void
private void
protected void
The user should override this method, and release lock if necessary.protected void
rollbackState
(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) called to perform the rollback of the specified stateprotected void
serializeStateData
(ProcedureStateSerializer serializer) The user-level code of the procedure may have some state to persist (e.g.protected boolean
Called by the ProcedureExecutor when the timeout set by setTimeout() is expired.private void
private void
private void
snapshotSplitOrClosedRegions
(MasterProcedureEnv env, List<RegionInfo> regions, String threadPoolName) private void
void
toStringClassDetails
(StringBuilder builder) Extend the toString() information with the procedure details e.g.private void
Methods inherited from class org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure
checkOnline, checkTableModifiable, getUser, getWALRegionDir, isTableEnabled, preflightChecks, releaseSyncLatch, setUser, waitInitialized
Methods inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure
abort, addChildProcedure, execute, failIfAborted, getCurrentState, getCurrentStateId, getCycles, isEofState, isRollbackSupported, isYieldAfterExecutionStep, isYieldBeforeExecuteFromState, rollback, setNextState, toStringState
Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure
addStackIndex, beforeReplay, bypass, compareTo, completionCleanup, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcedureMetrics, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, removeStackIndex, setAbortFailure, setChildrenLatch, setExecuted, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, shouldWaitClientAck, skipPersistence, suspend, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted
-
Field Details
-
LOG
-
metricsSnapshot
-
conf
-
snapshot
private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot -
rootDir
-
snapshotDir
-
workingDir
-
workingDirFS
-
rootFs
-
snapshotTable
-
status
-
snapshotManifest
-
htd
-
retryCounter
-
-
Constructor Details
-
SnapshotProcedure
public SnapshotProcedure() -
SnapshotProcedure
public SnapshotProcedure(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
-
-
Method Details
-
getTableName
Description copied from interface:TableProcedureInterface
Returns the name of the table the procedure is operating on- Specified by:
getTableName
in interfaceTableProcedureInterface
- Specified by:
getTableName
in classAbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
-
getTableOperationType
Description copied from interface:TableProcedureInterface
Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a delete and we have some table operation pending (e.g. add column) we can abort those operations.- Specified by:
getTableOperationType
in interfaceTableProcedureInterface
- Specified by:
getTableOperationType
in classAbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
- Returns:
- the operation type that the procedure is executing.
-
acquireLock
Description copied from class:Procedure
The user should override this method if they need a lock on an Entity. A lock can be anything, and it is up to the implementor. The Procedure Framework will call this method just before it invokesProcedure.execute(Object)
. It callsProcedure.releaseLock(Object)
after the call to execute. If you need to hold the lock for the life of the Procedure -- i.e. you do not want any other Procedure interfering while this Procedure is running, seeProcedure.holdLock(Object)
. Example: in our Master we can execute request in parallel for different tables. We can create t1 and create t2 and these creates can be executed at the same time. Anything else on t1/t2 is queued waiting that specific table create to happen. There are 3 LockState:- LOCK_ACQUIRED should be returned when the proc has the lock and the proc is ready to execute.
- LOCK_YIELD_WAIT should be returned when the proc has not the lock and the framework should take care of readding the procedure back to the runnable set for retry
- LOCK_EVENT_WAIT should be returned when the proc has not the lock and someone will take care of readding the procedure back to the runnable set when the lock is available.
- Overrides:
acquireLock
in classAbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
- Returns:
- the lock state as described above.
-
releaseLock
Description copied from class:Procedure
The user should override this method, and release lock if necessary.- Overrides:
releaseLock
in classAbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
-
holdLock
Description copied from class:Procedure
Used to keep the procedure lock even when the procedure is yielding or suspended.- Overrides:
holdLock
in classProcedure<MasterProcedureEnv>
- Returns:
- true if the procedure should hold on the lock until completionCleanup()
-
executeFromState
protected StateMachineProcedure.Flow executeFromState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException Description copied from class:StateMachineProcedure
called to perform a single step of the specified 'state' of the procedure- Specified by:
executeFromState
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState> state
- state to execute- Returns:
- Flow.NO_MORE_STATE if the procedure is completed, Flow.HAS_MORE_STATE if there is another step.
- Throws:
ProcedureSuspendedException
ProcedureYieldException
InterruptedException
-
rollbackState
protected void rollbackState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) throws IOException, InterruptedException Description copied from class:StateMachineProcedure
called to perform the rollback of the specified state- Specified by:
rollbackState
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState> state
- state to rollback- Throws:
IOException
- temporary failure, the rollback will retry laterInterruptedException
-
isRollbackSupported
protected boolean isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) Description copied from class:StateMachineProcedure
Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.- Overrides:
isRollbackSupported
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
-
getState
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState getState(int stateId) Description copied from class:StateMachineProcedure
Convert an ordinal (or state id) to an Enum (or more descriptive) state object.- Specified by:
getState
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState> - Parameters:
stateId
- the ordinal() of the state enum (or state id)- Returns:
- the state enum object
-
getStateId
protected int getStateId(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState state) Description copied from class:StateMachineProcedure
Convert the Enum (or more descriptive) state object to an ordinal (or state id).- Specified by:
getStateId
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState> - Parameters:
state
- the state enum object- Returns:
- stateId the ordinal() of the state enum (or state id)
-
getInitialState
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState getInitialState()Description copied from class:StateMachineProcedure
Return the initial state object that will be used for the first call to executeFromState().- Specified by:
getInitialState
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState> - Returns:
- the initial state enum object
-
prepareSnapshot
private void prepareSnapshot(MasterProcedureEnv env) throws ProcedureSuspendedException, IOException -
prepareSnapshotEnv
- Throws:
IOException
-
setTimeoutFailure
Description copied from class:Procedure
Called by the ProcedureExecutor when the timeout set by setTimeout() is expired. Another usage for this method is to implement retrying. A procedure can set the state toWAITING_TIMEOUT
by callingsetState
method, and throw aProcedureSuspendedException
to halt the execution of the procedure, and do not forget a callProcedure.setTimeout(int)
method to set the timeout. And you should also override this method to wake up the procedure, and also return false to tell the ProcedureExecutor that the timeout event has been handled.- Overrides:
setTimeoutFailure
in classProcedure<MasterProcedureEnv>
- Returns:
- true to let the framework handle the timeout as abort, false in case the procedure handled the timeout itself.
-
isAnySplitOrMergeProcedureRunning
-
loadTableDescriptorSnapshot
- Throws:
IOException
-
preSnapshot
- Throws:
IOException
-
postSnapshot
- Throws:
IOException
-
verifySnapshot
- Throws:
IOException
-
completeSnapshot
- Throws:
IOException
-
snapshotSplitRegions
- Throws:
IOException
-
snapshotClosedRegions
- Throws:
IOException
-
getDefaultRegionReplica
-
snapshotSplitOrClosedRegions
private void snapshotSplitOrClosedRegions(MasterProcedureEnv env, List<RegionInfo> regions, String threadPoolName) throws IOException - Throws:
IOException
-
snapshotMobRegion
- Throws:
IOException
-
serializeStateData
Description copied from class:Procedure
The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.- Overrides:
serializeStateData
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState> - Parameters:
serializer
- stores the serializable state- Throws:
IOException
-
deserializeStateData
Description copied from class:Procedure
Called on store load to allow the user to decode the previously serialized state.- Overrides:
deserializeStateData
in classStateMachineProcedure<MasterProcedureEnv,
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState> - Parameters:
serializer
- contains the serialized state- Throws:
IOException
-
createRemoteSnapshotProcedures
-
toStringClassDetails
Description copied from class:Procedure
Extend the toString() information with the procedure details e.g. className and parameters- Overrides:
toStringClassDetails
in classAbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SnapshotState>
- Parameters:
builder
- the string builder to use to append the proc specific information
-
getSnapshotDesc
public org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription getSnapshotDesc() -
afterReplay
Description copied from class:Procedure
Called when the procedure is ready to be added to the queue after the loading/replay operation.- Overrides:
afterReplay
in classProcedure<MasterProcedureEnv>
-
getSnapshot
public org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription getSnapshot() -
markSnapshotCorrupted
- Throws:
IOException
-
isSnapshotCorrupted
- Throws:
IOException
-