Package org.apache.hadoop.hbase.master.procedure

Class RefreshMetaProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<TEnvironment>

org.apache.hadoop.hbase.procedure2.StateMachineProcedure<MasterProcedureEnv,TState>

org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

org.apache.hadoop.hbase.master.procedure.RefreshMetaProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface

@Private
public class RefreshMetaProcedure
extends AbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

StateMachineProcedure.Flow

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field	Description
private List<RegionInfo>	currentRegions
private List<TableName>	deletedTables
private static final String	HIDDEN_DIR_PATTERN
private List<RegionInfo>	latestRegions
private static final org.slf4j.Logger	LOG
private static final int	MUTATION_BATCH_SIZE
private List<RegionInfo>	newlyAddedRegions
private List<Mutation>	pendingMutations
private RetryCounter	retryCounter

Fields inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

stateCount

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Fields inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

DUMMY_NAMESPACE_TABLE_NAME

Constructor Summary

Constructors

Constructor	Description
RefreshMetaProcedure()
RefreshMetaProcedure(MasterProcedureEnv env)

Method Summary

Modifier and Type	Method	Description
private void	applyMutations(Connection connection, List<Mutation> mutations)
protected void	deserializeStateData(ProcedureStateSerializer serializer)	Called on store load to allow the user to decode the previously serialized state.
private StateMachineProcedure.Flow	executeApply(MasterProcedureEnv env)
private StateMachineProcedure.Flow	executeFinish(MasterProcedureEnv env)
private StateMachineProcedure.Flow	executeFollowup(MasterProcedureEnv env)
protected StateMachineProcedure.Flow	executeFromState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState refreshMetaState)	called to perform a single step of the specified 'state' of the procedure
private StateMachineProcedure.Flow	executeInit(MasterProcedureEnv env)
private StateMachineProcedure.Flow	executePrepare()
private StateMachineProcedure.Flow	executeScanStorage(MasterProcedureEnv env)
(package private) List<RegionInfo>	getCurrentRegions(Connection connection)	Retrieves the current regions from the hbase:meta table.
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState	getInitialState()	Return the initial state object that will be used for the first call to executeFromState().
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState	getState(int stateId)	Convert an ordinal (or state id) to an Enum (or more descriptive) state object.
protected int	getStateId(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState refreshMetaState)	Convert the Enum (or more descriptive) state object to an ordinal (or state id).
TableName	getTableName()	Returns the name of the table the procedure is operating on
TableProcedureInterface.TableOperationType	getTableOperationType()	Given an operation type we can take decisions about what to do with pending operations.
(package private) boolean	hasBoundaryChanged(RegionInfo region1, RegionInfo region2)
private void	invalidateTableDescriptorCache(MasterProcedureEnv env)
private boolean	isValidRegionInfo(RegionInfo regionInfo, String expectedEncodedName)
private List<Mutation>	prepareMutations(Map<String,RegionInfo> currentMap, Map<String,RegionInfo> latestMap)	Prepares mutations by comparing the current regions in hbase:meta with the latest regions from backing storage.
private RegionInfo	readRegionInfo(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path regionInfoPath)
protected void	rollbackState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState refreshMetaState)	called to perform the rollback of the specified state
(package private) List<RegionInfo>	scanBackingStorage(Connection connection)	Scans the backing storage for all regions and returns a list of RegionInfo objects.
private List<RegionInfo>	scanRegionsInTable(org.apache.hadoop.fs.FileSystem fs, List<org.apache.hadoop.fs.Path> regionDirs)
private List<RegionInfo>	scanTablesInNamespace(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path namespacePath)
protected void	serializeStateData(ProcedureStateSerializer serializer)	The user-level code of the procedure may have some state to persist (e.g.
protected boolean	setTimeoutFailure(MasterProcedureEnv env)	Called by the ProcedureExecutor when the timeout set by setTimeout() is expired.

Methods inherited from class org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure

acquireLock, checkOnline, checkTableModifiable, getUser, getWALRegionDir, isTableEnabled, isTableModificationInProgress, preflightChecks, releaseLock, releaseSyncLatch, setUser, toStringClassDetails, waitInitialized

Methods inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

abort, addChildProcedure, execute, failIfAborted, getCurrentState, getCurrentStateId, getCycles, isEofState, isRollbackSupported, isRollbackSupported, isYieldAfterExecutionStep, isYieldBeforeExecuteFromState, rollback, setNextState, toStringState

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterExec, afterReplay, beforeExec, beforeReplay, bypass, compareTo, completionCleanup, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcedureMetrics, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, holdLock, incChildrenLatch, isBypass, isCriticalSystemTable, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, removeStackIndex, setAbortFailure, setChildrenLatch, setCriticalSystemTable, setExecuted, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, shouldWaitClientAck, skipPersistence, suspend, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

HIDDEN_DIR_PATTERN

private static final String HIDDEN_DIR_PATTERN

See Also:

• Constant Field Values

currentRegions

private List<RegionInfo> currentRegions

latestRegions

private List<RegionInfo> latestRegions

pendingMutations

private List<Mutation> pendingMutations

retryCounter

private RetryCounter retryCounter

MUTATION_BATCH_SIZE

private static final int MUTATION_BATCH_SIZE

See Also:

• Constant Field Values

newlyAddedRegions

private List<RegionInfo> newlyAddedRegions

deletedTables

private List<TableName> deletedTables

Constructor Details

RefreshMetaProcedure

public RefreshMetaProcedure()

RefreshMetaProcedure

public RefreshMetaProcedure(MasterProcedureEnv env)

Method Details

getTableName

public TableName getTableName()

Description copied from interface: TableProcedureInterface

Returns the name of the table the procedure is operating on

Specified by:

getTableName in interface TableProcedureInterface

Specified by:

getTableName in class AbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

getTableOperationType

public TableProcedureInterface.TableOperationType getTableOperationType()

Description copied from interface: TableProcedureInterface

Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a delete and we have some table operation pending (e.g. add column) we can abort those operations.

Specified by:

getTableOperationType in interface TableProcedureInterface

Specified by:

getTableOperationType in class AbstractStateMachineTableProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

Returns:

the operation type that the procedure is executing.

executeFromState

protected StateMachineProcedure.Flow executeFromState(MasterProcedureEnv env,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState refreshMetaState)

Description copied from class: StateMachineProcedure

called to perform a single step of the specified 'state' of the procedure

Specified by:

executeFromState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

refreshMetaState - state to execute

Returns:

Flow.NO_MORE_STATE if the procedure is completed, Flow.HAS_MORE_STATE if there is another step.

executeInit

private StateMachineProcedure.Flow executeInit(MasterProcedureEnv env)
                                        throws IOException

Throws:

IOException

executeScanStorage

private StateMachineProcedure.Flow executeScanStorage(MasterProcedureEnv env)
                                               throws IOException

Throws:

IOException

executePrepare

private StateMachineProcedure.Flow executePrepare()
                                           throws IOException

Throws:

IOException

executeApply

private StateMachineProcedure.Flow executeApply(MasterProcedureEnv env)
                                         throws ProcedureSuspendedException

Throws:

ProcedureSuspendedException

executeFollowup

private StateMachineProcedure.Flow executeFollowup(MasterProcedureEnv env)
                                            throws IOException

Throws:

IOException

executeFinish

private StateMachineProcedure.Flow executeFinish(MasterProcedureEnv env)

invalidateTableDescriptorCache

private void invalidateTableDescriptorCache(MasterProcedureEnv env)

prepareMutations

private List<Mutation> prepareMutations(Map<String,RegionInfo> currentMap,
 Map<String,RegionInfo> latestMap)
                                 throws IOException

Prepares mutations by comparing the current regions in hbase:meta with the latest regions from backing storage. Also populates newlyAddedRegions and deletedTables lists for follow-up actions.

Parameters:

currentMap - Current regions from hbase:meta

latestMap - Latest regions from backing storage

Returns:

List of mutations to apply to the meta table

Throws:

IOException - If there is an error creating mutations

applyMutations

private void applyMutations(Connection connection,
 List<Mutation> mutations)
                     throws IOException

Throws:

IOException

hasBoundaryChanged

boolean hasBoundaryChanged(RegionInfo region1,
 RegionInfo region2)

scanBackingStorage

List<RegionInfo> scanBackingStorage(Connection connection)
                             throws IOException

Scans the backing storage for all regions and returns a list of RegionInfo objects. This method scans the filesystem for region directories and reads their .regioninfo files.

Parameters:

connection - The HBase connection to use.

Returns:

List of RegionInfo objects found in the backing storage.

Throws:

IOException - If there is an error accessing the filesystem or reading region info files.

scanTablesInNamespace

private List<RegionInfo> scanTablesInNamespace(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path namespacePath)
                                        throws IOException

Throws:

IOException

scanRegionsInTable

private List<RegionInfo> scanRegionsInTable(org.apache.hadoop.fs.FileSystem fs,
 List<org.apache.hadoop.fs.Path> regionDirs)
                                     throws IOException

Throws:

IOException

isValidRegionInfo

private boolean isValidRegionInfo(RegionInfo regionInfo,
 String expectedEncodedName)

readRegionInfo

private RegionInfo readRegionInfo(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path regionInfoPath)

getCurrentRegions

List<RegionInfo> getCurrentRegions(Connection connection)
                            throws IOException

Retrieves the current regions from the hbase:meta table.

Parameters:

connection - The HBase connection to use.

Returns:

List of RegionInfo objects representing the current regions in meta.

Throws:

IOException - If there is an error accessing the meta table.

setTimeoutFailure

protected boolean setTimeoutFailure(MasterProcedureEnv env)

Description copied from class: Procedure

Called by the ProcedureExecutor when the timeout set by setTimeout() is expired.

Another usage for this method is to implement retrying. A procedure can set the state to WAITING_TIMEOUT by calling setState method, and throw a ProcedureSuspendedException to halt the execution of the procedure, and do not forget a call Procedure.setTimeout(int) method to set the timeout. And you should also override this method to wake up the procedure, and also return false to tell the ProcedureExecutor that the timeout event has been handled.

Overrides:

setTimeoutFailure in class Procedure<MasterProcedureEnv>

Returns:

true to let the framework handle the timeout as abort, false in case the procedure handled the timeout itself.

rollbackState

protected void rollbackState(MasterProcedureEnv env,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState refreshMetaState)
                      throws IOException,
InterruptedException

Description copied from class: StateMachineProcedure

called to perform the rollback of the specified state

Specified by:

rollbackState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

refreshMetaState - state to rollback

Throws:

IOException - temporary failure, the rollback will retry later

InterruptedException

getState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState getState(int stateId)

Description copied from class: StateMachineProcedure

Convert an ordinal (or state id) to an Enum (or more descriptive) state object.

Specified by:

getState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

Parameters:

stateId - the ordinal() of the state enum (or state id)

Returns:

the state enum object

getStateId

protected int getStateId(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState refreshMetaState)

Description copied from class: StateMachineProcedure

Convert the Enum (or more descriptive) state object to an ordinal (or state id).

Specified by:

getStateId in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

Parameters:

refreshMetaState - the state enum object

Returns:

stateId the ordinal() of the state enum (or state id)

getInitialState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState getInitialState()

Description copied from class: StateMachineProcedure

Return the initial state object that will be used for the first call to executeFromState().

Specified by:

getInitialState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

Returns:

the initial state enum object

serializeStateData

protected void serializeStateData(ProcedureStateSerializer serializer)
                           throws IOException

Description copied from class: Procedure

The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.

Overrides:

serializeStateData in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

Parameters:

serializer - stores the serializable state

Throws:

IOException

deserializeStateData

protected void deserializeStateData(ProcedureStateSerializer serializer)
                             throws IOException

Description copied from class: Procedure

Called on store load to allow the user to decode the previously serialized state.

Overrides:

deserializeStateData in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RefreshMetaState>

Parameters:

serializer - contains the serialized state

Throws:

IOException