org.apache.hadoop.hbase.master.assignment

Class SplitTableRegionProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<TEnvironment>

org.apache.hadoop.hbase.procedure2.StateMachineProcedure<MasterProcedureEnv,TState>

org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure<TState>

org.apache.hadoop.hbase.master.procedure.AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface

@InterfaceAudience.Private
public class SplitTableRegionProcedure
extends AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

The procedure to split a region in a table. Takes lock on the parent region. It holds the lock for the life of the procedure.

Throws exception on construction if determines context hostile to spllt (cluster going down or master is shutting down or table is disabled).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	SplitTableRegionProcedure.StoreFileSplitter Utility class used to do the file splitting / reference writing in parallel instead of sequentially.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

StateMachineProcedure.Flow

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field and Description
private byte[]	bestSplitRow
private RegionInfo	daughter_1_RI
private RegionInfo	daughter_2_RI
private static RegionState.State[]	EXPECTED_SPLIT_STATES
private static org.slf4j.Logger	LOG
private RegionSplitPolicy	splitPolicy
private Boolean	traceEnabled

Fields inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

stateCount

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
SplitTableRegionProcedure()
SplitTableRegionProcedure(MasterProcedureEnv env, RegionInfo regionToSplit, byte[] splitRow)

Method Summary

Modifier and Type	Method and Description
protected boolean	abort(MasterProcedureEnv env) The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable.
private void	assertReferenceFileCount(org.apache.hadoop.fs.FileSystem fs, int expectedReferenceFileCount, org.apache.hadoop.fs.Path dir)
private void	checkSplittable(MasterProcedureEnv env, RegionInfo regionToSplit, byte[] splitRow) Check whether the region is splittable
private AssignProcedure[]	createAssignProcedures(MasterProcedureEnv env, int regionReplication)
void	createDaughterRegions(MasterProcedureEnv env) Create daughter regions
private UnassignProcedure[]	createUnassignProcedures(MasterProcedureEnv env, int regionReplication)
protected void	deserializeStateData(ProcedureStateSerializer serializer) Called on store load to allow the user to decode the previously serialized state.
protected StateMachineProcedure.Flow	executeFromState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state) called to perform a single step of the specified 'state' of the procedure
private static long	getDaughterRegionIdTimestamp(RegionInfo hri) Calculate daughter regionid to use.
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState	getInitialState() Return the initial state object that will be used for the first call to executeFromState().
private RegionInfo	getParentRegion()
private ServerName	getParentRegionServerName(MasterProcedureEnv env)
protected ProcedureMetrics	getProcedureMetrics(MasterProcedureEnv env) Override this method to provide procedure specific counters for submitted count, failed count and time histogram.
private int	getRegionReplication(MasterProcedureEnv env)
private byte[]	getSplitRow()
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState	getState(int stateId) Convert an ordinal (or state id) to an Enum (or more descriptive) state object.
protected int	getStateId(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state) Convert the Enum (or more descriptive) state object to an ordinal (or state id).
TableProcedureInterface.TableOperationType	getTableOperationType() Given an operation type we can take decisions about what to do with pending operations.
(package private) static boolean	hasRecoveredEdits(MasterProcedureEnv env, RegionInfo ri) Check whether there are recovered.edits in the parent closed region.
protected boolean	isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state) Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.
private boolean	isTraceEnabled() The procedure could be restarted from a different machine.
private void	openParentRegion(MasterProcedureEnv env) Rollback close parent region
private void	postRollBackSplitRegion(MasterProcedureEnv env) Action after rollback a split table region action.
private void	postSplitRegion(MasterProcedureEnv env) Post split region actions
boolean	prepareSplitRegion(MasterProcedureEnv env) Prepare to Split region.
private void	preSplitRegion(MasterProcedureEnv env) Action before splitting region in a table.
private void	preSplitRegionAfterMETA(MasterProcedureEnv env) Pre split region actions after the Point-of-No-Return step
private void	preSplitRegionBeforeMETA(MasterProcedureEnv env) Post split region actions before the Point-of-No-Return step
protected void	rollbackState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state) To rollback SplitTableRegionProcedure, an AssignProcedure is asynchronously submitted for parent region to be split (rollback doesn't wait on the completion of the AssignProcedure) .
protected void	serializeStateData(ProcedureStateSerializer serializer) The user-level code of the procedure may have some state to persist (e.g.
private Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>	splitStoreFile(HRegionFileSystem regionFs, byte[] family, HStoreFile sf)
private Pair<Integer,Integer>	splitStoreFiles(MasterProcedureEnv env, HRegionFileSystem regionFs) Create Split directory
void	toStringClassDetails(StringBuilder sb) Extend the toString() information with the procedure details e.g.
private void	updateMetaForDaughterRegions(MasterProcedureEnv env) Add daughter regions to META
private void	writeMaxSequenceIdFile(MasterProcedureEnv env)

Methods inherited from class org.apache.hadoop.hbase.master.procedure.AbstractStateMachineRegionProcedure

acquireLock, checkTableModifiable, getRegion, getTableName, holdLock, releaseLock, setFailure, setRegion

Methods inherited from class org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure

checkOnline, getRegionDir, getUser, getWALRegionDir, preflightChecks, releaseSyncLatch, setUser, waitInitialized

Methods inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

addChildProcedure, execute, failIfAborted, getCurrentState, getCurrentStateId, getCycles, isEofState, isYieldAfterExecutionStep, isYieldBeforeExecuteFromState, rollback, setNextState, toStringState

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, bypass, compareTo, completionCleanup, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, removeStackIndex, setAbortFailure, setChildrenLatch, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, setTimeoutFailure, shouldWaitClientAck, skipPersistence, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

traceEnabled

private Boolean traceEnabled

daughter_1_RI

private RegionInfo daughter_1_RI

daughter_2_RI

private RegionInfo daughter_2_RI

bestSplitRow

private byte[] bestSplitRow

splitPolicy

private RegionSplitPolicy splitPolicy

EXPECTED_SPLIT_STATES

private static final RegionState.State[] EXPECTED_SPLIT_STATES

Constructor Detail

SplitTableRegionProcedure

public SplitTableRegionProcedure()

SplitTableRegionProcedure

public SplitTableRegionProcedure(MasterProcedureEnv env,
                                 RegionInfo regionToSplit,
                                 byte[] splitRow)
                          throws IOException

Throws:

IOException

Method Detail

hasRecoveredEdits

static boolean hasRecoveredEdits(MasterProcedureEnv env,
                                 RegionInfo ri)
                          throws IOException

Check whether there are recovered.edits in the parent closed region.

Parameters:

env - master env

Throws:

IOException - IOException

checkSplittable

private void checkSplittable(MasterProcedureEnv env,
                             RegionInfo regionToSplit,
                             byte[] splitRow)
                      throws IOException

Check whether the region is splittable

Parameters:

env - MasterProcedureEnv

regionToSplit - parent Region to be split

splitRow - if splitRow is not specified, will first try to get bestSplitRow from RS

Throws:

IOException

getDaughterRegionIdTimestamp

private static long getDaughterRegionIdTimestamp(RegionInfo hri)

Calculate daughter regionid to use.

Parameters:

hri - Parent RegionInfo

Returns:

Daughter region id (timestamp) to use.

executeFromState

protected StateMachineProcedure.Flow executeFromState(MasterProcedureEnv env,
                                                      org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)
                                               throws InterruptedException

Description copied from class: StateMachineProcedure

called to perform a single step of the specified 'state' of the procedure

Specified by:

executeFromState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

state - state to execute

Returns:

Flow.NO_MORE_STATE if the procedure is completed, Flow.HAS_MORE_STATE if there is another step.

Throws:

InterruptedException

rollbackState

protected void rollbackState(MasterProcedureEnv env,
                             org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)
                      throws IOException,
                             InterruptedException

To rollback SplitTableRegionProcedure, an AssignProcedure is asynchronously submitted for parent region to be split (rollback doesn't wait on the completion of the AssignProcedure) . This can be improved by changing rollback() to support sub-procedures. See HBASE-19851 for details.

Specified by:

rollbackState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

state - state to rollback

Throws:

IOException - temporary failure, the rollback will retry later

InterruptedException

isRollbackSupported

protected boolean isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)

Description copied from class: StateMachineProcedure

Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.

Overrides:

isRollbackSupported in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

getState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState getState(int stateId)

Description copied from class: StateMachineProcedure

Convert an ordinal (or state id) to an Enum (or more descriptive) state object.

Specified by:

getState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

stateId - the ordinal() of the state enum (or state id)

Returns:

the state enum object

getStateId

protected int getStateId(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)

Description copied from class: StateMachineProcedure

Convert the Enum (or more descriptive) state object to an ordinal (or state id).

Specified by:

getStateId in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

state - the state enum object

Returns:

stateId the ordinal() of the state enum (or state id)

getInitialState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState getInitialState()

Description copied from class: StateMachineProcedure

Return the initial state object that will be used for the first call to executeFromState().

Specified by:

getInitialState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Returns:

the initial state enum object

serializeStateData

protected void serializeStateData(ProcedureStateSerializer serializer)
                           throws IOException

Description copied from class: Procedure

The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.

Overrides:

serializeStateData in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

serializer - stores the serializable state

Throws:

IOException

deserializeStateData

protected void deserializeStateData(ProcedureStateSerializer serializer)
                             throws IOException

Description copied from class: Procedure

Called on store load to allow the user to decode the previously serialized state.

Overrides:

deserializeStateData in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

serializer - contains the serialized state

Throws:

IOException

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Description copied from class: Procedure

Extend the toString() information with the procedure details e.g. className and parameters

Overrides:

toStringClassDetails in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

sb - the string builder to use to append the proc specific information

getParentRegion

private RegionInfo getParentRegion()

getTableOperationType

public TableProcedureInterface.TableOperationType getTableOperationType()

Description copied from interface: TableProcedureInterface

Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a delete and we have some table operation pending (e.g. add column) we can abort those operations.

Specified by:

getTableOperationType in interface TableProcedureInterface

Specified by:

getTableOperationType in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Returns:

the operation type that the procedure is executing.

getProcedureMetrics

protected ProcedureMetrics getProcedureMetrics(MasterProcedureEnv env)

Description copied from class: Procedure

Override this method to provide procedure specific counters for submitted count, failed count and time histogram.

Overrides:

getProcedureMetrics in class Procedure<MasterProcedureEnv>

Parameters:

env - The environment passed to the procedure executor

Returns:

Container object for procedure related metric

getSplitRow

private byte[] getSplitRow()

prepareSplitRegion

public boolean prepareSplitRegion(MasterProcedureEnv env)
                           throws IOException

Prepare to Split region.

Parameters:

env - MasterProcedureEnv

Throws:

IOException

preSplitRegion

private void preSplitRegion(MasterProcedureEnv env)
                     throws IOException,
                            InterruptedException

Action before splitting region in a table.

Parameters:

env - MasterProcedureEnv

Throws:

IOException

InterruptedException

postRollBackSplitRegion

private void postRollBackSplitRegion(MasterProcedureEnv env)
                              throws IOException

Action after rollback a split table region action.

Parameters:

env - MasterProcedureEnv

Throws:

IOException

openParentRegion

private void openParentRegion(MasterProcedureEnv env)
                       throws IOException

Rollback close parent region

Parameters:

env - MasterProcedureEnv

Throws:

IOException

createDaughterRegions

public void createDaughterRegions(MasterProcedureEnv env)
                           throws IOException

Create daughter regions

Parameters:

env - MasterProcedureEnv

Throws:

IOException

splitStoreFiles

private Pair<Integer,Integer> splitStoreFiles(MasterProcedureEnv env,
                                              HRegionFileSystem regionFs)
                                       throws IOException

Create Split directory

Parameters:

env - MasterProcedureEnv

Throws:

IOException

assertReferenceFileCount

private void assertReferenceFileCount(org.apache.hadoop.fs.FileSystem fs,
                                      int expectedReferenceFileCount,
                                      org.apache.hadoop.fs.Path dir)
                               throws IOException

Throws:

IOException

splitStoreFile

private Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path> splitStoreFile(HRegionFileSystem regionFs,
                                                                                 byte[] family,
                                                                                 HStoreFile sf)
                                                                          throws IOException

Throws:

IOException

preSplitRegionBeforeMETA

private void preSplitRegionBeforeMETA(MasterProcedureEnv env)
                               throws IOException,
                                      InterruptedException

Post split region actions before the Point-of-No-Return step

Parameters:

env - MasterProcedureEnv

Throws:

IOException

InterruptedException

updateMetaForDaughterRegions

private void updateMetaForDaughterRegions(MasterProcedureEnv env)
                                   throws IOException

Add daughter regions to META

Parameters:

env - MasterProcedureEnv

Throws:

IOException

preSplitRegionAfterMETA

private void preSplitRegionAfterMETA(MasterProcedureEnv env)
                              throws IOException,
                                     InterruptedException

Pre split region actions after the Point-of-No-Return step

Parameters:

env - MasterProcedureEnv

Throws:

IOException

InterruptedException

postSplitRegion

private void postSplitRegion(MasterProcedureEnv env)
                      throws IOException

Post split region actions

Parameters:

env - MasterProcedureEnv

Throws:

IOException

getParentRegionServerName

private ServerName getParentRegionServerName(MasterProcedureEnv env)

createUnassignProcedures

private UnassignProcedure[] createUnassignProcedures(MasterProcedureEnv env,
                                                     int regionReplication)

createAssignProcedures

private AssignProcedure[] createAssignProcedures(MasterProcedureEnv env,
                                                 int regionReplication)

getRegionReplication

private int getRegionReplication(MasterProcedureEnv env)
                          throws IOException

Throws:

IOException

writeMaxSequenceIdFile

private void writeMaxSequenceIdFile(MasterProcedureEnv env)
                             throws IOException

Throws:

IOException

isTraceEnabled

private boolean isTraceEnabled()

The procedure could be restarted from a different machine. If the variable is null, we need to retrieve it.

Returns:

traceEnabled

abort

protected boolean abort(MasterProcedureEnv env)

Description copied from class: Procedure

The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable. The simplest implementation is to have an AtomicBoolean set in the abort() method and then the execute() will check if the abort flag is set or not. abort() may be called multiple times from the client, so the implementation must be idempotent.

NOTE: abort() is not like Thread.interrupt(). It is just a notification that allows the procedure implementor abort.

Overrides:

abort in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>