AssignmentManager (Apache HBase 2.3.0 API)

java.lang.Object
- org.apache.hadoop.hbase.master.assignment.AssignmentManager

```
@InterfaceAudience.Private
public class AssignmentManager
extends Object
```
The AssignmentManager is the coordinator for region assign/unassign operations.
- In-memory states of regions and servers are stored in RegionStates.
- hbase:meta state updates are handled by RegionStateStore.
Regions are created by CreateTable, Split, Merge. Regions are deleted by DeleteTable, Split, Merge. Assigns are triggered by CreateTable, EnableTable, Split, Merge, ServerCrash. Unassigns are triggered by DisableTable, Split, Merge

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`private static class`	`AssignmentManager.DeadServerMetricRegionChore`
`private static class`	`AssignmentManager.RegionInTransitionChore`
`static class`	`AssignmentManager.RegionInTransitionStat`
`private class`	`AssignmentManager.RegionMetaLoadingVisitor`

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`ASSIGN_DISPATCH_WAIT_MSEC_CONF_KEY`
`static String`	`ASSIGN_DISPATCH_WAITQ_MAX_CONF_KEY`
`static String`	`ASSIGN_MAX_ATTEMPTS`
`static String`	`ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS`
`private int`	`assignDispatchWaitMillis`
`private int`	`assignDispatchWaitQueueMaxSize`
`private int`	`assignMaxAttempts`
`private Condition`	`assignQueueFullCond`
`private ReentrantLock`	`assignQueueLock`
`private int`	`assignRetryImmediatelyMaxAttempts`
`private Thread`	`assignThread`
`static String`	`BOOTSTRAP_THREAD_POOL_SIZE_CONF_KEY`
`private Object`	`checkIfShouldMoveSystemRegionLock`
`static String`	`DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC_CONF_KEY`
`private AssignmentManager.DeadServerMetricRegionChore`	`deadMetricChore`
`private static int`	`DEFAULT_ASSIGN_DISPATCH_WAIT_MSEC`
`private static int`	`DEFAULT_ASSIGN_DISPATCH_WAITQ_MAX`
`private static int`	`DEFAULT_ASSIGN_MAX_ATTEMPTS`
`private static int`	`DEFAULT_ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS`
`private static int`	`DEFAULT_DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC`
`private static int`	`DEFAULT_RIT_CHORE_INTERVAL_MSEC`
`private static int`	`DEFAULT_RIT_STUCK_WARNING_THRESHOLD`
`private static org.slf4j.Logger`	`LOG`
`private MasterServices`	`master`
`private static Set<RegionInfo>`	`META_REGION_SET`
`private ProcedureEvent<?>`	`metaAssignEvent`
`private ProcedureEvent<?>`	`metaLoadEvent`
`private MetricsAssignmentManager`	`metrics`
`static String`	`METRICS_RIT_STUCK_WARNING_THRESHOLD` Region in Transition metrics threshold time
`private ArrayList<RegionStateNode>`	`pendingAssignQueue`
`private RegionStates`	`regionStates`
`private RegionStateStore`	`regionStateStore`
`static String`	`RIT_CHORE_INTERVAL_MSEC_CONF_KEY`
`private AssignmentManager.RegionInTransitionChore`	`ritChore`
`private Map<ServerName,Set<byte[]>>`	`rsReports`
`private AtomicBoolean`	`running`
`private boolean`	`shouldAssignRegionsWithFavoredNodes`
`private static RegionState.State[]`	`STATES_EXPECTED_ON_ASSIGN`
`private static RegionState.State[]`	`STATES_EXPECTED_ON_CLOSED`
`private static RegionState.State[]`	`STATES_EXPECTED_ON_CLOSING`
`private static RegionState.State[]`	`STATES_EXPECTED_ON_OPEN`
`private static RegionState.State[]`	`STATES_EXPECTED_ON_UNASSIGN_OR_MOVE`

Constructor Summary

Constructors
Constructor and Description

AssignmentManager(MasterServices master)

AssignmentManager(MasterServices master, RegionStateStore stateStore)

Constructors
Constructor and Description
`AssignmentManager(MasterServices master)`
`AssignmentManager(MasterServices master, RegionStateStore stateStore)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`private void`	`acceptPlan(HashMap<RegionInfo,RegionStateNode> regions, Map<ServerName,List<RegionInfo>> plan)`
`private void`	`addToPendingAssignment(HashMap<RegionInfo,RegionStateNode> regions, Collection<RegionInfo> pendingRegions)`
`long`	`assign(RegionInfo regionInfo)`
`long`	`assign(RegionInfo regionInfo, ServerName sn)`
`Future<byte[]>`	`assignAsync(RegionInfo regionInfo)` Submits a procedure that assigns a region without waiting for it to finish
`Future<byte[]>`	`assignAsync(RegionInfo regionInfo, ServerName sn)` Submits a procedure that assigns a region to a target server without waiting for it to finish
`private void`	`assignQueueSignal()`
`void`	`checkIfShouldMoveSystemRegionAsync()` Start a new thread to check if there are region servers whose versions are higher than others.
`private void`	`checkMetaLoaded(RegionInfo hri)` Used to check if the meta loading is done.
`private void`	`checkOnlineRegionsReport(ServerStateNode serverNode, Set<byte[]> regionNames)` Check that what the RegionServer reports aligns with the Master's image.
`private void`	`closeRegionSilently(ServerName sn, byte[] regionName)` Close `regionName` on `sn` silently and immediately without using a Procedure or going via hbase:meta.
`(package private) static int`	`compare(TransitRegionStateProcedure left, TransitRegionStateProcedure right)`
`AssignmentManager.RegionInTransitionStat`	`computeRegionInTransitionStat()`
`private boolean`	`containsBogusAssignments(Map<RegionInfo,RegionStateNode> regions, List<RegionInfo> hirs)`
`private TransitRegionStateProcedure`	`createAssignProcedure(RegionInfo regionInfo, ServerName sn)`
`private TransitRegionStateProcedure`	`createAssignProcedure(RegionStateNode regionNode, ServerName targetServer, boolean override)`
`TransitRegionStateProcedure[]`	`createAssignProcedures(List<RegionInfo> hris)` Create an array of TransitRegionStateProcedure w/o specifying a target server.
`private TransitRegionStateProcedure[]`	`createAssignProcedures(Map<ServerName,List<RegionInfo>> assignments)`
`MergeTableRegionsProcedure`	`createMergeProcedure(RegionInfo... ris)`
`TransitRegionStateProcedure`	`createMoveRegionProcedure(RegionInfo regionInfo, ServerName targetServer)`
`TransitRegionStateProcedure`	`createOneAssignProcedure(RegionInfo hri, boolean override)` Create one TransitRegionStateProcedure to assign a region w/o specifying a target server.
`TransitRegionStateProcedure`	`createOneUnassignProcedure(RegionInfo hri, boolean override)` Create one TransitRegionStateProcedure to unassign a region.
`TransitRegionStateProcedure[]`	`createRoundRobinAssignProcedures(List<RegionInfo> hris)` Create round-robin assigns.
`TransitRegionStateProcedure[]`	`createRoundRobinAssignProcedures(List<RegionInfo> hris, List<ServerName> serversToExclude)` Create round-robin assigns.
`SplitTableRegionProcedure`	`createSplitProcedure(RegionInfo regionToSplit, byte[] splitKey)`
`private TransitRegionStateProcedure`	`createUnassignProcedure(RegionStateNode regionNode, boolean override)`
`TransitRegionStateProcedure[]`	`createUnassignProceduresForDisabling(TableName tableName)` Called by DisableTableProcedure to unassign all the regions for a table.
`void`	`deleteTable(TableName tableName)` Delete the region states.
`List<RegionInfo>`	`getAssignedRegions()`
`(package private) int`	`getAssignMaxAttempts()`
`MetricsAssignmentManager`	`getAssignmentManagerMetrics()`
`(package private) int`	`getAssignRetryImmediatelyMaxAttempts()`
`private LoadBalancer`	`getBalancer()`
`org.apache.hadoop.conf.Configuration`	`getConfiguration()`
`List<ServerName>`	`getExcludedServersForSystemTable()` Get a list of servers that this region cannot be assigned to.
`List<ServerName>`	`getFavoredNodes(RegionInfo regionInfo)`
`(package private) MasterServices`	`getMaster()`
`private ProcedureEvent<?>`	`getMetaAssignEvent(RegionInfo metaRegionInfo)`
`private RegionInfo`	`getMetaForRegion(RegionInfo regionInfo)`
`RegionInfo`	`getMetaRegionFromName(byte[] regionName)`
`Set<RegionInfo>`	`getMetaRegionSet()`
`int`	`getNumRegionsOpened()`
`private MasterProcedureEnv`	`getProcedureEnvironment()`
`private MasterProcedureScheduler`	`getProcedureScheduler()`
`RegionInfo`	`getRegionInfo(byte[] regionName)`
`List<RegionStateNode>`	`getRegionsInTransition()`
`List<RegionInfo>`	`getRegionsOnServer(ServerName serverName)` Returns the regions hosted by the specified server.
`RegionStates`	`getRegionStates()`
`RegionStatesCount`	`getRegionStatesCount(TableName tableName)` Provide regions state count for given table.
`RegionStateStore`	`getRegionStateStore()`
`Pair<Integer,Integer>`	`getReopenStatus(TableName tableName)` Used by the client (via master) to identify if all regions have the schema updates
`Map<ServerName,Set<byte[]>>`	`getRSReports()`
`Map<ServerName,List<RegionInfo>>`	`getSnapShotOfAssignment(Collection<RegionInfo> regions)`
`private List<RegionInfo>`	`getSystemTables(ServerName serverName)`
`(package private) TableStateManager`	`getTableStateManager()`
`private void`	`handleRegionOverStuckWarningThreshold(RegionInfo regionInfo)`
`boolean`	`hasRegionsInTransition()`
`boolean`	`isCarryingMeta(ServerName serverName)`
`private boolean`	`isCarryingRegion(ServerName serverName, RegionInfo regionInfo)`
`boolean`	`isMetaAssigned()` Notice that, this only means the meta region is available on a RS, but the AM may still be loading the region states from meta, so usually you need to check `isMetaLoaded()` first before checking this method, unless you can make sure that your piece of code can only be executed after AM builds the region states.
`boolean`	`isMetaLoaded()` Return whether AM finishes the meta loading, i.e, the region states rebuilding.
`boolean`	`isMetaRegion(byte[] regionName)`
`private boolean`	`isMetaRegion(RegionInfo regionInfo)`
`boolean`	`isMetaRegionInTransition()`
`boolean`	`isRunning()`
`boolean`	`isTableDisabled(TableName tableName)`
`boolean`	`isTableEnabled(TableName tableName)`
`void`	`joinCluster()`
`private void`	`loadMeta()`
`RegionInfo`	`loadRegionFromMeta(String regionEncodedName)` Query META if the given `RegionInfo` exists, adding to `AssignmentManager.regionStateStore` cache if the region is found in META.
`void`	`markRegionAsMerged(RegionInfo child, ServerName serverName, RegionInfo[] mergeParents)` When called here, the merge has happened.
`void`	`markRegionAsSplit(RegionInfo parent, ServerName serverName, RegionInfo daughterA, RegionInfo daughterB)`
`void`	`move(RegionInfo regionInfo)`
`Future<byte[]>`	`moveAsync(RegionPlan regionPlan)`
`void`	`offlineRegion(RegionInfo regionInfo)`
`void`	`onlineRegion(RegionInfo regionInfo, ServerName serverName)`
`(package private) void`	`persistToMeta(RegionStateNode regionNode)`
`private void`	`preTransitCheck(RegionStateNode regionNode, RegionState.State[] expectedStates)`
`private void`	`processAssignmentPlans(HashMap<RegionInfo,RegionStateNode> regions, HashMap<RegionInfo,ServerName> retainMap, List<RegionInfo> hris, List<ServerName> servers)`
`private void`	`processAssignQueue()`
`void`	`processOfflineRegions()` Create assign procedure for offline regions.
`protected void`	`queueAssign(RegionStateNode regionNode)` Add the assign operation to the assignment queue.
`void`	`regionClosedAbnormally(RegionStateNode regionNode)`
`(package private) void`	`regionClosedWithoutPersistingToMeta(RegionStateNode regionNode)`
`(package private) void`	`regionClosing(RegionStateNode regionNode)`
`(package private) void`	`regionFailedOpen(RegionStateNode regionNode, boolean giveUp)`
`(package private) void`	`regionOpenedWithoutPersistingToMeta(RegionStateNode regionNode)`
`(package private) void`	`regionOpening(RegionStateNode regionNode)`
`void`	`reportOnlineRegions(ServerName serverName, Set<byte[]> regionNames)` The master will call this method when the RS send the regionServerReport().
`org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionResponse`	`reportRegionStateTransition(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionRequest req)`
`private void`	`reportRegionStateTransition(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionResponse.Builder builder, ServerName serverName, List<org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition> transitionList)`
`private boolean`	`reportTransition(RegionStateNode regionNode, ServerStateNode serverNode, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, long seqId, long procId)`
`private void`	`setMetaAssigned(RegionInfo metaRegionInfo, boolean assigned)`
`void`	`setupRIT(List<TransitRegionStateProcedure> procs)` Create RegionStateNode based on the TRSP list, and attach the TRSP to the RegionStateNode.
`private boolean`	`shouldAssignFavoredNodes(RegionInfo region)`
`void`	`start()`
`private void`	`startAssignmentThread()`
`void`	`stop()`
`private void`	`stopAssignmentThread()`
`long`	`submitServerCrash(ServerName serverName, boolean shouldSplitWal, boolean force)` Usually run by the Master in reaction to server crash during normal processing.
`private void`	`transitStateAndUpdate(RegionStateNode regionNode, RegionState.State newState, RegionState.State... expectedStates)`
`long`	`unassign(RegionInfo regionInfo)`
`private void`	`updateDeadServerRegionMetrics(int deadRegions, int unknownRegions)`
`private void`	`updateRegionMergeTransition(ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, RegionInfo merged, RegionInfo hriA, RegionInfo hriB)`
`private void`	`updateRegionsInTransitionMetrics(AssignmentManager.RegionInTransitionStat ritStat)`
`private void`	`updateRegionSplitTransition(ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, RegionInfo parent, RegionInfo hriA, RegionInfo hriB)`
`private void`	`updateRegionTransition(ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, RegionInfo regionInfo, long seqId, long procId)`
`boolean`	`waitMetaAssigned(Procedure<?> proc, RegionInfo regionInfo)` Notice that this event does not mean the AM has already finished region state rebuilding.
`boolean`	`waitMetaLoaded(Procedure<?> proc)` Wait until AM finishes the meta loading, i.e, the region states rebuilding.
`private HashMap<RegionInfo,RegionStateNode>`	`waitOnAssignQueue()`
`(package private) void`	`wakeMetaLoadedEvent()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

BOOTSTRAP_THREAD_POOL_SIZE_CONF_KEY

public static final String BOOTSTRAP_THREAD_POOL_SIZE_CONF_KEY

See Also:: Constant Field Values

ASSIGN_DISPATCH_WAIT_MSEC_CONF_KEY

public static final String ASSIGN_DISPATCH_WAIT_MSEC_CONF_KEY

See Also:: Constant Field Values

DEFAULT_ASSIGN_DISPATCH_WAIT_MSEC
```
private static final int DEFAULT_ASSIGN_DISPATCH_WAIT_MSEC
```
See Also:

Constant Field Values

ASSIGN_DISPATCH_WAITQ_MAX_CONF_KEY

public static final String ASSIGN_DISPATCH_WAITQ_MAX_CONF_KEY

See Also:: Constant Field Values

DEFAULT_ASSIGN_DISPATCH_WAITQ_MAX
```
private static final int DEFAULT_ASSIGN_DISPATCH_WAITQ_MAX
```
See Also:

Constant Field Values

RIT_CHORE_INTERVAL_MSEC_CONF_KEY

public static final String RIT_CHORE_INTERVAL_MSEC_CONF_KEY

See Also:: Constant Field Values

DEFAULT_RIT_CHORE_INTERVAL_MSEC
```
private static final int DEFAULT_RIT_CHORE_INTERVAL_MSEC
```
See Also:

Constant Field Values

DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC_CONF_KEY

public static final String DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC_CONF_KEY

See Also:: Constant Field Values

DEFAULT_DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC
```
private static final int DEFAULT_DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC
```
See Also:

Constant Field Values

ASSIGN_MAX_ATTEMPTS

public static final String ASSIGN_MAX_ATTEMPTS

See Also:: Constant Field Values

DEFAULT_ASSIGN_MAX_ATTEMPTS
```
private static final int DEFAULT_ASSIGN_MAX_ATTEMPTS
```
See Also:

Constant Field Values

ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS

public static final String ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS

See Also:: Constant Field Values

DEFAULT_ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS
```
private static final int DEFAULT_ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS
```
See Also:

Constant Field Values

METRICS_RIT_STUCK_WARNING_THRESHOLD
```
public static final String METRICS_RIT_STUCK_WARNING_THRESHOLD
```
Region in Transition metrics threshold time

See Also:

Constant Field Values

DEFAULT_RIT_STUCK_WARNING_THRESHOLD
```
private static final int DEFAULT_RIT_STUCK_WARNING_THRESHOLD
```
See Also:

Constant Field Values

metaAssignEvent

private final ProcedureEvent<?> metaAssignEvent

metaLoadEvent

private final ProcedureEvent<?> metaLoadEvent

metrics

private final MetricsAssignmentManager metrics

ritChore

private final AssignmentManager.RegionInTransitionChore ritChore

deadMetricChore

private final AssignmentManager.DeadServerMetricRegionChore deadMetricChore

master
```
private final MasterServices master
```

running
```
private final AtomicBoolean running
```

regionStates

private final RegionStates regionStates

regionStateStore

private final RegionStateStore regionStateStore

rsReports

private final Map<ServerName,Set<byte[]>> rsReports

shouldAssignRegionsWithFavoredNodes

private final boolean shouldAssignRegionsWithFavoredNodes

assignDispatchWaitQueueMaxSize

private final int assignDispatchWaitQueueMaxSize

assignDispatchWaitMillis

private final int assignDispatchWaitMillis

assignMaxAttempts
```
private final int assignMaxAttempts
```

assignRetryImmediatelyMaxAttempts

private final int assignRetryImmediatelyMaxAttempts

checkIfShouldMoveSystemRegionLock

private final Object checkIfShouldMoveSystemRegionLock

assignThread
```
private Thread assignThread
```

META_REGION_SET

private static final Set<RegionInfo> META_REGION_SET

STATES_EXPECTED_ON_OPEN

private static final RegionState.State[] STATES_EXPECTED_ON_OPEN

STATES_EXPECTED_ON_CLOSING

private static final RegionState.State[] STATES_EXPECTED_ON_CLOSING

STATES_EXPECTED_ON_CLOSED

private static final RegionState.State[] STATES_EXPECTED_ON_CLOSED

STATES_EXPECTED_ON_ASSIGN

private static final RegionState.State[] STATES_EXPECTED_ON_ASSIGN

STATES_EXPECTED_ON_UNASSIGN_OR_MOVE

private static final RegionState.State[] STATES_EXPECTED_ON_UNASSIGN_OR_MOVE

pendingAssignQueue

private final ArrayList<RegionStateNode> pendingAssignQueue

assignQueueLock

private final ReentrantLock assignQueueLock

assignQueueFullCond

private final Condition assignQueueFullCond

Constructor Detail

AssignmentManager

public AssignmentManager(MasterServices master)

AssignmentManager

AssignmentManager(MasterServices master,
                  RegionStateStore stateStore)

Method Detail

start

public void start()
           throws IOException,
                  org.apache.zookeeper.KeeperException

Throws:: IOException; org.apache.zookeeper.KeeperException

setupRIT
```
public void setupRIT(List<TransitRegionStateProcedure> procs)
```
Create RegionStateNode based on the TRSP list, and attach the TRSP to the RegionStateNode.
This is used to restore the RIT region list, so we do not need to restore it in the loadingMeta method below. And it is also very important as now before submitting a TRSP, we need to attach it to the RegionStateNode, which acts like a guard, so we need to restore this information at the very beginning, before we start processing any procedures.

stop
```
public void stop()
```

isRunning
```
public boolean isRunning()
```

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

getAssignmentManagerMetrics

public MetricsAssignmentManager getAssignmentManagerMetrics()

getBalancer
```
private LoadBalancer getBalancer()
```

getProcedureEnvironment

private MasterProcedureEnv getProcedureEnvironment()

getProcedureScheduler

private MasterProcedureScheduler getProcedureScheduler()

getAssignMaxAttempts
```
int getAssignMaxAttempts()
```

getAssignRetryImmediatelyMaxAttempts

int getAssignRetryImmediatelyMaxAttempts()

getRegionStates
```
public RegionStates getRegionStates()
```

getRegionsOnServer
```
public List<RegionInfo> getRegionsOnServer(ServerName serverName)
```
Returns the regions hosted by the specified server.
Notice that, for SCP, after we submit the SCP, no one can change the region list for the ServerStateNode so we do not need any locks here. And for other usage, this can only give you a snapshot of the current region list for this server, which means, right after you get the region list, new regions may be moved to this server or some regions may be moved out from this server, so you should not use it critically if you need strong consistency.

getRegionStateStore

public RegionStateStore getRegionStateStore()

getFavoredNodes

public List<ServerName> getFavoredNodes(RegionInfo regionInfo)

getTableStateManager

TableStateManager getTableStateManager()

isTableEnabled

public boolean isTableEnabled(TableName tableName)

isTableDisabled

public boolean isTableDisabled(TableName tableName)

isMetaRegion

private boolean isMetaRegion(RegionInfo regionInfo)

isMetaRegion

public boolean isMetaRegion(byte[] regionName)

getMetaRegionFromName

public RegionInfo getMetaRegionFromName(byte[] regionName)

isCarryingMeta

public boolean isCarryingMeta(ServerName serverName)

isCarryingRegion

private boolean isCarryingRegion(ServerName serverName,
                                 RegionInfo regionInfo)

getMetaForRegion

private RegionInfo getMetaForRegion(RegionInfo regionInfo)

getMetaRegionSet

public Set<RegionInfo> getMetaRegionSet()

isMetaAssigned
```
public boolean isMetaAssigned()
```
Notice that, this only means the meta region is available on a RS, but the AM may still be loading the region states from meta, so usually you need to check isMetaLoaded() first before checking this method, unless you can make sure that your piece of code can only be executed after AM builds the region states.

See Also:

isMetaLoaded()

isMetaRegionInTransition

public boolean isMetaRegionInTransition()

waitMetaAssigned
```
public boolean waitMetaAssigned(Procedure<?> proc,
                                RegionInfo regionInfo)
```
Notice that this event does not mean the AM has already finished region state rebuilding. See the comment of isMetaAssigned() for more details.

See Also:

isMetaAssigned()

setMetaAssigned

private void setMetaAssigned(RegionInfo metaRegionInfo,
                             boolean assigned)

getMetaAssignEvent

private ProcedureEvent<?> getMetaAssignEvent(RegionInfo metaRegionInfo)

waitMetaLoaded
```
public boolean waitMetaLoaded(Procedure<?> proc)
```
Wait until AM finishes the meta loading, i.e, the region states rebuilding.

See Also:

isMetaLoaded(), waitMetaAssigned(Procedure, RegionInfo)

wakeMetaLoadedEvent
```
void wakeMetaLoadedEvent()
```

isMetaLoaded
```
public boolean isMetaLoaded()
```
Return whether AM finishes the meta loading, i.e, the region states rebuilding.

See Also:

isMetaAssigned(), waitMetaLoaded(Procedure)

checkIfShouldMoveSystemRegionAsync
```
public void checkIfShouldMoveSystemRegionAsync()
```
Start a new thread to check if there are region servers whose versions are higher than others. If so, move all system table regions to RS with the highest version to keep compatibility. The reason is, RS in new version may not be able to access RS in old version when there are some incompatible changes.
This method is called when a new RegionServer is added to cluster only.

getSystemTables

private List<RegionInfo> getSystemTables(ServerName serverName)

preTransitCheck

private void preTransitCheck(RegionStateNode regionNode,
                             RegionState.State[] expectedStates)
                      throws HBaseIOException

Throws:: HBaseIOException

createAssignProcedure

private TransitRegionStateProcedure createAssignProcedure(RegionInfo regionInfo,
                                                          ServerName sn)
                                                   throws IOException

Throws:: IOException

assign

public long assign(RegionInfo regionInfo,
                   ServerName sn)
            throws IOException

Throws:: IOException

assign

public long assign(RegionInfo regionInfo)
            throws IOException

Throws:: IOException

assignAsync
```
public Future<byte[]> assignAsync(RegionInfo regionInfo,
                                  ServerName sn)
                           throws IOException
```
Submits a procedure that assigns a region to a target server without waiting for it to finish

Parameters:

regionInfo - the region we would like to assign

sn - target server name

Returns:

Throws:

IOException

assignAsync
```
public Future<byte[]> assignAsync(RegionInfo regionInfo)
                           throws IOException
```
Submits a procedure that assigns a region without waiting for it to finish

Parameters:

regionInfo - the region we would like to assign

Returns:

Throws:

IOException

unassign

public long unassign(RegionInfo regionInfo)
              throws IOException

Throws:: IOException

createMoveRegionProcedure

public TransitRegionStateProcedure createMoveRegionProcedure(RegionInfo regionInfo,
                                                             ServerName targetServer)
                                                      throws HBaseIOException

Throws:: HBaseIOException

move

public void move(RegionInfo regionInfo)
          throws IOException

Throws:: IOException

moveAsync

public Future<byte[]> moveAsync(RegionPlan regionPlan)
                         throws HBaseIOException

Throws:: HBaseIOException

createRoundRobinAssignProcedures
```
public TransitRegionStateProcedure[] createRoundRobinAssignProcedures(List<RegionInfo> hris,
                                                                      List<ServerName> serversToExclude)
```
Create round-robin assigns. Use on table creation to distribute out regions across cluster.

Returns:

AssignProcedures made out of the passed in hris and a call to the balancer to populate the assigns with targets chosen using round-robin (default balancer scheme). If at assign-time, the target chosen is no longer up, thats fine, the AssignProcedure will ask the balancer for a new target, and so on.

createRoundRobinAssignProcedures
```
public TransitRegionStateProcedure[] createRoundRobinAssignProcedures(List<RegionInfo> hris)
```
Create round-robin assigns. Use on table creation to distribute out regions across cluster.

Returns:

AssignProcedures made out of the passed in hris and a call to the balancer to populate the assigns with targets chosen using round-robin (default balancer scheme). If at assign-time, the target chosen is no longer up, thats fine, the AssignProcedure will ask the balancer for a new target, and so on.

compare

static int compare(TransitRegionStateProcedure left,
                   TransitRegionStateProcedure right)

createAssignProcedure

private TransitRegionStateProcedure createAssignProcedure(RegionStateNode regionNode,
                                                          ServerName targetServer,
                                                          boolean override)

createUnassignProcedure

private TransitRegionStateProcedure createUnassignProcedure(RegionStateNode regionNode,
                                                            boolean override)

createOneAssignProcedure

public TransitRegionStateProcedure createOneAssignProcedure(RegionInfo hri,
                                                            boolean override)

Create one TransitRegionStateProcedure to assign a region w/o specifying a target server. This method is specified for HBCK2

createOneUnassignProcedure

public TransitRegionStateProcedure createOneUnassignProcedure(RegionInfo hri,
                                                              boolean override)

Create one TransitRegionStateProcedure to unassign a region. This method is specified for HBCK2

createAssignProcedures
```
public TransitRegionStateProcedure[] createAssignProcedures(List<RegionInfo> hris)
```
Create an array of TransitRegionStateProcedure w/o specifying a target server.
If no target server, at assign time, we will try to use the former location of the region if one exists. This is how we 'retain' the old location across a server restart.
Should only be called when you can make sure that no one can touch these regions other than you. For example, when you are creating table.

createAssignProcedures
```
private TransitRegionStateProcedure[] createAssignProcedures(Map<ServerName,List<RegionInfo>> assignments)
```
Parameters:

assignments - Map of assignments from which we produce an array of AssignProcedures.

Returns:

Assignments made from the passed in assignments

createUnassignProceduresForDisabling
```
public TransitRegionStateProcedure[] createUnassignProceduresForDisabling(TableName tableName)
```
Called by DisableTableProcedure to unassign all the regions for a table.

createSplitProcedure

public SplitTableRegionProcedure createSplitProcedure(RegionInfo regionToSplit,
                                                      byte[] splitKey)
                                               throws IOException

Throws:: IOException

createMergeProcedure

public MergeTableRegionsProcedure createMergeProcedure(RegionInfo... ris)
                                                throws IOException

Throws:: IOException

deleteTable

public void deleteTable(TableName tableName)
                 throws IOException

Delete the region states. This is called by "DeleteTable"

Throws:: IOException

reportRegionStateTransition

private void reportRegionStateTransition(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionResponse.Builder builder,
                                         ServerName serverName,
                                         List<org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition> transitionList)
                                  throws IOException

Throws:: IOException

reportRegionStateTransition

public org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionResponse reportRegionStateTransition(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionRequest req)
                                                                                                                                           throws PleaseHoldException

Throws:: PleaseHoldException

updateRegionTransition

private void updateRegionTransition(ServerName serverName,
                                    org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                    RegionInfo regionInfo,
                                    long seqId,
                                    long procId)
                             throws IOException

Throws:: IOException

reportTransition

private boolean reportTransition(RegionStateNode regionNode,
                                 ServerStateNode serverNode,
                                 org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                 long seqId,
                                 long procId)
                          throws IOException

Throws:: IOException

updateRegionSplitTransition

private void updateRegionSplitTransition(ServerName serverName,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                         RegionInfo parent,
                                         RegionInfo hriA,
                                         RegionInfo hriB)
                                  throws IOException

Throws:: IOException

updateRegionMergeTransition

private void updateRegionMergeTransition(ServerName serverName,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                         RegionInfo merged,
                                         RegionInfo hriA,
                                         RegionInfo hriB)
                                  throws IOException

Throws:: IOException

reportOnlineRegions
```
public void reportOnlineRegions(ServerName serverName,
                                Set<byte[]> regionNames)
```
The master will call this method when the RS send the regionServerReport(). The report will contains the "online regions". This method will check the the online regions against the in-memory state of the AM, and we will log a warn message if there is a mismatch. This is because that there is no fencing between the reportRegionStateTransition method and regionServerReport method, so there could be race and introduce inconsistency here, but actually there is no problem.
Please see HBASE-21421 and HBASE-21463 for more details.

closeRegionSilently
```
private void closeRegionSilently(ServerName sn,
                                 byte[] regionName)
```
Close regionName on sn silently and immediately without using a Procedure or going via hbase:meta. For case where a RegionServer's hosting of a Region is not aligned w/ the Master's accounting of Region state. This is for cleaning up an error in accounting.

checkOnlineRegionsReport
```
private void checkOnlineRegionsReport(ServerStateNode serverNode,
                                      Set<byte[]> regionNames)
```
Check that what the RegionServer reports aligns with the Master's image. If disagreement, we will tell the RegionServer to expediently close a Region we do not think it should have.

computeRegionInTransitionStat

public AssignmentManager.RegionInTransitionStat computeRegionInTransitionStat()

updateRegionsInTransitionMetrics

private void updateRegionsInTransitionMetrics(AssignmentManager.RegionInTransitionStat ritStat)

updateDeadServerRegionMetrics

private void updateDeadServerRegionMetrics(int deadRegions,
                                           int unknownRegions)

handleRegionOverStuckWarningThreshold

private void handleRegionOverStuckWarningThreshold(RegionInfo regionInfo)

joinCluster

public void joinCluster()
                 throws IOException

Throws:: IOException

processOfflineRegions
```
public void processOfflineRegions()
```
Create assign procedure for offline regions. Just follow the old processofflineServersWithOnlineRegions method. Since now we do not need to deal with dead server any more, we only deal with the regions in OFFLINE state in this method. And this is a bit strange, that for new regions, we will add it in CLOSED state instead of OFFLINE state, and usually there will be a procedure to track them. The processofflineServersWithOnlineRegions is a legacy from long ago, as things are going really different now, maybe we do not need this method any more. Need to revisit later.

loadRegionFromMeta
```
public RegionInfo loadRegionFromMeta(String regionEncodedName)
                              throws UnknownRegionException
```
Query META if the given RegionInfo exists, adding to AssignmentManager.regionStateStore cache if the region is found in META.

Parameters:

regionEncodedName - encoded name for the region to be loaded from META into AssignmentManager.regionStateStore cache

Returns:

RegionInfo instance for the given region if it is present in META and got successfully loaded into AssignmentManager.regionStateStore cache, null otherwise.

Throws:

UnknownRegionException - if any errors occur while querying meta.

loadMeta

private void loadMeta()
               throws IOException

Throws:: IOException

checkMetaLoaded
```
private void checkMetaLoaded(RegionInfo hri)
                      throws PleaseHoldException
```
Used to check if the meta loading is done.
if not we throw PleaseHoldException since we are rebuilding the RegionStates

Parameters:

hri - region to check if it is already rebuild

Throws:

PleaseHoldException - if meta has not been loaded yet

getNumRegionsOpened
```
public int getNumRegionsOpened()
```

submitServerCrash
```
public long submitServerCrash(ServerName serverName,
                              boolean shouldSplitWal,
                              boolean force)
```
Usually run by the Master in reaction to server crash during normal processing. Can also be invoked via external RPC to effect repair; in the latter case, the 'force' flag is set so we push through the SCP though context may indicate already-running-SCP (An old SCP may have exited abnormally, or damaged cluster may still have references in hbase:meta to 'Unknown Servers' -- servers that are not online or in dead servers list, etc.)

Parameters:

force - Set if the request came in externally over RPC (via hbck2). Force means run the SCP even if it seems as though there might be an outstanding SCP running.

Returns:

pid of scheduled SCP or Procedure.NO_PROC_ID if none scheduled.

offlineRegion

public void offlineRegion(RegionInfo regionInfo)

onlineRegion

public void onlineRegion(RegionInfo regionInfo,
                         ServerName serverName)

getSnapShotOfAssignment

public Map<ServerName,List<RegionInfo>> getSnapShotOfAssignment(Collection<RegionInfo> regions)

getReopenStatus
```
public Pair<Integer,Integer> getReopenStatus(TableName tableName)
```
Used by the client (via master) to identify if all regions have the schema updates

Parameters:

tableName -

Returns:

Pair indicating the status of the alter command (pending/total)

Throws:

IOException

hasRegionsInTransition

public boolean hasRegionsInTransition()

getRegionsInTransition

public List<RegionStateNode> getRegionsInTransition()

getAssignedRegions

public List<RegionInfo> getAssignedRegions()

getRegionInfo

public RegionInfo getRegionInfo(byte[] regionName)

transitStateAndUpdate

private void transitStateAndUpdate(RegionStateNode regionNode,
                                   RegionState.State newState,
                                   RegionState.State... expectedStates)
                            throws IOException

Throws:: IOException

regionOpening

void regionOpening(RegionStateNode regionNode)
            throws IOException

Throws:: IOException

regionFailedOpen

void regionFailedOpen(RegionStateNode regionNode,
                      boolean giveUp)
               throws IOException

Throws:: IOException

regionClosing

void regionClosing(RegionStateNode regionNode)
            throws IOException

Throws:: IOException

regionOpenedWithoutPersistingToMeta

void regionOpenedWithoutPersistingToMeta(RegionStateNode regionNode)
                                  throws IOException

Throws:: IOException

regionClosedWithoutPersistingToMeta

void regionClosedWithoutPersistingToMeta(RegionStateNode regionNode)
                                  throws IOException

Throws:: IOException

regionClosedAbnormally

public void regionClosedAbnormally(RegionStateNode regionNode)
                            throws IOException

Throws:: IOException

persistToMeta

void persistToMeta(RegionStateNode regionNode)
            throws IOException

Throws:: IOException

markRegionAsSplit

public void markRegionAsSplit(RegionInfo parent,
                              ServerName serverName,
                              RegionInfo daughterA,
                              RegionInfo daughterB)
                       throws IOException

Throws:: IOException

markRegionAsMerged
```
public void markRegionAsMerged(RegionInfo child,
                               ServerName serverName,
                               RegionInfo[] mergeParents)
                        throws IOException
```
When called here, the merge has happened. The merged regions have been unassigned and the above markRegionClosed has been called on each so they have been disassociated from a hosting Server. The merged region will be open after this call. The merged regions are removed from hbase:meta below. Later they are deleted from the filesystem by the catalog janitor running against hbase:meta. It notices when the merged region no longer holds references to the old regions (References are deleted after a compaction rewrites what the Reference points at but not until the archiver chore runs, are the References removed).

Throws:

IOException

shouldAssignFavoredNodes

private boolean shouldAssignFavoredNodes(RegionInfo region)

queueAssign
```
protected void queueAssign(RegionStateNode regionNode)
```
Add the assign operation to the assignment queue. The pending assignment operation will be processed, and each region will be assigned by a server using the balancer.

startAssignmentThread
```
private void startAssignmentThread()
```

stopAssignmentThread
```
private void stopAssignmentThread()
```

assignQueueSignal
```
private void assignQueueSignal()
```

waitOnAssignQueue

private HashMap<RegionInfo,RegionStateNode> waitOnAssignQueue()

processAssignQueue
```
private void processAssignQueue()
```

containsBogusAssignments

private boolean containsBogusAssignments(Map<RegionInfo,RegionStateNode> regions,
                                         List<RegionInfo> hirs)

processAssignmentPlans

private void processAssignmentPlans(HashMap<RegionInfo,RegionStateNode> regions,
                                    HashMap<RegionInfo,ServerName> retainMap,
                                    List<RegionInfo> hris,
                                    List<ServerName> servers)

acceptPlan

private void acceptPlan(HashMap<RegionInfo,RegionStateNode> regions,
                        Map<ServerName,List<RegionInfo>> plan)
                 throws HBaseIOException

Throws:: HBaseIOException

addToPendingAssignment

private void addToPendingAssignment(HashMap<RegionInfo,RegionStateNode> regions,
                                    Collection<RegionInfo> pendingRegions)

getExcludedServersForSystemTable
```
public List<ServerName> getExcludedServersForSystemTable()
```
Get a list of servers that this region cannot be assigned to. For system tables, we must assign them to a server with highest version.

getMaster
```
MasterServices getMaster()
```

getRSReports

public Map<ServerName,Set<byte[]>> getRSReports()

Returns:: a snapshot of rsReports

getRegionStatesCount
```
public RegionStatesCount getRegionStatesCount(TableName tableName)
```
Provide regions state count for given table. e.g howmany regions of give table are opened/closed/rit etc

Parameters:

tableName - TableName

Returns:

region states count

Class AssignmentManager

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

BOOTSTRAP_THREAD_POOL_SIZE_CONF_KEY

ASSIGN_DISPATCH_WAIT_MSEC_CONF_KEY

DEFAULT_ASSIGN_DISPATCH_WAIT_MSEC

ASSIGN_DISPATCH_WAITQ_MAX_CONF_KEY

DEFAULT_ASSIGN_DISPATCH_WAITQ_MAX

RIT_CHORE_INTERVAL_MSEC_CONF_KEY

DEFAULT_RIT_CHORE_INTERVAL_MSEC

DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC_CONF_KEY

DEFAULT_DEAD_REGION_METRIC_CHORE_INTERVAL_MSEC

ASSIGN_MAX_ATTEMPTS

DEFAULT_ASSIGN_MAX_ATTEMPTS

ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS

DEFAULT_ASSIGN_RETRY_IMMEDIATELY_MAX_ATTEMPTS

METRICS_RIT_STUCK_WARNING_THRESHOLD

DEFAULT_RIT_STUCK_WARNING_THRESHOLD

metaAssignEvent

metaLoadEvent

metrics

ritChore

deadMetricChore

master

running

regionStates

regionStateStore

rsReports

shouldAssignRegionsWithFavoredNodes

assignDispatchWaitQueueMaxSize

assignDispatchWaitMillis

assignMaxAttempts

assignRetryImmediatelyMaxAttempts

checkIfShouldMoveSystemRegionLock

assignThread

META_REGION_SET

STATES_EXPECTED_ON_OPEN

STATES_EXPECTED_ON_CLOSING

STATES_EXPECTED_ON_CLOSED

STATES_EXPECTED_ON_ASSIGN

STATES_EXPECTED_ON_UNASSIGN_OR_MOVE

pendingAssignQueue

assignQueueLock

assignQueueFullCond

Constructor Detail

AssignmentManager

AssignmentManager

Method Detail

start

setupRIT

stop

isRunning

getConfiguration

getAssignmentManagerMetrics

getBalancer

getProcedureEnvironment

getProcedureScheduler

getAssignMaxAttempts

getAssignRetryImmediatelyMaxAttempts

getRegionStates

getRegionsOnServer

getRegionStateStore

getFavoredNodes

getTableStateManager

isTableEnabled

isTableDisabled

isMetaRegion

isMetaRegion

getMetaRegionFromName

isCarryingMeta

isCarryingRegion

getMetaForRegion

getMetaRegionSet

isMetaAssigned

isMetaRegionInTransition