org.apache.hadoop.hbase.master.assignment

Class AssignmentManager

java.lang.Object

org.apache.hadoop.hbase.master.assignment.AssignmentManager

All Implemented Interfaces:

ServerListener

@InterfaceAudience.Private
public class AssignmentManager
extends Object
implements ServerListener

The AssignmentManager is the coordinator for region assign/unassign operations.

• In-memory states of regions and servers are stored in RegionStates.
• hbase:meta state updates are handled by RegionStateStore.

Regions are created by CreateTable, Split, Merge. Regions are deleted by DeleteTable, Split, Merge. Assigns are triggered by CreateTable, EnableTable, Split, Merge, ServerCrash. Unassigns are triggered by DisableTable, Split, Merge

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	AssignmentManager.RegionInTransitionChore
static class	AssignmentManager.RegionInTransitionStat

Field Summary

Fields

Modifier and Type	Field and Description
static String	ASSIGN_DISPATCH_WAIT_MSEC_CONF_KEY
static String	ASSIGN_DISPATCH_WAITQ_MAX_CONF_KEY
static String	ASSIGN_MAX_ATTEMPTS
private static AssignProcedure[]	ASSIGN_PROCEDURE_ARRAY_TYPE
private int	assignDispatchWaitMillis
private int	assignDispatchWaitQueueMaxSize
private int	assignMaxAttempts
private Condition	assignQueueFullCond
private ReentrantLock	assignQueueLock
private Thread	assignThread
static String	BOOTSTRAP_THREAD_POOL_SIZE_CONF_KEY
private Object	checkIfShouldMoveSystemRegionLock
private static int	DEFAULT_ASSIGN_DISPATCH_WAIT_MSEC
private static int	DEFAULT_ASSIGN_DISPATCH_WAITQ_MAX
private static int	DEFAULT_ASSIGN_MAX_ATTEMPTS
private static int	DEFAULT_RIT_CHORE_INTERVAL_MSEC
private static int	DEFAULT_RIT_STUCK_WARNING_THRESHOLD
private CopyOnWriteArrayList<AssignmentListener>	listeners Listeners that are called on assignment events.
private static org.slf4j.Logger	LOG
private MasterServices	master
private static Set<RegionInfo>	META_REGION_SET
private ProcedureEvent<?>	metaAssignEvent
private ProcedureEvent<?>	metaLoadEvent
private MetricsAssignmentManager	metrics
static String	METRICS_RIT_STUCK_WARNING_THRESHOLD Region in Transition metrics threshold time
private ArrayList<RegionStates.RegionStateNode>	pendingAssignQueue
private RegionStates	regionStates
private RegionStateStore	regionStateStore
static String	RIT_CHORE_INTERVAL_MSEC_CONF_KEY
private AssignmentManager.RegionInTransitionChore	ritChore
private Map<ServerName,Set<byte[]>>	rsReports
private AtomicBoolean	running
private boolean	shouldAssignRegionsWithFavoredNodes
private static UnassignProcedure[]	UNASSIGN_PROCEDURE_ARRAY_TYPE

Constructor Summary

Constructors

Constructor and Description
AssignmentManager(MasterServices master)
AssignmentManager(MasterServices master, RegionStateStore stateStore)

Method Summary

Modifier and Type	Method and Description
private void	acceptPlan(HashMap<RegionInfo,RegionStates.RegionStateNode> regions, Map<ServerName,List<RegionInfo>> plan)
protected boolean	addRegionInTransition(RegionStates.RegionStateNode regionNode, RegionTransitionProcedure procedure)
private void	addToPendingAssignment(HashMap<RegionInfo,RegionStates.RegionStateNode> regions, Collection<RegionInfo> pendingRegions)
void	assign(RegionInfo regionInfo)
void	assign(RegionInfo regionInfo, ServerName sn)
private void	assignQueueSignal()
void	checkIfShouldMoveSystemRegionAsync() Start a new thread to check if there are region servers whose versions are higher than others.
private void	checkMetaLoaded(RegionInfo hri) Used to check if the meta loading is done.
(package private) void	checkOnlineRegionsReport(RegionStates.ServerStateNode serverNode, Set<byte[]> regionNames)
(package private) void	checkOnlineRegionsReportForMeta(ServerName serverName, Set<byte[]> regionNames)
AssignmentManager.RegionInTransitionStat	computeRegionInTransitionStat()
AssignProcedure	createAssignProcedure(RegionInfo regionInfo) Called by things like DisableTableProcedure to get a list of UnassignProcedure to unassign the regions of the table.
AssignProcedure	createAssignProcedure(RegionInfo regionInfo, boolean override)
AssignProcedure	createAssignProcedure(RegionInfo regionInfo, ServerName targetServer)
AssignProcedure	createAssignProcedure(RegionInfo regionInfo, ServerName targetServer, boolean override)
AssignProcedure[]	createAssignProcedures(List<RegionInfo> hris) Create an array of AssignProcedures w/o specifying a target server.
private AssignProcedure[]	createAssignProcedures(Map<ServerName,List<RegionInfo>> assignments, int size)
MergeTableRegionsProcedure	createMergeProcedure(RegionInfo... ris)
private MoveRegionProcedure	createMoveRegionProcedure(RegionPlan plan)
AssignProcedure[]	createRoundRobinAssignProcedures(List<RegionInfo> hris) Create round-robin assigns.
SplitTableRegionProcedure	createSplitProcedure(RegionInfo regionToSplit, byte[] splitKey)
UnassignProcedure	createUnassignProcedure(RegionInfo regionInfo)
UnassignProcedure	createUnassignProcedure(RegionInfo regionInfo, boolean override)
(package private) UnassignProcedure	createUnassignProcedure(RegionInfo regionInfo, ServerName destinationServer, boolean force)
(package private) UnassignProcedure	createUnassignProcedure(RegionInfo regionInfo, ServerName destinationServer, boolean force, boolean removeAfterUnassigning)
(package private) UnassignProcedure[]	createUnassignProcedures(Collection<RegionStates.RegionStateNode> nodes)
UnassignProcedure[]	createUnassignProcedures(TableName tableName)
void	deleteTable(TableName tableName) Delete the region states.
List<RegionInfo>	getAssignedRegions()
(package private) int	getAssignMaxAttempts()
MetricsAssignmentManager	getAssignmentManagerMetrics()
private LoadBalancer	getBalancer()
org.apache.hadoop.conf.Configuration	getConfiguration()
List<ServerName>	getExcludedServersForSystemTable() Get a list of servers that this region cannot be assigned to.
List<ServerName>	getFavoredNodes(RegionInfo regionInfo)
(package private) MasterServices	getMaster()
private ProcedureEvent<?>	getMetaAssignEvent(RegionInfo metaRegionInfo)
private RegionInfo	getMetaForRegion(RegionInfo regionInfo)
RegionInfo	getMetaRegionFromName(byte[] regionName)
Set<RegionInfo>	getMetaRegionSet()
int	getNumRegionsOpened()
private MasterProcedureEnv	getProcedureEnvironment()
private MasterProcedureScheduler	getProcedureScheduler()
RegionInfo	getRegionInfo(byte[] regionName)
List<RegionStates.RegionStateNode>	getRegionsInTransition()
RegionStates	getRegionStates()
RegionStateStore	getRegionStateStore()
Pair<Integer,Integer>	getReopenStatus(TableName tableName) Used by the client (via master) to identify if all regions have the schema updates
Map<ServerName,Set<byte[]>>	getRSReports()
Map<ServerName,List<RegionInfo>>	getSnapShotOfAssignment(Collection<RegionInfo> regions)
private List<RegionInfo>	getSystemTables(ServerName serverName)
(package private) TableStateManager	getTableStateManager()
private void	handleRegionOverStuckWarningThreshold(RegionInfo regionInfo)
boolean	hasRegionsInTransition()
boolean	isCarryingMeta(ServerName serverName)
private boolean	isCarryingRegion(ServerName serverName, RegionInfo regionInfo)
(package private) boolean	isLogSplittingDone(ServerName serverName, boolean meta) This is a very particular check.
boolean	isMetaAssigned() Notice that, this only means the meta region is available on a RS, but the AM may still be loading the region states from meta, so usually you need to check isMetaLoaded() first before checking this method, unless you can make sure that your piece of code can only be executed after AM builds the region states.
boolean	isMetaLoaded() Return whether AM finishes the meta loading, i.e, the region states rebuilding.
boolean	isMetaRegion(byte[] regionName)
private boolean	isMetaRegion(RegionInfo regionInfo)
boolean	isMetaRegionInTransition()
boolean	isRunning()
boolean	isTableDisabled(TableName tableName)
boolean	isTableEnabled(TableName tableName)
void	joinCluster()
private void	killRegionServer(ServerName serverName)
private void	loadMeta()
void	markRegionAsClosed(RegionStates.RegionStateNode regionNode)
void	markRegionAsClosing(RegionStates.RegionStateNode regionNode)
void	markRegionAsMerged(RegionInfo child, ServerName serverName, RegionInfo[] mergeParents) When called here, the merge has happened.
void	markRegionAsOpened(RegionStates.RegionStateNode regionNode)
void	markRegionAsOpening(RegionStates.RegionStateNode regionNode)
void	markRegionAsSplit(RegionInfo parent, ServerName serverName, RegionInfo daughterA, RegionInfo daughterB)
void	move(RegionInfo regionInfo)
Future<byte[]>	moveAsync(RegionPlan regionPlan)
void	offlineRegion(RegionInfo regionInfo)
void	onlineRegion(RegionInfo regionInfo, ServerName serverName)
private void	processAssignmentPlans(HashMap<RegionInfo,RegionStates.RegionStateNode> regions, HashMap<RegionInfo,ServerName> retainMap, List<RegionInfo> hris, List<ServerName> servers)
private void	processAssignQueue()
void	processOfflineRegions() Create assign procedure for offline regions.
protected void	queueAssign(RegionStates.RegionStateNode regionNode) Add the assign operation to the assignment queue.
void	registerListener(AssignmentListener listener) Add the listener to the notification list.
protected void	removeRegionInTransition(RegionStates.RegionStateNode regionNode, RegionTransitionProcedure procedure)
void	reportOnlineRegions(ServerName serverName, Set<byte[]> regionNames) the master will call this method when the RS send the regionServerReport().
org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionResponse	reportRegionStateTransition(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionRequest req)
private boolean	reportTransition(RegionStates.RegionStateNode regionNode, ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, long seqId)
private void	sendRegionClosedNotification(RegionInfo regionInfo)
private void	sendRegionOpenedNotification(RegionInfo regionInfo, ServerName serverName)
void	serverAdded(ServerName serverName) The server has joined the cluster.
void	serverRemoved(ServerName serverName) The server was removed from the cluster.
private void	setMetaAssigned(RegionInfo metaRegionInfo, boolean assigned)
private boolean	shouldAssignFavoredNodes(RegionInfo region)
void	start()
private void	startAssignmentThread()
void	stop()
private void	stopAssignmentThread()
long	submitServerCrash(ServerName serverName, boolean shouldSplitWal)
void	unassign(RegionInfo regionInfo)
void	unassign(RegionInfo regionInfo, boolean forceNewPlan)
void	undoRegionAsClosing(RegionStates.RegionStateNode regionNode)
void	undoRegionAsOpening(RegionStates.RegionStateNode regionNode)
boolean	unregisterListener(AssignmentListener listener) Remove the listener from the notification list.
private void	updateRegionMergeTransition(ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, RegionInfo merged, RegionInfo hriA, RegionInfo hriB)
private void	updateRegionsInTransitionMetrics(AssignmentManager.RegionInTransitionStat ritStat)
private void	updateRegionSplitTransition(ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, RegionInfo parent, RegionInfo hriA, RegionInfo hriB)
private void	updateRegionTransition(ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state, RegionInfo regionInfo, long seqId) Called when the RegionServer wants to report a Procedure transition.
boolean	waitMetaAssigned(Procedure<?> proc, RegionInfo regionInfo) Notice that this event does not mean the AM has already finished region state rebuilding.
boolean	waitMetaLoaded(Procedure<?> proc) Wait until AM finishes the meta loading, i.e, the region states rebuilding.
private HashMap<RegionInfo,RegionStates.RegionStateNode>	waitOnAssignQueue()
protected boolean	waitServerReportEvent(ServerName serverName, Procedure<?> proc)
(package private) void	wakeMetaLoadedEvent()
protected void	wakeServerReportEvent(RegionStates.ServerStateNode serverNode)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.ServerListener

waiting

Field Detail

LOG

private static final org.slf4j.Logger LOG

BOOTSTRAP_THREAD_POOL_SIZE_CONF_KEY

public static final String BOOTSTRAP_THREAD_POOL_SIZE_CONF_KEY

See Also:

Constant Field Values

ASSIGN_DISPATCH_WAIT_MSEC_CONF_KEY

public static final String ASSIGN_DISPATCH_WAIT_MSEC_CONF_KEY

See Also:

Constant Field Values

DEFAULT_ASSIGN_DISPATCH_WAIT_MSEC

private static final int DEFAULT_ASSIGN_DISPATCH_WAIT_MSEC

See Also:

Constant Field Values

ASSIGN_DISPATCH_WAITQ_MAX_CONF_KEY

public static final String ASSIGN_DISPATCH_WAITQ_MAX_CONF_KEY

See Also:

Constant Field Values

DEFAULT_ASSIGN_DISPATCH_WAITQ_MAX

private static final int DEFAULT_ASSIGN_DISPATCH_WAITQ_MAX

See Also:

Constant Field Values

RIT_CHORE_INTERVAL_MSEC_CONF_KEY

public static final String RIT_CHORE_INTERVAL_MSEC_CONF_KEY

See Also:

Constant Field Values

DEFAULT_RIT_CHORE_INTERVAL_MSEC

private static final int DEFAULT_RIT_CHORE_INTERVAL_MSEC

See Also:

Constant Field Values

ASSIGN_MAX_ATTEMPTS

public static final String ASSIGN_MAX_ATTEMPTS

See Also:

Constant Field Values

DEFAULT_ASSIGN_MAX_ATTEMPTS

private static final int DEFAULT_ASSIGN_MAX_ATTEMPTS

See Also:

Constant Field Values

METRICS_RIT_STUCK_WARNING_THRESHOLD

public static final String METRICS_RIT_STUCK_WARNING_THRESHOLD

Region in Transition metrics threshold time

See Also:

Constant Field Values

DEFAULT_RIT_STUCK_WARNING_THRESHOLD

private static final int DEFAULT_RIT_STUCK_WARNING_THRESHOLD

See Also:

Constant Field Values

metaAssignEvent

private final ProcedureEvent<?> metaAssignEvent

metaLoadEvent

private final ProcedureEvent<?> metaLoadEvent

listeners

private final CopyOnWriteArrayList<AssignmentListener> listeners

Listeners that are called on assignment events.

metrics

private final MetricsAssignmentManager metrics

ritChore

private final AssignmentManager.RegionInTransitionChore ritChore

master

private final MasterServices master

running

private final AtomicBoolean running

regionStates

private final RegionStates regionStates

regionStateStore

private final RegionStateStore regionStateStore

rsReports

private final Map<ServerName,Set<byte[]>> rsReports

shouldAssignRegionsWithFavoredNodes

private final boolean shouldAssignRegionsWithFavoredNodes

assignDispatchWaitQueueMaxSize

private final int assignDispatchWaitQueueMaxSize

assignDispatchWaitMillis

private final int assignDispatchWaitMillis

assignMaxAttempts

private final int assignMaxAttempts

checkIfShouldMoveSystemRegionLock

private final Object checkIfShouldMoveSystemRegionLock

assignThread

private Thread assignThread

META_REGION_SET

private static final Set<RegionInfo> META_REGION_SET

ASSIGN_PROCEDURE_ARRAY_TYPE

private static final AssignProcedure[] ASSIGN_PROCEDURE_ARRAY_TYPE

UNASSIGN_PROCEDURE_ARRAY_TYPE

private static final UnassignProcedure[] UNASSIGN_PROCEDURE_ARRAY_TYPE

pendingAssignQueue

private final ArrayList<RegionStates.RegionStateNode> pendingAssignQueue

assignQueueLock

private final ReentrantLock assignQueueLock

assignQueueFullCond

private final Condition assignQueueFullCond

Constructor Detail

AssignmentManager

public AssignmentManager(MasterServices master)

AssignmentManager

public AssignmentManager(MasterServices master,
                         RegionStateStore stateStore)

Method Detail

start

public void start()
           throws IOException,
                  org.apache.zookeeper.KeeperException

Throws:

IOException

org.apache.zookeeper.KeeperException

stop

public void stop()

isRunning

public boolean isRunning()

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

getAssignmentManagerMetrics

public MetricsAssignmentManager getAssignmentManagerMetrics()

getBalancer

private LoadBalancer getBalancer()

getProcedureEnvironment

private MasterProcedureEnv getProcedureEnvironment()

getProcedureScheduler

private MasterProcedureScheduler getProcedureScheduler()

getAssignMaxAttempts

int getAssignMaxAttempts()

registerListener

public void registerListener(AssignmentListener listener)

Add the listener to the notification list.

Parameters:

listener - The AssignmentListener to register

unregisterListener

public boolean unregisterListener(AssignmentListener listener)

Remove the listener from the notification list.

Parameters:

listener - The AssignmentListener to unregister

getRegionStates

public RegionStates getRegionStates()

getRegionStateStore

public RegionStateStore getRegionStateStore()

getFavoredNodes

public List<ServerName> getFavoredNodes(RegionInfo regionInfo)

getTableStateManager

TableStateManager getTableStateManager()

isTableEnabled

public boolean isTableEnabled(TableName tableName)

isTableDisabled

public boolean isTableDisabled(TableName tableName)

isMetaRegion

private boolean isMetaRegion(RegionInfo regionInfo)

isMetaRegion

public boolean isMetaRegion(byte[] regionName)

getMetaRegionFromName

public RegionInfo getMetaRegionFromName(byte[] regionName)

isCarryingMeta

public boolean isCarryingMeta(ServerName serverName)

isCarryingRegion

private boolean isCarryingRegion(ServerName serverName,
                                 RegionInfo regionInfo)

getMetaForRegion

private RegionInfo getMetaForRegion(RegionInfo regionInfo)

getMetaRegionSet

public Set<RegionInfo> getMetaRegionSet()

isMetaAssigned

public boolean isMetaAssigned()

Notice that, this only means the meta region is available on a RS, but the AM may still be loading the region states from meta, so usually you need to check isMetaLoaded() first before checking this method, unless you can make sure that your piece of code can only be executed after AM builds the region states.

See Also:

isMetaLoaded()

isMetaRegionInTransition

public boolean isMetaRegionInTransition()

waitMetaAssigned

public boolean waitMetaAssigned(Procedure<?> proc,
                                RegionInfo regionInfo)

Notice that this event does not mean the AM has already finished region state rebuilding. See the comment of isMetaAssigned() for more details.

See Also:

isMetaAssigned()

setMetaAssigned

private void setMetaAssigned(RegionInfo metaRegionInfo,
                             boolean assigned)

getMetaAssignEvent

private ProcedureEvent<?> getMetaAssignEvent(RegionInfo metaRegionInfo)

waitMetaLoaded

public boolean waitMetaLoaded(Procedure<?> proc)

Wait until AM finishes the meta loading, i.e, the region states rebuilding.

See Also:

isMetaLoaded(), waitMetaAssigned(Procedure, RegionInfo)

wakeMetaLoadedEvent

void wakeMetaLoadedEvent()

isMetaLoaded

public boolean isMetaLoaded()

Return whether AM finishes the meta loading, i.e, the region states rebuilding.

See Also:

isMetaAssigned(), waitMetaLoaded(Procedure)

checkIfShouldMoveSystemRegionAsync

public void checkIfShouldMoveSystemRegionAsync()

Start a new thread to check if there are region servers whose versions are higher than others. If so, move all system table regions to RS with the highest version to keep compatibility. The reason is, RS in new version may not be able to access RS in old version when there are some incompatible changes.

This method is called when a new RegionServer is added to cluster only.

getSystemTables

private List<RegionInfo> getSystemTables(ServerName serverName)

assign

public void assign(RegionInfo regionInfo,
                   ServerName sn)
            throws IOException

Throws:

IOException

assign

public void assign(RegionInfo regionInfo)
            throws IOException

Throws:

IOException

unassign

public void unassign(RegionInfo regionInfo)
              throws IOException

Throws:

IOException

unassign

public void unassign(RegionInfo regionInfo,
                     boolean forceNewPlan)
              throws IOException

Throws:

IOException

move

public void move(RegionInfo regionInfo)
          throws IOException

Throws:

IOException

moveAsync

public Future<byte[]> moveAsync(RegionPlan regionPlan)
                         throws HBaseIOException

Throws:

HBaseIOException

createRoundRobinAssignProcedures

public AssignProcedure[] createRoundRobinAssignProcedures(List<RegionInfo> hris)

Create round-robin assigns. Use on table creation to distribute out regions across cluster.

Returns:

AssignProcedures made out of the passed in hris and a call to the balancer to populate the assigns with targets chosen using round-robin (default balancer scheme). If at assign-time, the target chosen is no longer up, thats fine, the AssignProcedure will ask the balancer for a new target, and so on.

createAssignProcedures

public AssignProcedure[] createAssignProcedures(List<RegionInfo> hris)

Create an array of AssignProcedures w/o specifying a target server. If no target server, at assign time, we will try to use the former location of the region if one exists. This is how we 'retain' the old location across a server restart. Used by ServerCrashProcedure assigning regions on a server that has crashed (SCP is also used across a cluster-restart just-in-case to ensure we do cleanup of any old WALs or server processes).

createAssignProcedures

private AssignProcedure[] createAssignProcedures(Map<ServerName,List<RegionInfo>> assignments,
                                                 int size)

Parameters:

assignments - Map of assignments from which we produce an array of AssignProcedures.

size - Count of assignments to make (the caller may know the total count)

Returns:

Assignments made from the passed in assignments

createUnassignProcedures

UnassignProcedure[] createUnassignProcedures(Collection<RegionStates.RegionStateNode> nodes)

createAssignProcedure

public AssignProcedure createAssignProcedure(RegionInfo regionInfo)

Called by things like DisableTableProcedure to get a list of UnassignProcedure to unassign the regions of the table.

createAssignProcedure

public AssignProcedure createAssignProcedure(RegionInfo regionInfo,
                                             boolean override)

createAssignProcedure

public AssignProcedure createAssignProcedure(RegionInfo regionInfo,
                                             ServerName targetServer)

createAssignProcedure

public AssignProcedure createAssignProcedure(RegionInfo regionInfo,
                                             ServerName targetServer,
                                             boolean override)

createUnassignProcedure

public UnassignProcedure createUnassignProcedure(RegionInfo regionInfo)

createUnassignProcedure

public UnassignProcedure createUnassignProcedure(RegionInfo regionInfo,
                                                 boolean override)

createUnassignProcedures

public UnassignProcedure[] createUnassignProcedures(TableName tableName)

createUnassignProcedure

UnassignProcedure createUnassignProcedure(RegionInfo regionInfo,
                                          ServerName destinationServer,
                                          boolean force)

createUnassignProcedure

UnassignProcedure createUnassignProcedure(RegionInfo regionInfo,
                                          ServerName destinationServer,
                                          boolean force,
                                          boolean removeAfterUnassigning)

createMoveRegionProcedure

private MoveRegionProcedure createMoveRegionProcedure(RegionPlan plan)
                                               throws HBaseIOException

Throws:

HBaseIOException

createSplitProcedure

public SplitTableRegionProcedure createSplitProcedure(RegionInfo regionToSplit,
                                                      byte[] splitKey)
                                               throws IOException

Throws:

IOException

createMergeProcedure

public MergeTableRegionsProcedure createMergeProcedure(RegionInfo... ris)
                                                throws IOException

Throws:

IOException

deleteTable

public void deleteTable(TableName tableName)
                 throws IOException

Delete the region states. This is called by "DeleteTable"

Throws:

IOException

reportRegionStateTransition

public org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionResponse reportRegionStateTransition(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionRequest req)
                                                                                                                                           throws PleaseHoldException

Throws:

PleaseHoldException

updateRegionTransition

private void updateRegionTransition(ServerName serverName,
                                    org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                    RegionInfo regionInfo,
                                    long seqId)
                             throws PleaseHoldException,
                                    UnexpectedStateException

Called when the RegionServer wants to report a Procedure transition. Ends up calling #reportTransition(RegionStateNode, ServerName, TransitionCode, long)

Throws:

PleaseHoldException

UnexpectedStateException

reportTransition

private boolean reportTransition(RegionStates.RegionStateNode regionNode,
                                 ServerName serverName,
                                 org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                 long seqId)
                          throws UnexpectedStateException

Throws:

UnexpectedStateException

updateRegionSplitTransition

private void updateRegionSplitTransition(ServerName serverName,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                         RegionInfo parent,
                                         RegionInfo hriA,
                                         RegionInfo hriB)
                                  throws IOException

Throws:

IOException

updateRegionMergeTransition

private void updateRegionMergeTransition(ServerName serverName,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode state,
                                         RegionInfo merged,
                                         RegionInfo hriA,
                                         RegionInfo hriB)
                                  throws IOException

Throws:

IOException

reportOnlineRegions

public void reportOnlineRegions(ServerName serverName,
                                Set<byte[]> regionNames)
                         throws YouAreDeadException

the master will call this method when the RS send the regionServerReport(). the report will contains the "online regions". this method will check the the online regions against the in-memory state of the AM, if there is a mismatch we will try to fence out the RS with the assumption that something went wrong on the RS side.

Throws:

YouAreDeadException

checkOnlineRegionsReportForMeta

void checkOnlineRegionsReportForMeta(ServerName serverName,
                                     Set<byte[]> regionNames)

checkOnlineRegionsReport

void checkOnlineRegionsReport(RegionStates.ServerStateNode serverNode,
                              Set<byte[]> regionNames)

waitServerReportEvent

protected boolean waitServerReportEvent(ServerName serverName,
                                        Procedure<?> proc)

wakeServerReportEvent

protected void wakeServerReportEvent(RegionStates.ServerStateNode serverNode)

computeRegionInTransitionStat

public AssignmentManager.RegionInTransitionStat computeRegionInTransitionStat()

updateRegionsInTransitionMetrics

private void updateRegionsInTransitionMetrics(AssignmentManager.RegionInTransitionStat ritStat)

handleRegionOverStuckWarningThreshold

private void handleRegionOverStuckWarningThreshold(RegionInfo regionInfo)

joinCluster

public void joinCluster()
                 throws IOException

Throws:

IOException

processOfflineRegions

public void processOfflineRegions()

Create assign procedure for offline regions. Just follow the old processofflineServersWithOnlineRegions method. Since now we do not need to deal with dead server any more, we only deal with the regions in OFFLINE state in this method. And this is a bit strange, that for new regions, we will add it in CLOSED state instead of OFFLINE state, and usually there will be a procedure to track them. The processofflineServersWithOnlineRegions is a legacy from long ago, as things are going really different now, maybe we do not need this method any more. Need to revisit later.

loadMeta

private void loadMeta()
               throws IOException

Throws:

IOException

checkMetaLoaded

private void checkMetaLoaded(RegionInfo hri)
                      throws PleaseHoldException

Used to check if the meta loading is done.

if not we throw PleaseHoldException since we are rebuilding the RegionStates

Parameters:

hri - region to check if it is already rebuild

Throws:

PleaseHoldException - if meta has not been loaded yet

getNumRegionsOpened

public int getNumRegionsOpened()

submitServerCrash

public long submitServerCrash(ServerName serverName,
                              boolean shouldSplitWal)

offlineRegion

public void offlineRegion(RegionInfo regionInfo)

onlineRegion

public void onlineRegion(RegionInfo regionInfo,
                         ServerName serverName)

getSnapShotOfAssignment

public Map<ServerName,List<RegionInfo>> getSnapShotOfAssignment(Collection<RegionInfo> regions)

getReopenStatus

public Pair<Integer,Integer> getReopenStatus(TableName tableName)

Used by the client (via master) to identify if all regions have the schema updates

Parameters:

tableName -

Returns:

Pair indicating the status of the alter command (pending/total)

Throws:

IOException

addRegionInTransition

protected boolean addRegionInTransition(RegionStates.RegionStateNode regionNode,
                                        RegionTransitionProcedure procedure)

removeRegionInTransition

protected void removeRegionInTransition(RegionStates.RegionStateNode regionNode,
                                        RegionTransitionProcedure procedure)

hasRegionsInTransition

public boolean hasRegionsInTransition()

getRegionsInTransition

public List<RegionStates.RegionStateNode> getRegionsInTransition()

getAssignedRegions

public List<RegionInfo> getAssignedRegions()

getRegionInfo

public RegionInfo getRegionInfo(byte[] regionName)

sendRegionOpenedNotification

private void sendRegionOpenedNotification(RegionInfo regionInfo,
                                          ServerName serverName)

sendRegionClosedNotification

private void sendRegionClosedNotification(RegionInfo regionInfo)

markRegionAsOpening

public void markRegionAsOpening(RegionStates.RegionStateNode regionNode)
                         throws IOException

Throws:

IOException

undoRegionAsOpening

public void undoRegionAsOpening(RegionStates.RegionStateNode regionNode)

markRegionAsOpened

public void markRegionAsOpened(RegionStates.RegionStateNode regionNode)
                        throws IOException

Throws:

IOException

markRegionAsClosing

public void markRegionAsClosing(RegionStates.RegionStateNode regionNode)
                         throws IOException

Throws:

IOException

undoRegionAsClosing

public void undoRegionAsClosing(RegionStates.RegionStateNode regionNode)

markRegionAsClosed

public void markRegionAsClosed(RegionStates.RegionStateNode regionNode)
                        throws IOException

Throws:

IOException

markRegionAsSplit

public void markRegionAsSplit(RegionInfo parent,
                              ServerName serverName,
                              RegionInfo daughterA,
                              RegionInfo daughterB)
                       throws IOException

Throws:

IOException

markRegionAsMerged

public void markRegionAsMerged(RegionInfo child,
                               ServerName serverName,
                               RegionInfo[] mergeParents)
                        throws IOException

When called here, the merge has happened. The merged regions have been unassigned and the above markRegionClosed has been called on each so they have been disassociated from a hosting Server. The merged region will be open after this call. The merged regions are removed from hbase:meta below. Later they are deleted from the filesystem by the catalog janitor running against hbase:meta. It notices when the merged region no longer holds references to the old regions (References are deleted after a compaction rewrites what the Reference points at but not until the archiver chore runs, are the References removed).

Throws:

IOException

shouldAssignFavoredNodes

private boolean shouldAssignFavoredNodes(RegionInfo region)

queueAssign

protected void queueAssign(RegionStates.RegionStateNode regionNode)

Add the assign operation to the assignment queue. The pending assignment operation will be processed, and each region will be assigned by a server using the balancer.

startAssignmentThread

private void startAssignmentThread()

stopAssignmentThread

private void stopAssignmentThread()

assignQueueSignal

private void assignQueueSignal()

waitOnAssignQueue

private HashMap<RegionInfo,RegionStates.RegionStateNode> waitOnAssignQueue()

processAssignQueue

private void processAssignQueue()

processAssignmentPlans

private void processAssignmentPlans(HashMap<RegionInfo,RegionStates.RegionStateNode> regions,
                                    HashMap<RegionInfo,ServerName> retainMap,
                                    List<RegionInfo> hris,
                                    List<ServerName> servers)

acceptPlan

private void acceptPlan(HashMap<RegionInfo,RegionStates.RegionStateNode> regions,
                        Map<ServerName,List<RegionInfo>> plan)
                 throws HBaseIOException

Throws:

HBaseIOException

addToPendingAssignment

private void addToPendingAssignment(HashMap<RegionInfo,RegionStates.RegionStateNode> regions,
                                    Collection<RegionInfo> pendingRegions)

getExcludedServersForSystemTable

public List<ServerName> getExcludedServersForSystemTable()

Get a list of servers that this region cannot be assigned to. For system tables, we must assign them to a server with highest version.

serverAdded

public void serverAdded(ServerName serverName)

Description copied from interface: ServerListener

The server has joined the cluster.

Specified by:

serverAdded in interface ServerListener

Parameters:

serverName - The remote servers name.

serverRemoved

public void serverRemoved(ServerName serverName)

Description copied from interface: ServerListener

The server was removed from the cluster.

Specified by:

serverRemoved in interface ServerListener

Parameters:

serverName - The remote servers name.

killRegionServer

private void killRegionServer(ServerName serverName)

isLogSplittingDone

boolean isLogSplittingDone(ServerName serverName,
                           boolean meta)

This is a very particular check. The ServerManager is where you go to check on state of 'Servers', what Servers are online, etc.

Here we are checking the state of a server that is post expiration, a ServerManager function that moves a server from online to dead. Here we are seeing if the server has moved beyond a particular point in the recovery process such that it is safe to move on with assigns; etc.

For now it is only used in UnassignProcedure#remoteCallFailed(MasterProcedureEnv, RegionStateNode, IOException) to see whether we can safely quit without losing data.

Parameters:

meta - whether to check for meta log splitting

Returns:

true if the server does not exist or the log splitting is done, i.e, the server is in OFFLINE state, or for meta log, is in SPLITTING_META_DONE state. If null, presumes the ServerStateNode was cleaned up by SCP.

See Also:

UnassignProcedure#remoteCallFailed(MasterProcedureEnv, RegionStateNode, IOException)

getMaster

MasterServices getMaster()

getRSReports

public Map<ServerName,Set<byte[]>> getRSReports()

Returns:

a snapshot of rsReports