org.apache.hadoop.hbase.master

Class AssignmentManager

java.lang.Object

org.apache.hadoop.hbase.zookeeper.ZooKeeperListener

org.apache.hadoop.hbase.master.AssignmentManager

@InterfaceAudience.Private
public class AssignmentManager
extends ZooKeeperListener

Manages and performs region assignment.

Monitors ZooKeeper for events related to regions in transition.

Handles existing regions in transition during master failover.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AssignmentManager.ServerHostRegion

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicBoolean	failoverCleanupDone Indicator that AssignmentManager has recovered the region states so that ServerShutdownHandler can be fully enabled and re-assign regions of dead servers.
static ServerName	HBCK_CODE_SERVERNAME
protected Server	server
static boolean	TEST_SKIP_SPLIT_HANDLING For testing only! Set to true to skip handling of split.

Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener

watcher

Constructor Summary

Constructors

Constructor and Description
AssignmentManager(Server server, ServerManager serverManager, LoadBalancer balancer, ExecutorService service, MetricsMaster metricsMaster, TableLockManager tableLockManager) Constructs a new assignment manager.

Method Summary

Methods

Modifier and Type	Method and Description
Lock	acquireRegionLock(String encodedName) To avoid racing with AM, external entities may need to lock a region, for example, when SSH checks what regions to skip re-assigning.
void	addPlan(String encodedName, RegionPlan plan) Add a regionPlan for the specified region.
void	addPlans(Map<String,RegionPlan> plans) Add a map of region plans.
void	assign(HRegionInfo region, boolean setOfflineInZK) Assigns the specified region.
void	assign(HRegionInfo region, boolean setOfflineInZK, boolean forceNewPlan) Use care with forceNewPlan.
void	assign(List<HRegionInfo> regions) Assigns specified regions round robin, if any.
void	assign(Map<HRegionInfo,ServerName> regions) Assigns specified regions retaining assignments, if any.
void	assignMeta(HRegionInfo hri) Assigns the hbase:meta region or a replica.
void	balance(RegionPlan plan)
void	deleteClosingOrClosedNode(HRegionInfo region, ServerName sn)
LoadBalancer	getBalancer()
int	getNumRegionsOpened() Used by unit tests.
RegionPlan	getRegionReopenPlan(HRegionInfo hri)
RegionStates	getRegionStates() This SHOULD not be public.
Pair<Integer,Integer>	getReopenStatus(TableName tableName) Used by the client to identify if all regions have the schema updates
Set<HRegionInfo>	getReplicasToClose()
Map<ServerName,List<HRegionInfo>>	getSnapShotOfAssignment(Collection<HRegionInfo> infos)
TableStateManager	getTableStateManager()
AssignmentManager.ServerHostRegion	isCarryingMeta(ServerName serverName)
AssignmentManager.ServerHostRegion	isCarryingMetaReplica(ServerName serverName, HRegionInfo metaHri)
AssignmentManager.ServerHostRegion	isCarryingMetaReplica(ServerName serverName, int replicaId)
boolean	isFailoverCleanupDone() Used by ServerShutdownHandler to make sure AssignmentManager has completed the failover cleanup before re-assigning regions of dead servers.
void	nodeChildrenChanged(String path) New unassigned node has been created.
void	nodeCreated(String path) New unassigned node has been created.
void	nodeDataChanged(String path) Existing unassigned node has had data changed.
void	nodeDeleted(String path) Called when a node has been deleted
void	offlineDisabledRegion(HRegionInfo regionInfo)
protected String	onRegionTransition(ServerName serverName, RegionServerStatusProtos.RegionStateTransition transition) Try to update some region states.
List<HRegionInfo>	processServerShutdown(ServerName sn) Process shutdown server removing any assignments.
void	regionOffline(HRegionInfo regionInfo) Marks the region as offline.
void	registerListener(AssignmentListener listener) Add the listener to the notification list.
void	removeClosedRegion(HRegionInfo hri) When a region is closed, it should be removed from the regionsToReopen
static List<HRegionInfo>	replicaRegionsNotRecordedInMeta(Set<HRegionInfo> regionsRecordedInMeta, MasterServices master) Get a list of replica regions that are: not recorded in meta yet.
protected void	setEnabledTable(TableName tableName)
void	setRegionsToReopen(List<HRegionInfo> regions) Set the list of regions that will be reopened because of an update in table schema
void	shutdown() Shutdown the threadpool executor service
void	stop()
void	unassign(HRegionInfo region) Unassigns the specified region.
void	unassign(HRegionInfo region, boolean force)
void	unassign(HRegionInfo region, boolean force, ServerName dest) Unassigns the specified region.
boolean	unregisterListener(AssignmentListener listener) Remove the listener from the notification list.
void	updateRegionsInTransitionMetrics() Set Regions in transitions metrics.
protected boolean	waitForAssignment(Collection<HRegionInfo> regionSet, boolean waitTillAllAssigned, int reassigningRegions, long minEndTime) Waits until the specified region has completed assignment, or the deadline is reached.
protected boolean	waitForAssignment(Collection<HRegionInfo> regionSet, boolean waitTillAllAssigned, long deadline) Waits until the specified region has completed assignment, or the deadline is reached.
boolean	waitForAssignment(HRegionInfo regionInfo) Waits until the specified region has completed assignment.
void	waitOnRegionToClearRegionsInTransition(HRegionInfo hri) Wait on region to clear regions-in-transition.
boolean	waitOnRegionToClearRegionsInTransition(HRegionInfo hri, long timeOut) Wait on region to clear regions-in-transition or time out
protected boolean	wasRegionOnDeadServerByMeta(HRegionInfo region, ServerName sn)
protected void	zkEventWorkersSubmit(org.apache.hadoop.hbase.master.AssignmentManager.RegionRunnable regRunnable) Submit a task, ensuring that there is only one task at a time that working on a given region.

Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener

getWatcher

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HBCK_CODE_SERVERNAME

public static final ServerName HBCK_CODE_SERVERNAME

server

protected final Server server

failoverCleanupDone

protected final AtomicBoolean failoverCleanupDone

Indicator that AssignmentManager has recovered the region states so that ServerShutdownHandler can be fully enabled and re-assign regions of dead servers. So that when re-assignment happens, AssignmentManager has proper region states. Protected to ease testing.

TEST_SKIP_SPLIT_HANDLING

public static boolean TEST_SKIP_SPLIT_HANDLING

For testing only! Set to true to skip handling of split.

Constructor Detail

AssignmentManager

public AssignmentManager(Server server,
                 ServerManager serverManager,
                 LoadBalancer balancer,
                 ExecutorService service,
                 MetricsMaster metricsMaster,
                 TableLockManager tableLockManager)
                  throws org.apache.zookeeper.KeeperException,
                         IOException,
                         CoordinatedStateException

Constructs a new assignment manager.

Parameters:

server - instance of HMaster this AM running inside

serverManager - serverManager for associated HMaster

balancer - implementation of LoadBalancer

service - Executor service

metricsMaster - metrics manager

tableLockManager - TableLock manager

Throws:

org.apache.zookeeper.KeeperException

IOException

CoordinatedStateException

Method Detail

registerListener

public void registerListener(AssignmentListener listener)

Add the listener to the notification list.

Parameters:

listener - The AssignmentListener to register

unregisterListener

public boolean unregisterListener(AssignmentListener listener)

Remove the listener from the notification list.

Parameters:

listener - The AssignmentListener to unregister

getTableStateManager

public TableStateManager getTableStateManager()

Returns:

Instance of ZKTableStateManager.

getRegionStates

public RegionStates getRegionStates()

This SHOULD not be public. It is public now because of some unit tests. TODO: make it package private and keep RegionStates in the master package

getRegionReopenPlan

public RegionPlan getRegionReopenPlan(HRegionInfo hri)

addPlan

public void addPlan(String encodedName,
           RegionPlan plan)

Add a regionPlan for the specified region.

Parameters:

encodedName -

plan -

addPlans

public void addPlans(Map<String,RegionPlan> plans)

Add a map of region plans.

setRegionsToReopen

public void setRegionsToReopen(List<HRegionInfo> regions)

Set the list of regions that will be reopened because of an update in table schema

Parameters:

regions - list of regions that should be tracked for reopen

getReopenStatus

public Pair<Integer,Integer> getReopenStatus(TableName tableName)
                                      throws IOException

Used by the client to identify if all regions have the schema updates

Parameters:

tableName -

Returns:

Pair indicating the status of the alter command

Throws:

IOException

isFailoverCleanupDone

public boolean isFailoverCleanupDone()

Used by ServerShutdownHandler to make sure AssignmentManager has completed the failover cleanup before re-assigning regions of dead servers. So that when re-assignment happens, AssignmentManager has proper region states.

acquireRegionLock

public Lock acquireRegionLock(String encodedName)

To avoid racing with AM, external entities may need to lock a region, for example, when SSH checks what regions to skip re-assigning.

removeClosedRegion

public void removeClosedRegion(HRegionInfo hri)

When a region is closed, it should be removed from the regionsToReopen

Parameters:

hri - HRegionInfo of the region which was closed

nodeCreated

public void nodeCreated(String path)

New unassigned node has been created.

This happens when an RS begins the OPENING or CLOSING of a region by creating an unassigned node.

When this happens we must:

1. Watch the node for further events
2. Read and handle the state in the node

Overrides:

nodeCreated in class ZooKeeperListener

Parameters:

path - full path of the new node

nodeDataChanged

public void nodeDataChanged(String path)

Existing unassigned node has had data changed.

This happens when an RS transitions from OFFLINE to OPENING, or between OPENING/OPENED and CLOSING/CLOSED.

When this happens we must:

1. Watch the node for further events
2. Read and handle the state in the node

Overrides:

nodeDataChanged in class ZooKeeperListener

Parameters:

path - full path of the updated node

zkEventWorkersSubmit

protected void zkEventWorkersSubmit(org.apache.hadoop.hbase.master.AssignmentManager.RegionRunnable regRunnable)

Submit a task, ensuring that there is only one task at a time that working on a given region. Order is respected.

nodeDeleted

public void nodeDeleted(String path)

Description copied from class: ZooKeeperListener

Called when a node has been deleted

Overrides:

nodeDeleted in class ZooKeeperListener

Parameters:

path - full path of the deleted node

nodeChildrenChanged

public void nodeChildrenChanged(String path)

New unassigned node has been created.

This happens when an RS begins the OPENING, SPLITTING or CLOSING of a region by creating a znode.

When this happens we must:

1. Watch the node for further children changed events
2. Watch all new children for changed events

Overrides:

nodeChildrenChanged in class ZooKeeperListener

Parameters:

path - full path of the node whose children have changed

regionOffline

public void regionOffline(HRegionInfo regionInfo)

Marks the region as offline. Removes it from regions in transition and removes in-memory assignment information.

Used when a region has been closed and should remain closed.

Parameters:

regionInfo -

offlineDisabledRegion

public void offlineDisabledRegion(HRegionInfo regionInfo)

assign

public void assign(HRegionInfo region,
          boolean setOfflineInZK)

Assigns the specified region.

If a RegionPlan is available with a valid destination then it will be used to determine what server region is assigned to. If no RegionPlan is available, region will be assigned to a random available server.

Updates the RegionState and sends the OPEN RPC.

This will only succeed if the region is in transition and in a CLOSED or OFFLINE state or not in transition (in-memory not zk), and of course, the chosen server is up and running (It may have just crashed!). If the in-memory checks pass, the zk node is forced to OFFLINE before assigning.

Parameters:

region - server to be assigned

setOfflineInZK - whether ZK node should be created/transitioned to an OFFLINE state before assigning the region

assign

public void assign(HRegionInfo region,
          boolean setOfflineInZK,
          boolean forceNewPlan)

Use care with forceNewPlan. It could cause double assignment.

wasRegionOnDeadServerByMeta

protected boolean wasRegionOnDeadServerByMeta(HRegionInfo region,
                                  ServerName sn)

unassign

public void unassign(HRegionInfo region)

Unassigns the specified region.

Updates the RegionState and sends the CLOSE RPC unless region is being split by regionserver; then the unassign fails (silently) because we presume the region being unassigned no longer exists (its been split out of existence). TODO: What to do if split fails and is rolled back and parent is revivified?

If a RegionPlan is already set, it will remain.

Parameters:

region - server to be unassigned

unassign

public void unassign(HRegionInfo region,
            boolean force,
            ServerName dest)

Unassigns the specified region.

Updates the RegionState and sends the CLOSE RPC unless region is being split by regionserver; then the unassign fails (silently) because we presume the region being unassigned no longer exists (its been split out of existence). TODO: What to do if split fails and is rolled back and parent is revivified?

If a RegionPlan is already set, it will remain.

Parameters:

region - server to be unassigned

force - if region should be closed even if already closing

unassign

public void unassign(HRegionInfo region,
            boolean force)

deleteClosingOrClosedNode

public void deleteClosingOrClosedNode(HRegionInfo region,
                             ServerName sn)

Parameters:

region - regioninfo of znode to be deleted.

getNumRegionsOpened

public int getNumRegionsOpened()

Used by unit tests. Return the number of regions opened so far in the life of the master. Increases by one every time the master opens a region

Returns:

the counter value of the number of regions opened so far

waitForAssignment

public boolean waitForAssignment(HRegionInfo regionInfo)
                          throws InterruptedException

Waits until the specified region has completed assignment.

If the region is already assigned, returns immediately. Otherwise, method blocks until the region is assigned.

Parameters:

regionInfo - region to wait on assignment for

Returns:

true if the region is assigned false otherwise.

Throws:

InterruptedException

waitForAssignment

protected boolean waitForAssignment(Collection<HRegionInfo> regionSet,
                        boolean waitTillAllAssigned,
                        int reassigningRegions,
                        long minEndTime)
                             throws InterruptedException

Waits until the specified region has completed assignment, or the deadline is reached.

Throws:

InterruptedException

waitForAssignment

protected boolean waitForAssignment(Collection<HRegionInfo> regionSet,
                        boolean waitTillAllAssigned,
                        long deadline)
                             throws InterruptedException

Waits until the specified region has completed assignment, or the deadline is reached.

Parameters:

regionSet - set of region to wait on. the set is modified and the assigned regions removed

waitTillAllAssigned - true if we should wait all the regions to be assigned

deadline - the timestamp after which the wait is aborted

Returns:

true if all the regions are assigned false otherwise.

Throws:

InterruptedException

assignMeta

public void assignMeta(HRegionInfo hri)
                throws org.apache.zookeeper.KeeperException

Assigns the hbase:meta region or a replica.

Assumes that hbase:meta is currently closed and is not being actively served by any RegionServer.

Forcibly unsets the current meta region location in ZooKeeper and assigns hbase:meta to a random RegionServer.

Parameters:

hri - TODO

Throws:

org.apache.zookeeper.KeeperException

assign

public void assign(Map<HRegionInfo,ServerName> regions)
            throws IOException,
                   InterruptedException

Assigns specified regions retaining assignments, if any.

This is a synchronous call and will return once every region has been assigned. If anything fails, an exception is thrown

Throws:

InterruptedException

IOException

assign

public void assign(List<HRegionInfo> regions)
            throws IOException,
                   InterruptedException

Assigns specified regions round robin, if any.

This is a synchronous call and will return once every region has been assigned. If anything fails, an exception is thrown

Throws:

InterruptedException

IOException

replicaRegionsNotRecordedInMeta

public static List<HRegionInfo> replicaRegionsNotRecordedInMeta(Set<HRegionInfo> regionsRecordedInMeta,
                                                MasterServices master)
                                                         throws IOException

Get a list of replica regions that are: not recorded in meta yet. We might not have recorded the locations for the replicas since the replicas may not have been online yet, master restarted in the middle of assigning, ZK erased, etc.

Parameters:

regionsRecordedInMeta - the list of regions we know are recorded in meta either as a default, or, as the location of a replica

master -

Returns:

list of replica regions

Throws:

IOException

updateRegionsInTransitionMetrics

public void updateRegionsInTransitionMetrics()

Set Regions in transitions metrics. This takes an iterator on the RegionInTransition map (CLSM), and is not synchronized. This iterator is not fail fast, which may lead to stale read; but that's better than creating a copy of the map for metrics computation, as this method will be invoked on a frequent interval.

waitOnRegionToClearRegionsInTransition

public void waitOnRegionToClearRegionsInTransition(HRegionInfo hri)
                                            throws IOException,
                                                   InterruptedException

Wait on region to clear regions-in-transition.

Parameters:

hri - Region to wait on.

Throws:

IOException

InterruptedException

waitOnRegionToClearRegionsInTransition

public boolean waitOnRegionToClearRegionsInTransition(HRegionInfo hri,
                                             long timeOut)
                                               throws InterruptedException

Wait on region to clear regions-in-transition or time out

Parameters:

hri -

timeOut - Milliseconds to wait for current region to be out of transition state.

Returns:

True when a region clears regions-in-transition before timeout otherwise false

Throws:

InterruptedException

isCarryingMeta

public AssignmentManager.ServerHostRegion isCarryingMeta(ServerName serverName)

isCarryingMetaReplica

public AssignmentManager.ServerHostRegion isCarryingMetaReplica(ServerName serverName,
                                                       int replicaId)

isCarryingMetaReplica

public AssignmentManager.ServerHostRegion isCarryingMetaReplica(ServerName serverName,
                                                       HRegionInfo metaHri)

processServerShutdown

public List<HRegionInfo> processServerShutdown(ServerName sn)

Process shutdown server removing any assignments.

Parameters:

sn - Server that went down.

Returns:

list of regions in transition on this server

balance

public void balance(RegionPlan plan)

Parameters:

plan - Plan to execute.

stop

public void stop()

shutdown

public void shutdown()

Shutdown the threadpool executor service

setEnabledTable

protected void setEnabledTable(TableName tableName)

getReplicasToClose

public Set<HRegionInfo> getReplicasToClose()

onRegionTransition

protected String onRegionTransition(ServerName serverName,
                        RegionServerStatusProtos.RegionStateTransition transition)

Try to update some region states. If the state machine prevents such update, an error message is returned to explain the reason. It's expected that in each transition there should have just one region for opening/closing, 3 regions for splitting/merging. These regions should be on the server that requested the change. Region state machine. Only these transitions are expected to be triggered by a region server. On the state transition: (1) Open/Close should be initiated by master (a) Master sets the region to pending_open/pending_close in memory and hbase:meta after sending the request to the region server (b) Region server reports back to the master after open/close is done (either success/failure) (c) If region server has problem to report the status to master, it must be because the master is down or some temporary network issue. Otherwise, the region server should abort since it must be a bug. If the master is not accessible, the region server should keep trying until the server is stopped or till the status is reported to the (new) master (d) If region server dies in the middle of opening/closing a region, SSH picks it up and finishes it (e) If master dies in the middle, the new master recovers the state during initialization from hbase:meta. Region server can report any transition that has not been reported to the previous active master yet (2) Split/merge is initiated by region servers (a) To split a region, a region server sends a request to master to try to set a region to splitting, together with two daughters (to be created) to splitting new. If approved by the master, the splitting can then move ahead (b) To merge two regions, a region server sends a request to master to try to set the new merged region (to be created) to merging_new, together with two regions (to be merged) to merging. If it is ok with the master, the merge can then move ahead (c) Once the splitting/merging is done, the region server reports the status back to the master either success/failure. (d) Other scenarios should be handled similarly as for region open/close

getBalancer

public LoadBalancer getBalancer()

Returns:

Instance of load balancer

getSnapShotOfAssignment

public Map<ServerName,List<HRegionInfo>> getSnapShotOfAssignment(Collection<HRegionInfo> infos)