org.apache.hadoop.hbase.master.balancer

Class BaseLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, ConfigurationObserver, LoadBalancer, Stoppable

Direct Known Subclasses:

FavoredNodeLoadBalancer, SimpleLoadBalancer, StochasticLoadBalancer

public abstract class BaseLoadBalancer
extends Object
implements LoadBalancer

The base class for load balancers. It provides the the functions used to by AssignmentManager to assign regions in the edge cases. It doesn't provide an implementation of the actual balancing algorithm.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	BaseLoadBalancer.Cluster An efficient array based implementation similar to ClusterState for keeping the status of the cluster in terms of region assignment and distribution.

Field Summary

Fields

Modifier and Type	Field and Description
protected ClusterStatus	clusterStatus
protected org.apache.hadoop.conf.Configuration	config
protected ServerName	masterServerName
protected MetricsBalancer	metricsBalancer
protected RackManager	rackManager
protected org.apache.hadoop.hbase.master.balancer.RegionLocationFinder	regionFinder
protected MasterServices	services
protected float	slop
static String	TABLES_ON_MASTER
protected Set<String>	tablesOnMaster

Constructor Summary

Constructors

Constructor and Description
BaseLoadBalancer()

Method Summary

Methods

Modifier and Type	Method and Description
protected boolean	areSomeRegionReplicasColocated(BaseLoadBalancer.Cluster c) Subclasses should implement this to return true if the cluster has nodes that hosts multiple replicas for the same region, or, if there are multiple racks and the same rack hosts replicas of the same region
protected Map<ServerName,List<HRegionInfo>>	assignMasterRegions(Collection<HRegionInfo> regions, List<ServerName> servers) Assign the regions that should be on master regionserver.
protected List<RegionPlan>	balanceMasterRegions(Map<ServerName,List<HRegionInfo>> clusterMap) Balance the regions that should be on master regionserver.
protected BaseLoadBalancer.Cluster	createCluster(List<ServerName> servers, Collection<HRegionInfo> regions)
org.apache.hadoop.conf.Configuration	getConf()
protected Map<ServerName,List<HRegionInfo>>	getRegionAssignmentsByServer(Collection<HRegionInfo> regions)
protected static String[]	getTablesOnMaster(org.apache.hadoop.conf.Configuration conf)
Map<HRegionInfo,ServerName>	immediateAssignment(List<HRegionInfo> regions, List<ServerName> servers) Generates an immediate assignment plan to be used by a new master for regions in transition that do not have an already known destination.
void	initialize() Initialize the load balancer.
boolean	isStopped()
protected boolean	needsBalance(BaseLoadBalancer.Cluster c)
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf) This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
ServerName	randomAssignment(HRegionInfo regionInfo, List<ServerName> servers) Used to assign a single region to a random server.
void	regionOffline(HRegionInfo regionInfo) Marks the region as offline at balancer.
void	regionOnline(HRegionInfo regionInfo, ServerName sn) Marks the region as online at balancer.
Map<ServerName,List<HRegionInfo>>	retainAssignment(Map<HRegionInfo,ServerName> regions, List<ServerName> servers) Generates a bulk assignment startup plan, attempting to reuse the existing assignment information from META, but adjusting for the specified list of available/online servers available for assignment.
Map<ServerName,List<HRegionInfo>>	roundRobinAssignment(List<HRegionInfo> regions, List<ServerName> servers) Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.
void	setClusterStatus(ClusterStatus st) Set the current cluster status.
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setMasterServices(MasterServices masterServices) Set the master service.
void	setRackManager(RackManager rackManager)
protected void	setSlop(org.apache.hadoop.conf.Configuration conf)
boolean	shouldBeOnMaster(HRegionInfo region) Check if a region belongs to some small system table.
void	stop(String why) Stop this service.
static boolean	tablesOnMaster(org.apache.hadoop.conf.Configuration conf) Check if configured to put any tables on the active master

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

balanceCluster

Field Detail

regionFinder

protected final org.apache.hadoop.hbase.master.balancer.RegionLocationFinder regionFinder

slop

protected float slop

config

protected org.apache.hadoop.conf.Configuration config

rackManager

protected RackManager rackManager

TABLES_ON_MASTER

public static final String TABLES_ON_MASTER

See Also:

Constant Field Values

tablesOnMaster

protected final Set<String> tablesOnMaster

metricsBalancer

protected final MetricsBalancer metricsBalancer

clusterStatus

protected ClusterStatus clusterStatus

masterServerName

protected ServerName masterServerName

services

protected MasterServices services

Constructor Detail

BaseLoadBalancer

public BaseLoadBalancer()

Method Detail

getTablesOnMaster

protected static String[] getTablesOnMaster(org.apache.hadoop.conf.Configuration conf)

tablesOnMaster

public static boolean tablesOnMaster(org.apache.hadoop.conf.Configuration conf)

Check if configured to put any tables on the active master

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

setSlop

protected void setSlop(org.apache.hadoop.conf.Configuration conf)

shouldBeOnMaster

public boolean shouldBeOnMaster(HRegionInfo region)

Check if a region belongs to some small system table. If so, the primary replica may be expected to be put on the master regionserver.

balanceMasterRegions

protected List<RegionPlan> balanceMasterRegions(Map<ServerName,List<HRegionInfo>> clusterMap)

Balance the regions that should be on master regionserver.

assignMasterRegions

protected Map<ServerName,List<HRegionInfo>> assignMasterRegions(Collection<HRegionInfo> regions,
                                                    List<ServerName> servers)

Assign the regions that should be on master regionserver.

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setClusterStatus

public void setClusterStatus(ClusterStatus st)

Description copied from interface: LoadBalancer

Set the current cluster status. This allows a LoadBalancer to map host name to a server

Specified by:

setClusterStatus in interface LoadBalancer

setMasterServices

public void setMasterServices(MasterServices masterServices)

Description copied from interface: LoadBalancer

Set the master service.

Specified by:

setMasterServices in interface LoadBalancer

setRackManager

public void setRackManager(RackManager rackManager)

needsBalance

protected boolean needsBalance(BaseLoadBalancer.Cluster c)

areSomeRegionReplicasColocated

protected boolean areSomeRegionReplicasColocated(BaseLoadBalancer.Cluster c)

Subclasses should implement this to return true if the cluster has nodes that hosts multiple replicas for the same region, or, if there are multiple racks and the same rack hosts replicas of the same region

Parameters:

c - Cluster information

Returns:

whether region replicas are currently co-located

roundRobinAssignment

public Map<ServerName,List<HRegionInfo>> roundRobinAssignment(List<HRegionInfo> regions,
                                                     List<ServerName> servers)

Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.

Takes a list of all the regions and all the servers in the cluster and returns a map of each server to the regions that it should be assigned.

Currently implemented as a round-robin assignment. Same invariant as load balancing, all servers holding floor(avg) or ceiling(avg). TODO: Use block locations from HDFS to place regions with their blocks

Specified by:

roundRobinAssignment in interface LoadBalancer

Parameters:

regions - all regions

servers - all servers

Returns:

map of server to the regions it should take, or null if no assignment is possible (ie. no regions or no servers)

createCluster

protected BaseLoadBalancer.Cluster createCluster(List<ServerName> servers,
                                     Collection<HRegionInfo> regions)

immediateAssignment

public Map<HRegionInfo,ServerName> immediateAssignment(List<HRegionInfo> regions,
                                              List<ServerName> servers)

Generates an immediate assignment plan to be used by a new master for regions in transition that do not have an already known destination. Takes a list of regions that need immediate assignment and a list of all available servers. Returns a map of regions to the server they should be assigned to. This method will return quickly and does not do any intelligent balancing. The goal is to make a fast decision not the best decision possible. Currently this is random.

Specified by:

immediateAssignment in interface LoadBalancer

Parameters:

regions -

servers -

Returns:

map of regions to the server it should be assigned to

randomAssignment

public ServerName randomAssignment(HRegionInfo regionInfo,
                          List<ServerName> servers)

Used to assign a single region to a random server.

Specified by:

randomAssignment in interface LoadBalancer

Parameters:

regionInfo - Region for which this selection is being done.

Returns:

Servername

retainAssignment

public Map<ServerName,List<HRegionInfo>> retainAssignment(Map<HRegionInfo,ServerName> regions,
                                                 List<ServerName> servers)

Generates a bulk assignment startup plan, attempting to reuse the existing assignment information from META, but adjusting for the specified list of available/online servers available for assignment.

Takes a map of all regions to their existing assignment from META. Also takes a list of online servers for regions to be assigned to. Attempts to retain all assignment, so in some instances initial assignment will not be completely balanced.

Any leftover regions without an existing server to be assigned to will be assigned randomly to available servers.

Specified by:

retainAssignment in interface LoadBalancer

Parameters:

regions - regions and existing assignment from meta

servers - available servers

Returns:

map of servers and regions to be assigned to them

initialize

public void initialize()
                throws HBaseIOException

Description copied from interface: LoadBalancer

Initialize the load balancer. Must be called after setters.

Specified by:

initialize in interface LoadBalancer

Throws:

HBaseIOException

regionOnline

public void regionOnline(HRegionInfo regionInfo,
                ServerName sn)

Description copied from interface: LoadBalancer

Marks the region as online at balancer.

Specified by:

regionOnline in interface LoadBalancer

regionOffline

public void regionOffline(HRegionInfo regionInfo)

Description copied from interface: LoadBalancer

Marks the region as offline at balancer.

Specified by:

regionOffline in interface LoadBalancer

isStopped

public boolean isStopped()

Specified by:

isStopped in interface Stoppable

Returns:

True if Stoppable.stop(String) has been closed.

stop

public void stop(String why)

Description copied from interface: Stoppable

Stop this service.

Specified by:

stop in interface Stoppable

Parameters:

why - Why we're stopping.

getRegionAssignmentsByServer

protected Map<ServerName,List<HRegionInfo>> getRegionAssignmentsByServer(Collection<HRegionInfo> regions)

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

Specified by:

onConfigurationChange in interface LoadBalancer