Package org.apache.hadoop.hbase.master.balancer

Class BaseLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

All Implemented Interfaces:

ConfigurationObserver, LoadBalancer, Stoppable

Direct Known Subclasses:

FavoredNodeLoadBalancer, SimpleLoadBalancer, StochasticLoadBalancer

@Private
public abstract class BaseLoadBalancer
extends Object
implements LoadBalancer

The base class for load balancers. It provides the functions used to by AssignmentManager to assign regions in the edge cases. It doesn't provide an implementation of the actual balancing algorithm.

Since 3.0.0, all the balancers will be wrapped inside a RSGroupBasedLoadBalancer, it will be in charge of the synchronization of balancing and configuration changing, so we do not need to synchronize by ourselves.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	BALANCER_DECISION_BUFFER_ENABLED
static final String	BALANCER_REJECTION_BUFFER_ENABLED
protected ClusterMetrics	clusterStatus
static final boolean	DEFAULT_BALANCER_DECISION_BUFFER_ENABLED
static final boolean	DEFAULT_BALANCER_REJECTION_BUFFER_ENABLED
static final boolean	DEFAULT_HBASE_MASTER_LOADBALANCE_BYTABLE
protected boolean	isByTable
private static final org.slf4j.Logger	LOG
protected ServerName	masterServerName
protected MetricsBalancer	metricsBalancer
protected static final int	MIN_SERVER_BALANCE
protected ClusterInfoProvider	provider
protected RackManager	rackManager
protected RegionHDFSBlockLocationFinder	regionFinder
static final float	REGIONS_SLOP_DEFAULT
static final String	REGIONS_SLOP_KEY
protected float	slop
private boolean	stopped
protected boolean	useRegionFinder

Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

BOGUS_SERVER_NAME, HBASE_RSGROUP_LOADBALANCER_CLASS, MOVE_THROTTLING, MOVE_THROTTLING_DEFAULT

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BaseLoadBalancer()	The constructor that uses the basic MetricsBalancer
protected	BaseLoadBalancer(MetricsBalancer metricsBalancer)	This Constructor accepts an instance of MetricsBalancer, which will be used instead of creating a new one

Method Summary

Modifier and Type	Method	Description
final List<RegionPlan>	balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)	Perform the major balance operation for cluster, will invoke balanceTable(TableName, Map) to do actual balance.
protected abstract List<RegionPlan>	balanceTable(TableName tableName, Map<ServerName,List<RegionInfo>> loadOfOneTable)	Perform the major balance operation for table, all sub classes should override this method.
private BalancerClusterState	createCluster(List<ServerName> servers, Collection<RegionInfo> regions)
private RegionHDFSBlockLocationFinder	createRegionLocationFinder(org.apache.hadoop.conf.Configuration conf)
private List<ServerName>	findIdleServers(List<ServerName> servers)
protected final org.apache.hadoop.conf.Configuration	getConf()
protected float	getDefaultSlop()
private Map<ServerName,List<RegionInfo>>	getRegionAssignmentsByServer(Collection<RegionInfo> regions)
protected final boolean	idleRegionServerExist(BalancerClusterState c)
void	initialize()	Initialize the load balancer.
boolean	isStopped()	Returns True if Stoppable.stop(String) has been closed.
protected void	loadConf(org.apache.hadoop.conf.Configuration conf)
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf)	Notification that config has changed
void	postMasterStartupInitialize()	If balancer needs to do initialization after Master has started up, lets do that here.
protected void	preBalanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)	Called before actually executing balanceCluster.
ServerName	randomAssignment(RegionInfo regionInfo, List<ServerName> servers)	Used to assign a single region to a random server.
private ServerName	randomAssignment(BalancerClusterState cluster, RegionInfo regionInfo, List<ServerName> servers)	Used to assign a single region to a random server.
void	regionOffline(RegionInfo regionInfo)	Marks the region as offline at balancer.
void	regionOnline(RegionInfo regionInfo, ServerName sn)	Marks the region as online at balancer.
Map<ServerName,List<RegionInfo>>	retainAssignment(Map<RegionInfo,ServerName> regions, List<ServerName> servers)	Generates a bulk assignment startup plan, attempting to reuse the existing assignment information from META, but adjusting for the specified list of available/online servers available for assignment.
Map<ServerName,List<RegionInfo>>	roundRobinAssignment(List<RegionInfo> regions, List<ServerName> servers)	Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.
private void	roundRobinAssignment(BalancerClusterState cluster, List<RegionInfo> regions, List<ServerName> servers, Map<ServerName,List<RegionInfo>> assignments)	Round-robin a list of regions to a list of servers
void	setClusterInfoProvider(ClusterInfoProvider provider)	Set the cluster info provider.
protected final boolean	sloppyRegionServerExist(ClusterLoadState cs)
void	stop(String why)	Stop this service.
protected final Map<ServerName,List<RegionInfo>>	toEnsumbleTableLoad(Map<TableName,Map<ServerName,List<RegionInfo>>> LoadOfAllTable)
void	updateBalancerStatus(boolean status)	Updates the balancer status tag reported to JMX
void	updateClusterMetrics(ClusterMetrics st)	Set the current cluster status.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

throttle, updateBalancerLoadInfo

Field Details

LOG

private static final org.slf4j.Logger LOG

BALANCER_DECISION_BUFFER_ENABLED

public static final String BALANCER_DECISION_BUFFER_ENABLED

See Also:

• Constant Field Values

DEFAULT_BALANCER_DECISION_BUFFER_ENABLED

public static final boolean DEFAULT_BALANCER_DECISION_BUFFER_ENABLED

See Also:

• Constant Field Values

BALANCER_REJECTION_BUFFER_ENABLED

public static final String BALANCER_REJECTION_BUFFER_ENABLED

See Also:

• Constant Field Values

DEFAULT_BALANCER_REJECTION_BUFFER_ENABLED

public static final boolean DEFAULT_BALANCER_REJECTION_BUFFER_ENABLED

See Also:

• Constant Field Values

DEFAULT_HBASE_MASTER_LOADBALANCE_BYTABLE

public static final boolean DEFAULT_HBASE_MASTER_LOADBALANCE_BYTABLE

See Also:

• Constant Field Values

REGIONS_SLOP_KEY

public static final String REGIONS_SLOP_KEY

See Also:

• Constant Field Values

REGIONS_SLOP_DEFAULT

public static final float REGIONS_SLOP_DEFAULT

See Also:

• Constant Field Values

MIN_SERVER_BALANCE

protected static final int MIN_SERVER_BALANCE

See Also:

• Constant Field Values

stopped

private volatile boolean stopped

regionFinder

protected volatile RegionHDFSBlockLocationFinder regionFinder

useRegionFinder

protected boolean useRegionFinder

isByTable

protected boolean isByTable

slop

protected float slop

rackManager

protected volatile RackManager rackManager

metricsBalancer

protected MetricsBalancer metricsBalancer

clusterStatus

protected ClusterMetrics clusterStatus

masterServerName

protected ServerName masterServerName

provider

protected ClusterInfoProvider provider

Constructor Details

BaseLoadBalancer

protected BaseLoadBalancer()

The constructor that uses the basic MetricsBalancer

BaseLoadBalancer

protected BaseLoadBalancer(MetricsBalancer metricsBalancer)

This Constructor accepts an instance of MetricsBalancer, which will be used instead of creating a new one

Method Details

getConf

protected final org.apache.hadoop.conf.Configuration getConf()

updateClusterMetrics

public void updateClusterMetrics(ClusterMetrics st)

Description copied from interface: LoadBalancer

Set the current cluster status. This allows a LoadBalancer to map host name to a server

Specified by:

updateClusterMetrics in interface LoadBalancer

setClusterInfoProvider

public void setClusterInfoProvider(ClusterInfoProvider provider)

Description copied from interface: LoadBalancer

Set the cluster info provider. Usually it is just a wrapper of master.

Specified by:

setClusterInfoProvider in interface LoadBalancer

postMasterStartupInitialize

public void postMasterStartupInitialize()

Description copied from interface: LoadBalancer

If balancer needs to do initialization after Master has started up, lets do that here.

Specified by:

postMasterStartupInitialize in interface LoadBalancer

idleRegionServerExist

protected final boolean idleRegionServerExist(BalancerClusterState c)

sloppyRegionServerExist

protected final boolean sloppyRegionServerExist(ClusterLoadState cs)

roundRobinAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> roundRobinAssignment(List<RegionInfo> regions,
 List<ServerName> servers)
                                                      throws HBaseIOException

Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.

Takes a list of all the regions and all the servers in the cluster and returns a map of each server to the regions that it should be assigned.

Currently implemented as a round-robin assignment. Same invariant as load balancing, all servers holding floor(avg) or ceiling(avg). TODO: Use block locations from HDFS to place regions with their blocks

Specified by:

roundRobinAssignment in interface LoadBalancer

Parameters:

regions - all regions

servers - all servers

Returns:

map of server to the regions it should take, or emptyMap if no assignment is possible (ie. no servers)

Throws:

HBaseIOException

createCluster

private BalancerClusterState createCluster(List<ServerName> servers,
 Collection<RegionInfo> regions)
                                    throws HBaseIOException

Throws:

HBaseIOException

findIdleServers

private List<ServerName> findIdleServers(List<ServerName> servers)

randomAssignment

public ServerName randomAssignment(RegionInfo regionInfo,
 List<ServerName> servers)
                            throws HBaseIOException

Used to assign a single region to a random server.

Specified by:

randomAssignment in interface LoadBalancer

Parameters:

regionInfo - Region for which this selection is being done.

Throws:

HBaseIOException

retainAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> retainAssignment(Map<RegionInfo,ServerName> regions,
 List<ServerName> servers)
                                                  throws HBaseIOException

Generates a bulk assignment startup plan, attempting to reuse the existing assignment information from META, but adjusting for the specified list of available/online servers available for assignment.

Takes a map of all regions to their existing assignment from META. Also takes a list of online servers for regions to be assigned to. Attempts to retain all assignment, so in some instances initial assignment will not be completely balanced.

Any leftover regions without an existing server to be assigned to will be assigned randomly to available servers.

Specified by:

retainAssignment in interface LoadBalancer

Parameters:

regions - regions and existing assignment from meta

servers - available servers

Returns:

map of servers and regions to be assigned to them, or emptyMap if no assignment is possible (ie. no servers)

Throws:

HBaseIOException

getDefaultSlop

protected float getDefaultSlop()

createRegionLocationFinder

private RegionHDFSBlockLocationFinder createRegionLocationFinder(org.apache.hadoop.conf.Configuration conf)

loadConf

protected void loadConf(org.apache.hadoop.conf.Configuration conf)

initialize

public void initialize()

Description copied from interface: LoadBalancer

Initialize the load balancer. Must be called after setters.

Specified by:

initialize in interface LoadBalancer

regionOnline

public void regionOnline(RegionInfo regionInfo,
 ServerName sn)

Description copied from interface: LoadBalancer

Marks the region as online at balancer.

Specified by:

regionOnline in interface LoadBalancer

regionOffline

public void regionOffline(RegionInfo regionInfo)

Description copied from interface: LoadBalancer

Marks the region as offline at balancer.

Specified by:

regionOffline in interface LoadBalancer

isStopped

public boolean isStopped()

Description copied from interface: Stoppable

Returns True if Stoppable.stop(String) has been closed.

Specified by:

isStopped in interface Stoppable

stop

public void stop(String why)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Specified by:

stop in interface Stoppable

Parameters:

why - Why we're stopping.

updateBalancerStatus

public void updateBalancerStatus(boolean status)

Updates the balancer status tag reported to JMX

Specified by:

updateBalancerStatus in interface LoadBalancer

randomAssignment

private ServerName randomAssignment(BalancerClusterState cluster,
 RegionInfo regionInfo,
 List<ServerName> servers)

Used to assign a single region to a random server.

roundRobinAssignment

private void roundRobinAssignment(BalancerClusterState cluster,
 List<RegionInfo> regions,
 List<ServerName> servers,
 Map<ServerName,List<RegionInfo>> assignments)

Round-robin a list of regions to a list of servers

getRegionAssignmentsByServer

private Map<ServerName,List<RegionInfo>> getRegionAssignmentsByServer(Collection<RegionInfo> regions)

toEnsumbleTableLoad

protected final Map<ServerName,List<RegionInfo>> toEnsumbleTableLoad(Map<TableName,Map<ServerName,List<RegionInfo>>> LoadOfAllTable)

balanceTable

protected abstract List<RegionPlan> balanceTable(TableName tableName,
 Map<ServerName,List<RegionInfo>> loadOfOneTable)

Perform the major balance operation for table, all sub classes should override this method.

Will be invoked by balanceCluster(Map). If HConstants.HBASE_MASTER_LOADBALANCE_BYTABLE is enabled, we will call this method multiple times, one table a time, where we will only pass in the regions for a single table each time. If not, we will pass in all the regions at once, and the tableName will be HConstants.ENSEMBLE_TABLE_NAME.

Parameters:

tableName - the table to be balanced

loadOfOneTable - region load of servers for the specific one table

Returns:

List of plans

preBalanceCluster

protected void preBalanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)

Called before actually executing balanceCluster. The sub classes could override this method to do some initialization work.

balanceCluster

public final List<RegionPlan> balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)

Perform the major balance operation for cluster, will invoke balanceTable(TableName, Map) to do actual balance.

THIs method is marked as final which means you should not override this method. See the javadoc for balanceTable(TableName, Map) for more details.

Specified by:

balanceCluster in interface LoadBalancer

Parameters:

loadOfAllTable - region load of servers for all table

Returns:

a list of regions to be moved, including source and destination, or null if cluster is already balanced

See Also:

• balanceTable(TableName, Map)

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: LoadBalancer

Notification that config has changed

Specified by:

onConfigurationChange in interface ConfigurationObserver

Specified by:

onConfigurationChange in interface LoadBalancer