org.apache.hadoop.hbase.rsgroup

Class RSGroupBasedLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, ConfigurationObserver, LoadBalancer, RSGroupableBalancer, Stoppable

@InterfaceAudience.Private
public class RSGroupBasedLoadBalancer
extends Object
implements RSGroupableBalancer

GroupBasedLoadBalancer, used when Region Server Grouping is configured (HBase-6721) It does region balance based on a table's group membership. Most assignment methods contain two exclusive code paths: Online - when the group table is online and Offline - when it is unavailable. During Offline, assignments are assigned based on cached information in zookeeper. If unavailable (ie bootstrap) then regions are assigned randomly. Once the GROUP table has been assigned, the balancer switches to Online and will then start providing appropriate assignments for user tables.

Field Summary

Fields

Modifier and Type	Field and Description
private ClusterMetrics	clusterStatus
private org.apache.hadoop.conf.Configuration	config
static String	FALLBACK_GROUP_ENABLE_KEY Set this key to true to allow region fallback.
private boolean	fallbackEnabled
private LoadBalancer	internalBalancer
private static org.slf4j.Logger	LOG
private MasterServices	masterServices
private RSGroupInfoManager	rsGroupInfoManager

Fields inherited from interface org.apache.hadoop.hbase.rsgroup.RSGroupableBalancer

HBASE_RSGROUP_LOADBALANCER_CLASS

Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

BOGUS_SERVER_NAME, SYSTEM_TABLES_ON_MASTER, TABLES_ON_MASTER

Constructor Summary

Constructors

Constructor and Description
RSGroupBasedLoadBalancer() Used by reflection in LoadBalancerFactory.

Method Summary

Modifier and Type	Method and Description
List<RegionPlan>	balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable) Override to balance by RSGroup not invoke balanceTable(TableName, Map)
List<RegionPlan>	balanceTable(TableName tableName, Map<ServerName,List<RegionInfo>> loadOfOneTable) can achieve table balanced rather than overall balanced
private Pair<Map<TableName,Map<ServerName,List<RegionInfo>>>,List<RegionPlan>>	correctAssignments(Map<TableName,Map<ServerName,List<RegionInfo>>> existingAssignments)
private List<ServerName>	filterOfflineServers(RSGroupInfo RSGroupInfo, List<ServerName> onlineServers)
private List<ServerName>	filterServers(Set<Address> servers, List<ServerName> onlineServers) Filter servers based on the online servers.
private List<Pair<List<RegionInfo>,List<ServerName>>>	generateGroupAssignments(List<RegionInfo> regions, List<ServerName> servers)
org.apache.hadoop.conf.Configuration	getConf()
private List<ServerName>	getFallBackCandidates(List<ServerName> servers)
void	initialize() Initialize the load balancer.
boolean	isFallbackEnabled()
boolean	isOnline()
boolean	isStopped()
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf) This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
void	postMasterStartupInitialize() If balancer needs to do initialization after Master has started up, lets do that here.
ServerName	randomAssignment(RegionInfo region, List<ServerName> servers) Get a random region server from the list
void	regionOffline(RegionInfo regionInfo) Marks the region as offline at balancer.
void	regionOnline(RegionInfo regionInfo, ServerName sn) Marks the region as online at balancer.
Map<ServerName,List<RegionInfo>>	retainAssignment(Map<RegionInfo,ServerName> regions, List<ServerName> servers) Assign regions to the previously hosting region server
Map<ServerName,List<RegionInfo>>	roundRobinAssignment(List<RegionInfo> regions, List<ServerName> servers) Perform a Round Robin assignment of regions.
void	setClusterMetrics(ClusterMetrics sm) Set the current cluster status.
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setMasterServices(MasterServices masterServices) Set the master service.
void	setRsGroupInfoManager(RSGroupInfoManager rsGroupInfoManager)
void	stop(String why) Stop this service.
void	updateBalancerStatus(boolean status)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

isMasterCanHostUserRegions, isSystemTablesOnlyOnMaster, isTablesOnMaster

Field Detail

LOG

private static final org.slf4j.Logger LOG

config

private org.apache.hadoop.conf.Configuration config

clusterStatus

private ClusterMetrics clusterStatus

masterServices

private MasterServices masterServices

rsGroupInfoManager

private volatile RSGroupInfoManager rsGroupInfoManager

internalBalancer

private LoadBalancer internalBalancer

FALLBACK_GROUP_ENABLE_KEY

public static final String FALLBACK_GROUP_ENABLE_KEY

Set this key to true to allow region fallback. Fallback to the default rsgroup first, then fallback to any group if no online servers in default rsgroup. Please keep balancer switch on at the same time, which is relied on to correct misplaced regions

See Also:

Constant Field Values

fallbackEnabled

private boolean fallbackEnabled

Constructor Detail

RSGroupBasedLoadBalancer

@InterfaceAudience.Private
public RSGroupBasedLoadBalancer()

Used by reflection in LoadBalancerFactory.

Method Detail

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

setClusterMetrics

public void setClusterMetrics(ClusterMetrics sm)

Description copied from interface: LoadBalancer

Set the current cluster status. This allows a LoadBalancer to map host name to a server

Specified by:

setClusterMetrics in interface LoadBalancer

setMasterServices

public void setMasterServices(MasterServices masterServices)

Description copied from interface: LoadBalancer

Set the master service.

Specified by:

setMasterServices in interface LoadBalancer

balanceCluster

public List<RegionPlan> balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)
                                throws IOException

Override to balance by RSGroup not invoke balanceTable(TableName, Map)

Specified by:

balanceCluster in interface LoadBalancer

Parameters:

loadOfAllTable - region load of servers for all table

Returns:

a list of regions to be moved, including source and destination, or null if cluster is already balanced

Throws:

IOException

roundRobinAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> roundRobinAssignment(List<RegionInfo> regions,
                                                                      List<ServerName> servers)
                                                               throws HBaseIOException

Description copied from interface: LoadBalancer

Perform a Round Robin assignment of regions.

Specified by:

roundRobinAssignment in interface LoadBalancer

Returns:

Map of servername to regioninfos

Throws:

HBaseIOException

retainAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> retainAssignment(Map<RegionInfo,ServerName> regions,
                                                                  List<ServerName> servers)
                                                           throws HBaseIOException

Description copied from interface: LoadBalancer

Assign regions to the previously hosting region server

Specified by:

retainAssignment in interface LoadBalancer

Returns:

List of plans

Throws:

HBaseIOException

randomAssignment

public ServerName randomAssignment(RegionInfo region,
                                   List<ServerName> servers)
                            throws HBaseIOException

Description copied from interface: LoadBalancer

Get a random region server from the list

Specified by:

randomAssignment in interface LoadBalancer

Parameters:

region - Region for which this selection is being done.

Returns:

Servername

Throws:

HBaseIOException

generateGroupAssignments

private List<Pair<List<RegionInfo>,List<ServerName>>> generateGroupAssignments(List<RegionInfo> regions,
                                                                               List<ServerName> servers)
                                                                        throws HBaseIOException

Throws:

HBaseIOException

filterOfflineServers

private List<ServerName> filterOfflineServers(RSGroupInfo RSGroupInfo,
                                              List<ServerName> onlineServers)

filterServers

private List<ServerName> filterServers(Set<Address> servers,
                                       List<ServerName> onlineServers)

Filter servers based on the online servers.

servers is actually a TreeSet (see RSGroupInfo), having its contains()'s time complexity as O(logn), which is good enough.

TODO: consider using HashSet to pursue O(1) for contains() throughout the calling chain if needed.

Parameters:

servers - the servers

onlineServers - List of servers which are online.

Returns:

the list

correctAssignments

private Pair<Map<TableName,Map<ServerName,List<RegionInfo>>>,List<RegionPlan>> correctAssignments(Map<TableName,Map<ServerName,List<RegionInfo>>> existingAssignments)
                                                                                           throws IOException

Throws:

IOException

initialize

public void initialize()
                throws HBaseIOException

Description copied from interface: LoadBalancer

Initialize the load balancer. Must be called after setters.

Specified by:

initialize in interface LoadBalancer

Throws:

HBaseIOException

isOnline

public boolean isOnline()

isFallbackEnabled

public boolean isFallbackEnabled()

regionOnline

public void regionOnline(RegionInfo regionInfo,
                         ServerName sn)

Description copied from interface: LoadBalancer

Marks the region as online at balancer.

Specified by:

regionOnline in interface LoadBalancer

regionOffline

public void regionOffline(RegionInfo regionInfo)

Description copied from interface: LoadBalancer

Marks the region as offline at balancer.

Specified by:

regionOffline in interface LoadBalancer

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

Specified by:

onConfigurationChange in interface LoadBalancer

stop

public void stop(String why)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Specified by:

stop in interface Stoppable

Parameters:

why - Why we're stopping.

isStopped

public boolean isStopped()

Specified by:

isStopped in interface Stoppable

Returns:

True if Stoppable.stop(String) has been closed.

setRsGroupInfoManager

public void setRsGroupInfoManager(RSGroupInfoManager rsGroupInfoManager)

postMasterStartupInitialize

public void postMasterStartupInitialize()

Description copied from interface: LoadBalancer

If balancer needs to do initialization after Master has started up, lets do that here.

Specified by:

postMasterStartupInitialize in interface LoadBalancer

updateBalancerStatus

public void updateBalancerStatus(boolean status)

Specified by:

updateBalancerStatus in interface LoadBalancer

balanceTable

public List<RegionPlan> balanceTable(TableName tableName,
                                     Map<ServerName,List<RegionInfo>> loadOfOneTable)

can achieve table balanced rather than overall balanced

Specified by:

balanceTable in interface LoadBalancer

Parameters:

tableName - the table to be balanced

loadOfOneTable - region load of servers for the specific one table

Returns:

List of plans

getFallBackCandidates

private List<ServerName> getFallBackCandidates(List<ServerName> servers)