org.apache.hadoop.hbase.rsgroup

Class RSGroupBasedLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer

All Implemented Interfaces:

ConfigurationObserver, LoadBalancer, RSGroupableBalancer, Stoppable

@InterfaceAudience.Private
public class RSGroupBasedLoadBalancer
extends Object
implements RSGroupableBalancer

GroupBasedLoadBalancer, used when Region Server Grouping is configured (HBase-6721) It does region balance based on a table's group membership. Most assignment methods contain two exclusive code paths: Online - when the group table is online and Offline - when it is unavailable. During Offline, assignments are assigned based on cached information in zookeeper. If unavailable (ie bootstrap) then regions are assigned randomly. Once the GROUP table has been assigned, the balancer switches to Online and will then start providing appropriate assignments for user tables.

Field Summary

Fields

Modifier and Type	Field and Description
static String	FALLBACK_GROUP_ENABLE_KEY Set this key to true to allow region fallback.
private boolean	fallbackEnabled
private LoadBalancer	internalBalancer
private static org.slf4j.Logger	LOG
private MasterServices	masterServices
private RSGroupInfoManager	rsGroupInfoManager

Fields inherited from interface org.apache.hadoop.hbase.rsgroup.RSGroupableBalancer

HBASE_RSGROUP_LOADBALANCER_CLASS

Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

BOGUS_SERVER_NAME, SYSTEM_TABLES_ON_MASTER, TABLES_ON_MASTER

Constructor Summary

Constructors

Constructor and Description
RSGroupBasedLoadBalancer() Used by reflection in LoadBalancerFactory.

Method Summary

Modifier and Type	Method and Description
List<RegionPlan>	balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable) Balance by RSGroup.
private Pair<Map<TableName,Map<ServerName,List<RegionInfo>>>,List<RegionPlan>>	correctAssignments(Map<TableName,Map<ServerName,List<RegionInfo>>> existingAssignments)
private List<ServerName>	filterOfflineServers(RSGroupInfo RSGroupInfo, List<ServerName> onlineServers)
private List<ServerName>	filterServers(Set<Address> servers, List<ServerName> onlineServers) Filter servers based on the online servers.
private List<Pair<List<RegionInfo>,List<ServerName>>>	generateGroupAssignments(List<RegionInfo> regions, List<ServerName> servers)
private List<ServerName>	getFallBackCandidates(List<ServerName> servers)
void	initialize() Initialize the load balancer.
boolean	isFallbackEnabled()
boolean	isOnline()
boolean	isStopped() Returns True if Stoppable.stop(String) has been closed.
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf) This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
void	postMasterStartupInitialize() If balancer needs to do initialization after Master has started up, lets do that here.
ServerName	randomAssignment(RegionInfo region, List<ServerName> servers) Get a random region server from the list
void	regionOffline(RegionInfo regionInfo) Marks the region as offline at balancer.
void	regionOnline(RegionInfo regionInfo, ServerName sn) Marks the region as online at balancer.
Map<ServerName,List<RegionInfo>>	retainAssignment(Map<RegionInfo,ServerName> regions, List<ServerName> servers) Assign regions to the previously hosting region server nn * @return List of plans
Map<ServerName,List<RegionInfo>>	roundRobinAssignment(List<RegionInfo> regions, List<ServerName> servers) Perform a Round Robin assignment of regions.
void	setMasterServices(MasterServices masterServices) Set the master service.
void	setRsGroupInfoManager(RSGroupInfoManager rsGroupInfoManager)
void	stop(String why) Stop this service.
void	updateBalancerLoadInfo(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable) In some scenarios, Balancer needs to update internal status or information according to the current tables load
void	updateBalancerStatus(boolean status)
void	updateClusterMetrics(ClusterMetrics sm) Set the current cluster status.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

isMasterCanHostUserRegions, isSystemTablesOnlyOnMaster, isTablesOnMaster

Field Detail

LOG

private static final org.slf4j.Logger LOG

masterServices

private MasterServices masterServices

rsGroupInfoManager

private volatile RSGroupInfoManager rsGroupInfoManager

internalBalancer

private volatile LoadBalancer internalBalancer

FALLBACK_GROUP_ENABLE_KEY

public static final String FALLBACK_GROUP_ENABLE_KEY

Set this key to true to allow region fallback. Fallback to the default rsgroup first, then fallback to any group if no online servers in default rsgroup. Please keep balancer switch on at the same time, which is relied on to correct misplaced regions

See Also:

Constant Field Values

fallbackEnabled

private volatile boolean fallbackEnabled

Constructor Detail

RSGroupBasedLoadBalancer

@InterfaceAudience.Private
public RSGroupBasedLoadBalancer()

Used by reflection in LoadBalancerFactory.

Method Detail

updateClusterMetrics

public void updateClusterMetrics(ClusterMetrics sm)

Description copied from interface: LoadBalancer

Set the current cluster status. This allows a LoadBalancer to map host name to a server

Specified by:

updateClusterMetrics in interface LoadBalancer

updateBalancerLoadInfo

public void updateBalancerLoadInfo(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)

Description copied from interface: LoadBalancer

In some scenarios, Balancer needs to update internal status or information according to the current tables load

Specified by:

updateBalancerLoadInfo in interface LoadBalancer

Parameters:

loadOfAllTable - region load of servers for all table

setMasterServices

public void setMasterServices(MasterServices masterServices)

Description copied from interface: LoadBalancer

Set the master service. n

Specified by:

setMasterServices in interface LoadBalancer

setRsGroupInfoManager

public void setRsGroupInfoManager(RSGroupInfoManager rsGroupInfoManager)

balanceCluster

public List<RegionPlan> balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)
                                throws IOException

Balance by RSGroup.

Specified by:

balanceCluster in interface LoadBalancer

Parameters:

loadOfAllTable - region load of servers for all table

Returns:

a list of regions to be moved, including source and destination, or null if cluster is already balanced

Throws:

IOException

roundRobinAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> roundRobinAssignment(List<RegionInfo> regions,
                                                                      List<ServerName> servers)
                                                               throws HBaseIOException

Description copied from interface: LoadBalancer

Perform a Round Robin assignment of regions. nn * @return Map of servername to regioninfos

Specified by:

roundRobinAssignment in interface LoadBalancer

Throws:

HBaseIOException

retainAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> retainAssignment(Map<RegionInfo,ServerName> regions,
                                                                  List<ServerName> servers)
                                                           throws HBaseIOException

Description copied from interface: LoadBalancer

Assign regions to the previously hosting region server nn * @return List of plans

Specified by:

retainAssignment in interface LoadBalancer

Throws:

HBaseIOException

randomAssignment

public ServerName randomAssignment(RegionInfo region,
                                   List<ServerName> servers)
                            throws HBaseIOException

Description copied from interface: LoadBalancer

Get a random region server from the list

Specified by:

randomAssignment in interface LoadBalancer

Parameters:

region - Region for which this selection is being done. nn

Throws:

HBaseIOException

generateGroupAssignments

private List<Pair<List<RegionInfo>,List<ServerName>>> generateGroupAssignments(List<RegionInfo> regions,
                                                                               List<ServerName> servers)
                                                                        throws HBaseIOException

Throws:

HBaseIOException

filterOfflineServers

private List<ServerName> filterOfflineServers(RSGroupInfo RSGroupInfo,
                                              List<ServerName> onlineServers)

filterServers

private List<ServerName> filterServers(Set<Address> servers,
                                       List<ServerName> onlineServers)

Filter servers based on the online servers.

servers is actually a TreeSet (see RSGroupInfo), having its contains()'s time complexity as O(logn), which is good enough.

TODO: consider using HashSet to pursue O(1) for contains() throughout the calling chain if needed.

Parameters:

servers - the servers

onlineServers - List of servers which are online.

Returns:

the list

correctAssignments

private Pair<Map<TableName,Map<ServerName,List<RegionInfo>>>,List<RegionPlan>> correctAssignments(Map<TableName,Map<ServerName,List<RegionInfo>>> existingAssignments)
                                                                                           throws IOException

Throws:

IOException

initialize

public void initialize()
                throws HBaseIOException

Description copied from interface: LoadBalancer

Initialize the load balancer. Must be called after setters. n

Specified by:

initialize in interface LoadBalancer

Throws:

HBaseIOException

isOnline

public boolean isOnline()

isFallbackEnabled

public boolean isFallbackEnabled()

regionOnline

public void regionOnline(RegionInfo regionInfo,
                         ServerName sn)

Description copied from interface: LoadBalancer

Marks the region as online at balancer. nn

Specified by:

regionOnline in interface LoadBalancer

regionOffline

public void regionOffline(RegionInfo regionInfo)

Description copied from interface: LoadBalancer

Marks the region as offline at balancer. n

Specified by:

regionOffline in interface LoadBalancer

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

Specified by:

onConfigurationChange in interface LoadBalancer

stop

public void stop(String why)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Specified by:

stop in interface Stoppable

Parameters:

why - Why we're stopping.

isStopped

public boolean isStopped()

Description copied from interface: Stoppable

Returns True if Stoppable.stop(String) has been closed.

Specified by:

isStopped in interface Stoppable

postMasterStartupInitialize

public void postMasterStartupInitialize()

Description copied from interface: LoadBalancer

If balancer needs to do initialization after Master has started up, lets do that here.

Specified by:

postMasterStartupInitialize in interface LoadBalancer

updateBalancerStatus

public void updateBalancerStatus(boolean status)

Specified by:

updateBalancerStatus in interface LoadBalancer

getFallBackCandidates

private List<ServerName> getFallBackCandidates(List<ServerName> servers)