Package org.apache.hadoop.hbase.rsgroup

Class RSGroupBasedLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer

All Implemented Interfaces:

ConfigurationObserver, LoadBalancer, Stoppable

@Private
public class RSGroupBasedLoadBalancer
extends Object
implements LoadBalancer

GroupBasedLoadBalancer, used when Region Server Grouping is configured (HBase-6721) It does region balance based on a table's group membership.

Most assignment methods contain two exclusive code paths: Online - when the group table is online and Offline - when it is unavailable.

During Offline, assignments are assigned based on cached information in zookeeper. If unavailable (ie bootstrap) then regions are assigned randomly.

Once the GROUP table has been assigned, the balancer switches to Online and will then start providing appropriate assignments for user tables.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	FALLBACK_GROUP_ENABLE_KEY	Set this key to true to allow region fallback.
private boolean	fallbackEnabled
private FavoredNodesManager	favoredNodesManager
private LoadBalancer	internalBalancer
private static final org.slf4j.Logger	LOG
private MasterServices	masterServices
private ClusterInfoProvider	provider
private RSGroupInfoManager	rsGroupInfoManager

Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

BOGUS_SERVER_NAME, HBASE_RSGROUP_LOADBALANCER_CLASS, MOVE_THROTTLING, MOVE_THROTTLING_DEFAULT

Constructor Summary

Constructors

Constructor	Description
RSGroupBasedLoadBalancer()	Used by reflection in LoadBalancerFactory.

Method Summary

Modifier and Type	Method	Description
List<RegionPlan>	balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)	Balance by RSGroup.
private Pair<Map<TableName,Map<ServerName,List<RegionInfo>>>,List<RegionPlan>>	correctAssignments(Map<TableName,Map<ServerName,List<RegionInfo>>> existingAssignments)
private List<ServerName>	filterOfflineServers(RSGroupInfo RSGroupInfo, List<ServerName> onlineServers)
private List<ServerName>	filterServers(Set<Address> servers, List<ServerName> onlineServers)	Filter servers based on the online servers.
private List<Pair<List<RegionInfo>,List<ServerName>>>	generateGroupAssignments(List<RegionInfo> regions, List<ServerName> servers)
private List<ServerName>	getFallBackCandidates(List<ServerName> servers)
FavoredNodesManager	getFavoredNodesManager()
LoadBalancer	getInternalBalancer()
void	initialize()	Initialize the load balancer.
boolean	isFallbackEnabled()
boolean	isOnline()
boolean	isStopped()	Returns True if Stoppable.stop(String) has been closed.
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf)	Notification that config has changed
void	postMasterStartupInitialize()	If balancer needs to do initialization after Master has started up, lets do that here.
ServerName	randomAssignment(RegionInfo region, List<ServerName> servers)	Get a random region server from the list
void	regionOffline(RegionInfo regionInfo)	Marks the region as offline at balancer.
void	regionOnline(RegionInfo regionInfo, ServerName sn)	Marks the region as online at balancer.
Map<ServerName,List<RegionInfo>>	retainAssignment(Map<RegionInfo,ServerName> regions, List<ServerName> servers)	Assign regions to the previously hosting region server
Map<ServerName,List<RegionInfo>>	roundRobinAssignment(List<RegionInfo> regions, List<ServerName> servers)	Perform a Round Robin assignment of regions.
void	setClusterInfoProvider(ClusterInfoProvider provider)	Set the cluster info provider.
void	setMasterServices(MasterServices masterServices)
void	stop(String why)	Stop this service.
void	updateBalancerLoadInfo(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)	In some scenarios, Balancer needs to update internal status or information according to the current tables load
void	updateBalancerStatus(boolean status)
void	updateClusterMetrics(ClusterMetrics sm)	Set the current cluster status.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

throttle

Field Details

LOG

private static final org.slf4j.Logger LOG

masterServices

private MasterServices masterServices

provider

private ClusterInfoProvider provider

favoredNodesManager

private FavoredNodesManager favoredNodesManager

rsGroupInfoManager

private volatile RSGroupInfoManager rsGroupInfoManager

internalBalancer

private volatile LoadBalancer internalBalancer

FALLBACK_GROUP_ENABLE_KEY

public static final String FALLBACK_GROUP_ENABLE_KEY

Set this key to true to allow region fallback. Fallback to the default rsgroup first, then fallback to any group if no online servers in default rsgroup. Please keep balancer switch on at the same time, which is relied on to correct misplaced regions

See Also:

• Constant Field Values

fallbackEnabled

private volatile boolean fallbackEnabled

Constructor Details

RSGroupBasedLoadBalancer

@Private
public RSGroupBasedLoadBalancer()

Used by reflection in LoadBalancerFactory.

Method Details

updateClusterMetrics

public void updateClusterMetrics(ClusterMetrics sm)

Description copied from interface: LoadBalancer

Set the current cluster status. This allows a LoadBalancer to map host name to a server

Specified by:

updateClusterMetrics in interface LoadBalancer

updateBalancerLoadInfo

public void updateBalancerLoadInfo(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)

Description copied from interface: LoadBalancer

In some scenarios, Balancer needs to update internal status or information according to the current tables load

Specified by:

updateBalancerLoadInfo in interface LoadBalancer

Parameters:

loadOfAllTable - region load of servers for all table

setMasterServices

public void setMasterServices(MasterServices masterServices)

balanceCluster

public List<RegionPlan> balanceCluster(Map<TableName,Map<ServerName,List<RegionInfo>>> loadOfAllTable)
                                throws IOException

Balance by RSGroup.

Specified by:

balanceCluster in interface LoadBalancer

Parameters:

loadOfAllTable - region load of servers for all table

Returns:

a list of regions to be moved, including source and destination, or null if cluster is already balanced

Throws:

IOException

roundRobinAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> roundRobinAssignment(List<RegionInfo> regions,
 List<ServerName> servers)
                                                      throws IOException

Description copied from interface: LoadBalancer

Perform a Round Robin assignment of regions.

Specified by:

roundRobinAssignment in interface LoadBalancer

Returns:

Map of servername to regioninfos

Throws:

IOException

retainAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> retainAssignment(Map<RegionInfo,ServerName> regions,
 List<ServerName> servers)
                                                  throws HBaseIOException

Description copied from interface: LoadBalancer

Assign regions to the previously hosting region server

Specified by:

retainAssignment in interface LoadBalancer

Returns:

List of plans

Throws:

HBaseIOException

randomAssignment

public ServerName randomAssignment(RegionInfo region,
 List<ServerName> servers)
                            throws IOException

Description copied from interface: LoadBalancer

Get a random region server from the list

Specified by:

randomAssignment in interface LoadBalancer

Parameters:

region - Region for which this selection is being done.

Throws:

IOException

generateGroupAssignments

private List<Pair<List<RegionInfo>,List<ServerName>>> generateGroupAssignments(List<RegionInfo> regions,
 List<ServerName> servers)
                                                                        throws HBaseIOException

Throws:

HBaseIOException

filterOfflineServers

private List<ServerName> filterOfflineServers(RSGroupInfo RSGroupInfo,
 List<ServerName> onlineServers)

filterServers

private List<ServerName> filterServers(Set<Address> servers,
 List<ServerName> onlineServers)

Filter servers based on the online servers.

servers is actually a TreeSet (see RSGroupInfo), having its contains()'s time complexity as O(logn), which is good enough.

TODO: consider using HashSet to pursue O(1) for contains() throughout the calling chain if needed.

Parameters:

servers - the servers

onlineServers - List of servers which are online.

Returns:

the list

correctAssignments

private Pair<Map<TableName,Map<ServerName,List<RegionInfo>>>,List<RegionPlan>> correctAssignments(Map<TableName,Map<ServerName,List<RegionInfo>>> existingAssignments)
                                                                                           throws IOException

Throws:

IOException

initialize

public void initialize()
                throws IOException

Description copied from interface: LoadBalancer

Initialize the load balancer. Must be called after setters.

Specified by:

initialize in interface LoadBalancer

Throws:

IOException

isOnline

public boolean isOnline()

isFallbackEnabled

public boolean isFallbackEnabled()

regionOnline

public void regionOnline(RegionInfo regionInfo,
 ServerName sn)

Description copied from interface: LoadBalancer

Marks the region as online at balancer.

Specified by:

regionOnline in interface LoadBalancer

regionOffline

public void regionOffline(RegionInfo regionInfo)

Description copied from interface: LoadBalancer

Marks the region as offline at balancer.

Specified by:

regionOffline in interface LoadBalancer

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: LoadBalancer

Notification that config has changed

Specified by:

onConfigurationChange in interface ConfigurationObserver

Specified by:

onConfigurationChange in interface LoadBalancer

stop

public void stop(String why)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Specified by:

stop in interface Stoppable

Parameters:

why - Why we're stopping.

isStopped

public boolean isStopped()

Description copied from interface: Stoppable

Returns True if Stoppable.stop(String) has been closed.

Specified by:

isStopped in interface Stoppable

getInternalBalancer

public LoadBalancer getInternalBalancer()

getFavoredNodesManager

public FavoredNodesManager getFavoredNodesManager()

postMasterStartupInitialize

public void postMasterStartupInitialize()

Description copied from interface: LoadBalancer

If balancer needs to do initialization after Master has started up, lets do that here.

Specified by:

postMasterStartupInitialize in interface LoadBalancer

updateBalancerStatus

public void updateBalancerStatus(boolean status)

Specified by:

updateBalancerStatus in interface LoadBalancer

getFallBackCandidates

private List<ServerName> getFallBackCandidates(List<ServerName> servers)

setClusterInfoProvider

public void setClusterInfoProvider(ClusterInfoProvider provider)

Description copied from interface: LoadBalancer

Set the cluster info provider. Usually it is just a wrapper of master.

Specified by:

setClusterInfoProvider in interface LoadBalancer