Class BaseLoadBalancer
java.lang.Object
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer
- All Implemented Interfaces:
ConfigurationObserver
,LoadBalancer
,Stoppable
- Direct Known Subclasses:
FavoredNodeLoadBalancer
,SimpleLoadBalancer
,StochasticLoadBalancer
The base class for load balancers. It provides the functions used to by
AssignmentManager
to assign regions in the edge cases. It doesn't provide an implementation of the actual balancing
algorithm.
Since 3.0.0, all the balancers will be wrapped inside a RSGroupBasedLoadBalancer
, it will
be in charge of the synchronization of balancing and configuration changing, so we do not need to
synchronize by ourselves.-
Field Summary
Modifier and TypeFieldDescriptionstatic final String
static final String
protected ClusterMetrics
static final boolean
static final boolean
static final boolean
protected boolean
private static final org.slf4j.Logger
protected ServerName
protected MetricsBalancer
protected static final int
protected ClusterInfoProvider
protected RackManager
protected RegionHDFSBlockLocationFinder
protected float
private boolean
protected boolean
Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer
BOGUS_SERVER_NAME, HBASE_RSGROUP_LOADBALANCER_CLASS
-
Constructor Summary
ModifierConstructorDescriptionprotected
The constructor that uses the basic MetricsBalancerprotected
BaseLoadBalancer
(MetricsBalancer metricsBalancer) This Constructor accepts an instance of MetricsBalancer, which will be used instead of creating a new one -
Method Summary
Modifier and TypeMethodDescriptionfinal List<RegionPlan>
balanceCluster
(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) Perform the major balance operation for cluster, will invokebalanceTable(TableName, Map)
to do actual balance.protected abstract List<RegionPlan>
balanceTable
(TableName tableName, Map<ServerName, List<RegionInfo>> loadOfOneTable) Perform the major balance operation for table, all sub classes should override this method.private BalancerClusterState
createCluster
(List<ServerName> servers, Collection<RegionInfo> regions) private RegionHDFSBlockLocationFinder
createRegionLocationFinder
(org.apache.hadoop.conf.Configuration conf) private List<ServerName>
findIdleServers
(List<ServerName> servers) protected final org.apache.hadoop.conf.Configuration
getConf()
protected float
private Map<ServerName,
List<RegionInfo>> getRegionAssignmentsByServer
(Collection<RegionInfo> regions) protected final boolean
void
Initialize the load balancer.boolean
Returns True ifStoppable.stop(String)
has been closed.protected void
loadConf
(org.apache.hadoop.conf.Configuration conf) void
onConfigurationChange
(org.apache.hadoop.conf.Configuration conf) Notification that config has changedvoid
If balancer needs to do initialization after Master has started up, lets do that here.protected void
preBalanceCluster
(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) Called before actually executing balanceCluster.randomAssignment
(RegionInfo regionInfo, List<ServerName> servers) Used to assign a single region to a random server.private ServerName
randomAssignment
(BalancerClusterState cluster, RegionInfo regionInfo, List<ServerName> servers) Used to assign a single region to a random server.void
regionOffline
(RegionInfo regionInfo) Marks the region as offline at balancer.void
regionOnline
(RegionInfo regionInfo, ServerName sn) Marks the region as online at balancer.retainAssignment
(Map<RegionInfo, ServerName> regions, List<ServerName> servers) Generates a bulk assignment startup plan, attempting to reuse the existing assignment information from META, but adjusting for the specified list of available/online servers available for assignment.roundRobinAssignment
(List<RegionInfo> regions, List<ServerName> servers) Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.private void
roundRobinAssignment
(BalancerClusterState cluster, List<RegionInfo> regions, List<ServerName> servers, Map<ServerName, List<RegionInfo>> assignments) Round-robin a list of regions to a list of serversvoid
setClusterInfoProvider
(ClusterInfoProvider provider) Set the cluster info provider.protected final boolean
void
Stop this service.protected final Map<ServerName,
List<RegionInfo>> toEnsumbleTableLoad
(Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) void
updateBalancerStatus
(boolean status) Updates the balancer status tag reported to JMXvoid
Set the current cluster status.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer
updateBalancerLoadInfo
-
Field Details
-
LOG
-
BALANCER_DECISION_BUFFER_ENABLED
- See Also:
-
DEFAULT_BALANCER_DECISION_BUFFER_ENABLED
- See Also:
-
BALANCER_REJECTION_BUFFER_ENABLED
- See Also:
-
DEFAULT_BALANCER_REJECTION_BUFFER_ENABLED
- See Also:
-
DEFAULT_HBASE_MASTER_LOADBALANCE_BYTABLE
- See Also:
-
MIN_SERVER_BALANCE
- See Also:
-
stopped
-
regionFinder
-
useRegionFinder
-
isByTable
-
slop
-
rackManager
-
metricsBalancer
-
clusterStatus
-
masterServerName
-
provider
-
-
Constructor Details
-
BaseLoadBalancer
protected BaseLoadBalancer()The constructor that uses the basic MetricsBalancer -
BaseLoadBalancer
This Constructor accepts an instance of MetricsBalancer, which will be used instead of creating a new one
-
-
Method Details
-
getConf
-
updateClusterMetrics
Description copied from interface:LoadBalancer
Set the current cluster status. This allows a LoadBalancer to map host name to a server- Specified by:
updateClusterMetrics
in interfaceLoadBalancer
-
setClusterInfoProvider
Description copied from interface:LoadBalancer
Set the cluster info provider. Usually it is just a wrapper of master.- Specified by:
setClusterInfoProvider
in interfaceLoadBalancer
-
postMasterStartupInitialize
Description copied from interface:LoadBalancer
If balancer needs to do initialization after Master has started up, lets do that here.- Specified by:
postMasterStartupInitialize
in interfaceLoadBalancer
-
idleRegionServerExist
-
sloppyRegionServerExist
-
roundRobinAssignment
@NonNull public Map<ServerName,List<RegionInfo>> roundRobinAssignment(List<RegionInfo> regions, List<ServerName> servers) throws HBaseIOException Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment. Takes a list of all the regions and all the servers in the cluster and returns a map of each server to the regions that it should be assigned. Currently implemented as a round-robin assignment. Same invariant as load balancing, all servers holding floor(avg) or ceiling(avg). TODO: Use block locations from HDFS to place regions with their blocks- Specified by:
roundRobinAssignment
in interfaceLoadBalancer
- Parameters:
regions
- all regionsservers
- all servers- Returns:
- map of server to the regions it should take, or emptyMap if no assignment is possible (ie. no servers)
- Throws:
HBaseIOException
-
createCluster
private BalancerClusterState createCluster(List<ServerName> servers, Collection<RegionInfo> regions) throws HBaseIOException - Throws:
HBaseIOException
-
findIdleServers
-
randomAssignment
public ServerName randomAssignment(RegionInfo regionInfo, List<ServerName> servers) throws HBaseIOException Used to assign a single region to a random server.- Specified by:
randomAssignment
in interfaceLoadBalancer
- Parameters:
regionInfo
- Region for which this selection is being done.- Throws:
HBaseIOException
-
retainAssignment
@NonNull public Map<ServerName,List<RegionInfo>> retainAssignment(Map<RegionInfo, ServerName> regions, List<ServerName> servers) throws HBaseIOExceptionGenerates a bulk assignment startup plan, attempting to reuse the existing assignment information from META, but adjusting for the specified list of available/online servers available for assignment.Takes a map of all regions to their existing assignment from META. Also takes a list of online servers for regions to be assigned to. Attempts to retain all assignment, so in some instances initial assignment will not be completely balanced.
Any leftover regions without an existing server to be assigned to will be assigned randomly to available servers.
- Specified by:
retainAssignment
in interfaceLoadBalancer
- Parameters:
regions
- regions and existing assignment from metaservers
- available servers- Returns:
- map of servers and regions to be assigned to them, or emptyMap if no assignment is possible (ie. no servers)
- Throws:
HBaseIOException
-
getDefaultSlop
-
createRegionLocationFinder
private RegionHDFSBlockLocationFinder createRegionLocationFinder(org.apache.hadoop.conf.Configuration conf) -
loadConf
-
initialize
Description copied from interface:LoadBalancer
Initialize the load balancer. Must be called after setters.- Specified by:
initialize
in interfaceLoadBalancer
-
regionOnline
Description copied from interface:LoadBalancer
Marks the region as online at balancer.- Specified by:
regionOnline
in interfaceLoadBalancer
-
regionOffline
Description copied from interface:LoadBalancer
Marks the region as offline at balancer.- Specified by:
regionOffline
in interfaceLoadBalancer
-
isStopped
Description copied from interface:Stoppable
Returns True ifStoppable.stop(String)
has been closed. -
stop
Description copied from interface:Stoppable
Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions. -
updateBalancerStatus
Updates the balancer status tag reported to JMX- Specified by:
updateBalancerStatus
in interfaceLoadBalancer
-
randomAssignment
private ServerName randomAssignment(BalancerClusterState cluster, RegionInfo regionInfo, List<ServerName> servers) Used to assign a single region to a random server. -
roundRobinAssignment
private void roundRobinAssignment(BalancerClusterState cluster, List<RegionInfo> regions, List<ServerName> servers, Map<ServerName, List<RegionInfo>> assignments) Round-robin a list of regions to a list of servers -
getRegionAssignmentsByServer
private Map<ServerName,List<RegionInfo>> getRegionAssignmentsByServer(Collection<RegionInfo> regions) -
toEnsumbleTableLoad
protected final Map<ServerName,List<RegionInfo>> toEnsumbleTableLoad(Map<TableName, Map<ServerName, List<RegionInfo>>> LoadOfAllTable) -
balanceTable
protected abstract List<RegionPlan> balanceTable(TableName tableName, Map<ServerName, List<RegionInfo>> loadOfOneTable) Perform the major balance operation for table, all sub classes should override this method. Will be invoked bybalanceCluster(Map)
. IfHConstants.HBASE_MASTER_LOADBALANCE_BYTABLE
is enabled, we will call this method multiple times, one table a time, where we will only pass in the regions for a single table each time. If not, we will pass in all the regions at once, and thetableName
will beHConstants.ENSEMBLE_TABLE_NAME
.- Parameters:
tableName
- the table to be balancedloadOfOneTable
- region load of servers for the specific one table- Returns:
- List of plans
-
preBalanceCluster
Called before actually executing balanceCluster. The sub classes could override this method to do some initialization work. -
balanceCluster
public final List<RegionPlan> balanceCluster(Map<TableName, Map<ServerName, List<RegionInfo>>> loadOfAllTable) Perform the major balance operation for cluster, will invokebalanceTable(TableName, Map)
to do actual balance. THIs method is marked as final which means you should not override this method. See the javadoc forbalanceTable(TableName, Map)
for more details.- Specified by:
balanceCluster
in interfaceLoadBalancer
- Parameters:
loadOfAllTable
- region load of servers for all table- Returns:
- a list of regions to be moved, including source and destination, or null if cluster is already balanced
- See Also:
-
onConfigurationChange
Description copied from interface:LoadBalancer
Notification that config has changed- Specified by:
onConfigurationChange
in interfaceConfigurationObserver
- Specified by:
onConfigurationChange
in interfaceLoadBalancer
-