org.apache.hadoop.hbase.favored

Class FavoredNodeLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, ConfigurationObserver, FavoredNodesPromoter, LoadBalancer, Stoppable

@InterfaceAudience.LimitedPrivate(value="Configuration")
public class FavoredNodeLoadBalancer
extends BaseLoadBalancer
implements FavoredNodesPromoter

An implementation of the LoadBalancer that assigns favored nodes for each region. There is a Primary RegionServer that hosts the region, and then there is Secondary and Tertiary RegionServers. Currently, the favored nodes information is used in creating HDFS files - the Primary RegionServer passes the primary, secondary, tertiary node addresses as hints to the DistributedFileSystem API for creating files on the filesystem. These nodes are treated as hints by the HDFS to place the blocks of the file. This alleviates the problem to do with reading from remote nodes (since we can make the Secondary RegionServer as the new Primary RegionServer) after a region is recovered. This should help provide consistent read latencies for the regions even when their primary region servers die.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

BaseLoadBalancer.Cluster

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf
private FavoredNodesManager	fnm
private static org.slf4j.Logger	LOG
private RackManager	rackManager

Fields inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

clusterStatus, config, maintenanceMode, masterServerName, metricsBalancer, MIN_SERVER_BALANCE, onlySystemTablesOnMaster, overallSlop, regionFinder, services, slop, useRegionFinder

Fields inherited from interface org.apache.hadoop.hbase.favored.FavoredNodesPromoter

FAVORED_ALWAYS_ASSIGN_REGIONS

Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

BOGUS_SERVER_NAME, SYSTEM_TABLES_ON_MASTER, TABLES_ON_MASTER

Constructor Summary

Constructors

Constructor and Description
FavoredNodeLoadBalancer()

Method Summary

Modifier and Type	Method and Description
private void	addRegionToMap(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes, RegionInfo region, ServerName host)
private void	assignRegionToAvailableFavoredNode(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes, RegionInfo region, ServerName primaryHost, ServerName secondaryHost, ServerName tertiaryHost)
private void	assignSecondaryAndTertiaryNodesForRegion(FavoredNodeAssignmentHelper assignmentHelper, List<RegionInfo> regions, Map<RegionInfo,ServerName> primaryRSMap)
private ServerName	availableServersContains(List<ServerName> servers, ServerName favoredNode)
List<RegionPlan>	balanceCluster(Map<ServerName,List<RegionInfo>> clusterState) Perform the major balance operation
List<RegionPlan>	balanceCluster(TableName tableName, Map<ServerName,List<RegionInfo>> clusterState) Perform the major balance operation
void	generateFavoredNodesForDaughter(List<ServerName> servers, RegionInfo parent, RegionInfo regionA, RegionInfo regionB)
void	generateFavoredNodesForMergedRegion(RegionInfo merged, RegionInfo[] mergeParents)
List<ServerName>	getFavoredNodes(RegionInfo regionInfo)
private Set<ServerName>	getInheritedFNForDaughter(FavoredNodeAssignmentHelper helper, List<ServerName> parentFavoredNodes, FavoredNodesPlan.Position primary, FavoredNodesPlan.Position secondary)
void	initialize() Initialize the load balancer.
ServerName	randomAssignment(RegionInfo regionInfo, List<ServerName> servers) Used to assign a single region to a random server.
Map<ServerName,List<RegionInfo>>	roundRobinAssignment(List<RegionInfo> regions, List<ServerName> servers) Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.
private void	roundRobinAssignmentImpl(FavoredNodeAssignmentHelper assignmentHelper, Map<ServerName,List<RegionInfo>> assignmentMap, List<RegionInfo> regions, List<ServerName> servers)
private Pair<Map<ServerName,List<RegionInfo>>,List<RegionInfo>>	segregateRegionsAndAssignRegionsWithFavoredNodes(List<RegionInfo> regions, List<ServerName> availableServers)
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

areSomeRegionReplicasColocated, assignMasterSystemRegions, balanceMasterRegions, createCluster, getConf, getRegionAssignmentsByServer, isStopped, needsBalance, onConfigurationChange, postMasterStartupInitialize, regionOffline, regionOnline, retainAssignment, setClusterLoad, setClusterMetrics, setMasterServices, setRackManager, setSlop, shouldBeOnMaster, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

isMasterCanHostUserRegions, isSystemTablesOnlyOnMaster, isTablesOnMaster

Field Detail

LOG

private static final org.slf4j.Logger LOG

rackManager

private RackManager rackManager

conf

private org.apache.hadoop.conf.Configuration conf

fnm

private FavoredNodesManager fnm

Constructor Detail

FavoredNodeLoadBalancer

public FavoredNodeLoadBalancer()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class BaseLoadBalancer

initialize

public void initialize()
                throws HBaseIOException

Description copied from interface: LoadBalancer

Initialize the load balancer. Must be called after setters.

Specified by:

initialize in interface LoadBalancer

Overrides:

initialize in class BaseLoadBalancer

Throws:

HBaseIOException

balanceCluster

public List<RegionPlan> balanceCluster(Map<ServerName,List<RegionInfo>> clusterState)

Description copied from interface: LoadBalancer

Perform the major balance operation

Specified by:

balanceCluster in interface LoadBalancer

Returns:

List of plans

roundRobinAssignment

public Map<ServerName,List<RegionInfo>> roundRobinAssignment(List<RegionInfo> regions,
                                                             List<ServerName> servers)
                                                      throws HBaseIOException

Description copied from class: BaseLoadBalancer

Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.

Takes a list of all the regions and all the servers in the cluster and returns a map of each server to the regions that it should be assigned.

Currently implemented as a round-robin assignment. Same invariant as load balancing, all servers holding floor(avg) or ceiling(avg). TODO: Use block locations from HDFS to place regions with their blocks

Specified by:

roundRobinAssignment in interface LoadBalancer

Overrides:

roundRobinAssignment in class BaseLoadBalancer

Parameters:

regions - all regions

servers - all servers

Returns:

map of server to the regions it should take, or null if no assignment is possible (ie. no regions or no servers)

Throws:

HBaseIOException

randomAssignment

public ServerName randomAssignment(RegionInfo regionInfo,
                                   List<ServerName> servers)
                            throws HBaseIOException

Description copied from class: BaseLoadBalancer

Used to assign a single region to a random server.

Specified by:

randomAssignment in interface LoadBalancer

Overrides:

randomAssignment in class BaseLoadBalancer

Parameters:

regionInfo - Region for which this selection is being done.

Returns:

Servername

Throws:

HBaseIOException

segregateRegionsAndAssignRegionsWithFavoredNodes

private Pair<Map<ServerName,List<RegionInfo>>,List<RegionInfo>> segregateRegionsAndAssignRegionsWithFavoredNodes(List<RegionInfo> regions,
                                                                                                                 List<ServerName> availableServers)

availableServersContains

private ServerName availableServersContains(List<ServerName> servers,
                                            ServerName favoredNode)

assignRegionToAvailableFavoredNode

private void assignRegionToAvailableFavoredNode(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes,
                                                RegionInfo region,
                                                ServerName primaryHost,
                                                ServerName secondaryHost,
                                                ServerName tertiaryHost)

addRegionToMap

private void addRegionToMap(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes,
                            RegionInfo region,
                            ServerName host)

getFavoredNodes

public List<ServerName> getFavoredNodes(RegionInfo regionInfo)

roundRobinAssignmentImpl

private void roundRobinAssignmentImpl(FavoredNodeAssignmentHelper assignmentHelper,
                                      Map<ServerName,List<RegionInfo>> assignmentMap,
                                      List<RegionInfo> regions,
                                      List<ServerName> servers)
                               throws IOException

Throws:

IOException

assignSecondaryAndTertiaryNodesForRegion

private void assignSecondaryAndTertiaryNodesForRegion(FavoredNodeAssignmentHelper assignmentHelper,
                                                      List<RegionInfo> regions,
                                                      Map<RegionInfo,ServerName> primaryRSMap)
                                               throws IOException

Throws:

IOException

generateFavoredNodesForDaughter

public void generateFavoredNodesForDaughter(List<ServerName> servers,
                                            RegionInfo parent,
                                            RegionInfo regionA,
                                            RegionInfo regionB)
                                     throws IOException

Specified by:

generateFavoredNodesForDaughter in interface FavoredNodesPromoter

Throws:

IOException

getInheritedFNForDaughter

private Set<ServerName> getInheritedFNForDaughter(FavoredNodeAssignmentHelper helper,
                                                  List<ServerName> parentFavoredNodes,
                                                  FavoredNodesPlan.Position primary,
                                                  FavoredNodesPlan.Position secondary)
                                           throws IOException

Throws:

IOException

generateFavoredNodesForMergedRegion

public void generateFavoredNodesForMergedRegion(RegionInfo merged,
                                                RegionInfo[] mergeParents)
                                         throws IOException

Specified by:

generateFavoredNodesForMergedRegion in interface FavoredNodesPromoter

Throws:

IOException

balanceCluster

public List<RegionPlan> balanceCluster(TableName tableName,
                                       Map<ServerName,List<RegionInfo>> clusterState)
                                throws HBaseIOException

Description copied from interface: LoadBalancer

Perform the major balance operation

Specified by:

balanceCluster in interface LoadBalancer

Returns:

List of plans

Throws:

HBaseIOException