org.apache.hadoop.hbase.favored

Class FavoredNodeLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer

All Implemented Interfaces:

ConfigurationObserver, FavoredNodesPromoter, LoadBalancer, Stoppable

@InterfaceAudience.LimitedPrivate(value="Configuration")
public class FavoredNodeLoadBalancer
extends BaseLoadBalancer
implements FavoredNodesPromoter

An implementation of the LoadBalancer that assigns favored nodes for each region. There is a Primary RegionServer that hosts the region, and then there is Secondary and Tertiary RegionServers. Currently, the favored nodes information is used in creating HDFS files - the Primary RegionServer passes the primary, secondary, tertiary node addresses as hints to the DistributedFileSystem API for creating files on the filesystem. These nodes are treated as hints by the HDFS to place the blocks of the file. This alleviates the problem to do with reading from remote nodes (since we can make the Secondary RegionServer as the new Primary RegionServer) after a region is recovered. This should help provide consistent read latencies for the regions even when their primary region servers die.

Field Summary

Fields

Modifier and Type	Field and Description
private FavoredNodesManager	fnm
private static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

BALANCER_DECISION_BUFFER_ENABLED, BALANCER_REJECTION_BUFFER_ENABLED, clusterStatus, DEFAULT_BALANCER_DECISION_BUFFER_ENABLED, DEFAULT_BALANCER_REJECTION_BUFFER_ENABLED, DEFAULT_HBASE_MASTER_LOADBALANCE_BYTABLE, isByTable, masterServerName, metricsBalancer, MIN_SERVER_BALANCE, onlySystemTablesOnMaster, rackManager, regionFinder, services, slop, useRegionFinder

Fields inherited from interface org.apache.hadoop.hbase.favored.FavoredNodesPromoter

FAVORED_ALWAYS_ASSIGN_REGIONS

Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

BOGUS_SERVER_NAME, SYSTEM_TABLES_ON_MASTER, TABLES_ON_MASTER

Constructor Summary

Constructors

Constructor and Description
FavoredNodeLoadBalancer()

Method Summary

Modifier and Type	Method and Description
private void	addRegionToMap(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes, RegionInfo region, ServerName host)
private void	assignRegionToAvailableFavoredNode(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes, RegionInfo region, ServerName primaryHost, ServerName secondaryHost, ServerName tertiaryHost)
private void	assignSecondaryAndTertiaryNodesForRegion(FavoredNodeAssignmentHelper assignmentHelper, List<RegionInfo> regions, Map<RegionInfo,ServerName> primaryRSMap)
private ServerName	availableServersContains(List<ServerName> servers, ServerName favoredNode)
protected List<RegionPlan>	balanceTable(TableName tableName, Map<ServerName,List<RegionInfo>> loadOfOneTable) Perform the major balance operation for table, all sub classes should override this method.
void	generateFavoredNodesForDaughter(List<ServerName> servers, RegionInfo parent, RegionInfo regionA, RegionInfo regionB)
void	generateFavoredNodesForMergedRegion(RegionInfo merged, RegionInfo[] mergeParents)
List<ServerName>	getFavoredNodes(RegionInfo regionInfo)
private Set<ServerName>	getInheritedFNForDaughter(FavoredNodeAssignmentHelper helper, List<ServerName> parentFavoredNodes, FavoredNodesPlan.Position primary, FavoredNodesPlan.Position secondary)
void	initialize() Initialize the load balancer.
ServerName	randomAssignment(RegionInfo regionInfo, List<ServerName> servers) Used to assign a single region to a random server.
Map<ServerName,List<RegionInfo>>	roundRobinAssignment(List<RegionInfo> regions, List<ServerName> servers) Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.
private void	roundRobinAssignmentImpl(FavoredNodeAssignmentHelper assignmentHelper, Map<ServerName,List<RegionInfo>> assignmentMap, List<RegionInfo> regions, List<ServerName> servers)
private Pair<Map<ServerName,List<RegionInfo>>,List<RegionInfo>>	segregateRegionsAndAssignRegionsWithFavoredNodes(List<RegionInfo> regions, List<ServerName> availableServers)

Methods inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

assignMasterSystemRegions, balanceCluster, balanceMasterRegions, getConf, getDefaultSlop, idleRegionServerExist, isStopped, loadConf, onConfigurationChange, postMasterStartupInitialize, preBalanceCluster, regionOffline, regionOnline, retainAssignment, setMasterServices, shouldBeOnMaster, sloppyRegionServerExist, stop, toEnsumbleTableLoad, updateBalancerStatus, updateClusterMetrics

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

isMasterCanHostUserRegions, isSystemTablesOnlyOnMaster, isTablesOnMaster, updateBalancerLoadInfo

Field Detail

LOG

private static final org.slf4j.Logger LOG

fnm

private FavoredNodesManager fnm

Constructor Detail

FavoredNodeLoadBalancer

public FavoredNodeLoadBalancer()

Method Detail

initialize

public void initialize()

Description copied from interface: LoadBalancer

Initialize the load balancer. Must be called after setters. n

Specified by:

initialize in interface LoadBalancer

Overrides:

initialize in class BaseLoadBalancer

balanceTable

protected List<RegionPlan> balanceTable(TableName tableName,
                                        Map<ServerName,List<RegionInfo>> loadOfOneTable)

Description copied from class: BaseLoadBalancer

Perform the major balance operation for table, all sub classes should override this method.

Will be invoked by BaseLoadBalancer.balanceCluster(Map). If HConstants.HBASE_MASTER_LOADBALANCE_BYTABLE is enabled, we will call this method multiple times, one table a time, where we will only pass in the regions for a single table each time. If not, we will pass in all the regions at once, and the tableName will be HConstants.ENSEMBLE_TABLE_NAME.

Specified by:

balanceTable in class BaseLoadBalancer

Parameters:

tableName - the table to be balanced

loadOfOneTable - region load of servers for the specific one table

Returns:

List of plans

roundRobinAssignment

@NonNull
public Map<ServerName,List<RegionInfo>> roundRobinAssignment(List<RegionInfo> regions,
                                                                      List<ServerName> servers)
                                                               throws HBaseIOException

Description copied from class: BaseLoadBalancer

Generates a bulk assignment plan to be used on cluster startup using a simple round-robin assignment.

Takes a list of all the regions and all the servers in the cluster and returns a map of each server to the regions that it should be assigned.

Currently implemented as a round-robin assignment. Same invariant as load balancing, all servers holding floor(avg) or ceiling(avg). TODO: Use block locations from HDFS to place regions with their blocks

Specified by:

roundRobinAssignment in interface LoadBalancer

Overrides:

roundRobinAssignment in class BaseLoadBalancer

Parameters:

regions - all regions

servers - all servers

Returns:

map of server to the regions it should take, or emptyMap if no assignment is possible (ie. no servers)

Throws:

HBaseIOException

randomAssignment

public ServerName randomAssignment(RegionInfo regionInfo,
                                   List<ServerName> servers)
                            throws HBaseIOException

Description copied from class: BaseLoadBalancer

Used to assign a single region to a random server.

Specified by:

randomAssignment in interface LoadBalancer

Overrides:

randomAssignment in class BaseLoadBalancer

Parameters:

regionInfo - Region for which this selection is being done. nn

Throws:

HBaseIOException

segregateRegionsAndAssignRegionsWithFavoredNodes

private Pair<Map<ServerName,List<RegionInfo>>,List<RegionInfo>> segregateRegionsAndAssignRegionsWithFavoredNodes(List<RegionInfo> regions,
                                                                                                                 List<ServerName> availableServers)

availableServersContains

private ServerName availableServersContains(List<ServerName> servers,
                                            ServerName favoredNode)

assignRegionToAvailableFavoredNode

private void assignRegionToAvailableFavoredNode(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes,
                                                RegionInfo region,
                                                ServerName primaryHost,
                                                ServerName secondaryHost,
                                                ServerName tertiaryHost)

addRegionToMap

private void addRegionToMap(Map<ServerName,List<RegionInfo>> assignmentMapForFavoredNodes,
                            RegionInfo region,
                            ServerName host)

getFavoredNodes

public List<ServerName> getFavoredNodes(RegionInfo regionInfo)

roundRobinAssignmentImpl

private void roundRobinAssignmentImpl(FavoredNodeAssignmentHelper assignmentHelper,
                                      Map<ServerName,List<RegionInfo>> assignmentMap,
                                      List<RegionInfo> regions,
                                      List<ServerName> servers)
                               throws IOException

Throws:

IOException

assignSecondaryAndTertiaryNodesForRegion

private void assignSecondaryAndTertiaryNodesForRegion(FavoredNodeAssignmentHelper assignmentHelper,
                                                      List<RegionInfo> regions,
                                                      Map<RegionInfo,ServerName> primaryRSMap)
                                               throws IOException

Throws:

IOException

generateFavoredNodesForDaughter

public void generateFavoredNodesForDaughter(List<ServerName> servers,
                                            RegionInfo parent,
                                            RegionInfo regionA,
                                            RegionInfo regionB)
                                     throws IOException

Specified by:

generateFavoredNodesForDaughter in interface FavoredNodesPromoter

Throws:

IOException

getInheritedFNForDaughter

private Set<ServerName> getInheritedFNForDaughter(FavoredNodeAssignmentHelper helper,
                                                  List<ServerName> parentFavoredNodes,
                                                  FavoredNodesPlan.Position primary,
                                                  FavoredNodesPlan.Position secondary)
                                           throws IOException

Throws:

IOException

generateFavoredNodesForMergedRegion

public void generateFavoredNodesForMergedRegion(RegionInfo merged,
                                                RegionInfo[] mergeParents)
                                         throws IOException

Specified by:

generateFavoredNodesForMergedRegion in interface FavoredNodesPromoter

Throws:

IOException