Package org.apache.hadoop.hbase.master.balancer

Class CacheAwareLoadBalancer

java.lang.Object

org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer

org.apache.hadoop.hbase.master.balancer.CacheAwareLoadBalancer

All Implemented Interfaces:

ConfigurationObserver, LoadBalancer, Stoppable

@Private
public class CacheAwareLoadBalancer
extends StochasticLoadBalancer

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private class	CacheAwareLoadBalancer.CacheAwareCandidateGenerator
(package private) class	CacheAwareLoadBalancer.CacheAwareCostFunction
(package private) static class	CacheAwareLoadBalancer.CacheAwareRegionSkewnessCostFunction
private class	CacheAwareLoadBalancer.CacheAwareSkewnessCandidateGenerator

Field Summary

Fields

Modifier and Type	Field	Description
static final String	CACHE_RATIO_THRESHOLD
static final float	CACHE_RATIO_THRESHOLD_DEFAULT
private org.apache.hadoop.conf.Configuration	configuration
private static final org.slf4j.Logger	LOG
static final float	LOW_CACHE_RATIO_FOR_RELOCATION_DEFAULT
static final String	LOW_CACHE_RATIO_FOR_RELOCATION_KEY	Below this cache ratio on the current host, a move may be considered for the free-space heuristic.
private float	lowCacheRatioThreshold
static final float	MIN_FREE_CACHE_SPACE_FACTOR_DEFAULT
static final String	MIN_FREE_CACHE_SPACE_FACTOR_KEY	Minimum free block cache on a target server, as a multiple of the region's on-disk size in bytes, required to count that server as a relocation opportunity.
private float	minFreeCacheSpaceFactor
static final float	POTENTIAL_CACHE_RATIO_AFTER_MOVE_DEFAULT
static final String	POTENTIAL_CACHE_RATIO_AFTER_MOVE_KEY	Optimistic region cache ratio assumed for cost purposes when a better host has free cache space (actual warmup is not modeled).
private float	potentialCacheRatioAfterMove
Float	ratioThreshold
private BigDecimal	simulatedRatio
private Long	sleepTime

Fields inherited from class org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer

candidateGenerators, COST_FUNCTIONS_COST_FUNCTIONS_KEY, costFunctions, DEFAULT_KEEP_REGION_LOADS, DEFAULT_MAX_RUNNING_TIME, DEFAULT_MAX_STEPS, DEFAULT_MIN_COST_NEED_BALANCE, DEFAULT_RUN_MAX_STEPS, DEFAULT_STEPS_PER_REGION, KEEP_REGION_LOADS, loads, MAX_RUNNING_TIME_KEY, MAX_STEPS_KEY, MIN_COST_NEED_BALANCE_KEY, OVERALL_COST_FUNCTION_NAME, regionCacheRatioOnOldServerMap, RUN_MAX_STEPS_KEY, shuffledGeneratorClasses, STEPS_PER_REGION_KEY

Fields inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

BALANCER_DECISION_BUFFER_ENABLED, BALANCER_REJECTION_BUFFER_ENABLED, clusterStatus, DEFAULT_BALANCER_DECISION_BUFFER_ENABLED, DEFAULT_BALANCER_REJECTION_BUFFER_ENABLED, DEFAULT_HBASE_MASTER_LOADBALANCE_BYTABLE, isByTable, masterServerName, metricsBalancer, MIN_SERVER_BALANCE, provider, rackManager, regionFinder, REGIONS_SLOP_DEFAULT, REGIONS_SLOP_KEY, slop, useRegionFinder

Fields inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

BOGUS_SERVER_NAME, HBASE_RSGROUP_LOADBALANCER_CLASS, MOVE_THROTTLING, MOVE_THROTTLING_DEFAULT

Constructor Summary

Constructors

Constructor	Description
CacheAwareLoadBalancer()

Method Summary

Modifier and Type	Method	Description
private void	addCostFunction(List<CostFunction> costFunctions, CostFunction costFunction)
protected List<RegionPlan>	balanceTable(TableName tableName, Map<ServerName,List<RegionInfo>> loadOfOneTable)	Given the cluster state this will try and approach an optimal balance.
protected Map<Class<? extends CandidateGenerator>,CandidateGenerator>	createCandidateGenerators(org.apache.hadoop.conf.Configuration conf)
protected List<CostFunction>	createCostFunctions(org.apache.hadoop.conf.Configuration configuration)
protected BalancerClusterState	createState(Map<ServerName,List<RegionInfo>> clusterState, Map<String,Deque<BalancerRegionLoad>> loads, RegionHDFSBlockLocationFinder finder, RackManager rackManager)
private RegionInfo	getRegionInfoByEncodedName(BalancerClusterState cluster, String regionName)
protected Map<ServerName,Long>	getServerBlockCacheFreeBytes()
long	getThrottleDurationMs(RegionPlan plan)
void	loadConf(org.apache.hadoop.conf.Configuration configuration)
void	updateClusterMetrics(ClusterMetrics clusterMetrics)	Set the current cluster status.
private void	updateRegionLoad()	Collect the amount of region cached for all the regions from all the active region servers.

Methods inherited from class org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer

composeAttributeName, computeCost, functionCost, getCandidateGenerators, getCostFunctionNames, getCostFunctions, getRandomGenerator, initCosts, needsBalance, nextAction, sendRejectionReasonToRingBuffer, setRackManager, updateBalancerLoadInfo, updateCostsAndWeightsWithAction, updateMetricsSize

Methods inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer

balanceCluster, getConf, getDefaultSlop, idleRegionServerExist, initialize, isStopped, onConfigurationChange, postMasterStartupInitialize, preBalanceCluster, randomAssignment, regionOffline, regionOnline, retainAssignment, roundRobinAssignment, setClusterInfoProvider, sloppyRegionServerExist, stop, toEnsumbleTableLoad, updateBalancerStatus

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.LoadBalancer

onBalancingComplete, onBalancingStart, throttle

Field Details

LOG

private static final org.slf4j.Logger LOG

CACHE_RATIO_THRESHOLD

public static final String CACHE_RATIO_THRESHOLD

See Also:

• Constant Field Values

CACHE_RATIO_THRESHOLD_DEFAULT

public static final float CACHE_RATIO_THRESHOLD_DEFAULT

See Also:

• Constant Field Values

LOW_CACHE_RATIO_FOR_RELOCATION_KEY

public static final String LOW_CACHE_RATIO_FOR_RELOCATION_KEY

Below this cache ratio on the current host, a move may be considered for the free-space heuristic.

See Also:

• Constant Field Values

LOW_CACHE_RATIO_FOR_RELOCATION_DEFAULT

public static final float LOW_CACHE_RATIO_FOR_RELOCATION_DEFAULT

See Also:

• Constant Field Values

POTENTIAL_CACHE_RATIO_AFTER_MOVE_KEY

public static final String POTENTIAL_CACHE_RATIO_AFTER_MOVE_KEY

Optimistic region cache ratio assumed for cost purposes when a better host has free cache space (actual warmup is not modeled).

See Also:

• Constant Field Values

POTENTIAL_CACHE_RATIO_AFTER_MOVE_DEFAULT

public static final float POTENTIAL_CACHE_RATIO_AFTER_MOVE_DEFAULT

See Also:

• Constant Field Values

MIN_FREE_CACHE_SPACE_FACTOR_KEY

public static final String MIN_FREE_CACHE_SPACE_FACTOR_KEY

Minimum free block cache on a target server, as a multiple of the region's on-disk size in bytes, required to count that server as a relocation opportunity.

See Also:

• Constant Field Values

MIN_FREE_CACHE_SPACE_FACTOR_DEFAULT

public static final float MIN_FREE_CACHE_SPACE_FACTOR_DEFAULT

See Also:

• Constant Field Values

ratioThreshold

public Float ratioThreshold

sleepTime

private Long sleepTime

configuration

private org.apache.hadoop.conf.Configuration configuration

lowCacheRatioThreshold

private float lowCacheRatioThreshold

potentialCacheRatioAfterMove

private float potentialCacheRatioAfterMove

minFreeCacheSpaceFactor

private float minFreeCacheSpaceFactor

simulatedRatio

private BigDecimal simulatedRatio

Constructor Details

CacheAwareLoadBalancer

public CacheAwareLoadBalancer()

Method Details

loadConf

public void loadConf(org.apache.hadoop.conf.Configuration configuration)

Overrides:

loadConf in class StochasticLoadBalancer

createCandidateGenerators

protected Map<Class<? extends CandidateGenerator>,CandidateGenerator> createCandidateGenerators(org.apache.hadoop.conf.Configuration conf)

Overrides:

createCandidateGenerators in class StochasticLoadBalancer

createCostFunctions

protected List<CostFunction> createCostFunctions(org.apache.hadoop.conf.Configuration configuration)

Overrides:

createCostFunctions in class StochasticLoadBalancer

addCostFunction

private void addCostFunction(List<CostFunction> costFunctions,
 CostFunction costFunction)

updateClusterMetrics

public void updateClusterMetrics(ClusterMetrics clusterMetrics)

Description copied from interface: LoadBalancer

Set the current cluster status. This allows a LoadBalancer to map host name to a server

Specified by:

updateClusterMetrics in interface LoadBalancer

Overrides:

updateClusterMetrics in class StochasticLoadBalancer

getServerBlockCacheFreeBytes

protected Map<ServerName,Long> getServerBlockCacheFreeBytes()

createState

protected BalancerClusterState createState(Map<ServerName,List<RegionInfo>> clusterState,
 Map<String,Deque<BalancerRegionLoad>> loads,
 RegionHDFSBlockLocationFinder finder,
 RackManager rackManager)

Overrides:

createState in class StochasticLoadBalancer

updateRegionLoad

private void updateRegionLoad()

Collect the amount of region cached for all the regions from all the active region servers.

getRegionInfoByEncodedName

private RegionInfo getRegionInfoByEncodedName(BalancerClusterState cluster,
 String regionName)

getThrottleDurationMs

public long getThrottleDurationMs(RegionPlan plan)

balanceTable

protected List<RegionPlan> balanceTable(TableName tableName,
 Map<ServerName,List<RegionInfo>> loadOfOneTable)

Description copied from class: StochasticLoadBalancer

Given the cluster state this will try and approach an optimal balance. This should always approach the optimal state given enough steps.

Overrides:

balanceTable in class StochasticLoadBalancer

Parameters:

tableName - the table to be balanced

loadOfOneTable - region load of servers for the specific one table

Returns:

List of plans