Package org.apache.hadoop.hbase.master.balancer

Class TestCacheAwareLoadBalancerCostFunctions

java.lang.Object

org.apache.hadoop.hbase.master.balancer.BalancerTestBase

org.apache.hadoop.hbase.master.balancer.StochasticBalancerTestBase

org.apache.hadoop.hbase.master.balancer.TestCacheAwareLoadBalancerCostFunctions

@Tag("org.apache.hadoop.hbase.testclassification.MasterTests") @Tag("org.apache.hadoop.hbase.testclassification.MediumTests")
public class TestCacheAwareLoadBalancerCostFunctions
extends StochasticBalancerTestBase

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private class	TestCacheAwareLoadBalancerCostFunctions.MockClusterForCacheCost

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.master.balancer.BalancerTestBase

BalancerTestBase.MockMapping

Field Summary

Fields

Modifier and Type	Field	Description
private final int[][][]	clusterRegionCacheRatioMocks	Data set to testCacheCost: [test][0][0] = mapping of server to number of regions it hosts [test][region + 1][0] = server that region is hosted on [test][region + 1][server + 1] = size of region cached on server
private final float[]	expectedCacheCost
private org.apache.hadoop.hbase.master.balancer.CacheAwareLoadBalancer	loadBalancer
private static org.apache.hadoop.conf.Configuration	storedConfiguration

Fields inherited from class org.apache.hadoop.hbase.master.balancer.StochasticBalancerTestBase

dummyMetricsStochasticBalancer

Fields inherited from class org.apache.hadoop.hbase.master.balancer.BalancerTestBase

clusterStateMocks, clusterStateMocksWithNoSlop, clusterStateMocksWithSlop, conf, largeCluster, regionId

Constructor Summary

Constructors

Constructor	Description
TestCacheAwareLoadBalancerCostFunctions()

Method Summary

Modifier and Type	Method	Description
void	beforeEachTest()
private static Map<String,Deque<org.apache.hadoop.hbase.master.balancer.BalancerRegionLoad>>	buildRegionLoads(Collection<org.apache.hadoop.hbase.client.RegionInfo> regions, float cachedRatio, int regionSizeMb)
private static List<org.apache.hadoop.hbase.ServerName>	getServersInInsertionOrder(Map<org.apache.hadoop.hbase.ServerName,List<org.apache.hadoop.hbase.client.RegionInfo>> cluster)
private static org.apache.hadoop.hbase.master.balancer.CacheAwareLoadBalancer	newCacheAwareBalancer(org.apache.hadoop.conf.Configuration cfg)
static void	saveInitialConfiguration()
void	testCacheCost()
void	testNeedsBalanceFalseWhenWarmRegionsDespiteFreeBlockCacheSpace()	Checks that needsBalance isn't true when regions report high cache ratios
void	testNeedsBalanceWhenLowCacheRatioRegionsAndFreeBlockCacheSpace()	When block-cache persistence, cold regions (below CacheAwareLoadBalancer.LOW_CACHE_RATIO_FOR_RELOCATION_KEY) together with RS-reported block-cache free bytes inflate plausible best placement so weighted cache cost crosses minCostNeedBalance; StochasticLoadBalancer.needsBalance(org.apache.hadoop.hbase.TableName, org.apache.hadoop.hbase.master.balancer.BalancerClusterState) returns true even with evenly spread region-count skew.
void	testVerifyCacheAwareSkewnessCostFunctionDisabled()
void	testVerifyCacheAwareSkewnessCostFunctionEnabled()
void	testVerifyCacheCostFunctionDisabledByNoBucketCachePersistence()
void	testVerifyCacheCostFunctionDisabledByNoMultiplier()
void	testVerifyCacheCostFunctionEnabled()

Methods inherited from class org.apache.hadoop.hbase.master.balancer.StochasticBalancerTestBase

beforeAllTests, increaseMaxRunTimeOrFail, setMaxRunTime, testWithClusterWithIteration, testWithClusterWithIteration

Methods inherited from class org.apache.hadoop.hbase.master.balancer.BalancerTestBase

assertClusterAsBalanced, assertClusterOverallAsBalanced, assertRegionReplicaPlacement, convertToList, createRegions, createServerMap, mockCluster, mockClusterServers, mockClusterServers, mockClusterServers, mockClusterServersUnsorted, mockClusterServersWithTables, mockUniformClusterServers, printMock, printStats, randomRegions, randomRegions, randomServer, randomServers, reconcile, returnRegions, returnServer, returnServers, uniformRegions, updateLoad

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

expectedCacheCost

private final float[] expectedCacheCost

clusterRegionCacheRatioMocks

private final int[][][] clusterRegionCacheRatioMocks

Data set to testCacheCost: [test][0][0] = mapping of server to number of regions it hosts [test][region + 1][0] = server that region is hosted on [test][region + 1][server + 1] = size of region cached on server

storedConfiguration

private static org.apache.hadoop.conf.Configuration storedConfiguration

loadBalancer

private org.apache.hadoop.hbase.master.balancer.CacheAwareLoadBalancer loadBalancer

Constructor Details

TestCacheAwareLoadBalancerCostFunctions

public TestCacheAwareLoadBalancerCostFunctions()

Method Details

saveInitialConfiguration

@BeforeAll
public static void saveInitialConfiguration()

beforeEachTest

@BeforeEach
public void beforeEachTest()

testVerifyCacheAwareSkewnessCostFunctionEnabled

@Test
public void testVerifyCacheAwareSkewnessCostFunctionEnabled()

testVerifyCacheAwareSkewnessCostFunctionDisabled

@Test
public void testVerifyCacheAwareSkewnessCostFunctionDisabled()

testVerifyCacheCostFunctionEnabled

@Test
public void testVerifyCacheCostFunctionEnabled()

testVerifyCacheCostFunctionDisabledByNoBucketCachePersistence

@Test
public void testVerifyCacheCostFunctionDisabledByNoBucketCachePersistence()

testVerifyCacheCostFunctionDisabledByNoMultiplier

@Test
public void testVerifyCacheCostFunctionDisabledByNoMultiplier()

testCacheCost

@Test
public void testCacheCost()

testNeedsBalanceWhenLowCacheRatioRegionsAndFreeBlockCacheSpace

@Test
public void testNeedsBalanceWhenLowCacheRatioRegionsAndFreeBlockCacheSpace()

When block-cache persistence, cold regions (below CacheAwareLoadBalancer.LOW_CACHE_RATIO_FOR_RELOCATION_KEY) together with RS-reported block-cache free bytes inflate plausible best placement so weighted cache cost crosses minCostNeedBalance; StochasticLoadBalancer.needsBalance(org.apache.hadoop.hbase.TableName, org.apache.hadoop.hbase.master.balancer.BalancerClusterState) returns true even with evenly spread region-count skew.

testNeedsBalanceFalseWhenWarmRegionsDespiteFreeBlockCacheSpace

@Test
public void testNeedsBalanceFalseWhenWarmRegionsDespiteFreeBlockCacheSpace()

Checks that needsBalance isn't true when regions report high cache ratios

newCacheAwareBalancer

private static org.apache.hadoop.hbase.master.balancer.CacheAwareLoadBalancer newCacheAwareBalancer(org.apache.hadoop.conf.Configuration cfg)

buildRegionLoads

private static Map<String,Deque<org.apache.hadoop.hbase.master.balancer.BalancerRegionLoad>> buildRegionLoads(Collection<org.apache.hadoop.hbase.client.RegionInfo> regions,
 float cachedRatio,
 int regionSizeMb)

getServersInInsertionOrder

private static List<org.apache.hadoop.hbase.ServerName> getServersInInsertionOrder(Map<org.apache.hadoop.hbase.ServerName,List<org.apache.hadoop.hbase.client.RegionInfo>> cluster)