org.apache.hadoop.hbase.master.balancer

Class RegionLocationFinder

java.lang.Object

org.apache.hadoop.hbase.master.balancer.RegionLocationFinder

@InterfaceAudience.Private
class RegionLocationFinder
extends Object

This will find where data for a region is located in HDFS. It ranks ServerName's by the size of the store files they are holding for a given region.

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution>	cache
private static long	CACHE_TIME
private org.apache.hadoop.conf.Configuration	conf
private static HDFSBlocksDistribution	EMPTY_BLOCK_DISTRIBUTION
private static float	EPSILON
private org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListeningExecutorService	executor
private long	lastFullRefresh
private org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader<RegionInfo,HDFSBlocksDistribution>	loader
private static org.slf4j.Logger	LOG
private MasterServices	services
private ClusterMetrics	status

Constructor Summary

Constructors

Constructor and Description
RegionLocationFinder()

Method Summary

Modifier and Type	Method and Description
private org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListenableFuture<HDFSBlocksDistribution>	asyncGetBlockDistribution(RegionInfo hri)
private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution>	createCache() Create a cache for region to list of servers
HDFSBlocksDistribution	getBlockDistribution(RegionInfo hri)
(package private) org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution>	getCache()
org.apache.hadoop.conf.Configuration	getConf()
private float	getOldLocality(ServerName newServer, byte[] regionName, Map<ServerName,ServerMetrics> oldServers)
protected TableDescriptor	getTableDescriptor(TableName tableName) return TableDescriptor for a given tableName
protected List<ServerName>	getTopBlockLocations(RegionInfo region)
protected List<ServerName>	getTopBlockLocations(RegionInfo region, String currentHost) Returns an ordered list of hosts which have better locality for this region than the current host.
protected HDFSBlocksDistribution	internalGetTopBlockLocation(RegionInfo region) Returns an ordered list of hosts that are hosting the blocks for this region.
(package private) List<ServerName>	mapHostNameToServerName(List<String> hosts) Map hostname to ServerName, The output ServerName list will have the same order as input hosts.
void	refreshAndWait(Collection<RegionInfo> hris)
private void	refreshLocalityChangedRegions(ClusterMetrics oldStatus, ClusterMetrics newStatus) If locality for a region has changed, that pretty certainly means our cache is out of date.
private boolean	scheduleFullRefresh() Refresh all the region locations.
void	setClusterMetrics(ClusterMetrics status)
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setServices(MasterServices services)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

CACHE_TIME

private static final long CACHE_TIME

See Also:

Constant Field Values

EPSILON

private static final float EPSILON

See Also:

Constant Field Values

EMPTY_BLOCK_DISTRIBUTION

private static final HDFSBlocksDistribution EMPTY_BLOCK_DISTRIBUTION

conf

private org.apache.hadoop.conf.Configuration conf

status

private volatile ClusterMetrics status

services

private MasterServices services

executor

private final org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListeningExecutorService executor

lastFullRefresh

private long lastFullRefresh

loader

private org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader<RegionInfo,HDFSBlocksDistribution> loader

cache

private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution> cache

Constructor Detail

RegionLocationFinder

RegionLocationFinder()

Method Detail

createCache

private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution> createCache()

Create a cache for region to list of servers

Returns:

A new Cache.

getConf

public org.apache.hadoop.conf.Configuration getConf()

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

setServices

public void setServices(MasterServices services)

setClusterMetrics

public void setClusterMetrics(ClusterMetrics status)

refreshLocalityChangedRegions

private void refreshLocalityChangedRegions(ClusterMetrics oldStatus,
                                           ClusterMetrics newStatus)

If locality for a region has changed, that pretty certainly means our cache is out of date. Compare oldStatus and newStatus, refreshing any regions which have moved or changed locality.

getOldLocality

private float getOldLocality(ServerName newServer,
                             byte[] regionName,
                             Map<ServerName,ServerMetrics> oldServers)

scheduleFullRefresh

private boolean scheduleFullRefresh()

Refresh all the region locations.

Returns:

true if user created regions got refreshed.

getTopBlockLocations

protected List<ServerName> getTopBlockLocations(RegionInfo region)

getTopBlockLocations

protected List<ServerName> getTopBlockLocations(RegionInfo region,
                                                String currentHost)

Returns an ordered list of hosts which have better locality for this region than the current host.

internalGetTopBlockLocation

protected HDFSBlocksDistribution internalGetTopBlockLocation(RegionInfo region)

Returns an ordered list of hosts that are hosting the blocks for this region. The weight of each host is the sum of the block lengths of all files on that host, so the first host in the list is the server which holds the most bytes of the given region's HFiles.

Parameters:

region - region

Returns:

ordered list of hosts holding blocks of the specified region

getTableDescriptor

protected TableDescriptor getTableDescriptor(TableName tableName)
                                      throws IOException

return TableDescriptor for a given tableName

Parameters:

tableName - the table name nn

Throws:

IOException

mapHostNameToServerName

List<ServerName> mapHostNameToServerName(List<String> hosts)

Map hostname to ServerName, The output ServerName list will have the same order as input hosts.

Parameters:

hosts - the list of hosts

Returns:

ServerName list

getBlockDistribution

public HDFSBlocksDistribution getBlockDistribution(RegionInfo hri)

asyncGetBlockDistribution

private org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListenableFuture<HDFSBlocksDistribution> asyncGetBlockDistribution(RegionInfo hri)

refreshAndWait

public void refreshAndWait(Collection<RegionInfo> hris)

getCache

org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution> getCache()