Package org.apache.hadoop.hbase.master.balancer

Class RegionHDFSBlockLocationFinder

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.master.balancer.RegionHDFSBlockLocationFinder

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

@Private
class RegionHDFSBlockLocationFinder
extends org.apache.hadoop.conf.Configured

This will find where data for a region is located in HDFS. It ranks ServerName's by the size of the store files they are holding for a given region.

Field Summary

Fields

Modifier and Type	Field	Description
private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution>	cache
private static final long	CACHE_TIME
private static final HDFSBlocksDistribution	EMPTY_BLOCK_DISTRIBUTION
private static final float	EPSILON
private final org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListeningExecutorService	executor
private long	lastFullRefresh
private org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader<RegionInfo,HDFSBlocksDistribution>	loader
private static final org.slf4j.Logger	LOG
private ClusterInfoProvider	provider
private ClusterMetrics	status

Constructor Summary

Constructors

Constructor	Description
RegionHDFSBlockLocationFinder()

Method Summary

Modifier and Type	Method	Description
private org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListenableFuture<HDFSBlocksDistribution>	asyncGetBlockDistribution(RegionInfo hri)
private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution>	createCache()	Create a cache for region to list of servers
(package private) HDFSBlocksDistribution	getBlockDistribution(RegionInfo hri)
(package private) org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution>	getCache()
private TableDescriptor	getDescriptor(TableName tableName)	return TableDescriptor for a given tableName
private float	getOldLocality(ServerName newServer, byte[] regionName, Map<ServerName,ServerMetrics> oldServers)
(package private) List<ServerName>	getTopBlockLocations(RegionInfo region)
private HDFSBlocksDistribution	internalGetTopBlockLocation(RegionInfo region)	Returns an ordered list of hosts that are hosting the blocks for this region.
(package private) List<ServerName>	mapHostNameToServerName(List<String> hosts)	Map hostname to ServerName, The output ServerName list will have the same order as input hosts.
(package private) void	refreshAndWait(Collection<RegionInfo> hris)
private void	refreshLocalityChangedRegions(ClusterMetrics oldStatus, ClusterMetrics newStatus)	If locality for a region has changed, that pretty certainly means our cache is out of date.
private boolean	scheduleFullRefresh()	Refresh all the region locations.
(package private) void	setClusterInfoProvider(ClusterInfoProvider provider)
(package private) void	setClusterMetrics(ClusterMetrics status)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

CACHE_TIME

private static final long CACHE_TIME

See Also:

• Constant Field Values

EPSILON

private static final float EPSILON

See Also:

• Constant Field Values

EMPTY_BLOCK_DISTRIBUTION

private static final HDFSBlocksDistribution EMPTY_BLOCK_DISTRIBUTION

status

private volatile ClusterMetrics status

provider

private volatile ClusterInfoProvider provider

executor

private final org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListeningExecutorService executor

lastFullRefresh

private long lastFullRefresh

loader

private org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader<RegionInfo,HDFSBlocksDistribution> loader

cache

private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution> cache

Constructor Details

RegionHDFSBlockLocationFinder

RegionHDFSBlockLocationFinder()

Method Details

createCache

private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution> createCache()

Create a cache for region to list of servers

Returns:

A new Cache.

setClusterInfoProvider

void setClusterInfoProvider(ClusterInfoProvider provider)

setClusterMetrics

void setClusterMetrics(ClusterMetrics status)

refreshLocalityChangedRegions

private void refreshLocalityChangedRegions(ClusterMetrics oldStatus,
 ClusterMetrics newStatus)

If locality for a region has changed, that pretty certainly means our cache is out of date. Compare oldStatus and newStatus, refreshing any regions which have moved or changed locality.

getOldLocality

private float getOldLocality(ServerName newServer,
 byte[] regionName,
 Map<ServerName,ServerMetrics> oldServers)

scheduleFullRefresh

private boolean scheduleFullRefresh()

Refresh all the region locations.

Returns:

true if user created regions got refreshed.

getTopBlockLocations

List<ServerName> getTopBlockLocations(RegionInfo region)

internalGetTopBlockLocation

private HDFSBlocksDistribution internalGetTopBlockLocation(RegionInfo region)

Returns an ordered list of hosts that are hosting the blocks for this region. The weight of each host is the sum of the block lengths of all files on that host, so the first host in the list is the server which holds the most bytes of the given region's HFiles.

Parameters:

region - region

Returns:

ordered list of hosts holding blocks of the specified region

getDescriptor

private TableDescriptor getDescriptor(TableName tableName)
                               throws IOException

return TableDescriptor for a given tableName

Parameters:

tableName - the table name

Throws:

IOException

mapHostNameToServerName

List<ServerName> mapHostNameToServerName(List<String> hosts)

Map hostname to ServerName, The output ServerName list will have the same order as input hosts.

Parameters:

hosts - the list of hosts

Returns:

ServerName list

getBlockDistribution

HDFSBlocksDistribution getBlockDistribution(RegionInfo hri)

asyncGetBlockDistribution

private org.apache.hbase.thirdparty.com.google.common.util.concurrent.ListenableFuture<HDFSBlocksDistribution> asyncGetBlockDistribution(RegionInfo hri)

refreshAndWait

void refreshAndWait(Collection<RegionInfo> hris)

getCache

org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<RegionInfo,HDFSBlocksDistribution> getCache()