org.apache.hadoop.hbase.io.hfile

Class BlockCacheUtil

java.lang.Object

org.apache.hadoop.hbase.io.hfile.BlockCacheUtil

@InterfaceAudience.Private
public class BlockCacheUtil
extends Object

Utilty for aggregating counts in CachedBlocks and toString/toJSON CachedBlocks and BlockCaches. No attempt has been made at making this thread safe.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	BlockCacheUtil.CachedBlockCountsPerFile Little data structure to hold counts for a file.
static class	BlockCacheUtil.CachedBlocksByFile Use one of these to keep a running account of cached blocks by file.

Field Summary

Fields

Modifier and Type	Field and Description
private static int	DEFAULT_MAX
private static org.apache.hbase.thirdparty.com.google.gson.Gson	GSON Needed generating JSON.
private static org.slf4j.Logger	LOG
static long	NANOS_PER_SECOND

Constructor Summary

Constructors

Constructor and Description
BlockCacheUtil()

Method Summary

Modifier and Type	Method and Description
private static int	compareCacheBlock(Cacheable left, Cacheable right, boolean includeNextBlockMetadata)
static BlockCacheUtil.CachedBlocksByFile	getLoadedCachedBlocksByFile(org.apache.hadoop.conf.Configuration conf, BlockCache bc) Get a BlockCacheUtil.CachedBlocksByFile instance and load it up by iterating content in BlockCache.
static int	getMaxCachedBlocksByFile(org.apache.hadoop.conf.Configuration conf)
static boolean	shouldReplaceExistingCacheBlock(BlockCache blockCache, BlockCacheKey cacheKey, Cacheable newBlock) Because of the region splitting, it's possible that the split key locate in the middle of a block.
static String	toJSON(BlockCache bc) Returns JSON string of bc content.
static String	toJSON(BlockCacheUtil.CachedBlocksByFile cbsbf) Returns JSON string of cbsf aggregated
static String	toJSON(String filename, NavigableSet<CachedBlock> blocks) Returns A JSON String of filename and counts of blocks
static String	toString(CachedBlock cb, long now) n * @return The block content as String.
static String	toStringMinusFileName(CachedBlock cb, long now) n * @return The block content of bc as a String minus the filename.
static int	validateBlockAddition(Cacheable existing, Cacheable newBlock, BlockCacheKey cacheKey) Validate that the existing and newBlock are the same without including the nextBlockMetadata, if not, throw an exception.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

NANOS_PER_SECOND

public static final long NANOS_PER_SECOND

See Also:

Constant Field Values

GSON

private static final org.apache.hbase.thirdparty.com.google.gson.Gson GSON

Needed generating JSON.

DEFAULT_MAX

private static final int DEFAULT_MAX

See Also:

Constant Field Values

Constructor Detail

BlockCacheUtil

public BlockCacheUtil()

Method Detail

toString

public static String toString(CachedBlock cb,
                              long now)

n * @return The block content as String.

toJSON

public static String toJSON(String filename,
                            NavigableSet<CachedBlock> blocks)
                     throws IOException

Returns A JSON String of filename and counts of blocks

Throws:

IOException

toJSON

public static String toJSON(BlockCacheUtil.CachedBlocksByFile cbsbf)
                     throws IOException

Returns JSON string of cbsf aggregated

Throws:

IOException

toJSON

public static String toJSON(BlockCache bc)
                     throws IOException

Returns JSON string of bc content.

Throws:

IOException

toStringMinusFileName

public static String toStringMinusFileName(CachedBlock cb,
                                           long now)

n * @return The block content of bc as a String minus the filename.

getLoadedCachedBlocksByFile

public static BlockCacheUtil.CachedBlocksByFile getLoadedCachedBlocksByFile(org.apache.hadoop.conf.Configuration conf,
                                                                            BlockCache bc)

Get a BlockCacheUtil.CachedBlocksByFile instance and load it up by iterating content in BlockCache.

Parameters:

conf - Used to read configurations

bc - Block Cache to iterate.

Returns:

Laoded up instance of CachedBlocksByFile

compareCacheBlock

private static int compareCacheBlock(Cacheable left,
                                     Cacheable right,
                                     boolean includeNextBlockMetadata)

validateBlockAddition

public static int validateBlockAddition(Cacheable existing,
                                        Cacheable newBlock,
                                        BlockCacheKey cacheKey)

Validate that the existing and newBlock are the same without including the nextBlockMetadata, if not, throw an exception. If they are the same without the nextBlockMetadata, return the comparison.

Parameters:

existing - block that is existing in the cache.

newBlock - block that is trying to be cached.

cacheKey - the cache key of the blocks.

Returns:

comparison of the existing block to the newBlock.

shouldReplaceExistingCacheBlock

public static boolean shouldReplaceExistingCacheBlock(BlockCache blockCache,
                                                      BlockCacheKey cacheKey,
                                                      Cacheable newBlock)

Because of the region splitting, it's possible that the split key locate in the middle of a block. So it's possible that both the daughter regions load the same block from their parent HFile. When pread, we don't force the read to read all of the next block header. So when two threads try to cache the same block, it's possible that one thread read all of the next block header but the other one didn't. if the already cached block hasn't next block header but the new block to cache has, then we can replace the existing block with the new block for better performance.(HBASE-20447)

Parameters:

blockCache - BlockCache to check

cacheKey - the block cache key

newBlock - the new block which try to put into the block cache.

Returns:

true means need to replace existing block with new block for the same block cache key. false means just keep the existing block.

getMaxCachedBlocksByFile

public static int getMaxCachedBlocksByFile(org.apache.hadoop.conf.Configuration conf)