Package org.apache.hadoop.hbase.io.hfile
Class MemcachedBlockCache
java.lang.Object
org.apache.hadoop.hbase.io.hfile.MemcachedBlockCache
- All Implemented Interfaces:
Iterable<CachedBlock>
,BlockCache
Class to store blocks into memcached. This should only be used on a cluster of Memcached daemons
that are tuned well and have a good network connection to the HBase regionservers. Any other use
will likely slow down HBase greatly.
-
Nested Class Summary
Modifier and TypeClassDescriptionprivate static class
Class to encode and decode an HFileBlock to and from memecached's resulting byte arrays.private static class
-
Field Summary
Modifier and TypeFieldDescriptionprivate final AtomicLong
private final AtomicLong
private final CacheStats
private final net.spy.memcached.MemcachedClient
private static final org.slf4j.Logger
static final int
static final int
static final String
static final long
static final String
static final boolean
static final String
static final String
private final AtomicLong
private final ScheduledExecutorService
Statistics thread schedule pool (for heavy debugging, could remove)static final int
private final MemcachedBlockCache.HFileBlockTranscoder
private final AtomicLong
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
cacheBlock
(BlockCacheKey cacheKey, Cacheable buf) Add block to cache (defaults to not in-memory).void
cacheBlock
(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory) Add block to cache.boolean
evictBlock
(BlockCacheKey cacheKey) Evict block from cache.int
evictBlocksByHfileName
(String hfileName) This method does nothing so that memcached can handle all evictions.getBlock
(BlockCacheKey cacheKey, boolean caching, boolean repeat, boolean updateCacheMetrics) Fetch block from cache.Returns The list of sub blockcaches that make up this one; returns null if no sub caches.long
Returns the number of blocks currently cached in the block cache.long
Returns the occupied size of data blocks, in bytes.long
Returns the occupied size of the block cache, in bytes.long
Returns the number of data blocks currently cached in the block cache.long
Returns the free size of the block cache, in bytes.long
Returns the Max size of the block cache, in bytes.getStats()
Get the statistics for this block cache.iterator()
Returns Iterator over the blocks in the cache.void
logStats()
void
shutdown()
Shutdown the cache.long
size()
Returns the total size of the block cache, in bytes.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.hadoop.hbase.io.hfile.BlockCache
blockFitsIntoTheCache, cacheBlock, evictBlocksRangeByHfileName, getBlock, getBlockSize, getFullyCachedFiles, getRegionCachedInfo, isAlreadyCached, isCacheEnabled, isMetaBlock, notifyFileCachingCompleted, shouldCacheFile, waitForCacheInitialization
Methods inherited from interface java.lang.Iterable
forEach, spliterator
-
Field Details
-
LOG
-
MAX_SIZE
- See Also:
-
MAX_TIME
- See Also:
-
MEMCACHED_CONFIG_KEY
- See Also:
-
MEMCACHED_TIMEOUT_KEY
- See Also:
-
MEMCACHED_OPTIMEOUT_KEY
- See Also:
-
MEMCACHED_OPTIMIZE_KEY
- See Also:
-
MEMCACHED_DEFAULT_TIMEOUT
- See Also:
-
MEMCACHED_OPTIMIZE_DEFAULT
- See Also:
-
STAT_THREAD_PERIOD
- See Also:
-
client
-
tc
-
cacheStats
-
cachedCount
-
notCachedCount
-
cacheErrorCount
-
timeoutCount
-
scheduleThreadPool
Statistics thread schedule pool (for heavy debugging, could remove)
-
-
Constructor Details
-
MemcachedBlockCache
- Throws:
IOException
-
-
Method Details
-
cacheBlock
Description copied from interface:BlockCache
Add block to cache.- Specified by:
cacheBlock
in interfaceBlockCache
- Parameters:
cacheKey
- The block's cache key.buf
- The block contents wrapped in a ByteBuffer.inMemory
- Whether block should be treated as in-memory
-
cacheBlock
Description copied from interface:BlockCache
Add block to cache (defaults to not in-memory).- Specified by:
cacheBlock
in interfaceBlockCache
- Parameters:
cacheKey
- The block's cache key.buf
- The object to cache.
-
getBlock
public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean repeat, boolean updateCacheMetrics) Description copied from interface:BlockCache
Fetch block from cache.- Specified by:
getBlock
in interfaceBlockCache
- Parameters:
cacheKey
- Block to fetch.caching
- Whether this request has caching enabled (used for stats)repeat
- Whether this is a repeat lookup for the same block (used to avoid double counting cache misses when doing double-check locking)updateCacheMetrics
- Whether to update cache metrics or not- Returns:
- Block or null if block is not in 2 cache.
-
evictBlock
Description copied from interface:BlockCache
Evict block from cache.- Specified by:
evictBlock
in interfaceBlockCache
- Parameters:
cacheKey
- Block to evict- Returns:
- true if block existed and was evicted, false if not
-
evictBlocksByHfileName
This method does nothing so that memcached can handle all evictions.- Specified by:
evictBlocksByHfileName
in interfaceBlockCache
- Returns:
- the number of blocks evicted
-
getStats
Description copied from interface:BlockCache
Get the statistics for this block cache.- Specified by:
getStats
in interfaceBlockCache
-
shutdown
Description copied from interface:BlockCache
Shutdown the cache.- Specified by:
shutdown
in interfaceBlockCache
-
size
Description copied from interface:BlockCache
Returns the total size of the block cache, in bytes.- Specified by:
size
in interfaceBlockCache
- Returns:
- size of cache, in bytes
-
getMaxSize
Description copied from interface:BlockCache
Returns the Max size of the block cache, in bytes.- Specified by:
getMaxSize
in interfaceBlockCache
- Returns:
- size of cache, in bytes
-
getFreeSize
Description copied from interface:BlockCache
Returns the free size of the block cache, in bytes.- Specified by:
getFreeSize
in interfaceBlockCache
- Returns:
- free space in cache, in bytes
-
getCurrentSize
Description copied from interface:BlockCache
Returns the occupied size of the block cache, in bytes.- Specified by:
getCurrentSize
in interfaceBlockCache
- Returns:
- occupied space in cache, in bytes
-
getCurrentDataSize
Description copied from interface:BlockCache
Returns the occupied size of data blocks, in bytes.- Specified by:
getCurrentDataSize
in interfaceBlockCache
- Returns:
- occupied space in cache, in bytes
-
getBlockCount
Description copied from interface:BlockCache
Returns the number of blocks currently cached in the block cache.- Specified by:
getBlockCount
in interfaceBlockCache
- Returns:
- number of blocks in the cache
-
getDataBlockCount
Description copied from interface:BlockCache
Returns the number of data blocks currently cached in the block cache.- Specified by:
getDataBlockCount
in interfaceBlockCache
- Returns:
- number of blocks in the cache
-
iterator
Description copied from interface:BlockCache
Returns Iterator over the blocks in the cache.- Specified by:
iterator
in interfaceBlockCache
- Specified by:
iterator
in interfaceIterable<CachedBlock>
-
getBlockCaches
Description copied from interface:BlockCache
Returns The list of sub blockcaches that make up this one; returns null if no sub caches.- Specified by:
getBlockCaches
in interfaceBlockCache
-
logStats
-