org.apache.hadoop.hbase.io.hfile.bucket

Class TestBucketCacheRefCnt

java.lang.Object

org.apache.hadoop.hbase.io.hfile.bucket.TestBucketCacheRefCnt

public class TestBucketCacheRefCnt
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	TestBucketCacheRefCnt.MyBucketCache
(package private) static class	TestBucketCacheRefCnt.MyBucketCache2

Field Summary

Fields

Modifier and Type	Field and Description
private static int	BLOCK_SIZE
private static int[]	BLOCK_SIZE_ARRAY
private org.apache.hadoop.hbase.io.hfile.bucket.BucketCache	cache
private static long	CAPACITY_SIZE
static HBaseClassTestRule	CLASS_RULE
private static org.apache.hadoop.hbase.io.hfile.HFileContext	CONTEXT
private static String	IO_ENGINE
private static String	PERSISTENCE_PATH

Constructor Summary

Constructors

Constructor and Description
TestBucketCacheRefCnt()

Method Summary

Modifier and Type	Method and Description
private static org.apache.hadoop.hbase.io.hfile.bucket.BucketCache	create(int writerSize, int queueSize)
private static org.apache.hadoop.hbase.io.hfile.HFileBlock	createBlock(int offset, int size)
private static org.apache.hadoop.hbase.io.hfile.HFileBlock	createBlock(int offset, int size, org.apache.hadoop.hbase.io.ByteBuffAllocator alloc)
private static org.apache.hadoop.hbase.io.hfile.BlockCacheKey	createKey(String hfileName, long offset)
private static TestBucketCacheRefCnt.MyBucketCache	createMyBucketCache(int writerSize, int queueSize)
private static TestBucketCacheRefCnt.MyBucketCache2	createMyBucketCache2(int writerSize, int queueSize)
private void	disableWriter()
private static org.apache.hadoop.hbase.nio.ByteBuff	getOverwriteByteBuff(org.apache.hadoop.hbase.io.hfile.bucket.BucketEntry bucketEntry)
void	testBlockInBackingMap()
void	testBlockInRAMCache()
void	testEvictingBlockCachingBlockGettingBlockConcurrently() This test also is for HBASE-26281, test three threads for evicting Block,caching Block and getting Block execute concurrently.
void	testInBucketCache()
void	testMarkStaleAsEvicted()
void	testReplacingBlockAndGettingBlockConcurrently() This test is for HBASE-26281, test two threads for replacing Block and getting Block execute concurrently.
private static void	waitUntilFlushedToCache(org.apache.hadoop.hbase.io.hfile.bucket.BucketCache bucketCache, org.apache.hadoop.hbase.io.hfile.BlockCacheKey blockCacheKey)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

IO_ENGINE

private static final String IO_ENGINE

See Also:

Constant Field Values

CAPACITY_SIZE

private static final long CAPACITY_SIZE

See Also:

Constant Field Values

BLOCK_SIZE

private static final int BLOCK_SIZE

See Also:

Constant Field Values

BLOCK_SIZE_ARRAY

private static final int[] BLOCK_SIZE_ARRAY

PERSISTENCE_PATH

private static final String PERSISTENCE_PATH

CONTEXT

private static final org.apache.hadoop.hbase.io.hfile.HFileContext CONTEXT

cache

private org.apache.hadoop.hbase.io.hfile.bucket.BucketCache cache

Constructor Detail

TestBucketCacheRefCnt

public TestBucketCacheRefCnt()

Method Detail

create

private static org.apache.hadoop.hbase.io.hfile.bucket.BucketCache create(int writerSize,
                                                                          int queueSize)
                                                                   throws IOException

Throws:

IOException

createMyBucketCache

private static TestBucketCacheRefCnt.MyBucketCache createMyBucketCache(int writerSize,
                                                                       int queueSize)
                                                                throws IOException

Throws:

IOException

createMyBucketCache2

private static TestBucketCacheRefCnt.MyBucketCache2 createMyBucketCache2(int writerSize,
                                                                         int queueSize)
                                                                  throws IOException

Throws:

IOException

createBlock

private static org.apache.hadoop.hbase.io.hfile.HFileBlock createBlock(int offset,
                                                                       int size)

createBlock

private static org.apache.hadoop.hbase.io.hfile.HFileBlock createBlock(int offset,
                                                                       int size,
                                                                       org.apache.hadoop.hbase.io.ByteBuffAllocator alloc)

createKey

private static org.apache.hadoop.hbase.io.hfile.BlockCacheKey createKey(String hfileName,
                                                                        long offset)

disableWriter

private void disableWriter()

testBlockInRAMCache

public void testBlockInRAMCache()
                         throws IOException

Throws:

IOException

waitUntilFlushedToCache

private static void waitUntilFlushedToCache(org.apache.hadoop.hbase.io.hfile.bucket.BucketCache bucketCache,
                                            org.apache.hadoop.hbase.io.hfile.BlockCacheKey blockCacheKey)
                                     throws InterruptedException

Throws:

InterruptedException

testBlockInBackingMap

public void testBlockInBackingMap()
                           throws Exception

Throws:

Exception

testInBucketCache

public void testInBucketCache()
                       throws IOException

Throws:

IOException

testMarkStaleAsEvicted

public void testMarkStaleAsEvicted()
                            throws Exception

Throws:

Exception

testReplacingBlockAndGettingBlockConcurrently

public void testReplacingBlockAndGettingBlockConcurrently()
                                                   throws Exception

 This test is for HBASE-26281,
 test two threads for replacing Block and getting Block execute concurrently.
 The threads sequence is:
 1. Block1 was cached successfully,the RefCnt of Block1 is 1.
 2. Thread1 caching the same BlockCacheKey with Block2 satisfied
    BlockCacheUtil.shouldReplaceExistingCacheBlock(org.apache.hadoop.hbase.io.hfile.BlockCache, org.apache.hadoop.hbase.io.hfile.BlockCacheKey, org.apache.hadoop.hbase.io.hfile.Cacheable), so Block2 would
    replace Block1, but thread1 stopping before BucketCache.cacheBlockWithWaitInternal(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, org.apache.hadoop.hbase.io.hfile.Cacheable, boolean, boolean)
 3. Thread2 invoking BucketCache.getBlock(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, boolean, boolean, boolean) with the same BlockCacheKey,
    which returned Block1, the RefCnt of Block1 is 2.
 4. Thread1 continues caching Block2, in BucketCache.putIntoBackingMap(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, org.apache.hadoop.hbase.io.hfile.bucket.BucketEntry),
    the old Block1 is freed directly which RefCnt is 2, but the Block1 is still used
    by Thread2 and the content of Block1 would be overwritten after it is freed, which may
    cause a serious error.
 

n

Throws:

Exception

testEvictingBlockCachingBlockGettingBlockConcurrently

public void testEvictingBlockCachingBlockGettingBlockConcurrently()
                                                           throws Exception

 This test also is for HBASE-26281,
 test three threads for evicting Block,caching Block and getting Block
 execute concurrently.
 1. Thread1 caching Block1, stopping after BucketCache.putIntoBackingMap(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, org.apache.hadoop.hbase.io.hfile.bucket.BucketEntry),
    the RefCnt of Block1 is 1.
 2. Thread2 invoking BucketCache.evictBlock(org.apache.hadoop.hbase.io.hfile.BlockCacheKey) with the same BlockCacheKey,
    but stopping after BucketCache.removeFromRamCache(org.apache.hadoop.hbase.io.hfile.BlockCacheKey).
 3. Thread3 invoking BucketCache.getBlock(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, boolean, boolean, boolean) with the same BlockCacheKey,
    which returned Block1, the RefCnt of Block1 is 2.
 4. Thread1 continues caching block1,but finding that BucketCache.RAMCache.remove(org.apache.hadoop.hbase.io.hfile.BlockCacheKey)
    returning false, so invoking BucketCache.blockEvicted(org.apache.hadoop.hbase.io.hfile.BlockCacheKey, org.apache.hadoop.hbase.io.hfile.bucket.BucketEntry, boolean, boolean) to free the the Block1
    directly which RefCnt is 2 and the Block1 is still used by Thread3.
 

Throws:

Exception

getOverwriteByteBuff

private static org.apache.hadoop.hbase.nio.ByteBuff getOverwriteByteBuff(org.apache.hadoop.hbase.io.hfile.bucket.BucketEntry bucketEntry)