org.apache.hadoop.hbase.io.hfile

Class TestHFileBlockUnpack

java.lang.Object

org.apache.hadoop.hbase.io.hfile.TestHFileBlockUnpack

public class TestHFileBlockUnpack
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	TestHFileBlockUnpack.HFileBlockWrapper

Field Summary

Fields

Modifier and Type	Field and Description
(package private) org.apache.hadoop.hbase.io.ByteBuffAllocator	allocator
private static float	CHANCE_TO_REPEAT
static HBaseClassTestRule	CLASS_RULE
private org.apache.hadoop.fs.FileSystem	fs
private static int	MIN_ALLOCATION_SIZE
org.junit.rules.TestName	name
private static HBaseTestingUtility	TEST_UTIL

Constructor Summary

Constructors

Constructor and Description
TestHFileBlockUnpack()

Method Summary

Modifier and Type	Method and Description
private void	assertBuffersEqual(org.apache.hadoop.hbase.nio.ByteBuff bufferOne, org.apache.hadoop.hbase.nio.ByteBuff bufferTwo, int expectedSize)
private int	createTestBlock(org.apache.hadoop.fs.Path path)
void	itUnpacksIdenticallyEachTime() It's important that if you read and unpack the same HFileBlock twice, it results in an identical buffer each time.
void	itUsesSharedMemoryIfUnpackedBlockExceedsMinAllocationSize() If the block on disk size is less than ByteBuffAllocator's min allocation size, that block will be allocated to heap regardless of desire for off-heap.
private static int	maybeIncrement(Random random, int value)
private TestHFileBlockUnpack.HFileBlockWrapper	readBlock(org.apache.hadoop.fs.Path path, int totalSize)
void	setUp()
(package private) static int	writeTestKeyValues(org.apache.hadoop.hbase.io.hfile.HFileBlock.Writer hbw, int desiredSize)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

TEST_UTIL

private static final HBaseTestingUtility TEST_UTIL

CHANCE_TO_REPEAT

private static float CHANCE_TO_REPEAT

MIN_ALLOCATION_SIZE

private static final int MIN_ALLOCATION_SIZE

See Also:

Constant Field Values

allocator

org.apache.hadoop.hbase.io.ByteBuffAllocator allocator

name

public org.junit.rules.TestName name

fs

private org.apache.hadoop.fs.FileSystem fs

Constructor Detail

TestHFileBlockUnpack

public TestHFileBlockUnpack()

Method Detail

setUp

public void setUp()
           throws Exception

Throws:

Exception

itUnpacksIdenticallyEachTime

public void itUnpacksIdenticallyEachTime()
                                  throws IOException

It's important that if you read and unpack the same HFileBlock twice, it results in an identical buffer each time. Otherwise we end up with validation failures in block cache, since contents may not match if the same block is cached twice. See https://issues.apache.org/jira/browse/HBASE-27053

Throws:

IOException

assertBuffersEqual

private void assertBuffersEqual(org.apache.hadoop.hbase.nio.ByteBuff bufferOne,
                                org.apache.hadoop.hbase.nio.ByteBuff bufferTwo,
                                int expectedSize)

itUsesSharedMemoryIfUnpackedBlockExceedsMinAllocationSize

public void itUsesSharedMemoryIfUnpackedBlockExceedsMinAllocationSize()
                                                               throws IOException

If the block on disk size is less than ByteBuffAllocator's min allocation size, that block will be allocated to heap regardless of desire for off-heap. After de-compressing the block, the new size may now exceed the min allocation size. This test ensures that those de-compressed blocks, which will be allocated off-heap, are properly marked as HFileBlock.isSharedMem() == true See https://issues.apache.org/jira/browse/HBASE-27170

Throws:

IOException

readBlock

private TestHFileBlockUnpack.HFileBlockWrapper readBlock(org.apache.hadoop.fs.Path path,
                                                         int totalSize)
                                                  throws IOException

Throws:

IOException

createTestBlock

private int createTestBlock(org.apache.hadoop.fs.Path path)
                     throws IOException

Throws:

IOException

writeTestKeyValues

static int writeTestKeyValues(org.apache.hadoop.hbase.io.hfile.HFileBlock.Writer hbw,
                              int desiredSize)
                       throws IOException

Throws:

IOException

maybeIncrement

private static int maybeIncrement(Random random,
                                  int value)