Package org.apache.hadoop.hbase.io.hfile

Class TestHFileBlockUnpack

java.lang.Object

org.apache.hadoop.hbase.io.hfile.TestHFileBlockUnpack

@Tag("org.apache.hadoop.hbase.testclassification.IOTests") @Tag("org.apache.hadoop.hbase.testclassification.MediumTests")
public class TestHFileBlockUnpack
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static final class	TestHFileBlockUnpack.HFileBlockWrapper

Field Summary

Fields

Modifier and Type	Field	Description
(package private) org.apache.hadoop.hbase.io.ByteBuffAllocator	allocator
private static float	CHANCE_TO_REPEAT
private org.apache.hadoop.fs.FileSystem	fs
private static final int	MIN_ALLOCATION_SIZE
private static final HBaseTestingUtil	TEST_UTIL

Constructor Summary

Constructors

Constructor	Description
TestHFileBlockUnpack()

Method Summary

Modifier and Type	Method	Description
private void	assertBuffersEqual(org.apache.hadoop.hbase.nio.ByteBuff bufferOne, org.apache.hadoop.hbase.nio.ByteBuff bufferTwo, int expectedSize)
private int	createTestBlock(org.apache.hadoop.fs.Path path)
void	itUnpacksIdenticallyEachTime(org.junit.jupiter.api.TestInfo testInfo)	It's important that if you read and unpack the same HFileBlock twice, it results in an identical buffer each time.
void	itUsesSharedMemoryIfUnpackedBlockExceedsMinAllocationSize(org.junit.jupiter.api.TestInfo testInfo)	If the block on disk size is less than ByteBuffAllocator's min allocation size, that block will be allocated to heap regardless of desire for off-heap.
private static int	maybeIncrement(Random random, int value)
private TestHFileBlockUnpack.HFileBlockWrapper	readBlock(org.apache.hadoop.fs.Path path, int totalSize)
void	setUp()
(package private) static int	writeTestKeyValues(org.apache.hadoop.hbase.io.hfile.HFileBlock.Writer hbw, int desiredSize)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

TEST_UTIL

private static final HBaseTestingUtil TEST_UTIL

CHANCE_TO_REPEAT

private static float CHANCE_TO_REPEAT

MIN_ALLOCATION_SIZE

private static final int MIN_ALLOCATION_SIZE

See Also:

• Constant Field Values

allocator

org.apache.hadoop.hbase.io.ByteBuffAllocator allocator

fs

private org.apache.hadoop.fs.FileSystem fs

Constructor Details

TestHFileBlockUnpack

public TestHFileBlockUnpack()

Method Details

setUp

@BeforeEach
public void setUp()
           throws Exception

Throws:

Exception

itUnpacksIdenticallyEachTime

@Test
public void itUnpacksIdenticallyEachTime(org.junit.jupiter.api.TestInfo testInfo)
                                  throws IOException

It's important that if you read and unpack the same HFileBlock twice, it results in an identical buffer each time. Otherwise we end up with validation failures in block cache, since contents may not match if the same block is cached twice. See https://issues.apache.org/jira/browse/HBASE-27053

Throws:

IOException

assertBuffersEqual

private void assertBuffersEqual(org.apache.hadoop.hbase.nio.ByteBuff bufferOne,
 org.apache.hadoop.hbase.nio.ByteBuff bufferTwo,
 int expectedSize)

itUsesSharedMemoryIfUnpackedBlockExceedsMinAllocationSize

@Test
public void itUsesSharedMemoryIfUnpackedBlockExceedsMinAllocationSize(org.junit.jupiter.api.TestInfo testInfo)
                                                               throws IOException

If the block on disk size is less than ByteBuffAllocator's min allocation size, that block will be allocated to heap regardless of desire for off-heap. After de-compressing the block, the new size may now exceed the min allocation size. This test ensures that those de-compressed blocks, which will be allocated off-heap, are properly marked as HFileBlock.isSharedMem() == true See https://issues.apache.org/jira/browse/HBASE-27170

Throws:

IOException

readBlock

private TestHFileBlockUnpack.HFileBlockWrapper readBlock(org.apache.hadoop.fs.Path path,
 int totalSize)
                                                  throws IOException

Throws:

IOException

createTestBlock

private int createTestBlock(org.apache.hadoop.fs.Path path)
                     throws IOException

Throws:

IOException

writeTestKeyValues

static int writeTestKeyValues(org.apache.hadoop.hbase.io.hfile.HFileBlock.Writer hbw,
 int desiredSize)
                       throws IOException

Throws:

IOException

maybeIncrement

private static int maybeIncrement(Random random,
 int value)