HFileBlock (Apache HBase 1.1.7 API)

java.lang.Object
- org.apache.hadoop.hbase.io.hfile.HFileBlock

All Implemented Interfaces:

HeapSize, Cacheable
```
@InterfaceAudience.Private
public class HFileBlock
extends Object
implements Cacheable
```
Reading HFile version 1 and 2 blocks, and writing version 2 blocks.
- In version 1 all blocks are always compressed or uncompressed, as specified by the HFile's compression algorithm, with a type-specific magic record stored in the beginning of the compressed data (i.e. one needs to uncompress the compressed block to determine the block type). There is only a single compression algorithm setting for all blocks. Offset and size information from the block index are required to read a block.
- In version 2 a block is structured as follows:
  - header (see Writer#finishBlock())
    - Magic record identifying the block type (8 bytes)
    - Compressed block size, excluding header, including checksum (4 bytes)
    - Uncompressed block size, excluding header, excluding checksum (4 bytes)
    - The offset of the previous block of the same type (8 bytes). This is used to be able to navigate to the previous block without going to the block
    - For minorVersions >=1, the ordinal describing checksum type (1 byte)
    - For minorVersions >=1, the number of data bytes/checksum chunk (4 bytes)
    - For minorVersions >=1, the size of data on disk, including header, excluding checksums (4 bytes)
  - Raw/Compressed/Encrypted/Encoded data. The compression algorithm is the same for all the blocks in the HFile, similarly to what was done in version 1.
  - For minorVersions >=1, a series of 4 byte checksums, one each for the number of bytes specified by bytesPerChecksum.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static interface`	`HFileBlock.BlockIterator` An interface allowing to iterate `HFileBlock`s.
`static interface`	`HFileBlock.BlockWritable` Something that can be written into a block.
`static interface`	`HFileBlock.FSReader` A full-fledged reader with iteration ability.
`static class`	`HFileBlock.Writer` Unified version 2 `HFile` block writer.

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`BYTE_BUFFER_HEAP_SIZE`
`static boolean`	`DONT_FILL_HEADER`
`static int`	`ENCODED_HEADER_SIZE` The size of block header when blockType is `BlockType.ENCODED_DATA`.
`static int`	`EXTRA_SERIALIZATION_SPACE`
`static boolean`	`FILL_HEADER`

Method Summary

Methods
Modifier and Type	Method and Description
`boolean`	`equals(Object comparison)`
`void`	`expectType(BlockType expectedType)`
`BlockType`	`getBlockType()`
`ByteBuffer`	`getBufferReadOnly()` Returns the buffer this block stores internally.
`ByteBuffer`	`getBufferReadOnlyWithHeader()` Returns the buffer of this block, including header data.
`ByteBuffer`	`getBufferWithoutHeader()` Returns a buffer that does not include the header or checksum.
`DataInputStream`	`getByteStream()`
`DataBlockEncoding`	`getDataBlockEncoding()`
`short`	`getDataBlockEncodingId()`
`CacheableDeserializer<Cacheable>`	`getDeserializer()` Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.
`byte[]`	`getDummyHeaderForVersion()` Return the appropriate DUMMY_HEADER for the minor version
`HFileContext`	`getHFileContext()`
`int`	`getNextBlockOnDiskSizeWithHeader()`
`long`	`getOffset()`
`int`	`getOnDiskSizeWithHeader()`
`int`	`getOnDiskSizeWithoutHeader()`
`long`	`getPrevBlockOffset()`
`int`	`getSerializedLength()` Returns the length of the ByteBuffer required to serialized the object.
`int`	`getUncompressedSizeWithoutHeader()`
`int`	`headerSize()` Returns the size of this block header.
`static int`	`headerSize(boolean usesHBaseChecksum)` Maps a minor version to the size of the header.
`long`	`heapSize()`
`boolean`	`isUnpacked()` Return true when this block's buffer has been unpacked, false otherwise.
`static boolean`	`readWithExtra(InputStream in, byte[] buf, int bufOffset, int necessaryLen, int extraLen)` Read from an input stream.
`void`	`serialize(ByteBuffer destination)` Serializes its data into destination.
`void`	`serializeExtraInfo(ByteBuffer destination)`
`String`	`toString()`
`static String`	`toStringHeader(ByteBuffer buf)` Convert the contents of the block header into a human readable string.
`static void`	`verifyUncompressed(ByteBuffer buf, boolean useHBaseChecksum)` An additional sanity-check in case no compression or encryption is being used.

Methods inherited from class java.lang.Object
clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - FILL_HEADER
```
public static final boolean FILL_HEADER
```
    See Also:
    Constant Field Values
  - DONT_FILL_HEADER
```
public static final boolean DONT_FILL_HEADER
```
    See Also:
    Constant Field Values
  - ENCODED_HEADER_SIZE
```
public static final int ENCODED_HEADER_SIZE
```
    The size of block header when blockType is BlockType.ENCODED_DATA. This extends normal header by adding the id of encoder.
    
    See Also:
    Constant Field Values
  - BYTE_BUFFER_HEAP_SIZE
```
public static final int BYTE_BUFFER_HEAP_SIZE
```
  - EXTRA_SERIALIZATION_SPACE
```
public static final int EXTRA_SERIALIZATION_SPACE
```
    See Also:
    Constant Field Values
- Method Detail
  - getBlockType
```
public BlockType getBlockType()
```
    Specified by:
    
    getBlockType in interface Cacheable
    
    Returns:
    the block type of this cached HFile block
  - getDataBlockEncodingId
```
public short getDataBlockEncodingId()
```
    Returns:
    get data block encoding id that was used to encode this block
  - getOnDiskSizeWithHeader
```
public int getOnDiskSizeWithHeader()
```
    Returns:
    the on-disk size of header + data part + checksum.
  - getOnDiskSizeWithoutHeader
```
public int getOnDiskSizeWithoutHeader()
```
    Returns:
    the on-disk size of the data part + checksum (header excluded).
  - getUncompressedSizeWithoutHeader
```
public int getUncompressedSizeWithoutHeader()
```
    Returns:
    the uncompressed size of data part (header and checksum excluded).
  - getPrevBlockOffset
```
public long getPrevBlockOffset()
```
    Returns:
    the offset of the previous block of the same type in the file, or -1 if unknown
  - getBufferWithoutHeader
```
public ByteBuffer getBufferWithoutHeader()
```
    Returns a buffer that does not include the header or checksum.
    
    Returns:
    the buffer with header skipped and checksum omitted.
  - getBufferReadOnly
```
public ByteBuffer getBufferReadOnly()
```
    Returns the buffer this block stores internally. The clients must not modify the buffer object. This method has to be public because it is used in CompoundBloomFilter to avoid object creation on every Bloom filter lookup, but has to be used with caution. Checksum data is not included in the returned buffer but header data is.
    
    Returns:
    the buffer of this block for read-only operations
  - getBufferReadOnlyWithHeader
```
public ByteBuffer getBufferReadOnlyWithHeader()
```
    Returns the buffer of this block, including header data. The clients must not modify the buffer object. This method has to be public because it is used in BucketCache to avoid buffer copy.
    
    Returns:
    the buffer with header and checksum included for read-only operations
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - isUnpacked
```
public boolean isUnpacked()
```
    Return true when this block's buffer has been unpacked, false otherwise. Note this is a calculated heuristic, not tracked attribute of the block.
  - verifyUncompressed
```
public static void verifyUncompressed(ByteBuffer buf,
                      boolean useHBaseChecksum)
                               throws IOException
```
    An additional sanity-check in case no compression or encryption is being used.
    
    Throws:
    
    IOException
  - expectType
```
public void expectType(BlockType expectedType)
                throws IOException
```
    Parameters:
    expectedType - the expected type of this block
    
    Throws:
    
    IOException - if this block's type is different than expected
  - getOffset
```
public long getOffset()
```
    Returns:
    the offset of this block in the file it was read from
  - getByteStream
```
public DataInputStream getByteStream()
```
    Returns:
    a byte stream reading the data + checksum of this block
  - heapSize
```
public long heapSize()
```
    Specified by:
    
    heapSize in interface HeapSize
    
    Returns:
    Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.
  - readWithExtra
```
public static boolean readWithExtra(InputStream in,
                    byte[] buf,
                    int bufOffset,
                    int necessaryLen,
                    int extraLen)
                             throws IOException
```
    Read from an input stream. Analogous to IOUtils.readFully(InputStream, byte[], int, int), but specifies a number of "extra" bytes that would be desirable but not absolutely necessary to read.
    
    Parameters:
    in - the input stream to read from
    buf - the buffer to read into
    bufOffset - the destination offset in the buffer
    necessaryLen - the number of bytes that are absolutely necessary to read
    extraLen - the number of extra bytes that would be nice to read
    
    Returns:
    true if succeeded reading the extra bytes
    
    Throws:
    
    IOException - if failed to read the necessary bytes
  - getNextBlockOnDiskSizeWithHeader
```
public int getNextBlockOnDiskSizeWithHeader()
```
    Returns:
    the on-disk size of the next block (including the header size) that was read by peeking into the next block's header
  - getSerializedLength
```
public int getSerializedLength()
```
    Description copied from interface: Cacheable
    
    Returns the length of the ByteBuffer required to serialized the object. If the object cannot be serialized, it should return 0.
    
    Specified by:
    
    getSerializedLength in interface Cacheable
    
    Returns:
    int length in bytes of the serialized form or 0 if the object cannot be cached.
  - serialize
```
public void serialize(ByteBuffer destination)
```
    Description copied from interface: Cacheable
    
    Serializes its data into destination.
    
    Specified by:
    
    serialize in interface Cacheable
    
    Parameters:
    destination - Where to serialize to
  - serializeExtraInfo
```
public void serializeExtraInfo(ByteBuffer destination)
```
  - getDeserializer
```
public CacheableDeserializer<Cacheable> getDeserializer()
```
    Description copied from interface: Cacheable
    
    Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.
    
    Specified by:
    
    getDeserializer in interface Cacheable
    
    Returns:
    CacheableDeserialzer instance.
  - equals
```
public boolean equals(Object comparison)
```
    Overrides:
    
    equals in class Object
  - getDataBlockEncoding
```
public DataBlockEncoding getDataBlockEncoding()
```
  - headerSize
```
public int headerSize()
```
    Returns the size of this block header.
  - headerSize
```
public static int headerSize(boolean usesHBaseChecksum)
```
    Maps a minor version to the size of the header.
  - getDummyHeaderForVersion
```
public byte[] getDummyHeaderForVersion()
```
    Return the appropriate DUMMY_HEADER for the minor version
  - getHFileContext
```
public HFileContext getHFileContext()
```
    Returns:
    the HFileContext used to create this HFileBlock. Not necessary the fileContext for the file from which this block's data was originally read.
  - toStringHeader
```
public static String toStringHeader(ByteBuffer buf)
                             throws IOException
```
    Convert the contents of the block header into a human readable string. This is mostly helpful for debugging. This assumes that the block has minor version > 0.
    
    Throws:
    
    IOException

Class HFileBlock

Nested Class Summary

Field Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

FILL_HEADER

DONT_FILL_HEADER

ENCODED_HEADER_SIZE

BYTE_BUFFER_HEAP_SIZE

EXTRA_SERIALIZATION_SPACE

Method Detail

getBlockType

getDataBlockEncodingId

getOnDiskSizeWithHeader

getOnDiskSizeWithoutHeader

getUncompressedSizeWithoutHeader

getPrevBlockOffset

getBufferWithoutHeader

getBufferReadOnly

getBufferReadOnlyWithHeader

toString

isUnpacked

verifyUncompressed

expectType

getOffset

getByteStream

heapSize

readWithExtra

getNextBlockOnDiskSizeWithHeader

getSerializedLength

serialize

serializeExtraInfo

getDeserializer

equals

getDataBlockEncoding

headerSize

headerSize

getDummyHeaderForVersion

getHFileContext

toStringHeader