HFileBlock (Apache HBase 1.4.11 API)

java.lang.Object
- org.apache.hadoop.hbase.io.hfile.HFileBlock

All Implemented Interfaces:

HeapSize, Cacheable
```
@InterfaceAudience.Private
public class HFileBlock
extends Object
implements Cacheable
```
Reads HFile version 2 blocks to HFiles and via Cacheable Interface to caches. Version 2 was introduced in hbase-0.92.0. No longer has support for version 1 blocks since hbase-1.3.0.
Version 1 was the original file block. Version 2 was introduced when we changed the hbase file format to support multi-level block indexes and compound bloom filters (HBASE-3857).
HFileBlock: Version 2
In version 2, a block is structured as follows:
- Header: See Writer#putHeader() for where header is written; header total size is HFILEBLOCK_HEADER_SIZE
  - 0. blockType: Magic record identifying the BlockType (8 bytes): e.g. DATABLK*
  - 1. onDiskSizeWithoutHeader: Compressed -- a.k.a 'on disk' -- block size, excluding header, but including tailing checksum bytes (4 bytes)
  - 2. uncompressedSizeWithoutHeader: Uncompressed block size, excluding header, and excluding checksum bytes (4 bytes)
  - 3. prevBlockOffset: The offset of the previous block of the same type (8 bytes). This is used to navigate to the previous block without having to go to the block index
  - 4: For minorVersions >=1, the ordinal describing checksum type (1 byte)
  - 5: For minorVersions >=1, the number of data bytes/checksum chunk (4 bytes)
  - 6: onDiskDataSizeWithHeader: For minorVersions >=1, the size of data 'on disk', including header, excluding checksums (4 bytes)
- Raw/Compressed/Encrypted/Encoded data: The compression algorithm is the same for all the blocks in an HFile. If compression is NONE, this is just raw, serialized Cells.
- Tail: For minorVersions >=1, a series of 4 byte checksums, one each for the number of bytes specified by bytesPerChecksum.
Caching
Caches cache whole blocks with trailing checksums if any. We then tag on some metadata, the content of BLOCK_METADATA_SPACE which will be flag on if we are doing 'hbase' checksums and then the offset into the file which is needed when we re-make a cache key when we return the block to the cache as 'done'. See Cacheable.serialize(ByteBuffer, boolean) and Cacheable.getDeserializer().
TODO: Should we cache the checksums? Down in Writer#getBlockForCaching(CacheConfig) where we make a block to cache-on-write, there is an attempt at turning off checksums. This is not the only place we get blocks to cache. We also will cache the raw return from an hdfs read. In this case, the checksums may be present. If the cache is backed by something that doesn't do ECC, say an SSD, we might want to preserve checksums. For now this is open question.
TODO: Over in BucketCache, we save a block allocation by doing a custom serialization. Be sure to change it if serialization changes in here. Could we add a method here that takes an IOEngine and that then serializes to it rather than expose our internals over in BucketCache? IOEngine is in the bucket subpackage. Pull it up? Then this class knows about bucketcache. Ugh.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class HFileBlock.Writer
Unified version 2 HFile block writer.

Nested Classes
Modifier and Type	Class and Description
`static class`	`HFileBlock.Writer` Unified version 2 `HFile` block writer.

Field Summary

Fields
Modifier and Type Field and Description

static int BYTE_BUFFER_HEAP_SIZE

static boolean DONT_FILL_HEADER

static boolean FILL_HEADER

Fields
Modifier and Type	Field and Description
`static int`	`BYTE_BUFFER_HEAP_SIZE`
`static boolean`	`DONT_FILL_HEADER`
`static boolean`	`FILL_HEADER`

Constructor Summary

Constructors
Constructor and Description
`HFileBlock(BlockType blockType, int onDiskSizeWithoutHeader, int uncompressedSizeWithoutHeader, long prevBlockOffset, ByteBuffer b, boolean fillHeader, long offset, int nextBlockOnDiskSize, int onDiskDataSizeWithHeader, HFileContext fileContext)` Creates a new `HFile` block from the given fields.

Method Summary

Methods
Modifier and Type	Method and Description
`boolean`	`equals(Object comparison)`
`BlockType`	`getBlockType()`
`ByteBuffer`	`getBufferReadOnly()` Returns a read-only duplicate of the buffer this block stores internally ready to be read.
`ByteBuffer`	`getBufferWithoutHeader()` Returns a buffer that does not include the header or checksum.
`DataInputStream`	`getByteStream()`
`DataBlockEncoding`	`getDataBlockEncoding()`
`short`	`getDataBlockEncodingId()`
`CacheableDeserializer<Cacheable>`	`getDeserializer()` Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.
`ByteBuffer`	`getMetaData()` For use by bucketcache.
`int`	`getNextBlockOnDiskSize()`
`int`	`getOnDiskSizeWithHeader()`
`int`	`getSerializedLength()` Returns the length of the ByteBuffer required to serialized the object.
`int`	`getUncompressedSizeWithoutHeader()`
`int`	`hashCode()`
`int`	`headerSize()` Returns the size of this block header.
`static int`	`headerSize(boolean usesHBaseChecksum)` Maps a minor version to the size of the header.
`long`	`heapSize()`
`boolean`	`isUnpacked()` Return true when this block's buffer has been unpacked, false otherwise.
`void`	`sanityCheckUncompressedSize()` An additional sanity-check in case no compression or encryption is being used.
`void`	`serialize(ByteBuffer destination, boolean includeNextBlockOnDiskSize)` Serializes its data into destination.
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

- Field Detail
  - FILL_HEADER
```
public static final boolean FILL_HEADER
```
    See Also:
    Constant Field Values
  - DONT_FILL_HEADER
```
public static final boolean DONT_FILL_HEADER
```
    See Also:
    Constant Field Values
  - BYTE_BUFFER_HEAP_SIZE
```
public static final int BYTE_BUFFER_HEAP_SIZE
```
- Constructor Detail
  - HFileBlock
```
public HFileBlock(BlockType blockType,
          int onDiskSizeWithoutHeader,
          int uncompressedSizeWithoutHeader,
          long prevBlockOffset,
          ByteBuffer b,
          boolean fillHeader,
          long offset,
          int nextBlockOnDiskSize,
          int onDiskDataSizeWithHeader,
          HFileContext fileContext)
```
    Creates a new HFile block from the given fields. This constructor is used only while writing blocks and caching, and is sitting in a byte buffer and we want to stuff the block into cache. See HFileBlock.Writer.getBlockForCaching(CacheConfig).
    TODO: The caller presumes no checksumming required of this block instance since going into cache; checksum already verified on underlying block data pulled in from filesystem. Is that correct? What if cache is SSD?
    
    Parameters:
    blockType - the type of this block, see BlockType
    onDiskSizeWithoutHeader - see onDiskSizeWithoutHeader
    uncompressedSizeWithoutHeader - see uncompressedSizeWithoutHeader
    prevBlockOffset - see prevBlockOffset
    b - block header (HConstants.HFILEBLOCK_HEADER_SIZE bytes)
    fillHeader - when true, write the first 4 header fields into passed buffer.
    offset - the file offset the block was read from
    onDiskDataSizeWithHeader - see onDiskDataSizeWithHeader
    fileContext - HFile meta data
- Method Detail
  - getNextBlockOnDiskSize
```
public int getNextBlockOnDiskSize()
```
    Returns:
    the on-disk size of the next block (including the header size and any checksums if present) read by peeking into the next block's header; use as a hint when doing a read of the next block when scanning or running over a file.
  - getBlockType
```
public BlockType getBlockType()
```
    Specified by:
    
    getBlockType in interface Cacheable
    
    Returns:
    the block type of this cached HFile block
  - getDataBlockEncodingId
```
public short getDataBlockEncodingId()
```
    Returns:
    get data block encoding id that was used to encode this block
  - getOnDiskSizeWithHeader
```
public int getOnDiskSizeWithHeader()
```
    Returns:
    the on-disk size of header + data part + checksum.
  - getUncompressedSizeWithoutHeader
```
public int getUncompressedSizeWithoutHeader()
```
    Returns:
    the uncompressed size of data part (header and checksum excluded).
  - getBufferWithoutHeader
```
public ByteBuffer getBufferWithoutHeader()
```
    Returns a buffer that does not include the header or checksum.
    
    Returns:
    the buffer with header skipped and checksum omitted.
  - getBufferReadOnly
```
public ByteBuffer getBufferReadOnly()
```
    Returns a read-only duplicate of the buffer this block stores internally ready to be read. Clients must not modify the buffer object though they may set position and limit on the returned buffer since we pass back a duplicate. This method has to be public because it is used used in CompoundBloomFilter to avoid object creation on every Bloom filter lookup, but has to be used with caution. Buffer holds header, block content, and any follow-on checksums if present.
    
    Returns:
    the buffer of this block for read-only operations
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - isUnpacked
```
public boolean isUnpacked()
```
    Return true when this block's buffer has been unpacked, false otherwise. Note this is a calculated heuristic, not tracked attribute of the block.
  - sanityCheckUncompressedSize
```
public void sanityCheckUncompressedSize()
                                 throws IOException
```
    An additional sanity-check in case no compression or encryption is being used.
    
    Throws:
    
    IOException
  - getByteStream
```
public DataInputStream getByteStream()
```
    Returns:
    a byte stream reading the data + checksum of this block
  - heapSize
```
public long heapSize()
```
    Specified by:
    
    heapSize in interface HeapSize
    
    Returns:
    Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.
  - getSerializedLength
```
public int getSerializedLength()
```
    Description copied from interface: Cacheable
    
    Returns the length of the ByteBuffer required to serialized the object. If the object cannot be serialized, it should return 0.
    
    Specified by:
    
    getSerializedLength in interface Cacheable
    
    Returns:
    int length in bytes of the serialized form or 0 if the object cannot be cached.
  - serialize
```
public void serialize(ByteBuffer destination,
             boolean includeNextBlockOnDiskSize)
```
    Description copied from interface: Cacheable
    
    Serializes its data into destination.
    
    Specified by:
    
    serialize in interface Cacheable
    
    Parameters:
    destination - Where to serialize to
    includeNextBlockOnDiskSize - Whether to include nextBlockMetadata in the Cache block.
  - getMetaData
```
public ByteBuffer getMetaData()
```
    For use by bucketcache. This exposes internals.
  - getDeserializer
```
public CacheableDeserializer<Cacheable> getDeserializer()
```
    Description copied from interface: Cacheable
    
    Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.
    
    Specified by:
    
    getDeserializer in interface Cacheable
    
    Returns:
    CacheableDeserialzer instance.
  - hashCode
```
public int hashCode()
```
    Overrides:
    
    hashCode in class Object
  - equals
```
public boolean equals(Object comparison)
```
    Overrides:
    
    equals in class Object
  - getDataBlockEncoding
```
public DataBlockEncoding getDataBlockEncoding()
```
  - headerSize
```
public int headerSize()
```
    Returns the size of this block header.
  - headerSize
```
public static int headerSize(boolean usesHBaseChecksum)
```
    Maps a minor version to the size of the header.

Class HFileBlock

HFileBlock: Version 2

Caching

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

FILL_HEADER

DONT_FILL_HEADER

BYTE_BUFFER_HEAP_SIZE

Constructor Detail

HFileBlock

Method Detail

getNextBlockOnDiskSize

getBlockType

getDataBlockEncodingId

getOnDiskSizeWithHeader

getUncompressedSizeWithoutHeader

getBufferWithoutHeader

getBufferReadOnly

toString

isUnpacked

sanityCheckUncompressedSize

getByteStream

heapSize

getSerializedLength

serialize

getMetaData

getDeserializer

hashCode

equals

getDataBlockEncoding

headerSize

headerSize