HFileReaderV2 (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.io.hfile.AbstractHFileReader
- - org.apache.hadoop.hbase.io.hfile.HFileReaderV2

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, HFile.CachingBlockReader, HFile.Reader

Direct Known Subclasses:

HFileReaderV3
```
@InterfaceAudience.Private
public class HFileReaderV2
extends AbstractHFileReader
```
HFile reader for version 2.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`protected static class`	`HFileReaderV2.AbstractScannerV2`
`protected static class`	`HFileReaderV2.EncodedScannerV2` ScannerV2 that operates on encoded data blocks.
`protected static class`	`HFileReaderV2.ScannerV2` Implementation of `HFileScanner` interface.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader
AbstractHFileReader.BlockIndexNotLoadedException, AbstractHFileReader.NotSeekedException, AbstractHFileReader.Scanner

Field Summary

Fields
Modifier and Type	Field and Description
`protected boolean`	`decodeMemstoreTS`
`private HFileBlock.FSReader`	`fsBlockReader` Filesystem-level block reader.
`(package private) HFileContext`	`hfileContext`
`private boolean`	`includesMemstoreTS`
`static int`	`KEY_VALUE_LEN_SIZE` The size of a (key length, value length) tuple that prefixes each entry in a data block.
`private List<HFileBlock>`	`loadOnOpenBlocks` Blocks read from the load-on-open section, excluding data root index, meta index, and file info.
`private static org.apache.commons.logging.Log`	`LOG`
`(package private) static int`	`MAX_MINOR_VERSION` Maximum minor version supported by this HFile format
`(package private) static int`	`MIN_MINOR_VERSION` Minimum minor version supported by this HFile format
`static int`	`MINOR_VERSION_NO_CHECKSUM` In HFile V2 minor version that does not support checksums
`static int`	`MINOR_VERSION_WITH_CHECKSUM` Minor versions in HFile V2 starting with this number have hbase checksums
`(package private) static int`	`MINOR_VERSION_WITH_FAKED_KEY` Minor versions starting with this number have faked index key
`private IdLock`	`offsetLock` A "sparse lock" implementation allowing to lock on a particular block identified by offset.
`static int`	`PBUF_TRAILER_MINOR_VERSION` HFile minor version that introduced pbuf filetrailer

Fields inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader
avgKeyLen, avgValueLen, cacheConf, comparator, compressAlgo, conf, dataBlockEncoder, dataBlockIndexReader, fileInfo, fileSize, hfs, lastKey, metaBlockIndexReader, name, path, trailer

Constructor Summary

Constructors
Constructor and Description
`HFileReaderV2(org.apache.hadoop.fs.Path path, FixedFileTrailer trailer, FSDataInputStreamWrapper fsdis, long size, CacheConfig cacheConf, HFileSystem hfs, org.apache.hadoop.conf.Configuration conf)` Opens a HFile.

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`close()`
`void`	`close(boolean evictOnClose)` Close method with optional evictOnClose
`protected HFileContext`	`createHFileContext(FSDataInputStreamWrapper fsdis, long fileSize, HFileSystem hfs, org.apache.hadoop.fs.Path path, FixedFileTrailer trailer)`
`private DataInput`	`getBloomFilterMetadata(BlockType blockType)`
`private HFileBlock`	`getCachedBlock(BlockCacheKey cacheKey, boolean cacheBlock, boolean useLock, boolean isCompaction, boolean updateCacheMetrics, BlockType expectedBlockType, DataBlockEncoding expectedDataBlockEncoding)` Retrieve block from cache.
`DataInput`	`getDeleteBloomFilterMetadata()` Retrieves delete family Bloom filter metadata as appropriate for each `HFile` version.
`DataBlockEncoding`	`getEffectiveEncodingInCache(boolean isCompaction)`
`HFileContext`	`getFileContext()` Return the file context of the HFile this reader belongs to
`DataInput`	`getGeneralBloomFilterMetadata()` Returns a buffer with the Bloom filter metadata.
`byte[]`	`getLastKey()`
`int`	`getMajorVersion()`
`ByteBuffer`	`getMetaBlock(String metaBlockName, boolean cacheBlock)`
`private static String`	`getPathOffsetEndStr(org.apache.hadoop.fs.Path path, long offset, long end)`
`HFileScanner`	`getScanner(boolean cacheBlocks, boolean pread, boolean isCompaction)` Create a Scanner on this file.
`(package private) HFileBlock.FSReader`	`getUncachedBlockReader()` For testing
`boolean`	`hasMVCCInfo()`
`boolean`	`isFileInfoLoaded()`
`byte[]`	`midkey()`
`(package private) boolean`	`prefetchComplete()` Returns false if block prefetching was requested for this file and has not completed, true otherwise
`HFileBlock`	`readBlock(long dataBlockOffset, long onDiskBlockSize, boolean cacheBlock, boolean pread, boolean isCompaction, boolean updateCacheMetrics, BlockType expectedBlockType, DataBlockEncoding expectedDataBlockEncoding)` Read in a file block.
`protected boolean`	`shouldIncludeMemstoreTS()`
`void`	`unbufferStream()` To close the stream's socket.
`private void`	`validateBlockType(HFileBlock block, BlockType expectedBlockType)` Compares the actual type of a block retrieved from cache or disk with its expected type and throws an exception in case of a mismatch.
`private void`	`validateMinorVersion(org.apache.hadoop.fs.Path path, int minorVersion)` Validates that the minor version is within acceptable limits.

Methods inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader
getComparator, getCompressionAlgorithm, getConf, getDataBlockEncoding, getDataBlockIndexReader, getEntries, getFirstKey, getFirstRowKey, getLastRowKey, getName, getPath, getScanner, getTrailer, indexSize, isPrimaryReplicaReader, length, loadFileInfo, setConf, setPrimaryReplicaReader, toString, toStringFirstKey, toStringLastKey

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.apache.commons.logging.Log LOG
```
  - MINOR_VERSION_WITH_CHECKSUM
```
public static final int MINOR_VERSION_WITH_CHECKSUM
```
    Minor versions in HFile V2 starting with this number have hbase checksums
    
    See Also:
    Constant Field Values
  - MINOR_VERSION_NO_CHECKSUM
```
public static final int MINOR_VERSION_NO_CHECKSUM
```
    In HFile V2 minor version that does not support checksums
    
    See Also:
    Constant Field Values
  - PBUF_TRAILER_MINOR_VERSION
```
public static final int PBUF_TRAILER_MINOR_VERSION
```
    HFile minor version that introduced pbuf filetrailer
    
    See Also:
    Constant Field Values
  - KEY_VALUE_LEN_SIZE
```
public static final int KEY_VALUE_LEN_SIZE
```
    The size of a (key length, value length) tuple that prefixes each entry in a data block.
    
    See Also:
    Constant Field Values
  - includesMemstoreTS
```
private boolean includesMemstoreTS
```
  - decodeMemstoreTS
```
protected boolean decodeMemstoreTS
```
  - fsBlockReader
```
private HFileBlock.FSReader fsBlockReader
```
    Filesystem-level block reader.
  - offsetLock
```
private IdLock offsetLock
```
    A "sparse lock" implementation allowing to lock on a particular block identified by offset. The purpose of this is to avoid two clients loading the same block, and have all but one client wait to get the block from the cache.
  - loadOnOpenBlocks
```
private List<HFileBlock> loadOnOpenBlocks
```
    Blocks read from the load-on-open section, excluding data root index, meta index, and file info.
  - MIN_MINOR_VERSION
```
static final int MIN_MINOR_VERSION
```
    Minimum minor version supported by this HFile format
    
    See Also:
    Constant Field Values
  - MAX_MINOR_VERSION
```
static final int MAX_MINOR_VERSION
```
    Maximum minor version supported by this HFile format
    
    See Also:
    Constant Field Values
  - MINOR_VERSION_WITH_FAKED_KEY
```
static final int MINOR_VERSION_WITH_FAKED_KEY
```
    Minor versions starting with this number have faked index key
    
    See Also:
    Constant Field Values
  - hfileContext
```
HFileContext hfileContext
```
- Constructor Detail
  - HFileReaderV2
```
public HFileReaderV2(org.apache.hadoop.fs.Path path,
             FixedFileTrailer trailer,
             FSDataInputStreamWrapper fsdis,
             long size,
             CacheConfig cacheConf,
             HFileSystem hfs,
             org.apache.hadoop.conf.Configuration conf)
              throws IOException
```
    Opens a HFile. You must load the index before you can use it by calling AbstractHFileReader.loadFileInfo().
    
    Parameters:
    path - Path to HFile.
    trailer - File trailer.
    fsdis - input stream.
    size - Length of the stream.
    cacheConf - Cache configuration.
    hfs -
    conf -
    
    Throws:
    
    IOException
- Method Detail
  - shouldIncludeMemstoreTS
```
protected boolean shouldIncludeMemstoreTS()
```
  - createHFileContext
```
protected HFileContext createHFileContext(FSDataInputStreamWrapper fsdis,
                              long fileSize,
                              HFileSystem hfs,
                              org.apache.hadoop.fs.Path path,
                              FixedFileTrailer trailer)
                                   throws IOException
```
    Throws:
    
    IOException
  - getPathOffsetEndStr
```
private static String getPathOffsetEndStr(org.apache.hadoop.fs.Path path,
                         long offset,
                         long end)
```
  - getScanner
```
public HFileScanner getScanner(boolean cacheBlocks,
                      boolean pread,
                      boolean isCompaction)
```
    Create a Scanner on this file. No seeks or reads are done on creation. Call HFileScanner.seekTo(byte[]) to position an start the read. There is nothing to clean up in a Scanner. Letting go of your references to the scanner is sufficient.
    
    Parameters:
    cacheBlocks - True if we should cache blocks read in by this scanner.
    pread - Use positional read rather than seek+read if true (pread is better for random reads, seek+read is better scanning).
    isCompaction - is scanner being used for a compaction?
    
    Returns:
    Scanner on this file.
  - getCachedBlock
```
private HFileBlock getCachedBlock(BlockCacheKey cacheKey,
                        boolean cacheBlock,
                        boolean useLock,
                        boolean isCompaction,
                        boolean updateCacheMetrics,
                        BlockType expectedBlockType,
                        DataBlockEncoding expectedDataBlockEncoding)
                           throws IOException
```
    Retrieve block from cache. Validates the retrieved block's type vs expectedBlockType and its encoding vs. expectedDataBlockEncoding. Unpacks the block as necessary.
    
    Throws:
    
    IOException
  - getMetaBlock
```
public ByteBuffer getMetaBlock(String metaBlockName,
                      boolean cacheBlock)
                        throws IOException
```
    Parameters:
    metaBlockName -
    cacheBlock - Add block to cache, if found
    
    Returns:
    block wrapped in a ByteBuffer, with header skipped
    
    Throws:
    
    IOException
  - readBlock
```
public HFileBlock readBlock(long dataBlockOffset,
                   long onDiskBlockSize,
                   boolean cacheBlock,
                   boolean pread,
                   boolean isCompaction,
                   boolean updateCacheMetrics,
                   BlockType expectedBlockType,
                   DataBlockEncoding expectedDataBlockEncoding)
                     throws IOException
```
    Description copied from interface: HFile.CachingBlockReader
    
    Read in a file block.
    
    Parameters:
    dataBlockOffset - offset to read.
    onDiskBlockSize - size of the block
    isCompaction - is this block being read as part of a compaction
    expectedBlockType - the block type we are expecting to read with this read operation, or null to read whatever block type is available and avoid checking (that might reduce caching efficiency of encoded data blocks)
    expectedDataBlockEncoding - the data block encoding the caller is expecting data blocks to be in, or null to not perform this check and return the block irrespective of the encoding. This check only applies to data blocks and can be set to null when the caller is expecting to read a non-data block and has set expectedBlockType accordingly.
    
    Returns:
    Block wrapped in a ByteBuffer.
    
    Throws:
    
    IOException
  - hasMVCCInfo
```
public boolean hasMVCCInfo()
```
  - validateBlockType
```
private void validateBlockType(HFileBlock block,
                     BlockType expectedBlockType)
                        throws IOException
```
    Compares the actual type of a block retrieved from cache or disk with its expected type and throws an exception in case of a mismatch. Expected block type of BlockType.DATA is considered to match the actual block type [@link BlockType.ENCODED_DATA as well.
    
    Parameters:
    block - a block retrieved from cache or disk
    expectedBlockType - the expected block type, or null to skip the check
    
    Throws:
    
    IOException
  - getLastKey
```
public byte[] getLastKey()
```
    Returns:
    Last key in the file. May be null if file has no entries. Note that this is not the last row key, but rather the byte form of the last KeyValue.
  - midkey
```
public byte[] midkey()
              throws IOException
```
    Returns:
    Midkey for this file. We work with block boundaries only so returned midkey is an approximation only.
    
    Throws:
    
    IOException
  - close
```
public void close()
           throws IOException
```
    Throws:
    
    IOException
  - close
```
public void close(boolean evictOnClose)
           throws IOException
```
    Description copied from interface: HFile.Reader
    
    Close method with optional evictOnClose
    
    Throws:
    
    IOException
  - getEffectiveEncodingInCache
```
public DataBlockEncoding getEffectiveEncodingInCache(boolean isCompaction)
```
  - getUncachedBlockReader
```
HFileBlock.FSReader getUncachedBlockReader()
```
    For testing
    
    Specified by:
    
    getUncachedBlockReader in class AbstractHFileReader
  - getGeneralBloomFilterMetadata
```
public DataInput getGeneralBloomFilterMetadata()
                                        throws IOException
```
    Returns a buffer with the Bloom filter metadata. The caller takes ownership of the buffer.
    
    Throws:
    
    IOException
  - getDeleteBloomFilterMetadata
```
public DataInput getDeleteBloomFilterMetadata()
                                       throws IOException
```
    Description copied from interface: HFile.Reader
    
    Retrieves delete family Bloom filter metadata as appropriate for each HFile version. Knows nothing about how that metadata is structured.
    
    Throws:
    
    IOException
  - getBloomFilterMetadata
```
private DataInput getBloomFilterMetadata(BlockType blockType)
                                  throws IOException
```
    Throws:
    
    IOException
  - isFileInfoLoaded
```
public boolean isFileInfoLoaded()
```
    Specified by:
    
    isFileInfoLoaded in class AbstractHFileReader
  - validateMinorVersion
```
private void validateMinorVersion(org.apache.hadoop.fs.Path path,
                        int minorVersion)
```
    Validates that the minor version is within acceptable limits. Otherwise throws an Runtime exception
  - getMajorVersion
```
public int getMajorVersion()
```
    Specified by:
    
    getMajorVersion in class AbstractHFileReader
  - getFileContext
```
public HFileContext getFileContext()
```
    Description copied from interface: HFile.Reader
    
    Return the file context of the HFile this reader belongs to
  - prefetchComplete
```
boolean prefetchComplete()
```
    Returns false if block prefetching was requested for this file and has not completed, true otherwise
  - unbufferStream
```
public void unbufferStream()
```
    Description copied from interface: HFile.Reader
    
    To close the stream's socket. Note: This can be concurrently called from multiple threads and implementation should take care of thread safety.

Class HFileReaderV2

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

Field Summary

Fields inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

Methods inherited from class java.lang.Object

Field Detail

LOG

MINOR_VERSION_WITH_CHECKSUM

MINOR_VERSION_NO_CHECKSUM

PBUF_TRAILER_MINOR_VERSION

KEY_VALUE_LEN_SIZE

includesMemstoreTS

decodeMemstoreTS

fsBlockReader

offsetLock

loadOnOpenBlocks

MIN_MINOR_VERSION

MAX_MINOR_VERSION

MINOR_VERSION_WITH_FAKED_KEY

hfileContext

Constructor Detail

HFileReaderV2

Method Detail

shouldIncludeMemstoreTS

createHFileContext

getPathOffsetEndStr

getScanner

getCachedBlock

getMetaBlock

readBlock

hasMVCCInfo

validateBlockType

getLastKey

midkey

close

close

getEffectiveEncodingInCache

getUncachedBlockReader

getGeneralBloomFilterMetadata

getDeleteBloomFilterMetadata

getBloomFilterMetadata

isFileInfoLoaded

validateMinorVersion

getMajorVersion

getFileContext

prefetchComplete

unbufferStream