org.apache.hadoop.hbase.io.hfile

Class HFileReaderV2

java.lang.Object

org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

org.apache.hadoop.hbase.io.hfile.HFileReaderV2

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, HFile.CachingBlockReader, HFile.Reader

Direct Known Subclasses:

HFileReaderV3

@InterfaceAudience.Private
public class HFileReaderV2
extends AbstractHFileReader

HFile reader for version 2.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	HFileReaderV2.AbstractScannerV2
protected static class	HFileReaderV2.EncodedScannerV2 ScannerV2 that operates on encoded data blocks.
protected static class	HFileReaderV2.ScannerV2 Implementation of HFileScanner interface.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

AbstractHFileReader.BlockIndexNotLoadedException, AbstractHFileReader.NotSeekedException, AbstractHFileReader.Scanner

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	decodeMemstoreTS
static int	KEY_VALUE_LEN_SIZE The size of a (key length, value length) tuple that prefixes each entry in a data block.
static int	MINOR_VERSION_NO_CHECKSUM In HFile V2 minor version that does not support checksums
static int	MINOR_VERSION_WITH_CHECKSUM Minor versions in HFile V2 starting with this number have hbase checksums
static int	PBUF_TRAILER_MINOR_VERSION HFile minor version that introduced pbuf filetrailer

Fields inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

avgKeyLen, avgValueLen, cacheConf, comparator, compressAlgo, conf, dataBlockEncoder, dataBlockIndexReader, fileInfo, fileSize, hfs, istream, istreamNoFsChecksum, lastKey, metaBlockIndexReader, name, path, trailer

Constructor Summary

Constructors

Constructor and Description
HFileReaderV2(org.apache.hadoop.fs.Path path, FixedFileTrailer trailer, FSDataInputStreamWrapper fsdis, long size, CacheConfig cacheConf, HFileSystem hfs, org.apache.hadoop.conf.Configuration conf) Opens a HFile.

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	close(boolean evictOnClose) Close method with optional evictOnClose
protected HFileContext	createHFileContext(FSDataInputStreamWrapper fsdis, long fileSize, HFileSystem hfs, org.apache.hadoop.fs.Path path, FixedFileTrailer trailer)
DataInput	getDeleteBloomFilterMetadata() Retrieves delete family Bloom filter metadata as appropriate for each HFile version.
DataBlockEncoding	getEffectiveEncodingInCache(boolean isCompaction)
HFileContext	getFileContext() Return the file context of the HFile this reader belongs to
DataInput	getGeneralBloomFilterMetadata() Returns a buffer with the Bloom filter metadata.
byte[]	getLastKey()
int	getMajorVersion()
ByteBuffer	getMetaBlock(String metaBlockName, boolean cacheBlock)
HFileScanner	getScanner(boolean cacheBlocks, boolean pread, boolean isCompaction) Create a Scanner on this file.
boolean	hasMVCCInfo()
boolean	isFileInfoLoaded()
byte[]	midkey()
HFileBlock	readBlock(long dataBlockOffset, long onDiskBlockSize, boolean cacheBlock, boolean pread, boolean isCompaction, boolean updateCacheMetrics, BlockType expectedBlockType, DataBlockEncoding expectedDataBlockEncoding) Read in a file block.
protected boolean	shouldIncludeMemstoreTS()

Methods inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

getComparator, getCompressionAlgorithm, getConf, getDataBlockEncoding, getDataBlockIndexReader, getEntries, getFirstKey, getFirstRowKey, getLastRowKey, getName, getPath, getScanner, getTrailer, indexSize, length, loadFileInfo, setConf, toString, toStringFirstKey, toStringLastKey

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MINOR_VERSION_WITH_CHECKSUM

public static final int MINOR_VERSION_WITH_CHECKSUM

Minor versions in HFile V2 starting with this number have hbase checksums

See Also:

Constant Field Values

MINOR_VERSION_NO_CHECKSUM

public static final int MINOR_VERSION_NO_CHECKSUM

In HFile V2 minor version that does not support checksums

See Also:

Constant Field Values

PBUF_TRAILER_MINOR_VERSION

public static final int PBUF_TRAILER_MINOR_VERSION

HFile minor version that introduced pbuf filetrailer

See Also:

Constant Field Values

KEY_VALUE_LEN_SIZE

public static final int KEY_VALUE_LEN_SIZE

The size of a (key length, value length) tuple that prefixes each entry in a data block.

See Also:

Constant Field Values

decodeMemstoreTS

protected boolean decodeMemstoreTS

Constructor Detail

HFileReaderV2

public HFileReaderV2(org.apache.hadoop.fs.Path path,
             FixedFileTrailer trailer,
             FSDataInputStreamWrapper fsdis,
             long size,
             CacheConfig cacheConf,
             HFileSystem hfs,
             org.apache.hadoop.conf.Configuration conf)
              throws IOException

Opens a HFile. You must load the index before you can use it by calling AbstractHFileReader.loadFileInfo().

Parameters:

path - Path to HFile.

trailer - File trailer.

fsdis - input stream.

size - Length of the stream.

cacheConf - Cache configuration.

hfs -

conf -

Throws:

IOException

Method Detail

shouldIncludeMemstoreTS

protected boolean shouldIncludeMemstoreTS()

createHFileContext

protected HFileContext createHFileContext(FSDataInputStreamWrapper fsdis,
                              long fileSize,
                              HFileSystem hfs,
                              org.apache.hadoop.fs.Path path,
                              FixedFileTrailer trailer)
                                   throws IOException

Throws:

IOException

getScanner

public HFileScanner getScanner(boolean cacheBlocks,
                      boolean pread,
                      boolean isCompaction)

Create a Scanner on this file. No seeks or reads are done on creation. Call HFileScanner.seekTo(byte[]) to position an start the read. There is nothing to clean up in a Scanner. Letting go of your references to the scanner is sufficient.

Parameters:

cacheBlocks - True if we should cache blocks read in by this scanner.

pread - Use positional read rather than seek+read if true (pread is better for random reads, seek+read is better scanning).

isCompaction - is scanner being used for a compaction?

Returns:

Scanner on this file.

getMetaBlock

public ByteBuffer getMetaBlock(String metaBlockName,
                      boolean cacheBlock)
                        throws IOException

Parameters:

metaBlockName -

cacheBlock - Add block to cache, if found

Returns:

block wrapped in a ByteBuffer, with header skipped

Throws:

IOException

readBlock

public HFileBlock readBlock(long dataBlockOffset,
                   long onDiskBlockSize,
                   boolean cacheBlock,
                   boolean pread,
                   boolean isCompaction,
                   boolean updateCacheMetrics,
                   BlockType expectedBlockType,
                   DataBlockEncoding expectedDataBlockEncoding)
                     throws IOException

Description copied from interface: HFile.CachingBlockReader

Read in a file block.

Parameters:

dataBlockOffset - offset to read.

onDiskBlockSize - size of the block

isCompaction - is this block being read as part of a compaction

expectedBlockType - the block type we are expecting to read with this read operation, or null to read whatever block type is available and avoid checking (that might reduce caching efficiency of encoded data blocks)

expectedDataBlockEncoding - the data block encoding the caller is expecting data blocks to be in, or null to not perform this check and return the block irrespective of the encoding. This check only applies to data blocks and can be set to null when the caller is expecting to read a non-data block and has set expectedBlockType accordingly.

Returns:

Block wrapped in a ByteBuffer.

Throws:

IOException

hasMVCCInfo

public boolean hasMVCCInfo()

getLastKey

public byte[] getLastKey()

Returns:

Last key in the file. May be null if file has no entries. Note that this is not the last row key, but rather the byte form of the last KeyValue.

midkey

public byte[] midkey()
              throws IOException

Returns:

Midkey for this file. We work with block boundaries only so returned midkey is an approximation only.

Throws:

IOException

close

public void close()
           throws IOException

Throws:

IOException

close

public void close(boolean evictOnClose)
           throws IOException

Description copied from interface: HFile.Reader

Close method with optional evictOnClose

Throws:

IOException

getEffectiveEncodingInCache

public DataBlockEncoding getEffectiveEncodingInCache(boolean isCompaction)

getGeneralBloomFilterMetadata

public DataInput getGeneralBloomFilterMetadata()
                                        throws IOException

Returns a buffer with the Bloom filter metadata. The caller takes ownership of the buffer.

Throws:

IOException

getDeleteBloomFilterMetadata

public DataInput getDeleteBloomFilterMetadata()
                                       throws IOException

Description copied from interface: HFile.Reader

Retrieves delete family Bloom filter metadata as appropriate for each HFile version. Knows nothing about how that metadata is structured.

Throws:

IOException

isFileInfoLoaded

public boolean isFileInfoLoaded()

Specified by:

isFileInfoLoaded in class AbstractHFileReader

getMajorVersion

public int getMajorVersion()

Specified by:

getMajorVersion in class AbstractHFileReader

getFileContext

public HFileContext getFileContext()

Description copied from interface: HFile.Reader

Return the file context of the HFile this reader belongs to