HStoreFile (Apache HBase 2.0.6 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.HStoreFile

All Implemented Interfaces:

StoreFile, StoreFileReader.Listener
```
@InterfaceAudience.Private
public class HStoreFile
extends Object
implements StoreFile, StoreFileReader.Listener
```
A Store data file. Stores usually have one or more of these files. They are produced by flushing the memstore to disk. To create, instantiate a writer using StoreFileWriter.Builder and append data. Be sure to add any metadata before calling close on the Writer (Use the appendMetadata convenience methods). On close, a StoreFile is sitting in the Filesystem. To refer to it, create a StoreFile instance passing filesystem and path. To read, call initReader()
StoreFiles may also reference store files in another Store. The reason for this weird pattern where you use a different instance for the writer and a reader is that we write once but read a lot more.

Field Summary

Fields
Modifier and Type	Field and Description
`static byte[]`	`BLOOM_FILTER_TYPE_KEY` Bloom filter Type in FileInfo
`static byte[]`	`BULKLOAD_TASK_KEY` Meta key set when store file is a result of a bulk load
`static byte[]`	`BULKLOAD_TIME_KEY`
`private CacheConfig`	`cacheConf`
`private BloomType`	`cfBloomType` Bloom filter type specified in column family configuration.
`private boolean`	`compactedAway`
`private CellComparator`	`comparator`
`private static boolean`	`DEFAULT_STORE_FILE_READER_NO_READAHEAD`
`static byte[]`	`DELETE_FAMILY_COUNT` Delete Family Count in FileInfo
`static byte[]`	`EARLIEST_PUT_TS` Key for timestamp of earliest-put in metadata
`static byte[]`	`EXCLUDE_FROM_MINOR_COMPACTION_KEY` Minor compaction flag in FileInfo
`private boolean`	`excludeFromMinorCompaction`
`private StoreFileInfo`	`fileInfo`
`private Optional<Cell>`	`firstKey`
`private org.apache.hadoop.fs.FileSystem`	`fs`
`static byte[]`	`LAST_BLOOM_KEY` Last Bloom filter key in FileInfo
`private Optional<Cell>`	`lastKey`
`private static org.slf4j.Logger`	`LOG`
`static byte[]`	`MAJOR_COMPACTION_KEY` Major compaction flag in FileInfo
`private AtomicBoolean`	`majorCompaction`
`static byte[]`	`MAX_SEQ_ID_KEY` Max Sequence ID in FileInfo
`private long`	`maxMemstoreTS`
`private Map<byte[],byte[]>`	`metadataMap` Map of the metadata entries in the corresponding HFile.
`static byte[]`	`MOB_CELLS_COUNT` Key for the number of mob cells in metadata
`private boolean`	`noReadahead`
`private boolean`	`primaryReplica`
`private StoreFileReader`	`reader`
`private AtomicInteger`	`refCount`
`private long`	`sequenceid`
`static byte[]`	`SKIP_RESET_SEQ_ID` Key for skipping resetting sequence id in metadata.
`static String`	`STORE_FILE_READER_NO_READAHEAD`
`(package private) Set<StoreFileReader>`	`streamReaders`
`static byte[]`	`TIMERANGE_KEY` Key for Timerange information in metadata

Constructor Summary

Constructors
Constructor and Description
`HStoreFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType cfBloomType, boolean primaryReplica)` Constructor, loads a reader and it's indices, etc.
`HStoreFile(org.apache.hadoop.fs.FileSystem fs, StoreFileInfo fileInfo, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType cfBloomType, boolean primaryReplica)` Constructor, loads a reader and it's indices, etc.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`closeStoreFile(boolean evictOnClose)`
`void`	`closeStreamReaders(boolean evictOnClose)`
`private StoreFileReader`	`createStreamReader(boolean canUseDropBehind)`
`void`	`deleteStoreFile()` Delete this file
`boolean`	`excludeFromMinorCompaction()`
`OptionalLong`	`getBulkLoadTimestamp()` Return the timestamp at which this bulk load file was generated.
`CacheConfig`	`getCacheConf()`
`CellComparator`	`getComparator()` Get the comparator for comparing two cells.
`StoreFileInfo`	`getFileInfo()`
`Optional<Cell>`	`getFirstKey()` Get the first key in this store file.
`HDFSBlocksDistribution`	`getHDFSBlockDistribution()`
`Optional<Cell>`	`getLastKey()` Get the last key in this store file.
`OptionalLong`	`getMaximumTimestamp()` Get the max timestamp of all the cells in the store file.
`long`	`getMaxMemStoreTS()` Get max of the MemstoreTS in the KV's in this store file.
`long`	`getMaxSequenceId()`
`byte[]`	`getMetadataValue(byte[] key)` Only used by the Striped Compaction Policy
`OptionalLong`	`getMinimumTimestamp()` Get the min timestamp of all the cells in the store file.
`long`	`getModificationTimestamp()` Get the modification time of this store file.
`long`	`getModificationTimeStamp()` Get the modification time of this store file.
`org.apache.hadoop.fs.Path`	`getPath()`
`StoreFileScanner`	`getPreadScanner(boolean cacheBlocks, long readPt, long scannerOrder, boolean canOptimizeForNonNullColumn)` Get a scanner which uses pread.
`org.apache.hadoop.fs.Path`	`getQualifiedPath()`
`StoreFileReader`	`getReader()`
`int`	`getRefCount()`
`StoreFileScanner`	`getStreamScanner(boolean canUseDropBehind, boolean cacheBlocks, boolean isCompaction, long readPt, long scannerOrder, boolean canOptimizeForNonNullColumn)` Get a scanner which uses streaming read.
`void`	`initReader()` Initialize the reader used for pread.
`boolean`	`isBulkLoadResult()` Check if this storefile was created by bulk load.
`boolean`	`isCompactedAway()`
`boolean`	`isHFile()`
`boolean`	`isMajorCompactionResult()`
`boolean`	`isReference()`
`boolean`	`isReferencedInReads()`
`private boolean`	`isSkipResetSeqId(byte[] skipResetSeqId)` Gets whether to skip resetting the sequence id for cells.
`void`	`markCompactedAway()`
`private void`	`open()` Opens reader on this store file.
`void`	`storeFileReaderClosed(StoreFileReader reader)`
`String`	`toString()`
`String`	`toStringDetailed()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.slf4j.Logger LOG
```
  - STORE_FILE_READER_NO_READAHEAD
```
public static final String STORE_FILE_READER_NO_READAHEAD
```
    See Also:
    
    Constant Field Values
  - DEFAULT_STORE_FILE_READER_NO_READAHEAD
```
private static final boolean DEFAULT_STORE_FILE_READER_NO_READAHEAD
```
    See Also:
    
    Constant Field Values
  - MAX_SEQ_ID_KEY
```
public static final byte[] MAX_SEQ_ID_KEY
```
    Max Sequence ID in FileInfo
  - MAJOR_COMPACTION_KEY
```
public static final byte[] MAJOR_COMPACTION_KEY
```
    Major compaction flag in FileInfo
  - EXCLUDE_FROM_MINOR_COMPACTION_KEY
```
public static final byte[] EXCLUDE_FROM_MINOR_COMPACTION_KEY
```
    Minor compaction flag in FileInfo
  - BLOOM_FILTER_TYPE_KEY
```
public static final byte[] BLOOM_FILTER_TYPE_KEY
```
    Bloom filter Type in FileInfo
  - DELETE_FAMILY_COUNT
```
public static final byte[] DELETE_FAMILY_COUNT
```
    Delete Family Count in FileInfo
  - LAST_BLOOM_KEY
```
public static final byte[] LAST_BLOOM_KEY
```
    Last Bloom filter key in FileInfo
  - TIMERANGE_KEY
```
public static final byte[] TIMERANGE_KEY
```
    Key for Timerange information in metadata
  - EARLIEST_PUT_TS
```
public static final byte[] EARLIEST_PUT_TS
```
    Key for timestamp of earliest-put in metadata
  - MOB_CELLS_COUNT
```
public static final byte[] MOB_CELLS_COUNT
```
    Key for the number of mob cells in metadata
  - BULKLOAD_TASK_KEY
```
public static final byte[] BULKLOAD_TASK_KEY
```
    Meta key set when store file is a result of a bulk load
  - BULKLOAD_TIME_KEY
```
public static final byte[] BULKLOAD_TIME_KEY
```
  - SKIP_RESET_SEQ_ID
```
public static final byte[] SKIP_RESET_SEQ_ID
```
    Key for skipping resetting sequence id in metadata. For bulk loaded hfiles, the scanner resets the cell seqId with the latest one, if this metadata is set as true, the reset is skipped.
  - fileInfo
```
private final StoreFileInfo fileInfo
```
  - fs
```
private final org.apache.hadoop.fs.FileSystem fs
```
  - cacheConf
```
private final CacheConfig cacheConf
```
  - refCount
```
private final AtomicInteger refCount
```
  - streamReaders
```
final Set<StoreFileReader> streamReaders
```
  - noReadahead
```
private final boolean noReadahead
```
  - primaryReplica
```
private final boolean primaryReplica
```
  - compactedAway
```
private volatile boolean compactedAway
```
  - sequenceid
```
private long sequenceid
```
  - maxMemstoreTS
```
private long maxMemstoreTS
```
  - firstKey
```
private Optional<Cell> firstKey
```
  - lastKey
```
private Optional<Cell> lastKey
```
  - comparator
```
private CellComparator comparator
```
  - majorCompaction
```
private AtomicBoolean majorCompaction
```
  - excludeFromMinorCompaction
```
private boolean excludeFromMinorCompaction
```
  - metadataMap
```
private Map<byte[],byte[]> metadataMap
```
    Map of the metadata entries in the corresponding HFile. Populated when Reader is opened after which it is not modified again.
  - reader
```
private volatile StoreFileReader reader
```
  - cfBloomType
```
private final BloomType cfBloomType
```
    Bloom filter type specified in column family configuration. Does not necessarily correspond to the Bloom filter type present in the HFile.
- Constructor Detail
  - HStoreFile
```
public HStoreFile(org.apache.hadoop.fs.FileSystem fs,
                  org.apache.hadoop.fs.Path p,
                  org.apache.hadoop.conf.Configuration conf,
                  CacheConfig cacheConf,
                  BloomType cfBloomType,
                  boolean primaryReplica)
           throws IOException
```
    Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).
    
    Parameters:
    
    fs - The current file system to use.
    
    p - The path of the file.
    
    conf - The current configuration.
    
    cacheConf - The cache configuration and block cache reference.
    
    cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.
    
    primaryReplica - true if this is a store file for primary replica, otherwise false.
    
    Throws:
    
    IOException
  - HStoreFile
```
public HStoreFile(org.apache.hadoop.fs.FileSystem fs,
                  StoreFileInfo fileInfo,
                  org.apache.hadoop.conf.Configuration conf,
                  CacheConfig cacheConf,
                  BloomType cfBloomType,
                  boolean primaryReplica)
```
    Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).
    
    Parameters:
    
    fs - fs The current file system to use.
    
    fileInfo - The store file information.
    
    conf - The current configuration.
    
    cacheConf - The cache configuration and block cache reference.
    
    cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.
    
    primaryReplica - true if this is a store file for primary replica, otherwise false.
- Method Detail
  - getCacheConf
```
public CacheConfig getCacheConf()
```
  - getFirstKey
```
public Optional<Cell> getFirstKey()
```
    Description copied from interface: StoreFile
    
    Get the first key in this store file.
    
    Specified by:
    
    getFirstKey in interface StoreFile
  - getLastKey
```
public Optional<Cell> getLastKey()
```
    Description copied from interface: StoreFile
    
    Get the last key in this store file.
    
    Specified by:
    
    getLastKey in interface StoreFile
  - getComparator
```
public CellComparator getComparator()
```
    Description copied from interface: StoreFile
    
    Get the comparator for comparing two cells.
    
    Specified by:
    
    getComparator in interface StoreFile
  - getMaxMemStoreTS
```
public long getMaxMemStoreTS()
```
    Description copied from interface: StoreFile
    
    Get max of the MemstoreTS in the KV's in this store file.
    
    Specified by:
    
    getMaxMemStoreTS in interface StoreFile
  - getFileInfo
```
public StoreFileInfo getFileInfo()
```
    Returns:
    
    the StoreFile object associated to this StoreFile. null if the StoreFile is not a reference.
  - getPath
```
public org.apache.hadoop.fs.Path getPath()
```
    Specified by:
    
    getPath in interface StoreFile
    
    Returns:
    
    Path or null if this StoreFile was made with a Stream.
  - getQualifiedPath
```
public org.apache.hadoop.fs.Path getQualifiedPath()
```
    Specified by:
    
    getQualifiedPath in interface StoreFile
    
    Returns:
    
    Returns the qualified path of this StoreFile
  - isReference
```
public boolean isReference()
```
    Specified by:
    
    isReference in interface StoreFile
    
    Returns:
    
    True if this is a StoreFile Reference.
  - isHFile
```
public boolean isHFile()
```
    Specified by:
    
    isHFile in interface StoreFile
    
    Returns:
    
    True if this is HFile.
  - isMajorCompactionResult
```
public boolean isMajorCompactionResult()
```
    Specified by:
    
    isMajorCompactionResult in interface StoreFile
    
    Returns:
    
    True if this file was made by a major compaction.
  - excludeFromMinorCompaction
```
public boolean excludeFromMinorCompaction()
```
    Specified by:
    
    excludeFromMinorCompaction in interface StoreFile
    
    Returns:
    
    True if this file should not be part of a minor compaction.
  - getMaxSequenceId
```
public long getMaxSequenceId()
```
    Specified by:
    
    getMaxSequenceId in interface StoreFile
    
    Returns:
    
    This files maximum edit sequence id.
  - getModificationTimeStamp
```
public long getModificationTimeStamp()
                              throws IOException
```
    Description copied from interface: StoreFile
    
    Get the modification time of this store file. Usually will access the file system so throws IOException.
    
    Specified by:
    
    getModificationTimeStamp in interface StoreFile
    
    Throws:
    
    IOException
    
    See Also:
    
    StoreFile.getModificationTimestamp()
  - getModificationTimestamp
```
public long getModificationTimestamp()
                              throws IOException
```
    Description copied from interface: StoreFile
    
    Get the modification time of this store file. Usually will access the file system so throws IOException.
    
    Specified by:
    
    getModificationTimestamp in interface StoreFile
    
    Throws:
    
    IOException
  - getMetadataValue
```
public byte[] getMetadataValue(byte[] key)
```
    Only used by the Striped Compaction Policy
    
    Parameters:
    
    key -
    
    Returns:
    
    value associated with the metadata key
  - isBulkLoadResult
```
public boolean isBulkLoadResult()
```
    Description copied from interface: StoreFile
    
    Check if this storefile was created by bulk load. When a hfile is bulk loaded into HBase, we append '_SeqId_<id-when-loaded>' to the hfile name, unless "hbase.mapreduce.bulkload.assign.sequenceNumbers" is explicitly turned off. If "hbase.mapreduce.bulkload.assign.sequenceNumbers" is turned off, fall back to BULKLOAD_TIME_KEY.
    
    Specified by:
    
    isBulkLoadResult in interface StoreFile
    
    Returns:
    
    true if this storefile was created by bulk load.
  - isCompactedAway
```
public boolean isCompactedAway()
```
  - getRefCount
```
public int getRefCount()
```
  - isReferencedInReads
```
public boolean isReferencedInReads()
```
    Returns:
    
    true if the file is still used in reads
  - getBulkLoadTimestamp
```
public OptionalLong getBulkLoadTimestamp()
```
    Description copied from interface: StoreFile
    
    Return the timestamp at which this bulk load file was generated.
    
    Specified by:
    
    getBulkLoadTimestamp in interface StoreFile
  - getHDFSBlockDistribution
```
public HDFSBlocksDistribution getHDFSBlockDistribution()
```
    Returns:
    
    the cached value of HDFS blocks distribution. The cached value is calculated when store file is opened.
  - open
```
private void open()
           throws IOException
```
    Opens reader on this store file. Called by Constructor.
    
    Throws:
    
    IOException
    
    See Also:
    
    closeStoreFile(boolean)
  - initReader
```
public void initReader()
                throws IOException
```
    Initialize the reader used for pread.
    
    Throws:
    
    IOException
  - createStreamReader
```
private StoreFileReader createStreamReader(boolean canUseDropBehind)
                                    throws IOException
```
    Throws:
    
    IOException
  - getPreadScanner
```
public StoreFileScanner getPreadScanner(boolean cacheBlocks,
                                        long readPt,
                                        long scannerOrder,
                                        boolean canOptimizeForNonNullColumn)
```
    Get a scanner which uses pread.
    Must be called after initReader.
  - getStreamScanner
```
public StoreFileScanner getStreamScanner(boolean canUseDropBehind,
                                         boolean cacheBlocks,
                                         boolean isCompaction,
                                         long readPt,
                                         long scannerOrder,
                                         boolean canOptimizeForNonNullColumn)
                                  throws IOException
```
    Get a scanner which uses streaming read.
    Must be called after initReader.
    
    Throws:
    
    IOException
  - getReader
```
public StoreFileReader getReader()
```
    Returns:
    
    Current reader. Must call initReader first else returns null.
    
    See Also:
    
    initReader()
  - closeStoreFile
```
public void closeStoreFile(boolean evictOnClose)
                    throws IOException
```
    Parameters:
    
    evictOnClose - whether to evict blocks belonging to this file
    
    Throws:
    
    IOException
  - closeStreamReaders
```
public void closeStreamReaders(boolean evictOnClose)
                        throws IOException
```
    Throws:
    
    IOException
  - deleteStoreFile
```
public void deleteStoreFile()
                     throws IOException
```
    Delete this file
    
    Throws:
    
    IOException
  - markCompactedAway
```
public void markCompactedAway()
```
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - toStringDetailed
```
public String toStringDetailed()
```
    Specified by:
    
    toStringDetailed in interface StoreFile
    
    Returns:
    
    a length description of this StoreFile, suitable for debug output
  - isSkipResetSeqId
```
private boolean isSkipResetSeqId(byte[] skipResetSeqId)
```
    Gets whether to skip resetting the sequence id for cells.
    
    Parameters:
    
    skipResetSeqId - The byte array of boolean.
    
    Returns:
    
    Whether to skip resetting the sequence id.
  - getMinimumTimestamp
```
public OptionalLong getMinimumTimestamp()
```
    Description copied from interface: StoreFile
    
    Get the min timestamp of all the cells in the store file.
    
    Specified by:
    
    getMinimumTimestamp in interface StoreFile
  - getMaximumTimestamp
```
public OptionalLong getMaximumTimestamp()
```
    Description copied from interface: StoreFile
    
    Get the max timestamp of all the cells in the store file.
    
    Specified by:
    
    getMaximumTimestamp in interface StoreFile
  - storeFileReaderClosed
```
public void storeFileReaderClosed(StoreFileReader reader)
```
    Specified by:
    
    storeFileReaderClosed in interface StoreFileReader.Listener

Class HStoreFile

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

STORE_FILE_READER_NO_READAHEAD

DEFAULT_STORE_FILE_READER_NO_READAHEAD

MAX_SEQ_ID_KEY

MAJOR_COMPACTION_KEY

EXCLUDE_FROM_MINOR_COMPACTION_KEY

BLOOM_FILTER_TYPE_KEY

DELETE_FAMILY_COUNT

LAST_BLOOM_KEY

TIMERANGE_KEY

EARLIEST_PUT_TS

MOB_CELLS_COUNT

BULKLOAD_TASK_KEY

BULKLOAD_TIME_KEY

SKIP_RESET_SEQ_ID

fileInfo

fs

cacheConf

refCount

streamReaders

noReadahead

primaryReplica

compactedAway

sequenceid

maxMemstoreTS

firstKey

lastKey

comparator

majorCompaction

excludeFromMinorCompaction

metadataMap

reader

cfBloomType

Constructor Detail

HStoreFile

HStoreFile

Method Detail

getCacheConf

getFirstKey

getLastKey

getComparator

getMaxMemStoreTS

getFileInfo

getPath

getQualifiedPath

isReference

isHFile

isMajorCompactionResult

excludeFromMinorCompaction

getMaxSequenceId

getModificationTimeStamp

getModificationTimestamp

getMetadataValue

isBulkLoadResult

isCompactedAway

getRefCount

isReferencedInReads

getBulkLoadTimestamp

getHDFSBlockDistribution

open

initReader

createStreamReader

getPreadScanner

getStreamScanner

getReader

closeStoreFile

closeStreamReaders

deleteStoreFile

markCompactedAway

toString

toStringDetailed

isSkipResetSeqId

getMinimumTimestamp

getMaximumTimestamp