HStoreFile (Apache HBase 2.5.0 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.HStoreFile

All Implemented Interfaces:

StoreFile
```
@InterfaceAudience.Private
public class HStoreFile
extends Object
implements StoreFile
```
A Store data file. Stores usually have one or more of these files. They are produced by flushing the memstore to disk. To create, instantiate a writer using StoreFileWriter.Builder and append data. Be sure to add any metadata before calling close on the Writer (Use the appendMetadata convenience methods). On close, a StoreFile is sitting in the Filesystem. To refer to it, create a StoreFile instance passing filesystem and path. To read, call initReader()
StoreFiles may also reference store files in another Store. The reason for this weird pattern where you use a different instance for the writer and a reader is that we write once but read a lot more.

Field Summary

Fields
Modifier and Type	Field and Description
`static byte[]`	`BLOOM_FILTER_PARAM_KEY` Bloom filter param in FileInfo
`static byte[]`	`BLOOM_FILTER_TYPE_KEY` Bloom filter Type in FileInfo
`static byte[]`	`BULKLOAD_TASK_KEY` Meta key set when store file is a result of a bulk load
`static byte[]`	`BULKLOAD_TIME_KEY`
`private CacheConfig`	`cacheConf`
`private BloomType`	`cfBloomType` Bloom filter type specified in column family configuration.
`private boolean`	`compactedAway`
`private Set<String>`	`compactedStoreFiles`
`static byte[]`	`COMPACTION_EVENT_KEY` Key for compaction event which contains the compacted storefiles in FileInfo
`private CellComparator`	`comparator`
`static byte[]`	`DELETE_FAMILY_COUNT` Delete Family Count in FileInfo
`static byte[]`	`EARLIEST_PUT_TS` Key for timestamp of earliest-put in metadata
`static byte[]`	`EXCLUDE_FROM_MINOR_COMPACTION_KEY` Minor compaction flag in FileInfo
`private boolean`	`excludeFromMinorCompaction`
`private StoreFileInfo`	`fileInfo`
`private Optional<Cell>`	`firstKey`
`private StoreFileReader`	`initialReader`
`private InputStreamBlockDistribution`	`initialReaderBlockDistribution`
`static byte[]`	`LAST_BLOOM_KEY` Last Bloom filter key in FileInfo
`private Optional<Cell>`	`lastKey`
`private static org.slf4j.Logger`	`LOG`
`static byte[]`	`MAJOR_COMPACTION_KEY` Major compaction flag in FileInfo
`private AtomicBoolean`	`majorCompaction`
`static byte[]`	`MAX_SEQ_ID_KEY` Max Sequence ID in FileInfo
`private long`	`maxMemstoreTS`
`private Map<byte[],byte[]>`	`metadataMap` Map of the metadata entries in the corresponding HFile.
`private BloomFilterMetrics`	`metrics`
`static byte[]`	`MOB_CELLS_COUNT` Key for the number of mob cells in metadata
`static byte[]`	`MOB_FILE_REFS` Key for the list of MOB file references
`static byte[]`	`NULL_VALUE` Null data
`private long`	`sequenceid`
`static byte[]`	`SKIP_RESET_SEQ_ID` Key for skipping resetting sequence id in metadata.
`static byte[]`	`TIMERANGE_KEY` Key for Timerange information in metadata

Constructor Summary

Constructors
Constructor and Description
`HStoreFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType cfBloomType, boolean primaryReplica)` Constructor, loads a reader and it's indices, etc.
`HStoreFile(StoreFileInfo fileInfo, BloomType cfBloomType, CacheConfig cacheConf)` Constructor, loads a reader and it's indices, etc.
`HStoreFile(StoreFileInfo fileInfo, BloomType cfBloomType, CacheConfig cacheConf, BloomFilterMetrics metrics)` Constructor, loads a reader and it's indices, etc.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`closeStoreFile(boolean evictOnClose)`
`private StoreFileReader`	`createStreamReader(boolean canUseDropBehind)`
`void`	`deleteStoreFile()` Delete this file n
`boolean`	`excludeFromMinorCompaction()` Returns True if this file should not be part of a minor compaction.
`OptionalLong`	`getBulkLoadTimestamp()` Return the timestamp at which this bulk load file was generated.
`CacheConfig`	`getCacheConf()`
`(package private) Set<String>`	`getCompactedStoreFiles()`
`CellComparator`	`getComparator()` Get the comparator for comparing two cells.
`org.apache.hadoop.fs.Path`	`getEncodedPath()` Returns Encoded Path if this StoreFile was made with a Stream.
`StoreFileInfo`	`getFileInfo()`
`Optional<Cell>`	`getFirstKey()` Get the first key in this store file.
`HDFSBlocksDistribution`	`getHDFSBlockDistribution()`
`Optional<Cell>`	`getLastKey()` Get the last key in this store file.
`OptionalLong`	`getMaximumTimestamp()` Get the max timestamp of all the cells in the store file.
`long`	`getMaxMemStoreTS()` Get max of the MemstoreTS in the KV's in this store file.
`long`	`getMaxSequenceId()` Returns This files maximum edit sequence id.
`byte[]`	`getMetadataValue(byte[] key)`
`OptionalLong`	`getMinimumTimestamp()` Get the min timestamp of all the cells in the store file.
`long`	`getModificationTimestamp()` Get the modification time of this store file.
`long`	`getModificationTimeStamp()` Get the modification time of this store file.
`org.apache.hadoop.fs.Path`	`getPath()` Returns Path or null if this StoreFile was made with a Stream.
`StoreFileScanner`	`getPreadScanner(boolean cacheBlocks, long readPt, long scannerOrder, boolean canOptimizeForNonNullColumn)` Get a scanner which uses pread.
`org.apache.hadoop.fs.Path`	`getQualifiedPath()` Returns Returns the qualified path of this StoreFile
`StoreFileReader`	`getReader()`
`int`	`getRefCount()`
`StoreFileScanner`	`getStreamScanner(boolean canUseDropBehind, boolean cacheBlocks, boolean isCompaction, long readPt, long scannerOrder, boolean canOptimizeForNonNullColumn)` Get a scanner which uses streaming read.
`void`	`initReader()` Initialize the reader used for pread.
`boolean`	`isBulkLoadResult()` Check if this storefile was created by bulk load.
`boolean`	`isCompactedAway()`
`boolean`	`isHFile()` Returns True if this is HFile.
`boolean`	`isMajorCompactionResult()` Returns True if this file was made by a major compaction.
`boolean`	`isReference()` Returns True if this is a StoreFile Reference.
`boolean`	`isReferencedInReads()` Returns true if the file is still used in reads
`private boolean`	`isSkipResetSeqId(byte[] skipResetSeqId)` Gets whether to skip resetting the sequence id for cells.
`void`	`markCompactedAway()`
`private void`	`open()` Opens reader on this store file.
`String`	`toString()`
`String`	`toStringDetailed()` Returns a length description of this StoreFile, suitable for debug output

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.slf4j.Logger LOG
```
  - MAX_SEQ_ID_KEY
```
public static final byte[] MAX_SEQ_ID_KEY
```
    Max Sequence ID in FileInfo
  - MAJOR_COMPACTION_KEY
```
public static final byte[] MAJOR_COMPACTION_KEY
```
    Major compaction flag in FileInfo
  - EXCLUDE_FROM_MINOR_COMPACTION_KEY
```
public static final byte[] EXCLUDE_FROM_MINOR_COMPACTION_KEY
```
    Minor compaction flag in FileInfo
  - COMPACTION_EVENT_KEY
```
public static final byte[] COMPACTION_EVENT_KEY
```
    Key for compaction event which contains the compacted storefiles in FileInfo
  - BLOOM_FILTER_TYPE_KEY
```
public static final byte[] BLOOM_FILTER_TYPE_KEY
```
    Bloom filter Type in FileInfo
  - BLOOM_FILTER_PARAM_KEY
```
public static final byte[] BLOOM_FILTER_PARAM_KEY
```
    Bloom filter param in FileInfo
  - DELETE_FAMILY_COUNT
```
public static final byte[] DELETE_FAMILY_COUNT
```
    Delete Family Count in FileInfo
  - LAST_BLOOM_KEY
```
public static final byte[] LAST_BLOOM_KEY
```
    Last Bloom filter key in FileInfo
  - TIMERANGE_KEY
```
public static final byte[] TIMERANGE_KEY
```
    Key for Timerange information in metadata
  - EARLIEST_PUT_TS
```
public static final byte[] EARLIEST_PUT_TS
```
    Key for timestamp of earliest-put in metadata
  - MOB_CELLS_COUNT
```
public static final byte[] MOB_CELLS_COUNT
```
    Key for the number of mob cells in metadata
  - NULL_VALUE
```
public static final byte[] NULL_VALUE
```
    Null data
  - MOB_FILE_REFS
```
public static final byte[] MOB_FILE_REFS
```
    Key for the list of MOB file references
  - BULKLOAD_TASK_KEY
```
public static final byte[] BULKLOAD_TASK_KEY
```
    Meta key set when store file is a result of a bulk load
  - BULKLOAD_TIME_KEY
```
public static final byte[] BULKLOAD_TIME_KEY
```
  - SKIP_RESET_SEQ_ID
```
public static final byte[] SKIP_RESET_SEQ_ID
```
    Key for skipping resetting sequence id in metadata. For bulk loaded hfiles, the scanner resets the cell seqId with the latest one, if this metadata is set as true, the reset is skipped.
  - fileInfo
```
private final StoreFileInfo fileInfo
```
  - initialReader
```
private volatile StoreFileReader initialReader
```
  - initialReaderBlockDistribution
```
private volatile InputStreamBlockDistribution initialReaderBlockDistribution
```
  - cacheConf
```
private final CacheConfig cacheConf
```
  - metrics
```
private final BloomFilterMetrics metrics
```
  - compactedAway
```
private volatile boolean compactedAway
```
  - sequenceid
```
private long sequenceid
```
  - maxMemstoreTS
```
private long maxMemstoreTS
```
  - firstKey
```
private Optional<Cell> firstKey
```
  - lastKey
```
private Optional<Cell> lastKey
```
  - comparator
```
private CellComparator comparator
```
  - majorCompaction
```
private AtomicBoolean majorCompaction
```
  - excludeFromMinorCompaction
```
private boolean excludeFromMinorCompaction
```
  - compactedStoreFiles
```
private final Set<String> compactedStoreFiles
```
  - metadataMap
```
private Map<byte[],byte[]> metadataMap
```
    Map of the metadata entries in the corresponding HFile. Populated when Reader is opened after which it is not modified again.
  - cfBloomType
```
private final BloomType cfBloomType
```
    Bloom filter type specified in column family configuration. Does not necessarily correspond to the Bloom filter type present in the HFile.
- Constructor Detail
  - HStoreFile
```
public HStoreFile(org.apache.hadoop.fs.FileSystem fs,
                  org.apache.hadoop.fs.Path p,
                  org.apache.hadoop.conf.Configuration conf,
                  CacheConfig cacheConf,
                  BloomType cfBloomType,
                  boolean primaryReplica)
           throws IOException
```
    Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).
    
    Parameters:
    
    fs - The current file system to use.
    
    p - The path of the file.
    
    conf - The current configuration.
    
    cacheConf - The cache configuration and block cache reference.
    
    cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.
    
    primaryReplica - true if this is a store file for primary replica, otherwise false. n
    
    Throws:
    
    IOException
  - HStoreFile
```
public HStoreFile(StoreFileInfo fileInfo,
                  BloomType cfBloomType,
                  CacheConfig cacheConf)
```
    Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).
    
    Parameters:
    
    fileInfo - The store file information.
    
    cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.
    
    cacheConf - The cache configuration and block cache reference.
  - HStoreFile
```
public HStoreFile(StoreFileInfo fileInfo,
                  BloomType cfBloomType,
                  CacheConfig cacheConf,
                  BloomFilterMetrics metrics)
```
    Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).
    
    Parameters:
    
    fileInfo - The store file information.
    
    cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.
    
    cacheConf - The cache configuration and block cache reference.
    
    metrics - Tracks bloom filter requests and results. May be null.
- Method Detail
  - getCacheConf
```
public CacheConfig getCacheConf()
```
  - getFirstKey
```
public Optional<Cell> getFirstKey()
```
    Description copied from interface: StoreFile
    
    Get the first key in this store file.
    
    Specified by:
    
    getFirstKey in interface StoreFile
  - getLastKey
```
public Optional<Cell> getLastKey()
```
    Description copied from interface: StoreFile
    
    Get the last key in this store file.
    
    Specified by:
    
    getLastKey in interface StoreFile
  - getComparator
```
public CellComparator getComparator()
```
    Description copied from interface: StoreFile
    
    Get the comparator for comparing two cells.
    
    Specified by:
    
    getComparator in interface StoreFile
  - getMaxMemStoreTS
```
public long getMaxMemStoreTS()
```
    Description copied from interface: StoreFile
    
    Get max of the MemstoreTS in the KV's in this store file.
    
    Specified by:
    
    getMaxMemStoreTS in interface StoreFile
  - getFileInfo
```
public StoreFileInfo getFileInfo()
```
    Returns:
    
    the StoreFile object associated to this StoreFile. null if the StoreFile is not a reference.
  - getPath
```
public org.apache.hadoop.fs.Path getPath()
```
    Description copied from interface: StoreFile
    
    Returns Path or null if this StoreFile was made with a Stream.
    
    Specified by:
    
    getPath in interface StoreFile
  - getEncodedPath
```
public org.apache.hadoop.fs.Path getEncodedPath()
```
    Description copied from interface: StoreFile
    
    Returns Encoded Path if this StoreFile was made with a Stream.
    
    Specified by:
    
    getEncodedPath in interface StoreFile
  - getQualifiedPath
```
public org.apache.hadoop.fs.Path getQualifiedPath()
```
    Description copied from interface: StoreFile
    
    Returns Returns the qualified path of this StoreFile
    
    Specified by:
    
    getQualifiedPath in interface StoreFile
  - isReference
```
public boolean isReference()
```
    Description copied from interface: StoreFile
    
    Returns True if this is a StoreFile Reference.
    
    Specified by:
    
    isReference in interface StoreFile
  - isHFile
```
public boolean isHFile()
```
    Description copied from interface: StoreFile
    
    Returns True if this is HFile.
    
    Specified by:
    
    isHFile in interface StoreFile
  - isMajorCompactionResult
```
public boolean isMajorCompactionResult()
```
    Description copied from interface: StoreFile
    
    Returns True if this file was made by a major compaction.
    
    Specified by:
    
    isMajorCompactionResult in interface StoreFile
  - excludeFromMinorCompaction
```
public boolean excludeFromMinorCompaction()
```
    Description copied from interface: StoreFile
    
    Returns True if this file should not be part of a minor compaction.
    
    Specified by:
    
    excludeFromMinorCompaction in interface StoreFile
  - getMaxSequenceId
```
public long getMaxSequenceId()
```
    Description copied from interface: StoreFile
    
    Returns This files maximum edit sequence id.
    
    Specified by:
    
    getMaxSequenceId in interface StoreFile
  - getModificationTimeStamp
```
public long getModificationTimeStamp()
                              throws IOException
```
    Description copied from interface: StoreFile
    
    Get the modification time of this store file. Usually will access the file system so throws IOException.
    
    Specified by:
    
    getModificationTimeStamp in interface StoreFile
    
    Throws:
    
    IOException
    
    See Also:
    
    StoreFile.getModificationTimestamp()
  - getModificationTimestamp
```
public long getModificationTimestamp()
                              throws IOException
```
    Description copied from interface: StoreFile
    
    Get the modification time of this store file. Usually will access the file system so throws IOException.
    
    Specified by:
    
    getModificationTimestamp in interface StoreFile
    
    Throws:
    
    IOException
  - getMetadataValue
```
public byte[] getMetadataValue(byte[] key)
```
    Parameters:
    
    key - to look up
    
    Returns:
    
    value associated with the metadata key
  - isBulkLoadResult
```
public boolean isBulkLoadResult()
```
    Description copied from interface: StoreFile
    
    Check if this storefile was created by bulk load. When a hfile is bulk loaded into HBase, we append '_SeqId_<id-when-loaded>' to the hfile name, unless "hbase.mapreduce.bulkload.assign.sequenceNumbers" is explicitly turned off. If "hbase.mapreduce.bulkload.assign.sequenceNumbers" is turned off, fall back to BULKLOAD_TIME_KEY.
    
    Specified by:
    
    isBulkLoadResult in interface StoreFile
    
    Returns:
    
    true if this storefile was created by bulk load.
  - isCompactedAway
```
public boolean isCompactedAway()
```
  - getRefCount
```
public int getRefCount()
```
  - isReferencedInReads
```
public boolean isReferencedInReads()
```
    Returns true if the file is still used in reads
  - getBulkLoadTimestamp
```
public OptionalLong getBulkLoadTimestamp()
```
    Description copied from interface: StoreFile
    
    Return the timestamp at which this bulk load file was generated.
    
    Specified by:
    
    getBulkLoadTimestamp in interface StoreFile
  - getHDFSBlockDistribution
```
public HDFSBlocksDistribution getHDFSBlockDistribution()
```
    Returns:
    
    the cached value of HDFS blocks distribution. The cached value is calculated when store file is opened.
  - open
```
private void open()
           throws IOException
```
    Opens reader on this store file. Called by Constructor.
    
    Throws:
    
    IOException
    
    See Also:
    
    closeStoreFile(boolean)
  - initReader
```
public void initReader()
                throws IOException
```
    Initialize the reader used for pread.
    
    Throws:
    
    IOException
  - createStreamReader
```
private StoreFileReader createStreamReader(boolean canUseDropBehind)
                                    throws IOException
```
    Throws:
    
    IOException
  - getPreadScanner
```
public StoreFileScanner getPreadScanner(boolean cacheBlocks,
                                        long readPt,
                                        long scannerOrder,
                                        boolean canOptimizeForNonNullColumn)
```
    Get a scanner which uses pread.
    Must be called after initReader.
  - getStreamScanner
```
public StoreFileScanner getStreamScanner(boolean canUseDropBehind,
                                         boolean cacheBlocks,
                                         boolean isCompaction,
                                         long readPt,
                                         long scannerOrder,
                                         boolean canOptimizeForNonNullColumn)
                                  throws IOException
```
    Get a scanner which uses streaming read.
    Must be called after initReader.
    
    Throws:
    
    IOException
  - getReader
```
public StoreFileReader getReader()
```
    Returns:
    
    Current reader. Must call initReader first else returns null.
    
    See Also:
    
    initReader()
  - closeStoreFile
```
public void closeStoreFile(boolean evictOnClose)
                    throws IOException
```
    Parameters:
    
    evictOnClose - whether to evict blocks belonging to this file n
    
    Throws:
    
    IOException
  - deleteStoreFile
```
public void deleteStoreFile()
                     throws IOException
```
    Delete this file n
    
    Throws:
    
    IOException
  - markCompactedAway
```
public void markCompactedAway()
```
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - toStringDetailed
```
public String toStringDetailed()
```
    Description copied from interface: StoreFile
    
    Returns a length description of this StoreFile, suitable for debug output
    
    Specified by:
    
    toStringDetailed in interface StoreFile
  - isSkipResetSeqId
```
private boolean isSkipResetSeqId(byte[] skipResetSeqId)
```
    Gets whether to skip resetting the sequence id for cells.
    
    Parameters:
    
    skipResetSeqId - The byte array of boolean.
    
    Returns:
    
    Whether to skip resetting the sequence id.
  - getMinimumTimestamp
```
public OptionalLong getMinimumTimestamp()
```
    Description copied from interface: StoreFile
    
    Get the min timestamp of all the cells in the store file.
    
    Specified by:
    
    getMinimumTimestamp in interface StoreFile
  - getMaximumTimestamp
```
public OptionalLong getMaximumTimestamp()
```
    Description copied from interface: StoreFile
    
    Get the max timestamp of all the cells in the store file.
    
    Specified by:
    
    getMaximumTimestamp in interface StoreFile
  - getCompactedStoreFiles
```
Set<String> getCompactedStoreFiles()
```

Class HStoreFile

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

MAX_SEQ_ID_KEY

MAJOR_COMPACTION_KEY

EXCLUDE_FROM_MINOR_COMPACTION_KEY

COMPACTION_EVENT_KEY

BLOOM_FILTER_TYPE_KEY

BLOOM_FILTER_PARAM_KEY

DELETE_FAMILY_COUNT

LAST_BLOOM_KEY

TIMERANGE_KEY

EARLIEST_PUT_TS

MOB_CELLS_COUNT

NULL_VALUE

MOB_FILE_REFS

BULKLOAD_TASK_KEY

BULKLOAD_TIME_KEY

SKIP_RESET_SEQ_ID

fileInfo

initialReader

initialReaderBlockDistribution

cacheConf

metrics

compactedAway

sequenceid

maxMemstoreTS

firstKey

lastKey

comparator

majorCompaction

excludeFromMinorCompaction

compactedStoreFiles

metadataMap

cfBloomType

Constructor Detail

HStoreFile

HStoreFile

HStoreFile

Method Detail

getCacheConf

getFirstKey

getLastKey

getComparator

getMaxMemStoreTS

getFileInfo

getPath

getEncodedPath

getQualifiedPath

isReference

isHFile

isMajorCompactionResult

excludeFromMinorCompaction

getMaxSequenceId

getModificationTimeStamp

getModificationTimestamp

getMetadataValue

isBulkLoadResult

isCompactedAway

getRefCount

isReferencedInReads

getBulkLoadTimestamp

getHDFSBlockDistribution

open

initReader

createStreamReader

getPreadScanner

getStreamScanner

getReader

closeStoreFile

deleteStoreFile

markCompactedAway

toString

toStringDetailed

isSkipResetSeqId

getMinimumTimestamp