HStore (Apache HBase 2.2.3 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.HStore

All Implemented Interfaces:

ConfigurationObserver, PropagatingConfigurationObserver, HeapSize, Store, StoreConfigInformation

Direct Known Subclasses:

HMobStore
```
@InterfaceAudience.Private
public class HStore
extends Object
implements Store, HeapSize, StoreConfigInformation, PropagatingConfigurationObserver
```
A Store holds a column family in a Region. Its a memstore and a set of zero or more StoreFiles, which stretch backwards over time.
There's no reason to consider append-logging at this level; all logging and locking is handled at the HRegion level. Store just provides services to manage sets of StoreFiles. One of the most important of those services is compaction services where files are aggregated once they pass a configurable threshold.
Locking and transactions are handled at a higher level. This API should not be called directly but by an HRegion manager.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

private class HStore.StoreFlusherImpl

Nested Classes
Modifier and Type	Class and Description
`private class`	`HStore.StoreFlusherImpl`

Field Summary

Fields
Modifier and Type	Field and Description
`(package private) ReentrantLock`	`archiveLock` Lock specific to archiving compacted store files.
`static String`	`BLOCK_STORAGE_POLICY_KEY`
`static String`	`BLOCKING_STOREFILES_KEY`
`private long`	`blockingFileCount`
`protected int`	`blocksize`
`protected int`	`bytesPerChecksum`
`protected CacheConfig`	`cacheConf`
`private Set<ChangedReadersObserver>`	`changedReaderObservers`
`protected ChecksumType`	`checksumType` Checksum configuration
`(package private) static int`	`closeCheckInterval`
`static String`	`COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY`
`private AtomicLong`	`compactedCellsCount`
`private AtomicLong`	`compactedCellsSize`
`private int`	`compactionCheckMultiplier`
`protected CellComparator`	`comparator`
`protected org.apache.hadoop.conf.Configuration`	`conf`
`protected Encryption.Context`	`cryptoContext`
`private AtomicInteger`	`currentParallelPutCount` Use this counter to track concurrent puts.
`private HFileDataBlockEncoder`	`dataBlockEncoder`
`static long`	`DEEP_OVERHEAD`
`static String`	`DEFAULT_BLOCK_STORAGE_POLICY`
`static int`	`DEFAULT_BLOCKING_STOREFILE_COUNT`
`static int`	`DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER`
`private static int`	`DEFAULT_FLUSH_RETRIES_NUMBER`
`private ColumnFamilyDescriptor`	`family`
`private List<HStoreFile>`	`filesCompacting`
`static long`	`FIXED_OVERHEAD`
`private AtomicLong`	`flushedCellsCount`
`private AtomicLong`	`flushedCellsSize`
`private AtomicLong`	`flushedOutputFileSize`
`private int`	`flushRetriesNumber`
`(package private) boolean`	`forceMajor`
`private HRegionFileSystem`	`fs`
`private long`	`lastCompactSize`
`(package private) ReentrantReadWriteLock`	`lock` RWLock for store operations.
`private static org.slf4j.Logger`	`LOG`
`private AtomicLong`	`majorCompactedCellsCount`
`private AtomicLong`	`majorCompactedCellsSize`
`protected MemStore`	`memstore`
`static String`	`MEMSTORE_CLASS_NAME`
`private static AtomicBoolean`	`offPeakCompactionTracker`
`private OffPeakHours`	`offPeakHours`
`private int`	`parallelPutCountPrintThreshold`
`private int`	`pauseTime`
`protected HRegion`	`region`
`private ScanInfo`	`scanInfo`
`(package private) StoreEngine<?,?,?,?>`	`storeEngine`
`private AtomicLong`	`storeSize`
`private AtomicLong`	`totalUncompressedBytes`
`private boolean`	`verifyBulkLoads`

Fields inherited from interface org.apache.hadoop.hbase.regionserver.Store
NO_PRIORITY, PRIORITY_USER

Constructor Summary

Constructors
Modifier	Constructor and Description
`protected`	`HStore(HRegion region, ColumnFamilyDescriptor family, org.apache.hadoop.conf.Configuration confParam, boolean warmup)` Constructor

Method Summary

All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`void`	`add(Cell cell, MemStoreSizing memstoreSizing)` Adds a value to the memstore
`void`	`add(Iterable<Cell> cells, MemStoreSizing memstoreSizing)` Adds the specified value to the memstore
`void`	`addChangedReaderObserver(ChangedReadersObserver o)`
`private void`	`addToCompactingFiles(Collection<HStoreFile> filesToAdd)` Adds the files to compacting files.
`boolean`	`areWritesEnabled()`
`void`	`assertBulkLoadHFileOk(org.apache.hadoop.fs.Path srcPath)` This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.
`org.apache.hadoop.fs.Path`	`bulkLoadHFile(byte[] family, String srcPathStr, org.apache.hadoop.fs.Path dstPath)`
`private void`	`bulkLoadHFile(HStoreFile sf)`
`void`	`bulkLoadHFile(StoreFileInfo fileInfo)`
`void`	`cancelRequestedCompaction(CompactionContext compaction)`
`boolean`	`canSplit()` Returns whether this store is splittable, i.e., no reference file in this store.
`private static void`	`clearAndClose(List<KeyValueScanner> scanners)`
`private void`	`clearCompactedfiles(List<HStoreFile> filesToRemove)`
`org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>`	`close()` Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.
`void`	`closeAndArchiveCompactedFiles()` Closes and archives the compacted files under this store
`private HStoreFile`	`commitFile(org.apache.hadoop.fs.Path path, long logCacheFlushId, MonitoredTask status)`
`List<HStoreFile>`	`compact(CompactionContext compaction, ThroughputController throughputController, User user)` Compact the StoreFiles.
`void`	`compactRecentForTestingAssumingDefaultPolicy(int N)` This method tries to compact N recent files for testing.
`protected void`	`completeCompaction(Collection<HStoreFile> compactedFiles)` Update counts.
`protected void`	`createCacheConf(ColumnFamilyDescriptor family)` Creates the cache config.
`private HFileContext`	`createFileContext(Compression.Algorithm compression, boolean includeMVCCReadpoint, boolean includesTag, Encryption.Context cryptoContext)`
`StoreFlushContext`	`createFlushContext(long cacheFlushId, FlushLifeCycleTracker tracker)`
`protected KeyValueScanner`	`createScanner(Scan scan, ScanInfo scanInfo, NavigableSet<byte[]> targetCols, long readPt)`
`protected StoreEngine<?,?,?,?>`	`createStoreEngine(HStore store, org.apache.hadoop.conf.Configuration conf, CellComparator kvComparator)` Creates the store engine configured for the given Store.
`protected HStoreFile`	`createStoreFileAndReader(org.apache.hadoop.fs.Path p)`
`private HStoreFile`	`createStoreFileAndReader(StoreFileInfo info)`
`StoreFileWriter`	`createWriterInTmp(long maxKeyCount, Compression.Algorithm compression, boolean isCompaction, boolean includeMVCCReadpoint, boolean includesTag, boolean shouldDropBehind)`
`void`	`deleteChangedReaderObserver(ChangedReadersObserver o)`
`void`	`deregisterChildren(ConfigurationManager manager)` Needs to be called to deregister the children from the manager.
`static long`	`determineTTLFromFamily(ColumnFamilyDescriptor family)`
`protected List<HStoreFile>`	`doCompaction(CompactionRequestImpl cr, Collection<HStoreFile> filesToCompact, User user, long compactionStartTime, List<org.apache.hadoop.fs.Path> newFiles)`
`private void`	`finishCompactionRequest(CompactionRequestImpl cr)`
`protected List<org.apache.hadoop.fs.Path>`	`flushCache(long logCacheFlushId, MemStoreSnapshot snapshot, MonitoredTask status, ThroughputController throughputController, FlushLifeCycleTracker tracker)` Write out current snapshot.
`OptionalDouble`	`getAvgStoreFileAge()`
`long`	`getBlockingFileCount()` The number of files required before flushes for this store will be blocked.
`static int`	`getBytesPerChecksum(org.apache.hadoop.conf.Configuration conf)` Returns the configured bytesPerChecksum value.
`CacheConfig`	`getCacheConfig()` Used for tests.
`static ChecksumType`	`getChecksumType(org.apache.hadoop.conf.Configuration conf)` Returns the configured checksum algorithm.
`static int`	`getCloseCheckInterval()`
`ColumnFamilyDescriptor`	`getColumnFamilyDescriptor()`
`String`	`getColumnFamilyName()`
`long`	`getCompactedCellsCount()`
`long`	`getCompactedCellsSize()`
`Collection<HStoreFile>`	`getCompactedFiles()`
`int`	`getCompactedFilesCount()`
`long`	`getCompactionCheckMultiplier()`
`double`	`getCompactionPressure()` This value can represent the degree of emergency of compaction for this store.
`CompactionProgress`	`getCompactionProgress()` getter for CompactionProgress object
`int`	`getCompactPriority()`
`CellComparator`	`getComparator()`
`RegionCoprocessorHost`	`getCoprocessorHost()`
`int`	`getCurrentParallelPutCount()`
`HFileDataBlockEncoder`	`getDataBlockEncoder()`
`org.apache.hadoop.fs.FileSystem`	`getFileSystem()`
`MemStoreSize`	`getFlushableSize()`
`long`	`getFlushedCellsCount()`
`long`	`getFlushedCellsSize()`
`long`	`getFlushedOutputFileSize()`
`long`	`getHFilesSize()`
`HRegion`	`getHRegion()`
`long`	`getLastCompactSize()`
`long`	`getMajorCompactedCellsCount()`
`long`	`getMajorCompactedCellsSize()`
`OptionalLong`	`getMaxMemStoreTS()`
`OptionalLong`	`getMaxSequenceId()`
`OptionalLong`	`getMaxStoreFileAge()`
`private MemStore`	`getMemstore()`
`long`	`getMemStoreFlushSize()`
`MemStoreSize`	`getMemStoreSize()`
`OptionalLong`	`getMinStoreFileAge()`
`long`	`getNumHFiles()`
`long`	`getNumReferenceFiles()`
`protected OffPeakHours`	`getOffPeakHours()`
`HRegionFileSystem`	`getRegionFileSystem()`
`RegionInfo`	`getRegionInfo()`
`ScanInfo`	`getScanInfo()`
`KeyValueScanner`	`getScanner(Scan scan, NavigableSet<byte[]> targetCols, long readPt)` Return a scanner for both the memstore and the HStore files.
`List<KeyValueScanner>`	`getScanners(boolean cacheBlocks, boolean isGet, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, byte[] stopRow, long readPt)` Get all scanners with no filtering based on TTL (that happens further down the line).
`List<KeyValueScanner>`	`getScanners(boolean cacheBlocks, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, long readPt)` Get all scanners with no filtering based on TTL (that happens further down the line).
`List<KeyValueScanner>`	`getScanners(List<HStoreFile> files, boolean cacheBlocks, boolean isGet, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, byte[] stopRow, long readPt, boolean includeMemstoreScanner)` Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).
`List<KeyValueScanner>`	`getScanners(List<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, long readPt, boolean includeMemstoreScanner)` Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).
`long`	`getSize()`
`long`	`getSmallestReadPoint()`
`MemStoreSize`	`getSnapshotSize()`
`Optional<byte[]>`	`getSplitPoint()` Determines if Store should be split.
`StoreEngine<?,?,?,?>`	`getStoreEngine()` Returns the StoreEngine that is backing this concrete implementation of Store.
`private LongStream`	`getStoreFileAgeStream()`
`private long`	`getStorefileFieldSize(HStoreFile file, ToLongFunction<StoreFileReader> f)`
`Collection<HStoreFile>`	`getStorefiles()`
`int`	`getStorefilesCount()`
`private long`	`getStorefilesFieldSize(ToLongFunction<StoreFileReader> f)`
`(package private) long`	`getStoreFileSize(HStoreFile file)` Computes the length of a store file without succumbing to any errors along the way.
`long`	`getStorefilesRootLevelIndexSize()`
`long`	`getStorefilesSize()`
`private long`	`getStorefilesSize(Collection<HStoreFile> files, Predicate<HStoreFile> predicate)`
`long`	`getStoreFileTtl()`
`static org.apache.hadoop.fs.Path`	`getStoreHomedir(org.apache.hadoop.fs.Path tabledir, RegionInfo hri, byte[] family)` Deprecated.
`static org.apache.hadoop.fs.Path`	`getStoreHomedir(org.apache.hadoop.fs.Path tabledir, String encodedName, byte[] family)` Deprecated.
`long`	`getStoreSizeUncompressed()`
`TableName`	`getTableName()`
`private long`	`getTotalSize(Collection<HStoreFile> sfs)`
`long`	`getTotalStaticBloomSize()` Returns the total byte size of all Bloom filter bit arrays.
`long`	`getTotalStaticIndexSize()` Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.
`private long`	`getTotalUncompressedBytes(List<HStoreFile> files)`
`boolean`	`hasReferences()`
`boolean`	`hasTooManyStoreFiles()`
`long`	`heapSize()`
`boolean`	`isPrimaryReplicaStore()`
`boolean`	`isSloppyMemStore()`
`private List<HStoreFile>`	`loadStoreFiles(boolean warmup)` Creates an unsorted list of StoreFile loaded in parallel from the given directory.
`private void`	`logCompactionEndMessage(CompactionRequestImpl cr, List<HStoreFile> sfs, long now, long compactionStartTime)` Log a very elaborate compaction completion message.
`private List<HStoreFile>`	`moveCompactedFilesIntoPlace(CompactionRequestImpl cr, List<org.apache.hadoop.fs.Path> newFiles, User user)`
`(package private) HStoreFile`	`moveFileIntoPlace(org.apache.hadoop.fs.Path newFile)`
`boolean`	`needsCompaction()` See if there's too much store files in this store
`private void`	`notifyChangedReadersObservers(List<HStoreFile> sfs)` Notify all observers that set of Readers has changed.
`void`	`onConfigurationChange(org.apache.hadoop.conf.Configuration conf)` This method would be called by the `ConfigurationManager` object when the `Configuration` object is reloaded from disk.
`private List<HStoreFile>`	`openStoreFiles(Collection<StoreFileInfo> files, boolean warmup)`
`void`	`postSnapshotOperation()` Perform tasks needed after the completion of snapshot operation.
`Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>`	`preBulkLoadHFile(String srcPathStr, long seqNum)` This method should only be called from Region.
`Long`	`preFlushSeqIDEstimation()`
`void`	`preSnapshotOperation()` Sets the store up for a region level snapshot operation.
`List<KeyValueScanner>`	`recreateScanners(List<KeyValueScanner> currentFileScanners, boolean cacheBlocks, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, long readPt, boolean includeMemstoreScanner)` Recreates the scanners on the current list of active store file scanners
`void`	`refreshStoreFiles()` Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available.
`void`	`refreshStoreFiles(Collection<String> newFiles)` Replaces the store files that the store has with the given files.
`private void`	`refreshStoreFilesInternal(Collection<StoreFileInfo> newFiles)` Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available.
`void`	`registerChildren(ConfigurationManager manager)` Needs to be called to register the children to the manager.
`private void`	`removeCompactedfiles(Collection<HStoreFile> compactedfiles)` Archives and removes the compacted files
`private void`	`removeUnneededFiles()`
`(package private) void`	`replaceStoreFiles(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> result)`
`void`	`replayCompactionMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor compaction, boolean pickCompactionFiles, boolean removeFiles)` Call to complete a compaction.
`(package private) void`	`reportArchivedFilesForQuota(List<? extends StoreFile> archivedFiles, List<Long> fileSizes)`
`Optional<CompactionContext>`	`requestCompaction()`
`Optional<CompactionContext>`	`requestCompaction(int priority, CompactionLifeCycleTracker tracker, User user)`
`(package private) void`	`setDataBlockEncoderInTest(HFileDataBlockEncoder blockEncoder)` Should be used only in tests.
`(package private) void`	`setScanInfo(ScanInfo scanInfo)` Set scan info, used by test
`boolean`	`shouldPerformMajorCompaction()` Tests whether we should run a major compaction.
`(package private) void`	`snapshot()` Snapshot this stores memstore.
`void`	`startReplayingFromWAL()` This message intends to inform the MemStore that next coming updates are going to be part of the replaying edits from WAL
`void`	`stopReplayingFromWAL()` This message intends to inform the MemStore that the replaying edits from WAL are done
`boolean`	`throttleCompaction(long compactionSize)`
`long`	`timeOfOldestEdit()` When was the last edit done in the memstore
`String`	`toString()`
`void`	`triggerMajorCompaction()`
`(package private) void`	`updateSpaceQuotaAfterFileReplacement(RegionSizeStore sizeStore, RegionInfo regionInfo, Collection<HStoreFile> oldFiles, Collection<HStoreFile> newFiles)` Updates the space quota usage for this region, removing the size for files compacted away and adding in the size for new files.
`private boolean`	`updateStorefiles(List<HStoreFile> sfs, long snapshotId)` Change storeFiles adding into place the Reader produced by this new flush.
`void`	`upsert(Iterable<Cell> cells, long readpoint, MemStoreSizing memstoreSizing)` Adds or replaces the specified KeyValues.
`private void`	`validateStoreFile(org.apache.hadoop.fs.Path path)` Validates a store file by opening and closing it.
`(package private) int`	`versionsToReturn(int wantedVersions)`
`private void`	`writeCompactionWalRecord(Collection<HStoreFile> filesCompacted, Collection<HStoreFile> newFiles)` Writes the compaction WAL record.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - MEMSTORE_CLASS_NAME
```
public static final String MEMSTORE_CLASS_NAME
```
    See Also:
    
    Constant Field Values
  - COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY
```
public static final String COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY
```
    See Also:
    
    Constant Field Values
  - BLOCKING_STOREFILES_KEY
```
public static final String BLOCKING_STOREFILES_KEY
```
    See Also:
    
    Constant Field Values
  - BLOCK_STORAGE_POLICY_KEY
```
public static final String BLOCK_STORAGE_POLICY_KEY
```
    See Also:
    
    Constant Field Values
  - DEFAULT_BLOCK_STORAGE_POLICY
```
public static final String DEFAULT_BLOCK_STORAGE_POLICY
```
    See Also:
    
    Constant Field Values
  - DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER
```
public static final int DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER
```
    See Also:
    
    Constant Field Values
  - DEFAULT_BLOCKING_STOREFILE_COUNT
```
public static final int DEFAULT_BLOCKING_STOREFILE_COUNT
```
    See Also:
    
    Constant Field Values
  - LOG
```
private static final org.slf4j.Logger LOG
```
  - memstore
```
protected final MemStore memstore
```
  - region
```
protected final HRegion region
```
  - family
```
private final ColumnFamilyDescriptor family
```
  - fs
```
private final HRegionFileSystem fs
```
  - conf
```
protected org.apache.hadoop.conf.Configuration conf
```
  - cacheConf
```
protected CacheConfig cacheConf
```
  - lastCompactSize
```
private long lastCompactSize
```
  - forceMajor
```
volatile boolean forceMajor
```
  - closeCheckInterval
```
static int closeCheckInterval
```
  - storeSize
```
private AtomicLong storeSize
```
  - totalUncompressedBytes
```
private AtomicLong totalUncompressedBytes
```
  - lock
```
final ReentrantReadWriteLock lock
```
    RWLock for store operations. Locked in shared mode when the list of component stores is looked at: - all reads/writes to table data - checking for split Locked in exclusive mode when the list of component stores is modified: - closing - completing a compaction
  - archiveLock
```
final ReentrantLock archiveLock
```
    Lock specific to archiving compacted store files. This avoids races around the combination of retrieving the list of compacted files and moving them to the archive directory. Since this is usually a background process (other than on close), we don't want to handle this with the store write lock, which would block readers and degrade performance. Locked by: - CompactedHFilesDispatchHandler via closeAndArchiveCompactedFiles() - close()
  - verifyBulkLoads
```
private final boolean verifyBulkLoads
```
  - currentParallelPutCount
```
private final AtomicInteger currentParallelPutCount
```
    Use this counter to track concurrent puts. If TRACE-log is enabled, if we are over the threshold set by hbase.region.store.parallel.put.print.threshold (Default is 50) we will log a message that identifies the Store experience this high-level of concurrency.
  - parallelPutCountPrintThreshold
```
private final int parallelPutCountPrintThreshold
```
  - scanInfo
```
private ScanInfo scanInfo
```
  - filesCompacting
```
private final List<HStoreFile> filesCompacting
```
  - changedReaderObservers
```
private final Set<ChangedReadersObserver> changedReaderObservers
```
  - blocksize
```
protected final int blocksize
```
  - dataBlockEncoder
```
private HFileDataBlockEncoder dataBlockEncoder
```
  - checksumType
```
protected ChecksumType checksumType
```
    Checksum configuration
  - bytesPerChecksum
```
protected int bytesPerChecksum
```
  - comparator
```
protected final CellComparator comparator
```
  - storeEngine
```
final StoreEngine<?,?,?,?> storeEngine
```
  - offPeakCompactionTracker
```
private static final AtomicBoolean offPeakCompactionTracker
```
  - offPeakHours
```
private volatile OffPeakHours offPeakHours
```
  - DEFAULT_FLUSH_RETRIES_NUMBER
```
private static final int DEFAULT_FLUSH_RETRIES_NUMBER
```
    See Also:
    
    Constant Field Values
  - flushRetriesNumber
```
private int flushRetriesNumber
```
  - pauseTime
```
private int pauseTime
```
  - blockingFileCount
```
private long blockingFileCount
```
  - compactionCheckMultiplier
```
private int compactionCheckMultiplier
```
  - cryptoContext
```
protected Encryption.Context cryptoContext
```
  - flushedCellsCount
```
private AtomicLong flushedCellsCount
```
  - compactedCellsCount
```
private AtomicLong compactedCellsCount
```
  - majorCompactedCellsCount
```
private AtomicLong majorCompactedCellsCount
```
  - flushedCellsSize
```
private AtomicLong flushedCellsSize
```
  - flushedOutputFileSize
```
private AtomicLong flushedOutputFileSize
```
  - compactedCellsSize
```
private AtomicLong compactedCellsSize
```
  - majorCompactedCellsSize
```
private AtomicLong majorCompactedCellsSize
```
  - FIXED_OVERHEAD
```
public static final long FIXED_OVERHEAD
```
  - DEEP_OVERHEAD
```
public static final long DEEP_OVERHEAD
```
- Constructor Detail
  - HStore
```
protected HStore(HRegion region,
                 ColumnFamilyDescriptor family,
                 org.apache.hadoop.conf.Configuration confParam,
                 boolean warmup)
          throws IOException
```
    Constructor
    
    Parameters:
    
    region -
    
    family - HColumnDescriptor for this column
    
    confParam - configuration object failed. Can be null.
    
    Throws:
    
    IOException
- Method Detail
  - getMemstore
```
private MemStore getMemstore()
```
    Returns:
    
    MemStore Instance to use in this store.
  - createCacheConf
```
protected void createCacheConf(ColumnFamilyDescriptor family)
```
    Creates the cache config.
    
    Parameters:
    
    family - The current column family.
  - createStoreEngine
```
protected StoreEngine<?,?,?,?> createStoreEngine(HStore store,
                                                 org.apache.hadoop.conf.Configuration conf,
                                                 CellComparator kvComparator)
                                          throws IOException
```
    Creates the store engine configured for the given Store.
    
    Parameters:
    
    store - The store. An unfortunate dependency needed due to it being passed to coprocessors via the compactor.
    
    conf - Store configuration.
    
    kvComparator - KVComparator for storeFileManager.
    
    Returns:
    
    StoreEngine to use.
    
    Throws:
    
    IOException
  - determineTTLFromFamily
```
public static long determineTTLFromFamily(ColumnFamilyDescriptor family)
```
    Parameters:
    
    family -
    
    Returns:
    
    TTL in seconds of the specified family
  - getColumnFamilyName
```
public String getColumnFamilyName()
```
    Specified by:
    
    getColumnFamilyName in interface Store
  - getTableName
```
public TableName getTableName()
```
    Specified by:
    
    getTableName in interface Store
  - getFileSystem
```
public org.apache.hadoop.fs.FileSystem getFileSystem()
```
    Specified by:
    
    getFileSystem in interface Store
  - getRegionFileSystem
```
public HRegionFileSystem getRegionFileSystem()
```
  - getStoreFileTtl
```
public long getStoreFileTtl()
```
    Specified by:
    
    getStoreFileTtl in interface StoreConfigInformation
    
    Returns:
    
    Gets the cf-specific time-to-live for store files.
  - getMemStoreFlushSize
```
public long getMemStoreFlushSize()
```
    Specified by:
    
    getMemStoreFlushSize in interface StoreConfigInformation
    
    Returns:
    
    Gets the Memstore flush size for the region that this store works with.
  - getFlushableSize
```
public MemStoreSize getFlushableSize()
```
    Specified by:
    
    getFlushableSize in interface Store
    
    Returns:
    
    The amount of memory we could flush from this memstore; usually this is equal to Store.getMemStoreSize() unless we are carrying snapshots and then it will be the size of outstanding snapshots.
  - getSnapshotSize
```
public MemStoreSize getSnapshotSize()
```
    Specified by:
    
    getSnapshotSize in interface Store
    
    Returns:
    
    size of the memstore snapshot
  - getCompactionCheckMultiplier
```
public long getCompactionCheckMultiplier()
```
    Specified by:
    
    getCompactionCheckMultiplier in interface StoreConfigInformation
    
    Returns:
    
    Gets the cf-specific compaction check frequency multiplier. The need for compaction (outside of normal checks during flush, open, etc.) will be ascertained every multiplier * HConstants.THREAD_WAKE_FREQUENCY milliseconds.
  - getBlockingFileCount
```
public long getBlockingFileCount()
```
    Description copied from interface: StoreConfigInformation
    
    The number of files required before flushes for this store will be blocked.
    
    Specified by:
    
    getBlockingFileCount in interface StoreConfigInformation
  - getBytesPerChecksum
```
public static int getBytesPerChecksum(org.apache.hadoop.conf.Configuration conf)
```
    Returns the configured bytesPerChecksum value.
    
    Parameters:
    
    conf - The configuration
    
    Returns:
    
    The bytesPerChecksum that is set in the configuration
  - getChecksumType
```
public static ChecksumType getChecksumType(org.apache.hadoop.conf.Configuration conf)
```
    Returns the configured checksum algorithm.
    
    Parameters:
    
    conf - The configuration
    
    Returns:
    
    The checksum algorithm that is set in the configuration
  - getCloseCheckInterval
```
public static int getCloseCheckInterval()
```
    Returns:
    
    how many bytes to write between status checks
  - getColumnFamilyDescriptor
```
public ColumnFamilyDescriptor getColumnFamilyDescriptor()
```
    Specified by:
    
    getColumnFamilyDescriptor in interface Store
  - getMaxSequenceId
```
public OptionalLong getMaxSequenceId()
```
    Specified by:
    
    getMaxSequenceId in interface Store
    
    Returns:
    
    The maximum sequence id in all store files.
  - getMaxMemStoreTS
```
public OptionalLong getMaxMemStoreTS()
```
    Specified by:
    
    getMaxMemStoreTS in interface Store
    
    Returns:
    
    The maximum memstoreTS in all store files.
  - getStoreHomedir
```
@Deprecated
public static org.apache.hadoop.fs.Path getStoreHomedir(org.apache.hadoop.fs.Path tabledir,
                                                                    RegionInfo hri,
                                                                    byte[] family)
```
    Deprecated.
    
    Parameters:
    
    tabledir - Path to where the table is being stored
    
    hri - RegionInfo for the region.
    
    family - ColumnFamilyDescriptor describing the column family
    
    Returns:
    
    Path to family/Store home directory.
  - getStoreHomedir
```
@Deprecated
public static org.apache.hadoop.fs.Path getStoreHomedir(org.apache.hadoop.fs.Path tabledir,
                                                                    String encodedName,
                                                                    byte[] family)
```
    Deprecated.
    
    Parameters:
    
    tabledir - Path to where the table is being stored
    
    encodedName - Encoded region name.
    
    family - ColumnFamilyDescriptor describing the column family
    
    Returns:
    
    Path to family/Store home directory.
  - getDataBlockEncoder
```
public HFileDataBlockEncoder getDataBlockEncoder()
```
    Returns:
    
    the data block encoder
  - setDataBlockEncoderInTest
```
void setDataBlockEncoderInTest(HFileDataBlockEncoder blockEncoder)
```
    Should be used only in tests.
    
    Parameters:
    
    blockEncoder - the block delta encoder to use
  - loadStoreFiles
```
private List<HStoreFile> loadStoreFiles(boolean warmup)
                                 throws IOException
```
    Creates an unsorted list of StoreFile loaded in parallel from the given directory.
    
    Throws:
    
    IOException
  - openStoreFiles
```
private List<HStoreFile> openStoreFiles(Collection<StoreFileInfo> files,
                                        boolean warmup)
                                 throws IOException
```
    Throws:
    
    IOException
  - refreshStoreFiles
```
public void refreshStoreFiles()
                       throws IOException
```
    Description copied from interface: Store
    
    Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available. Mainly used by secondary region replicas to keep up to date with the primary region files.
    
    Specified by:
    
    refreshStoreFiles in interface Store
    
    Throws:
    
    IOException
  - refreshStoreFiles
```
public void refreshStoreFiles(Collection<String> newFiles)
                       throws IOException
```
    Replaces the store files that the store has with the given files. Mainly used by secondary region replicas to keep up to date with the primary region files.
    
    Throws:
    
    IOException
  - refreshStoreFilesInternal
```
private void refreshStoreFilesInternal(Collection<StoreFileInfo> newFiles)
                                throws IOException
```
    Checks the underlying store files, and opens the files that have not been opened, and removes the store file readers for store files no longer available. Mainly used by secondary region replicas to keep up to date with the primary region files.
    
    Throws:
    
    IOException
  - createStoreFileAndReader
```
protected HStoreFile createStoreFileAndReader(org.apache.hadoop.fs.Path p)
                                       throws IOException
```
    Throws:
    
    IOException
  - createStoreFileAndReader
```
private HStoreFile createStoreFileAndReader(StoreFileInfo info)
                                     throws IOException
```
    Throws:
    
    IOException
  - startReplayingFromWAL
```
public void startReplayingFromWAL()
```
    This message intends to inform the MemStore that next coming updates are going to be part of the replaying edits from WAL
  - stopReplayingFromWAL
```
public void stopReplayingFromWAL()
```
    This message intends to inform the MemStore that the replaying edits from WAL are done
  - add
```
public void add(Cell cell,
                MemStoreSizing memstoreSizing)
```
    Adds a value to the memstore
  - add
```
public void add(Iterable<Cell> cells,
                MemStoreSizing memstoreSizing)
```
    Adds the specified value to the memstore
  - timeOfOldestEdit
```
public long timeOfOldestEdit()
```
    Description copied from interface: Store
    
    When was the last edit done in the memstore
    
    Specified by:
    
    timeOfOldestEdit in interface Store
  - getStorefiles
```
public Collection<HStoreFile> getStorefiles()
```
    Specified by:
    
    getStorefiles in interface Store
    
    Returns:
    
    All store files.
  - getCompactedFiles
```
public Collection<HStoreFile> getCompactedFiles()
```
    Specified by:
    
    getCompactedFiles in interface Store
  - assertBulkLoadHFileOk
```
public void assertBulkLoadHFileOk(org.apache.hadoop.fs.Path srcPath)
                           throws IOException
```
    This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.
    
    Throws:
    
    IOException
  - preBulkLoadHFile
```
public Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path> preBulkLoadHFile(String srcPathStr,
                                                                                  long seqNum)
                                                                           throws IOException
```
    This method should only be called from Region. It is assumed that the ranges of values in the HFile fit within the stores assigned region. (assertBulkLoadHFileOk checks this)
    
    Parameters:
    
    srcPathStr -
    
    seqNum - sequence Id associated with the HFile
    
    Throws:
    
    IOException
  - bulkLoadHFile
```
public org.apache.hadoop.fs.Path bulkLoadHFile(byte[] family,
                                               String srcPathStr,
                                               org.apache.hadoop.fs.Path dstPath)
                                        throws IOException
```
    Throws:
    
    IOException
  - bulkLoadHFile
```
public void bulkLoadHFile(StoreFileInfo fileInfo)
                   throws IOException
```
    Throws:
    
    IOException
  - bulkLoadHFile
```
private void bulkLoadHFile(HStoreFile sf)
                    throws IOException
```
    Throws:
    
    IOException
  - close
```
public org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> close()
                                                                                            throws IOException
```
    Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.
    
    Returns:
    
    the StoreFiles that were previously being used.
    
    Throws:
    
    IOException - on failure
  - snapshot
```
void snapshot()
```
    Snapshot this stores memstore. Call before running flushCache(long, MemStoreSnapshot, MonitoredTask, ThroughputController, FlushLifeCycleTracker) so it has some work to do.
  - flushCache
```
protected List<org.apache.hadoop.fs.Path> flushCache(long logCacheFlushId,
                                                     MemStoreSnapshot snapshot,
                                                     MonitoredTask status,
                                                     ThroughputController throughputController,
                                                     FlushLifeCycleTracker tracker)
                                              throws IOException
```
    Write out current snapshot. Presumes snapshot() has been called previously.
    
    Parameters:
    
    logCacheFlushId - flush sequence number
    
    snapshot -
    
    status -
    
    throughputController -
    
    Returns:
    
    The path name of the tmp file to which the store was flushed
    
    Throws:
    
    IOException - if exception occurs during process
  - commitFile
```
private HStoreFile commitFile(org.apache.hadoop.fs.Path path,
                              long logCacheFlushId,
                              MonitoredTask status)
                       throws IOException
```
    Parameters:
    
    path - The pathname of the tmp file into which the store was flushed
    
    logCacheFlushId -
    
    status -
    
    Returns:
    
    store file created.
    
    Throws:
    
    IOException
  - createWriterInTmp
```
public StoreFileWriter createWriterInTmp(long maxKeyCount,
                                         Compression.Algorithm compression,
                                         boolean isCompaction,
                                         boolean includeMVCCReadpoint,
                                         boolean includesTag,
                                         boolean shouldDropBehind)
                                  throws IOException
```
    Parameters:
    
    maxKeyCount -
    
    compression - Compression algorithm to use
    
    isCompaction - whether we are creating a new file in a compaction
    
    includeMVCCReadpoint - - whether to include MVCC or not
    
    includesTag - - includesTag or not
    
    Returns:
    
    Writer for a new StoreFile in the tmp dir.
    
    Throws:
    
    IOException
  - createFileContext
```
private HFileContext createFileContext(Compression.Algorithm compression,
                                       boolean includeMVCCReadpoint,
                                       boolean includesTag,
                                       Encryption.Context cryptoContext)
```
  - getTotalSize
```
private long getTotalSize(Collection<HStoreFile> sfs)
```
  - updateStorefiles
```
private boolean updateStorefiles(List<HStoreFile> sfs,
                                 long snapshotId)
                          throws IOException
```
    Change storeFiles adding into place the Reader produced by this new flush.
    
    Parameters:
    
    sfs - Store files
    
    snapshotId -
    
    Returns:
    
    Whether compaction is required.
    
    Throws:
    
    IOException
  - notifyChangedReadersObservers
```
private void notifyChangedReadersObservers(List<HStoreFile> sfs)
                                    throws IOException
```
    Notify all observers that set of Readers has changed.
    
    Throws:
    
    IOException
  - getScanners
```
public List<KeyValueScanner> getScanners(boolean cacheBlocks,
                                         boolean isGet,
                                         boolean usePread,
                                         boolean isCompaction,
                                         ScanQueryMatcher matcher,
                                         byte[] startRow,
                                         byte[] stopRow,
                                         long readPt)
                                  throws IOException
```
    Get all scanners with no filtering based on TTL (that happens further down the line).
    
    Parameters:
    
    cacheBlocks - cache the blocks or not
    
    usePread - true to use pread, false if not
    
    isCompaction - true if the scanner is created for compaction
    
    matcher - the scan query matcher
    
    startRow - the start row
    
    stopRow - the stop row
    
    readPt - the read point of the current scan
    
    Returns:
    
    all scanners for this store
    
    Throws:
    
    IOException
  - getScanners
```
public List<KeyValueScanner> getScanners(boolean cacheBlocks,
                                         boolean usePread,
                                         boolean isCompaction,
                                         ScanQueryMatcher matcher,
                                         byte[] startRow,
                                         boolean includeStartRow,
                                         byte[] stopRow,
                                         boolean includeStopRow,
                                         long readPt)
                                  throws IOException
```
    Get all scanners with no filtering based on TTL (that happens further down the line).
    
    Parameters:
    
    cacheBlocks - cache the blocks or not
    
    usePread - true to use pread, false if not
    
    isCompaction - true if the scanner is created for compaction
    
    matcher - the scan query matcher
    
    startRow - the start row
    
    includeStartRow - true to include start row, false if not
    
    stopRow - the stop row
    
    includeStopRow - true to include stop row, false if not
    
    readPt - the read point of the current scan
    
    Returns:
    
    all scanners for this store
    
    Throws:
    
    IOException
  - clearAndClose
```
private static void clearAndClose(List<KeyValueScanner> scanners)
```
  - getScanners
```
public List<KeyValueScanner> getScanners(List<HStoreFile> files,
                                         boolean cacheBlocks,
                                         boolean isGet,
                                         boolean usePread,
                                         boolean isCompaction,
                                         ScanQueryMatcher matcher,
                                         byte[] startRow,
                                         byte[] stopRow,
                                         long readPt,
                                         boolean includeMemstoreScanner)
                                  throws IOException
```
    Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).
    
    Parameters:
    
    files - the list of files on which the scanners has to be created
    
    cacheBlocks - cache the blocks or not
    
    usePread - true to use pread, false if not
    
    isCompaction - true if the scanner is created for compaction
    
    matcher - the scan query matcher
    
    startRow - the start row
    
    stopRow - the stop row
    
    readPt - the read point of the current scan
    
    includeMemstoreScanner - true if memstore has to be included
    
    Returns:
    
    scanners on the given files and on the memstore if specified
    
    Throws:
    
    IOException
  - getScanners
```
public List<KeyValueScanner> getScanners(List<HStoreFile> files,
                                         boolean cacheBlocks,
                                         boolean usePread,
                                         boolean isCompaction,
                                         ScanQueryMatcher matcher,
                                         byte[] startRow,
                                         boolean includeStartRow,
                                         byte[] stopRow,
                                         boolean includeStopRow,
                                         long readPt,
                                         boolean includeMemstoreScanner)
                                  throws IOException
```
    Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).
    
    Parameters:
    
    files - the list of files on which the scanners has to be created
    
    cacheBlocks - ache the blocks or not
    
    usePread - true to use pread, false if not
    
    isCompaction - true if the scanner is created for compaction
    
    matcher - the scan query matcher
    
    startRow - the start row
    
    includeStartRow - true to include start row, false if not
    
    stopRow - the stop row
    
    includeStopRow - true to include stop row, false if not
    
    readPt - the read point of the current scan
    
    includeMemstoreScanner - true if memstore has to be included
    
    Returns:
    
    scanners on the given files and on the memstore if specified
    
    Throws:
    
    IOException
  - addChangedReaderObserver
```
public void addChangedReaderObserver(ChangedReadersObserver o)
```
    Parameters:
    
    o - Observer who wants to know about changes in set of Readers
  - deleteChangedReaderObserver
```
public void deleteChangedReaderObserver(ChangedReadersObserver o)
```
    Parameters:
    
    o - Observer no longer interested in changes in set of Readers.
  - compact
```
public List<HStoreFile> compact(CompactionContext compaction,
                                ThroughputController throughputController,
                                User user)
                         throws IOException
```
    Compact the StoreFiles. This method may take some time, so the calling thread must be able to block for long periods.
    During this time, the Store can work as usual, getting values from StoreFiles and writing new StoreFiles from the memstore. Existing StoreFiles are not destroyed until the new compacted StoreFile is completely written-out to disk.
    The compactLock prevents multiple simultaneous compactions. The structureLock prevents us from interfering with other write operations.
    We don't want to hold the structureLock for the whole time, as a compact() can be lengthy and we want to allow cache-flushes during this period.
    Compaction event should be idempotent, since there is no IO Fencing for the region directory in hdfs. A region server might still try to complete the compaction after it lost the region. That is why the following events are carefully ordered for a compaction: 1. Compaction writes new files under region/.tmp directory (compaction output) 2. Compaction atomically moves the temporary file under region directory 3. Compaction appends a WAL edit containing the compaction input and output files. Forces sync on WAL. 4. Compaction deletes the input files from the region directory. Failure conditions are handled like this: - If RS fails before 2, compaction wont complete. Even if RS lives on and finishes the compaction later, it will only write the new data file to the region directory. Since we already have this data, this will be idempotent but we will have a redundant copy of the data. - If RS fails between 2 and 3, the region will have a redundant copy of the data. The RS that failed won't be able to finish snyc() for WAL because of lease recovery in WAL. - If RS fails after 3, the region region server who opens the region will pick up the the compaction marker from the WAL and replay it by removing the compaction input files. Failed RS can also attempt to delete those files, but the operation will be idempotent See HBASE-2231 for details.
    
    Parameters:
    
    compaction - compaction details obtained from requestCompaction()
    
    Returns:
    
    Storefile we compacted into or null if we failed or opted out early.
    
    Throws:
    
    IOException
  - doCompaction
```
protected List<HStoreFile> doCompaction(CompactionRequestImpl cr,
                                        Collection<HStoreFile> filesToCompact,
                                        User user,
                                        long compactionStartTime,
                                        List<org.apache.hadoop.fs.Path> newFiles)
                                 throws IOException
```
    Throws:
    
    IOException
  - moveCompactedFilesIntoPlace
```
private List<HStoreFile> moveCompactedFilesIntoPlace(CompactionRequestImpl cr,
                                                     List<org.apache.hadoop.fs.Path> newFiles,
                                                     User user)
                                              throws IOException
```
    Throws:
    
    IOException
  - moveFileIntoPlace
```
HStoreFile moveFileIntoPlace(org.apache.hadoop.fs.Path newFile)
                      throws IOException
```
    Throws:
    
    IOException
  - writeCompactionWalRecord
```
private void writeCompactionWalRecord(Collection<HStoreFile> filesCompacted,
                                      Collection<HStoreFile> newFiles)
                               throws IOException
```
    Writes the compaction WAL record.
    
    Parameters:
    
    filesCompacted - Files compacted (input).
    
    newFiles - Files from compaction.
    
    Throws:
    
    IOException
  - replaceStoreFiles
```
void replaceStoreFiles(Collection<HStoreFile> compactedFiles,
                       Collection<HStoreFile> result)
                throws IOException
```
    Throws:
    
    IOException
  - updateSpaceQuotaAfterFileReplacement
```
void updateSpaceQuotaAfterFileReplacement(RegionSizeStore sizeStore,
                                          RegionInfo regionInfo,
                                          Collection<HStoreFile> oldFiles,
                                          Collection<HStoreFile> newFiles)
```
    Updates the space quota usage for this region, removing the size for files compacted away and adding in the size for new files.
    
    Parameters:
    
    sizeStore - The object tracking changes in region size for space quotas.
    
    regionInfo - The identifier for the region whose size is being updated.
    
    oldFiles - Files removed from this store's region.
    
    newFiles - Files added to this store's region.
  - logCompactionEndMessage
```
private void logCompactionEndMessage(CompactionRequestImpl cr,
                                     List<HStoreFile> sfs,
                                     long now,
                                     long compactionStartTime)
```
    Log a very elaborate compaction completion message.
    
    Parameters:
    
    cr - Request.
    
    sfs - Resulting files.
    
    compactionStartTime - Start time.
  - replayCompactionMarker
```
public void replayCompactionMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor compaction,
                                   boolean pickCompactionFiles,
                                   boolean removeFiles)
                            throws IOException
```
    Call to complete a compaction. Its for the case where we find in the WAL a compaction that was not finished. We could find one recovering a WAL after a regionserver crash. See HBASE-2231.
    
    Parameters:
    
    compaction -
    
    Throws:
    
    IOException
  - compactRecentForTestingAssumingDefaultPolicy
```
public void compactRecentForTestingAssumingDefaultPolicy(int N)
                                                  throws IOException
```
    This method tries to compact N recent files for testing. Note that because compacting "recent" files only makes sense for some policies, e.g. the default one, it assumes default policy is used. It doesn't use policy, but instead makes a compaction candidate list by itself.
    
    Parameters:
    
    N - Number of files.
    
    Throws:
    
    IOException
  - hasReferences
```
public boolean hasReferences()
```
    Specified by:
    
    hasReferences in interface Store
    
    Returns:
    
    true if the store has any underlying reference files to older HFiles
  - getCompactionProgress
```
public CompactionProgress getCompactionProgress()
```
    getter for CompactionProgress object
    
    Returns:
    
    CompactionProgress object; can be null
  - shouldPerformMajorCompaction
```
public boolean shouldPerformMajorCompaction()
                                     throws IOException
```
    Description copied from interface: Store
    
    Tests whether we should run a major compaction. For example, if the configured major compaction interval is reached.
    
    Specified by:
    
    shouldPerformMajorCompaction in interface Store
    
    Returns:
    
    true if we should run a major compaction.
    
    Throws:
    
    IOException
  - requestCompaction
```
public Optional<CompactionContext> requestCompaction()
                                              throws IOException
```
    Throws:
    
    IOException
  - requestCompaction
```
public Optional<CompactionContext> requestCompaction(int priority,
                                                     CompactionLifeCycleTracker tracker,
                                                     User user)
                                              throws IOException
```
    Throws:
    
    IOException
  - addToCompactingFiles
```
private void addToCompactingFiles(Collection<HStoreFile> filesToAdd)
```
    Adds the files to compacting files. filesCompacting must be locked.
  - removeUnneededFiles
```
private void removeUnneededFiles()
                          throws IOException
```
    Throws:
    
    IOException
  - cancelRequestedCompaction
```
public void cancelRequestedCompaction(CompactionContext compaction)
```
  - finishCompactionRequest
```
private void finishCompactionRequest(CompactionRequestImpl cr)
```
  - validateStoreFile
```
private void validateStoreFile(org.apache.hadoop.fs.Path path)
                        throws IOException
```
    Validates a store file by opening and closing it. In HFileV2 this should not be an expensive operation.
    
    Parameters:
    
    path - the path to the store file
    
    Throws:
    
    IOException
  - completeCompaction
```
protected void completeCompaction(Collection<HStoreFile> compactedFiles)
                           throws IOException
```
    Update counts.
    
    Parameters:
    
    compactedFiles - list of files that were compacted
    
    Throws:
    
    IOException
  - versionsToReturn
```
int versionsToReturn(int wantedVersions)
```
  - canSplit
```
public boolean canSplit()
```
    Description copied from interface: Store
    
    Returns whether this store is splittable, i.e., no reference file in this store.
    
    Specified by:
    
    canSplit in interface Store
  - getSplitPoint
```
public Optional<byte[]> getSplitPoint()
```
    Determines if Store should be split.
  - getLastCompactSize
```
public long getLastCompactSize()
```
    Specified by:
    
    getLastCompactSize in interface Store
    
    Returns:
    
    aggregate size of all HStores used in the last compaction
  - getSize
```
public long getSize()
```
    Specified by:
    
    getSize in interface Store
    
    Returns:
    
    aggregate size of HStore
  - triggerMajorCompaction
```
public void triggerMajorCompaction()
```
  - getScanner
```
public KeyValueScanner getScanner(Scan scan,
                                  NavigableSet<byte[]> targetCols,
                                  long readPt)
                           throws IOException
```
    Return a scanner for both the memstore and the HStore files. Assumes we are not in a compaction.
    
    Parameters:
    
    scan - Scan to apply when scanning the stores
    
    targetCols - columns to scan
    
    Returns:
    
    a scanner over the current key values
    
    Throws:
    
    IOException - on failure
  - createScanner
```
protected KeyValueScanner createScanner(Scan scan,
                                        ScanInfo scanInfo,
                                        NavigableSet<byte[]> targetCols,
                                        long readPt)
                                 throws IOException
```
    Throws:
    
    IOException
  - recreateScanners
```
public List<KeyValueScanner> recreateScanners(List<KeyValueScanner> currentFileScanners,
                                              boolean cacheBlocks,
                                              boolean usePread,
                                              boolean isCompaction,
                                              ScanQueryMatcher matcher,
                                              byte[] startRow,
                                              boolean includeStartRow,
                                              byte[] stopRow,
                                              boolean includeStopRow,
                                              long readPt,
                                              boolean includeMemstoreScanner)
                                       throws IOException
```
    Recreates the scanners on the current list of active store file scanners
    
    Parameters:
    
    currentFileScanners - the current set of active store file scanners
    
    cacheBlocks - cache the blocks or not
    
    usePread - use pread or not
    
    isCompaction - is the scanner for compaction
    
    matcher - the scan query matcher
    
    startRow - the scan's start row
    
    includeStartRow - should the scan include the start row
    
    stopRow - the scan's stop row
    
    includeStopRow - should the scan include the stop row
    
    readPt - the read point of the current scane
    
    includeMemstoreScanner - whether the current scanner should include memstorescanner
    
    Returns:
    
    list of scanners recreated on the current Scanners
    
    Throws:
    
    IOException
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - getStorefilesCount
```
public int getStorefilesCount()
```
    Specified by:
    
    getStorefilesCount in interface Store
    
    Returns:
    
    Count of store files
  - getCompactedFilesCount
```
public int getCompactedFilesCount()
```
    Specified by:
    
    getCompactedFilesCount in interface Store
    
    Returns:
    
    Count of compacted store files
  - getStoreFileAgeStream
```
private LongStream getStoreFileAgeStream()
```
  - getMaxStoreFileAge
```
public OptionalLong getMaxStoreFileAge()
```
    Specified by:
    
    getMaxStoreFileAge in interface Store
    
    Returns:
    
    Max age of store files in this store
  - getMinStoreFileAge
```
public OptionalLong getMinStoreFileAge()
```
    Specified by:
    
    getMinStoreFileAge in interface Store
    
    Returns:
    
    Min age of store files in this store
  - getAvgStoreFileAge
```
public OptionalDouble getAvgStoreFileAge()
```
    Specified by:
    
    getAvgStoreFileAge in interface Store
    
    Returns:
    
    Average age of store files in this store
  - getNumReferenceFiles
```
public long getNumReferenceFiles()
```
    Specified by:
    
    getNumReferenceFiles in interface Store
    
    Returns:
    
    Number of reference files in this store
  - getNumHFiles
```
public long getNumHFiles()
```
    Specified by:
    
    getNumHFiles in interface Store
    
    Returns:
    
    Number of HFiles in this store
  - getStoreSizeUncompressed
```
public long getStoreSizeUncompressed()
```
    Specified by:
    
    getStoreSizeUncompressed in interface Store
    
    Returns:
    
    The size of the store files, in bytes, uncompressed.
  - getStorefilesSize
```
public long getStorefilesSize()
```
    Specified by:
    
    getStorefilesSize in interface Store
    
    Returns:
    
    The size of the store files, in bytes.
  - getHFilesSize
```
public long getHFilesSize()
```
    Specified by:
    
    getHFilesSize in interface Store
    
    Returns:
    
    The size of only the store files which are HFiles, in bytes.
  - getTotalUncompressedBytes
```
private long getTotalUncompressedBytes(List<HStoreFile> files)
```
  - getStorefilesSize
```
private long getStorefilesSize(Collection<HStoreFile> files,
                               Predicate<HStoreFile> predicate)
```
  - getStorefileFieldSize
```
private long getStorefileFieldSize(HStoreFile file,
                                   ToLongFunction<StoreFileReader> f)
```
  - getStorefilesFieldSize
```
private long getStorefilesFieldSize(ToLongFunction<StoreFileReader> f)
```
  - getStorefilesRootLevelIndexSize
```
public long getStorefilesRootLevelIndexSize()
```
    Specified by:
    
    getStorefilesRootLevelIndexSize in interface Store
    
    Returns:
    
    The size of the store file root-level indexes, in bytes.
  - getTotalStaticIndexSize
```
public long getTotalStaticIndexSize()
```
    Description copied from interface: Store
    
    Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.
    
    Specified by:
    
    getTotalStaticIndexSize in interface Store
    
    Returns:
    
    the total size of block indexes in the store
  - getTotalStaticBloomSize
```
public long getTotalStaticBloomSize()
```
    Description copied from interface: Store
    
    Returns the total byte size of all Bloom filter bit arrays. For compound Bloom filters even the Bloom blocks currently not loaded into the block cache are counted.
    
    Specified by:
    
    getTotalStaticBloomSize in interface Store
    
    Returns:
    
    the total size of all Bloom filters in the store
  - getMemStoreSize
```
public MemStoreSize getMemStoreSize()
```
    Specified by:
    
    getMemStoreSize in interface Store
    
    Returns:
    
    The size of this store's memstore.
  - getCompactPriority
```
public int getCompactPriority()
```
    Specified by:
    
    getCompactPriority in interface Store
  - throttleCompaction
```
public boolean throttleCompaction(long compactionSize)
```
  - getHRegion
```
public HRegion getHRegion()
```
  - getCoprocessorHost
```
public RegionCoprocessorHost getCoprocessorHost()
```
  - getRegionInfo
```
public RegionInfo getRegionInfo()
```
    Specified by:
    
    getRegionInfo in interface Store
    
    Returns:
    
    the parent region info hosting this store
  - areWritesEnabled
```
public boolean areWritesEnabled()
```
    Specified by:
    
    areWritesEnabled in interface Store
  - getSmallestReadPoint
```
public long getSmallestReadPoint()
```
    Specified by:
    
    getSmallestReadPoint in interface Store
    
    Returns:
    
    The smallest mvcc readPoint across all the scanners in this region. Writes older than this readPoint, are included in every read operation.
  - upsert
```
public void upsert(Iterable<Cell> cells,
                   long readpoint,
                   MemStoreSizing memstoreSizing)
            throws IOException
```
    Adds or replaces the specified KeyValues.
    For each KeyValue specified, if a cell with the same row, family, and qualifier exists in MemStore, it will be replaced. Otherwise, it will just be inserted to MemStore.
    This operation is atomic on each KeyValue (row/family/qualifier) but not necessarily atomic across all of them.
    
    Parameters:
    
    readpoint - readpoint below which we can safely remove duplicate KVs
    
    Throws:
    
    IOException
  - createFlushContext
```
public StoreFlushContext createFlushContext(long cacheFlushId,
                                            FlushLifeCycleTracker tracker)
```
  - needsCompaction
```
public boolean needsCompaction()
```
    Description copied from interface: Store
    
    See if there's too much store files in this store
    
    Specified by:
    
    needsCompaction in interface Store
    
    Returns:
    
    true if number of store files is greater than the number defined in minFilesToCompact
  - getCacheConfig
```
public CacheConfig getCacheConfig()
```
    Used for tests.
    
    Returns:
    
    cache configuration for this Store.
  - heapSize
```
public long heapSize()
```
    Specified by:
    
    heapSize in interface HeapSize
    
    Returns:
    
    Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.
  - getComparator
```
public CellComparator getComparator()
```
    Specified by:
    
    getComparator in interface Store
  - getScanInfo
```
public ScanInfo getScanInfo()
```
  - setScanInfo
```
void setScanInfo(ScanInfo scanInfo)
```
    Set scan info, used by test
    
    Parameters:
    
    scanInfo - new scan info to use for test
  - hasTooManyStoreFiles
```
public boolean hasTooManyStoreFiles()
```
    Specified by:
    
    hasTooManyStoreFiles in interface Store
    
    Returns:
    
    Whether this store has too many store files.
  - getFlushedCellsCount
```
public long getFlushedCellsCount()
```
    Specified by:
    
    getFlushedCellsCount in interface Store
    
    Returns:
    
    The number of cells flushed to disk
  - getFlushedCellsSize
```
public long getFlushedCellsSize()
```
    Specified by:
    
    getFlushedCellsSize in interface Store
    
    Returns:
    
    The total size of data flushed to disk, in bytes
  - getFlushedOutputFileSize
```
public long getFlushedOutputFileSize()
```
    Specified by:
    
    getFlushedOutputFileSize in interface Store
    
    Returns:
    
    The total size of out output files on disk, in bytes
  - getCompactedCellsCount
```
public long getCompactedCellsCount()
```
    Specified by:
    
    getCompactedCellsCount in interface Store
    
    Returns:
    
    The number of cells processed during minor compactions
  - getCompactedCellsSize
```
public long getCompactedCellsSize()
```
    Specified by:
    
    getCompactedCellsSize in interface Store
    
    Returns:
    
    The total amount of data processed during minor compactions, in bytes
  - getMajorCompactedCellsCount
```
public long getMajorCompactedCellsCount()
```
    Specified by:
    
    getMajorCompactedCellsCount in interface Store
    
    Returns:
    
    The number of cells processed during major compactions
  - getMajorCompactedCellsSize
```
public long getMajorCompactedCellsSize()
```
    Specified by:
    
    getMajorCompactedCellsSize in interface Store
    
    Returns:
    
    The total amount of data processed during major compactions, in bytes
  - getStoreEngine
```
public StoreEngine<?,?,?,?> getStoreEngine()
```
    Returns the StoreEngine that is backing this concrete implementation of Store.
    
    Returns:
    
    Returns the StoreEngine object used internally inside this HStore object.
  - getOffPeakHours
```
protected OffPeakHours getOffPeakHours()
```
  - onConfigurationChange
```
public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)
```
    This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
    
    Specified by:
    
    onConfigurationChange in interface ConfigurationObserver
  - registerChildren
```
public void registerChildren(ConfigurationManager manager)
```
    Needs to be called to register the children to the manager.
    
    Specified by:
    
    registerChildren in interface PropagatingConfigurationObserver
    
    Parameters:
    
    manager - : to register to
  - deregisterChildren
```
public void deregisterChildren(ConfigurationManager manager)
```
    Needs to be called to deregister the children from the manager.
    
    Specified by:
    
    deregisterChildren in interface PropagatingConfigurationObserver
    
    Parameters:
    
    manager - : to deregister from
  - getCompactionPressure
```
public double getCompactionPressure()
```
    Description copied from interface: Store
    This value can represent the degree of emergency of compaction for this store. It should be greater than or equal to 0.0, any value greater than 1.0 means we have too many store files.
    - if getStorefilesCount <= getMinFilesToCompact, return 0.0
    - return (getStorefilesCount - getMinFilesToCompact) / (blockingFileCount - getMinFilesToCompact)
    And for striped stores, we should calculate this value by the files in each stripe separately and return the maximum value.
    It is similar to Store.getCompactPriority() except that it is more suitable to use in a linear formula.
    Specified by:
    
    getCompactionPressure in interface Store
  - isPrimaryReplicaStore
```
public boolean isPrimaryReplicaStore()
```
    Specified by:
    
    isPrimaryReplicaStore in interface Store
  - preSnapshotOperation
```
public void preSnapshotOperation()
```
    Sets the store up for a region level snapshot operation.
    
    See Also:
    
    postSnapshotOperation()
  - postSnapshotOperation
```
public void postSnapshotOperation()
```
    Perform tasks needed after the completion of snapshot operation.
    
    See Also:
    
    preSnapshotOperation()
  - closeAndArchiveCompactedFiles
```
public void closeAndArchiveCompactedFiles()
                                   throws IOException
```
    Closes and archives the compacted files under this store
    
    Throws:
    
    IOException
  - removeCompactedfiles
```
private void removeCompactedfiles(Collection<HStoreFile> compactedfiles)
                           throws IOException
```
    Archives and removes the compacted files
    
    Parameters:
    
    compactedfiles - The compacted files in this store that are not active in reads
    
    Throws:
    
    IOException
  - getStoreFileSize
```
long getStoreFileSize(HStoreFile file)
```
    Computes the length of a store file without succumbing to any errors along the way. If an error is encountered, the implementation returns 0 instead of the actual size.
    
    Parameters:
    
    file - The file to compute the size of.
    
    Returns:
    
    The size in bytes of the provided file.
  - preFlushSeqIDEstimation
```
public Long preFlushSeqIDEstimation()
```
  - isSloppyMemStore
```
public boolean isSloppyMemStore()
```
    Specified by:
    
    isSloppyMemStore in interface Store
    
    Returns:
    
    true if the memstore may need some extra memory space
  - clearCompactedfiles
```
private void clearCompactedfiles(List<HStoreFile> filesToRemove)
                          throws IOException
```
    Throws:
    
    IOException
  - getCurrentParallelPutCount
```
public int getCurrentParallelPutCount()
```
    Specified by:
    
    getCurrentParallelPutCount in interface Store
  - reportArchivedFilesForQuota
```
void reportArchivedFilesForQuota(List<? extends StoreFile> archivedFiles,
                                 List<Long> fileSizes)
```

Class HStore

Nested Class Summary

Field Summary

Fields inherited from interface org.apache.hadoop.hbase.regionserver.Store

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

MEMSTORE_CLASS_NAME

COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY

BLOCKING_STOREFILES_KEY

BLOCK_STORAGE_POLICY_KEY

DEFAULT_BLOCK_STORAGE_POLICY

DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER

DEFAULT_BLOCKING_STOREFILE_COUNT

LOG

memstore

region

family

fs

conf

cacheConf

lastCompactSize

forceMajor

closeCheckInterval

storeSize

totalUncompressedBytes

lock

archiveLock

verifyBulkLoads

currentParallelPutCount

parallelPutCountPrintThreshold

scanInfo

filesCompacting

changedReaderObservers

blocksize

dataBlockEncoder

checksumType

bytesPerChecksum

comparator

storeEngine

offPeakCompactionTracker

offPeakHours

DEFAULT_FLUSH_RETRIES_NUMBER

flushRetriesNumber

pauseTime

blockingFileCount

compactionCheckMultiplier

cryptoContext

flushedCellsCount

compactedCellsCount

majorCompactedCellsCount

flushedCellsSize

flushedOutputFileSize

compactedCellsSize

majorCompactedCellsSize

FIXED_OVERHEAD

DEEP_OVERHEAD

Constructor Detail

HStore

Method Detail

getMemstore

createCacheConf

createStoreEngine

determineTTLFromFamily

getColumnFamilyName

getTableName

getFileSystem

getRegionFileSystem

getStoreFileTtl

getMemStoreFlushSize

getFlushableSize

getSnapshotSize

getCompactionCheckMultiplier

getBlockingFileCount

getBytesPerChecksum

getChecksumType

getCloseCheckInterval

getColumnFamilyDescriptor

getMaxSequenceId