Uses of Class
org.apache.hadoop.hbase.regionserver.HStoreFile
Package
Description
-
Uses of HStoreFile in org.apache.hadoop.hbase.backup
Modifier and TypeFieldDescription(package private) HStoreFile
HFileArchiver.FileableStoreFile.file
Modifier and TypeMethodDescriptionprivate static void
HFileArchiver.archive
(org.apache.hadoop.fs.FileSystem fs, RegionInfo regionInfo, byte[] family, Collection<HStoreFile> compactedFiles, org.apache.hadoop.fs.Path storeArchiveDir) static void
HFileArchiver.archiveRecoveredEdits
(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, RegionInfo regionInfo, byte[] family, Collection<HStoreFile> replayedEdits) Archive recovered edits using existing logic for archiving store files.static void
HFileArchiver.archiveStoreFiles
(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, RegionInfo regionInfo, org.apache.hadoop.fs.Path tableDir, byte[] family, Collection<HStoreFile> compactedFiles) Remove the store files, either by archiving them or outright deletionprivate static void
HFileArchiver.deleteStoreFilesWithoutArchiving
(Collection<HStoreFile> compactedFiles) Just do a simple delete of the given store filesModifierConstructorDescriptionFileableStoreFile
(org.apache.hadoop.fs.FileSystem fs, HStoreFile store) -
Uses of HStoreFile in org.apache.hadoop.hbase.master.assignment
Modifier and TypeFieldDescriptionprivate final HStoreFile
SplitTableRegionProcedure.StoreFileSplitter.sf
Modifier and TypeMethodDescriptionprivate Pair<org.apache.hadoop.fs.Path,
org.apache.hadoop.fs.Path> SplitTableRegionProcedure.splitStoreFile
(HRegionFileSystem regionFs, StoreFileTracker tracker, byte[] family, HStoreFile sf) ModifierConstructorDescriptionStoreFileSplitter
(HRegionFileSystem regionFs, StoreFileTracker tracker, byte[] family, HStoreFile sf) Constructor that takes what it needs to split -
Uses of HStoreFile in org.apache.hadoop.hbase.mob
Modifier and TypeMethodDescriptionstatic boolean
MobUtils.removeMobFiles
(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, TableName tableName, org.apache.hadoop.fs.Path tableDir, byte[] family, Collection<HStoreFile> storeFiles) Archives the mob files. -
Uses of HStoreFile in org.apache.hadoop.hbase.regionserver
Modifier and TypeFieldDescription(package private) final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>
DefaultStoreFileManager.StoreFileList.all
List of store files inside this store.private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>
StripeStoreFileManager.State.allCompactedFilesCached
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>
StripeStoreFileManager.State.allFilesCached
Cached list of all files in the structure, to return from some callsprivate org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>
DefaultStoreFileManager.compactedfiles
List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.private Collection<HStoreFile>
StripeStoreFileManager.CompactionOrFlushMergeCopy.compactedFiles
private Supplier<Collection<HStoreFile>>
StoreContext.Builder.compactedFilesSupplier
private final Supplier<Collection<HStoreFile>>
StoreContext.compactedFilesSupplier
private Supplier<Collection<HStoreFile>>
StoreFileWriter.Builder.compactedFilesSupplier
private final Supplier<Collection<HStoreFile>>
StoreFileWriter.compactedFilesSupplier
private final Supplier<Collection<HStoreFile>>
StoreFileWriter.SingleStoreFileWriter.compactedFilesSupplier
private HashMap<HStoreFile,
byte[]> StripeStoreFileManager.fileEnds
private final List<HStoreFile>
HStore.filesCompacting
private HashMap<HStoreFile,
byte[]> StripeStoreFileManager.fileStarts
Cached file metadata (or overrides as the case may be)private List<HStoreFile>
StripeStoreFileManager.CompactionOrFlushMergeCopy.l0Results
private ArrayList<HStoreFile>
StripeStoreFileManager.CompactionOrFlushMergeCopy.level0Files
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>
StripeStoreFileManager.State.level0Files
Level 0.(package private) final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>
DefaultStoreFileManager.StoreFileList.live
List of store files that include the latest cells inside this store.private Collection<HStoreFile>
StripeStoreFileManager.CompactionOrFlushMergeCopy.results
static final Comparator<HStoreFile>
StoreFileComparators.SEQ_ID
Comparator that compares based on the Sequence Ids of the the store files.static final Comparator<HStoreFile>
StoreFileComparators.SEQ_ID_MAX_TIMESTAMP
Comparator for time-aware compaction.private final Comparator<HStoreFile>
DefaultStoreFileManager.storeFileComparator
private ArrayList<List<HStoreFile>>
StripeStoreFileManager.CompactionOrFlushMergeCopy.stripeFiles
ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>>
StripeStoreFileManager.State.stripeFiles
Files by stripe.Modifier and TypeMethodDescriptionStoreEngine.createStoreFileAndReader
(org.apache.hadoop.fs.Path p) StoreEngine.createStoreFileAndReader
(StoreFileInfo info) HStore.tryCommitRecoveredHFile
(org.apache.hadoop.fs.Path path) Modifier and TypeMethodDescriptionDefaultStoreFileManager.clearCompactedFiles()
StoreFileManager.clearCompactedFiles()
Clears all the compacted files and returns them.org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>
StripeStoreFileManager.clearCompactedFiles()
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>
DefaultStoreFileManager.clearFiles()
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>
StoreFileManager.clearFiles()
Clears all the files currently in use and returns them.org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>
StripeStoreFileManager.clearFiles()
Map<byte[],
List<HStoreFile>> HRegion.close()
Close down this HRegion.Map<byte[],
List<HStoreFile>> HRegion.close
(boolean abort) Map<byte[],
List<HStoreFile>> HRegion.close
(boolean abort, boolean ignoreStatus) Close this HRegion.Map<byte[],
List<HStoreFile>> HRegion.close
(boolean abort, boolean ignoreStatus, boolean isGracefulStop) Close down this HRegion.org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>
HStore.close()
Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>
HStore.closeWithoutLock()
StoreEngine.commitStoreFiles
(List<org.apache.hadoop.fs.Path> files, boolean validate) Commit the givenfiles
.HStore.compact
(CompactionContext compaction, ThroughputController throughputController, User user) Compact the StoreFiles.private Map<byte[],
List<HStoreFile>> HRegion.doClose
(boolean abort, MonitoredTask status) protected List<HStoreFile>
HStore.doCompaction
(CompactionRequestImpl cr, Collection<HStoreFile> filesToCompact, User user, long compactionStartTime, List<org.apache.hadoop.fs.Path> newFiles) private Collection<HStoreFile>
StripeStoreFileManager.findExpiredFiles
(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) final Iterator<HStoreFile>
DefaultStoreFileManager.getCandidateFilesForRowKeyBefore
(KeyValue targetKey) StoreFileManager.getCandidateFilesForRowKeyBefore
(KeyValue targetKey) Gets initial, full list of candidate store files to check for row-key-before.StripeStoreFileManager.getCandidateFilesForRowKeyBefore
(KeyValue targetKey) SeeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue)
for details on this methods.DefaultStoreFileManager.getCompactedfiles()
StoreFileManager.getCompactedfiles()
List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.StripeStoreFileManager.getCompactedfiles()
HStore.getCompactedFiles()
StoreContext.getCompactedFilesSupplier()
StripeStoreFileManager.KeyBeforeConcatenatedLists.Iterator.getComponents()
DefaultStoreFileManager.getFilesForScan
(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, boolean onlyLatestVersion) StoreFileManager.getFilesForScan
(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, boolean onlyLatestVersion) Gets the store files to scan for a Scan or Get request.StripeStoreFileManager.getFilesForScan
(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, boolean onlyLatestVersion) (package private) static Optional<HStoreFile>
StoreUtils.getLargestFile
(Collection<HStoreFile> candidates) Gets the largest file (with reader) out of the list of files.private final ArrayList<HStoreFile>
StripeStoreFileManager.CompactionOrFlushMergeCopy.getLevel0Copy()
Returns A lazy L0 copy from current state.StripeStoreFileManager.getLevel0Files()
private List<HStoreFile>
DefaultStoreFileManager.getLiveFiles
(Collection<HStoreFile> storeFiles) DefaultStoreFileManager.getStoreFileComparator()
StoreFileManager.getStoreFileComparator()
StripeStoreFileManager.getStoreFileComparator()
HStore.getStorefiles()
Returns All store files.final Collection<HStoreFile>
DefaultStoreFileManager.getStoreFiles()
StoreFileManager.getStoreFiles()
Gets the snapshot of the store files currently in use.StripeStoreFileManager.getStoreFiles()
private final ArrayList<HStoreFile>
StripeStoreFileManager.CompactionOrFlushMergeCopy.getStripeCopy
(int index) ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>>
StripeStoreFileManager.getStripes()
DefaultStoreFileManager.getUnneededFiles
(long maxTs, List<HStoreFile> filesCompacting) StoreFileManager.getUnneededFiles
(long maxTs, List<HStoreFile> filesCompacting) StripeStoreFileManager.getUnneededFiles
(long maxTs, List<HStoreFile> filesCompacting) StripeStoreFileManager.KeyBeforeConcatenatedLists.iterator()
private List<HStoreFile>
StoreEngine.openStoreFiles
(Collection<StoreFileInfo> files, boolean warmup) DateTieredStoreEngine.DateTieredCompactionContext.preSelect
(List<HStoreFile> filesCompacting) DefaultStoreEngine.DefaultCompactionContext.preSelect
(List<HStoreFile> filesCompacting) StripeStoreEngine.StripeCompaction.preSelect
(List<HStoreFile> filesCompacting) private TreeMap<byte[],
HStoreFile> StripeStoreFileManager.CompactionOrFlushMergeCopy.processResults()
Process new files, and add them either to the structure of existing stripes, or to the list of new candidate stripes.DefaultStoreFileManager.updateCandidateFilesForRowKeyBefore
(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) StoreFileManager.updateCandidateFilesForRowKeyBefore
(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) Updates the candidate list for finding row key before.StripeStoreFileManager.updateCandidateFilesForRowKeyBefore
(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) SeeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue)
andStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator, KeyValue, Cell)
for details on this methods.Modifier and TypeMethodDescriptionStoreFileComparators.GetPathName.apply
(HStoreFile sf) long
StoreFileComparators.GetBulkTime.applyAsLong
(HStoreFile sf) long
StoreFileComparators.GetFileSize.applyAsLong
(HStoreFile sf) long
StoreFileComparators.GetMaxTimestamp.applyAsLong
(HStoreFile sf) private void
HStore.bulkLoadHFile
(HStoreFile sf) private byte[]
StripeStoreFileManager.endOf
(HStoreFile sf) private void
StripeStoreFileManager.ensureLevel0Metadata
(HStoreFile sf) (package private) static Optional<byte[]>
StoreUtils.getFileSplitPoint
(HStoreFile file, CellComparator comparator) Gets the approximate mid-point of the given file that is optimal for use in splitting it.static long
StoreUtils.getStorefileFieldSize
(HStoreFile file, ToLongFunction<StoreFileReader> f) (package private) long
HStore.getStoreFileSize
(HStoreFile file) Computes the length of a store file without succumbing to any errors along the way.private static void
StripeStoreFileManager.insertFileIntoStripe
(ArrayList<HStoreFile> stripe, HStoreFile sf) Inserts a file in the correct place (by seqnum) in a stripe copy.org.apache.hadoop.fs.Path
HRegionFileSystem.mergeStoreFile
(RegionInfo mergingRegion, String familyName, HStoreFile f, StoreFileTracker tracker) Write out a merge reference under the given merges directory.void
RegionCoprocessorHost.postCompact
(HStore store, HStoreFile resultFile, CompactionLifeCycleTracker tracker, CompactionRequest request, User user) Called after the store compaction has completed.void
RegionCoprocessorHost.postFlush
(HStore store, HStoreFile storeFile, FlushLifeCycleTracker tracker) Invoked after a memstore flushorg.apache.hadoop.fs.Path
HRegionFileSystem.splitStoreFile
(RegionInfo hri, String familyName, HStoreFile f, byte[] splitRow, boolean top, RegionSplitPolicy splitPolicy, StoreFileTracker tracker) Write out a split reference.private byte[]
StripeStoreFileManager.startOf
(HStoreFile sf) Modifier and TypeMethodDescriptionvoid
DefaultStoreFileManager.addCompactionResults
(Collection<HStoreFile> newCompactedfiles, Collection<HStoreFile> results) void
StoreFileManager.addCompactionResults
(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> results) Adds only the new compaction results into the structure.void
StripeStoreFileManager.addCompactionResults
(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> results) void
StoreEngine.addStoreFiles
(Collection<HStoreFile> storeFiles, IOExceptionRunnable actionAfterAdding) Add the store files to store file manager, and also record it in the store file tracker.private void
HStore.addToCompactingFiles
(Collection<HStoreFile> filesToAdd) Adds the files to compacting files.void
StoreFileWriter.appendMetadata
(long maxSequenceId, boolean majorCompaction, Collection<HStoreFile> storeFiles) Writes meta data.private void
StoreFileWriter.SingleStoreFileWriter.appendMetadata
(long maxSequenceId, boolean majorCompaction, Collection<HStoreFile> storeFiles) Writes meta data.void
HRegionWALFileSystem.archiveRecoveredEdits
(String familyName, Collection<HStoreFile> storeFiles) Closes and archives the specified store files from the specified family.private void
HStore.clearCompactedfiles
(List<HStoreFile> filesToRemove) List<org.apache.hadoop.fs.Path>
AbstractMultiFileWriter.commitWriters
(long maxSeqId, boolean majorCompaction, Collection<HStoreFile> storeFiles) private boolean
HStore.completeFlush
(List<HStoreFile> sfs, long snapshotId) (package private) static void
HStoreFile.decreaseStoreFilesRefeCount
(Collection<HStoreFile> storeFiles) private void
StripeStoreFileManager.CompactionOrFlushMergeCopy.deleteResults
(Collection<HStoreFile> compactedFiles) protected List<HStoreFile>
HStore.doCompaction
(CompactionRequestImpl cr, Collection<HStoreFile> filesToCompact, User user, long compactionStartTime, List<org.apache.hadoop.fs.Path> newFiles) private void
StripeStoreFileManager.ensureEdgeStripeMetadata
(ArrayList<HStoreFile> stripe, boolean isFirst) private Collection<HStoreFile>
StripeStoreFileManager.findExpiredFiles
(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) private Collection<HStoreFile>
StripeStoreFileManager.findExpiredFiles
(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) private Collection<HStoreFile>
StripeStoreFileManager.findExpiredFiles
(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) static OptionalInt
StoreUtils.getDeterministicRandomSeed
(Collection<HStoreFile> files) Creates a deterministic hash code for store file collection.(package private) static Optional<HStoreFile>
StoreUtils.getLargestFile
(Collection<HStoreFile> candidates) Gets the largest file (with reader) out of the list of files.private List<HStoreFile>
DefaultStoreFileManager.getLiveFiles
(Collection<HStoreFile> storeFiles) static long
StoreUtils.getLowestTimestamp
(Collection<HStoreFile> candidates) Gets lowest timestamp from candidate StoreFilesstatic OptionalLong
StoreUtils.getMaxMemStoreTSInList
(Collection<HStoreFile> sfs) Return the largest memstoreTS found across all storefiles in the given list.static OptionalLong
StoreUtils.getMaxSequenceIdInList
(Collection<HStoreFile> sfs) Return the highest sequence ID found across all storefiles in the given list.HStore.getScanners
(List<HStoreFile> files, boolean cacheBlocks, boolean isGet, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, byte[] stopRow, long readPt, boolean includeMemstoreScanner, boolean onlyLatestVersion) Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).HStore.getScanners
(List<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, long readPt, boolean includeMemstoreScanner, boolean onlyLatestVersion) Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).static List<StoreFileScanner>
StoreFileScanner.getScannersForCompaction
(Collection<HStoreFile> files, boolean canUseDropBehind, long readPt) Get scanners for compaction.static List<StoreFileScanner>
StoreFileScanner.getScannersForStoreFiles
(Collection<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, boolean useDropBehind, long readPt) Return an array of scanners corresponding to the given set of store files.static List<StoreFileScanner>
StoreFileScanner.getScannersForStoreFiles
(Collection<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, boolean canUseDrop, ScanQueryMatcher matcher, long readPt) Return an array of scanners corresponding to the given set of store files, And set the ScanQueryMatcher for each store file scanner for further optimization(package private) static Optional<byte[]>
StoreUtils.getSplitPoint
(Collection<HStoreFile> storefiles, CellComparator comparator) Gets the mid point of the largest file passed in as split point.static long
StoreUtils.getStorefilesSize
(Collection<HStoreFile> files, Predicate<HStoreFile> predicate) static long
StoreUtils.getStorefilesSize
(Collection<HStoreFile> files, Predicate<HStoreFile> predicate) private long
HStore.getTotalSize
(Collection<HStoreFile> sfs) static long
StoreUtils.getTotalUncompressedBytes
(List<HStoreFile> files) DefaultStoreFileManager.getUnneededFiles
(long maxTs, List<HStoreFile> filesCompacting) StoreFileManager.getUnneededFiles
(long maxTs, List<HStoreFile> filesCompacting) StripeStoreFileManager.getUnneededFiles
(long maxTs, List<HStoreFile> filesCompacting) static boolean
StoreUtils.hasReferences
(Collection<HStoreFile> files) Determines whether any files in the collection are references.(package private) static void
HStoreFile.increaseStoreFilesRefeCount
(Collection<HStoreFile> storeFiles) private static void
StripeStoreFileManager.insertFileIntoStripe
(ArrayList<HStoreFile> stripe, HStoreFile sf) Inserts a file in the correct place (by seqnum) in a stripe copy.void
DefaultStoreFileManager.insertNewFiles
(Collection<HStoreFile> sfs) void
StoreFileManager.insertNewFiles
(Collection<HStoreFile> sfs) Adds new files, either for from MemStore flush or bulk insert, into the structure.void
StripeStoreFileManager.insertNewFiles
(Collection<HStoreFile> sfs) void
DefaultStoreFileManager.loadFiles
(List<HStoreFile> storeFiles) void
StoreFileManager.loadFiles
(List<HStoreFile> storeFiles) Loads the initial store files into empty StoreFileManager.void
StripeStoreFileManager.loadFiles
(List<HStoreFile> storeFiles) private void
StripeStoreFileManager.loadUnclassifiedStoreFiles
(List<HStoreFile> storeFiles) Loads initial store files that were picked up from some physical location pertaining to this store (presumably).private void
HStore.logCompactionEndMessage
(CompactionRequestImpl cr, List<HStoreFile> sfs, long now, long compactionStartTime) Log a very elaborate compaction completion message.private void
StripeStoreFileManager.markCompactedAway
(Collection<HStoreFile> compactedFiles) private void
StripeStoreFileManager.CompactionOrFlushMergeCopy.mergeResults
(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> results) boolean
DateTieredStoreEngine.needsCompaction
(List<HStoreFile> filesCompacting) boolean
DefaultStoreEngine.needsCompaction
(List<HStoreFile> filesCompacting) abstract boolean
StoreEngine.needsCompaction
(List<HStoreFile> filesCompacting) boolean
StripeStoreEngine.needsCompaction
(List<HStoreFile> filesCompacting) private void
HStore.notifyChangedReadersObservers
(List<HStoreFile> sfs) Notify all observers that set of Readers has changed.void
RegionCoprocessorHost.postCompactSelection
(HStore store, List<HStoreFile> selected, CompactionLifeCycleTracker tracker, CompactionRequest request, User user) Called after theHStoreFile
s to be compacted have been selected from the available candidates.boolean
RegionCoprocessorHost.preCompactSelection
(HStore store, List<HStoreFile> candidates, CompactionLifeCycleTracker tracker, User user) Called prior to selecting theHStoreFile
s for compaction from the list of currently available candidates.DateTieredStoreEngine.DateTieredCompactionContext.preSelect
(List<HStoreFile> filesCompacting) DefaultStoreEngine.DefaultCompactionContext.preSelect
(List<HStoreFile> filesCompacting) StripeStoreEngine.StripeCompaction.preSelect
(List<HStoreFile> filesCompacting) private void
StripeStoreFileManager.CompactionOrFlushMergeCopy.processNewCandidateStripes
(TreeMap<byte[], HStoreFile> newStripes) SeeStripeStoreFileManager.addCompactionResults(Collection, Collection)
- updates the stripe list with new candidate stripes/removes old stripes; produces new set of stripe end rows.private void
HStore.removeCompactedfiles
(Collection<HStoreFile> compactedfiles, boolean evictOnClose) Archives and removes the compacted filesvoid
DefaultStoreFileManager.removeCompactedFiles
(Collection<HStoreFile> removedCompactedfiles) void
StoreEngine.removeCompactedFiles
(Collection<HStoreFile> compactedFiles) void
StoreFileManager.removeCompactedFiles
(Collection<HStoreFile> compactedFiles) Remove the compacted filesvoid
StripeStoreFileManager.removeCompactedFiles
(Collection<HStoreFile> compactedFiles) void
HRegionFileSystem.removeStoreFiles
(String familyName, Collection<HStoreFile> storeFiles) Closes and archives the specified store files from the specified family.(package private) void
HStore.replaceStoreFiles
(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> result, boolean writeCompactionMarker) void
StoreEngine.replaceStoreFiles
(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> newFiles, IOExceptionRunnable walMarkerWriter, Runnable actionUnderLock) boolean
DateTieredStoreEngine.DateTieredCompactionContext.select
(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) boolean
DefaultStoreEngine.DefaultCompactionContext.select
(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) boolean
StripeStoreEngine.StripeCompaction.select
(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) private byte[]
StoreFileWriter.SingleStoreFileWriter.toCompactionEventTrackerBytes
(Collection<HStoreFile> storeFiles) Used when writeCOMPACTION_EVENT_KEY
to new file's file info.static List<StoreFileInfo>
StoreUtils.toStoreFileInfo
(Collection<HStoreFile> storefiles) DefaultStoreFileManager.updateCandidateFilesForRowKeyBefore
(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) StoreFileManager.updateCandidateFilesForRowKeyBefore
(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) Updates the candidate list for finding row key before.StripeStoreFileManager.updateCandidateFilesForRowKeyBefore
(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) SeeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue)
andStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator, KeyValue, Cell)
for details on this methods.void
ChangedReadersObserver.updateReaders
(List<HStoreFile> sfs, List<KeyValueScanner> memStoreScanners) Notify observers.void
StoreScanner.updateReaders
(List<HStoreFile> sfs, List<KeyValueScanner> memStoreScanners) (package private) void
HStore.updateSpaceQuotaAfterFileReplacement
(RegionSizeStore sizeStore, RegionInfo regionInfo, Collection<HStoreFile> oldFiles, Collection<HStoreFile> newFiles) Updates the space quota usage for this region, removing the size for files compacted away and adding in the size for new files.StoreContext.Builder.withCompactedFilesSupplier
(Supplier<Collection<HStoreFile>> compactedFilesSupplier) StoreFileWriter.Builder.withCompactedFilesSupplier
(Supplier<Collection<HStoreFile>> compactedFilesSupplier) private void
HStore.writeCompactionWalRecord
(Collection<HStoreFile> filesCompacted, Collection<HStoreFile> newFiles) Writes the compaction WAL record.ModifierConstructorDescriptionDefaultStoreFileManager
(CellComparator cellComparator, Comparator<HStoreFile> storeFileComparator, org.apache.hadoop.conf.Configuration conf, CompactionConfiguration comConf) private
SingleStoreFileWriter
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType bloomType, long maxKeys, InetSocketAddress[] favoredNodes, HFileContext fileContext, boolean shouldDropCacheBehind, Supplier<Collection<HStoreFile>> compactedFilesSupplier) Creates an HFile.Writer that also write helpful meta data.(package private)
StoreFileList
(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> storeFiles, org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> liveStoreFiles) private
StoreFileWriter
(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path liveFilePath, org.apache.hadoop.fs.Path historicalFilePath, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType bloomType, long maxKeys, InetSocketAddress[] favoredNodes, HFileContext fileContext, boolean shouldDropCacheBehind, Supplier<Collection<HStoreFile>> compactedFilesSupplier, CellComparator comparator, int maxVersions, boolean newVersionBehavior) Creates an HFile.Writer that also write helpful meta data. -
Uses of HStoreFile in org.apache.hadoop.hbase.regionserver.compactions
Modifier and TypeFieldDescriptionprivate Collection<HStoreFile>
CompactionRequestImpl.filesToCompact
Modifier and TypeMethodDescriptionprotected final ArrayList<HStoreFile>
ExploringCompactionPolicy.applyCompactionPolicy
(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mightBeStuck) ExploringCompactionPolicy.applyCompactionPolicy
(List<HStoreFile> candidates, boolean mightBeStuck, boolean mayUseOffPeak, int minFiles, int maxFiles) protected ArrayList<HStoreFile>
RatioBasedCompactionPolicy.applyCompactionPolicy
(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mayBeStuck) -- Default minor compaction selection algorithm: choose CompactSelection from candidates -- First exclude bulk-load files if indicated in configuration.protected ArrayList<HStoreFile>
SortedCompactionPolicy.checkMinFilesCriteria
(ArrayList<HStoreFile> candidates, int minFiles) protected ArrayList<HStoreFile>
SortedCompactionPolicy.getCurrentEligibleFiles
(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) private Collection<HStoreFile>
FIFOCompactionPolicy.getExpiredStores
(Collection<HStoreFile> files, Collection<HStoreFile> filesCompacting) CompactionRequestImpl.getFiles()
StripeCompactionPolicy.StripeInformationProvider.getLevel0Files()
Returns Level 0 files.StripeCompactionPolicy.StripeInformationProvider.getStoreFiles()
ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>>
StripeCompactionPolicy.StripeInformationProvider.getStripes()
Returns The stripes.abstract List<HStoreFile>
CompactionContext.preSelect
(List<HStoreFile> filesCompacting) Called before coprocessor preCompactSelection and should filter the candidates for coprocessor; i.e.SortedCompactionPolicy.preSelectCompactionForCoprocessor
(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) StripeCompactionPolicy.preSelectFilesForCoprocessor
(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting) ExploringCompactionPolicy.selectCompactFiles
(List<HStoreFile> candidates, int maxFiles, boolean isOffpeak) Select at least one file in the candidates list to compact, through choosing files from the head to the index that the accumulation length larger the max compaction size.private List<HStoreFile>
StripeCompactionPolicy.selectSimpleCompaction
(List<HStoreFile> sfs, boolean allFilesOnly, boolean isOffpeak, boolean forceCompact) Selects the compaction of a single stripe using default policy.protected ArrayList<HStoreFile>
SortedCompactionPolicy.skipLargeFiles
(ArrayList<HStoreFile> candidates, boolean mayUseOffpeak) Modifier and TypeMethodDescriptionprivate boolean
FIFOCompactionPolicy.isEmptyStoreFile
(HStoreFile sf) The FIFOCompactionPolicy only choose the TTL expired store files as the compaction candidates.Modifier and TypeMethodDescriptionprivate boolean
StripeCompactionPolicy.allFilesExpired
(List<HStoreFile> storeFiles) protected final ArrayList<HStoreFile>
ExploringCompactionPolicy.applyCompactionPolicy
(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mightBeStuck) ExploringCompactionPolicy.applyCompactionPolicy
(List<HStoreFile> candidates, boolean mightBeStuck, boolean mayUseOffPeak, int minFiles, int maxFiles) protected ArrayList<HStoreFile>
RatioBasedCompactionPolicy.applyCompactionPolicy
(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mayBeStuck) -- Default minor compaction selection algorithm: choose CompactSelection from candidates -- First exclude bulk-load files if indicated in configuration.protected ArrayList<HStoreFile>
SortedCompactionPolicy.checkMinFilesCriteria
(ArrayList<HStoreFile> candidates, int minFiles) protected CompactionRequestImpl
DateTieredCompactionPolicy.createCompactionRequest
(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) protected CompactionRequestImpl
RatioBasedCompactionPolicy.createCompactionRequest
(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) protected abstract CompactionRequestImpl
SortedCompactionPolicy.createCompactionRequest
(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) private List<StoreFileScanner>
Compactor.createFileScanners
(Collection<HStoreFile> filesToCompact, long smallestReadPoint, boolean useDropBehind) Creates file scanners for compaction.StripeCompactionPolicy.estimateTargetKvs
(Collection<HStoreFile> files, double splitCount) private boolean
ExploringCompactionPolicy.filesInRatio
(List<HStoreFile> files, double currentRatio) Check that all files satisfy the constraintprotected void
SortedCompactionPolicy.filterBulk
(ArrayList<HStoreFile> candidates) private DateTieredCompactionRequest
DateTieredCompactionPolicy.generateCompactionRequest
(ArrayList<HStoreFile> storeFiles, CompactionWindow window, boolean mayUseOffPeak, boolean mayBeStuck, int minThreshold, long now) DateTieredCompactionPolicy.getCompactBoundariesForMajor
(Collection<HStoreFile> filesToCompact, long now) Return a list of boundaries for multiple compaction output in ascending order.protected ArrayList<HStoreFile>
SortedCompactionPolicy.getCurrentEligibleFiles
(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) protected ArrayList<HStoreFile>
SortedCompactionPolicy.getCurrentEligibleFiles
(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) private Collection<HStoreFile>
FIFOCompactionPolicy.getExpiredStores
(Collection<HStoreFile> files, Collection<HStoreFile> filesCompacting) private Compactor.FileDetails
Compactor.getFileDetails
(Collection<HStoreFile> filesToCompact, boolean allFiles, boolean major) Extracts some details about the files to compact that are commonly needed by compactors.long
SortedCompactionPolicy.getNextMajorCompactTime
(Collection<HStoreFile> filesToCompact) Returns When to run next major compactionstatic long
StripeCompactionPolicy.getTotalFileSize
(Collection<HStoreFile> candidates) private static long
StripeCompactionPolicy.getTotalKvCount
(Collection<HStoreFile> candidates) private long
ExploringCompactionPolicy.getTotalStoreSize
(List<HStoreFile> potentialMatchFiles) Find the total size of a list of store files.private boolean
FIFOCompactionPolicy.hasExpiredStores
(Collection<HStoreFile> files) private boolean
ExploringCompactionPolicy.isBetterSelection
(List<HStoreFile> bestSelection, long bestSize, List<HStoreFile> selection, long size, boolean mightBeStuck) boolean
DateTieredCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) Heuristics for guessing whether we need minor compaction.boolean
DateTieredCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) Heuristics for guessing whether we need minor compaction.boolean
FIFOCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) boolean
FIFOCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) boolean
RatioBasedCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) A heuristic method to decide whether to schedule a compaction requestboolean
RatioBasedCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) A heuristic method to decide whether to schedule a compaction requestabstract boolean
SortedCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) abstract boolean
SortedCompactionPolicy.needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) boolean
StripeCompactionPolicy.needsCompactions
(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting) abstract List<HStoreFile>
CompactionContext.preSelect
(List<HStoreFile> filesCompacting) Called before coprocessor preCompactSelection and should filter the candidates for coprocessor; i.e.SortedCompactionPolicy.preSelectCompactionForCoprocessor
(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) SortedCompactionPolicy.preSelectCompactionForCoprocessor
(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) StripeCompactionPolicy.preSelectFilesForCoprocessor
(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting) protected void
SortedCompactionPolicy.removeExcessFiles
(ArrayList<HStoreFile> candidates, boolean isUserCompaction, boolean isMajorCompaction) abstract boolean
CompactionContext.select
(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) Called to select files for compaction.ExploringCompactionPolicy.selectCompactFiles
(List<HStoreFile> candidates, int maxFiles, boolean isOffpeak) Select at least one file in the candidates list to compact, through choosing files from the head to the index that the accumulation length larger the max compaction size.FIFOCompactionPolicy.selectCompaction
(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) FIFOCompactionPolicy.selectCompaction
(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) SortedCompactionPolicy.selectCompaction
(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) SortedCompactionPolicy.selectCompaction
(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) StripeCompactionPolicy.selectCompaction
(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting, boolean isOffpeak) DateTieredCompactionPolicy.selectMajorCompaction
(ArrayList<HStoreFile> candidateSelection) DateTieredCompactionPolicy.selectMinorCompaction
(ArrayList<HStoreFile> candidateSelection, boolean mayUseOffPeak, boolean mayBeStuck) We receive store files sorted in ascending order by seqId then scan the list of files.private List<HStoreFile>
StripeCompactionPolicy.selectSimpleCompaction
(List<HStoreFile> sfs, boolean allFilesOnly, boolean isOffpeak, boolean forceCompact) Selects the compaction of a single stripe using default policy.abstract boolean
CompactionPolicy.shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) boolean
DateTieredCompactionPolicy.shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) boolean
FIFOCompactionPolicy.shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) boolean
RatioBasedCompactionPolicy.shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) abstract boolean
SortedCompactionPolicy.shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) boolean
StripeCompactionPolicy.shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) protected ArrayList<HStoreFile>
SortedCompactionPolicy.skipLargeFiles
(ArrayList<HStoreFile> candidates, boolean mayUseOffpeak) void
CompactionRequestImpl.updateFiles
(Collection<HStoreFile> files) ModifierConstructorDescriptionBoundaryStripeCompactionRequest
(Collection<HStoreFile> files, List<byte[]> targetBoundaries) DateTieredCompactionRequest
(Collection<HStoreFile> files, List<Long> boundaryList, Map<Long, String> boundaryPolicyMap) SplitStripeCompactionRequest
(Collection<HStoreFile> files, byte[] startRow, byte[] endRow, int targetCount, long targetKvs) SplitStripeCompactionRequest
(Collection<HStoreFile> files, byte[] startRow, byte[] endRow, long targetKvs)