Uses of Class
org.apache.hadoop.hbase.regionserver.HStoreFile
Packages that use HStoreFile
Package
Description
-
Uses of HStoreFile in org.apache.hadoop.hbase.backup
Fields in org.apache.hadoop.hbase.backup declared as HStoreFileModifier and TypeFieldDescription(package private) HStoreFileHFileArchiver.FileableStoreFile.fileMethods in org.apache.hadoop.hbase.backup with parameters of type HStoreFileMethod parameters in org.apache.hadoop.hbase.backup with type arguments of type HStoreFileModifier and TypeMethodDescriptionprivate static voidHFileArchiver.archive(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, RegionInfo regionInfo, byte[] family, Collection<HStoreFile> compactedFiles, org.apache.hadoop.fs.Path storeArchiveDir) static voidHFileArchiver.archiveRecoveredEdits(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, RegionInfo regionInfo, byte[] family, Collection<HStoreFile> replayedEdits) Archive recovered edits using existing logic for archiving store files.static voidHFileArchiver.archiveStoreFiles(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, RegionInfo regionInfo, org.apache.hadoop.fs.Path tableDir, byte[] family, Collection<HStoreFile> compactedFiles) Remove the store files, either by archiving them or outright deletionprivate static voidHFileArchiver.deleteStoreFilesWithoutArchiving(Collection<HStoreFile> compactedFiles) Just do a simple delete of the given store filesConstructors in org.apache.hadoop.hbase.backup with parameters of type HStoreFileModifierConstructorDescriptionFileableStoreFile(org.apache.hadoop.fs.FileSystem fs, HStoreFile store) -
Uses of HStoreFile in org.apache.hadoop.hbase.master.assignment
Fields in org.apache.hadoop.hbase.master.assignment declared as HStoreFileModifier and TypeFieldDescriptionprivate final HStoreFileSplitTableRegionProcedure.StoreFileSplitter.sfMethods in org.apache.hadoop.hbase.master.assignment with parameters of type HStoreFileModifier and TypeMethodDescriptionprivate Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path> SplitTableRegionProcedure.splitStoreFile(HRegionFileSystem regionFs, TableDescriptor htd, ColumnFamilyDescriptor hcd, HStoreFile sf) Constructors in org.apache.hadoop.hbase.master.assignment with parameters of type HStoreFileModifierConstructorDescriptionStoreFileSplitter(HRegionFileSystem regionFs, TableDescriptor htd, ColumnFamilyDescriptor hcd, HStoreFile sf) Constructor that takes what it needs to split -
Uses of HStoreFile in org.apache.hadoop.hbase.mob
Fields in org.apache.hadoop.hbase.mob declared as HStoreFileMethod parameters in org.apache.hadoop.hbase.mob with type arguments of type HStoreFileModifier and TypeMethodDescriptionstatic booleanMobUtils.removeMobFiles(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, TableName tableName, org.apache.hadoop.fs.Path tableDir, byte[] family, Collection<HStoreFile> storeFiles) Archives the mob files.Constructors in org.apache.hadoop.hbase.mob with parameters of type HStoreFile -
Uses of HStoreFile in org.apache.hadoop.hbase.regionserver
Fields in org.apache.hadoop.hbase.regionserver with type parameters of type HStoreFileModifier and TypeFieldDescription(package private) final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>DefaultStoreFileManager.StoreFileList.allList of store files inside this store.private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>StripeStoreFileManager.State.allCompactedFilesCachedorg.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>StripeStoreFileManager.State.allFilesCachedCached list of all files in the structure, to return from some callsprivate org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>DefaultStoreFileManager.compactedfilesList of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.private Collection<HStoreFile>StripeStoreFileManager.CompactionOrFlushMergeCopy.compactedFilesprivate Supplier<Collection<HStoreFile>>StoreContext.Builder.compactedFilesSupplierprivate final Supplier<Collection<HStoreFile>>StoreContext.compactedFilesSupplierprivate Supplier<Collection<HStoreFile>>StoreFileWriter.Builder.compactedFilesSupplierprivate final Supplier<Collection<HStoreFile>>StoreFileWriter.compactedFilesSupplierprivate final Supplier<Collection<HStoreFile>>StoreFileWriter.SingleStoreFileWriter.compactedFilesSupplierprivate HashMap<HStoreFile,byte[]> StripeStoreFileManager.fileEndsprivate final List<HStoreFile>HStore.filesCompactingprivate HashMap<HStoreFile,byte[]> StripeStoreFileManager.fileStartsCached file metadata (or overrides as the case may be)private List<HStoreFile>StripeStoreFileManager.CompactionOrFlushMergeCopy.l0Resultsprivate ArrayList<HStoreFile>StripeStoreFileManager.CompactionOrFlushMergeCopy.level0Filesorg.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>StripeStoreFileManager.State.level0FilesLevel 0.(package private) final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>DefaultStoreFileManager.StoreFileList.liveList of store files that include the latest cells inside this store.private Collection<HStoreFile>StripeStoreFileManager.CompactionOrFlushMergeCopy.resultsstatic final Comparator<HStoreFile>StoreFileComparators.SEQ_IDComparator that compares based on the Sequence Ids of the the store files.static final Comparator<HStoreFile>StoreFileComparators.SEQ_ID_MAX_TIMESTAMPComparator for time-aware compaction.private final Comparator<HStoreFile>DefaultStoreFileManager.storeFileComparatorprivate ArrayList<List<HStoreFile>>StripeStoreFileManager.CompactionOrFlushMergeCopy.stripeFilesArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>>StripeStoreFileManager.State.stripeFilesFiles by stripe.Methods in org.apache.hadoop.hbase.regionserver that return HStoreFileModifier and TypeMethodDescriptionStoreEngine.createStoreFileAndReader(org.apache.hadoop.fs.Path p) StoreEngine.createStoreFileAndReader(StoreFileInfo info) private HStoreFileDataTieringManager.getHStoreFile(org.apache.hadoop.fs.Path hFilePath) HStore.tryCommitRecoveredHFile(org.apache.hadoop.fs.Path path) Methods in org.apache.hadoop.hbase.regionserver that return types with arguments of type HStoreFileModifier and TypeMethodDescriptionDefaultStoreFileManager.clearCompactedFiles()StoreFileManager.clearCompactedFiles()Clears all the compacted files and returns them.org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>StripeStoreFileManager.clearCompactedFiles()org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>DefaultStoreFileManager.clearFiles()org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>StoreFileManager.clearFiles()Clears all the files currently in use and returns them.org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>StripeStoreFileManager.clearFiles()Map<byte[],List<HStoreFile>> HRegion.close()Close down this HRegion.Map<byte[],List<HStoreFile>> HRegion.close(boolean abort) Map<byte[],List<HStoreFile>> HRegion.close(boolean abort, boolean ignoreStatus) Close this HRegion.Map<byte[],List<HStoreFile>> HRegion.close(boolean abort, boolean ignoreStatus, boolean isGracefulStop) Close down this HRegion.org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>HStore.close()Close all the readers We don't need to worry about subsequent requests because the Region holds a write lock that will prevent any more reads or writes.private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>HStore.closeWithoutLock()StoreEngine.commitStoreFiles(List<org.apache.hadoop.fs.Path> files, boolean isCompaction, boolean validate) Commit the givenfiles.HStore.compact(CompactionContext compaction, ThroughputController throughputController, User user) Compact the StoreFiles.private Map<byte[],List<HStoreFile>> HRegion.doClose(boolean abort, MonitoredTask status) protected List<HStoreFile>HStore.doCompaction(CompactionRequestImpl cr, Collection<HStoreFile> filesToCompact, User user, long compactionStartTime, List<org.apache.hadoop.fs.Path> newFiles) private Collection<HStoreFile>StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) final Iterator<HStoreFile>DefaultStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue targetKey) StoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue targetKey) Gets initial, full list of candidate store files to check for row-key-before.StripeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue targetKey) SeeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue)for details on this methods.DefaultStoreFileManager.getCompactedfiles()StoreFileManager.getCompactedfiles()List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.StripeStoreFileManager.getCompactedfiles()HStore.getCompactedFiles()StoreContext.getCompactedFilesSupplier()StripeStoreFileManager.KeyBeforeConcatenatedLists.Iterator.getComponents()DefaultStoreFileManager.getFilesForScan(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, boolean onlyLatestVersion) StoreFileManager.getFilesForScan(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, boolean onlyLatestVersion) Gets the store files to scan for a Scan or Get request.StripeStoreFileManager.getFilesForScan(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, boolean onlyLatestVersion) (package private) static Optional<HStoreFile>StoreUtils.getLargestFile(Collection<HStoreFile> candidates) Gets the largest file (with reader) out of the list of files.private final ArrayList<HStoreFile>StripeStoreFileManager.CompactionOrFlushMergeCopy.getLevel0Copy()Returns A lazy L0 copy from current state.StripeStoreFileManager.getLevel0Files()private List<HStoreFile>DefaultStoreFileManager.getLiveFiles(Collection<HStoreFile> storeFiles) DefaultStoreFileManager.getStoreFileComparator()StoreFileManager.getStoreFileComparator()StripeStoreFileManager.getStoreFileComparator()HStore.getStorefiles()Returns All store files.final Collection<HStoreFile>DefaultStoreFileManager.getStoreFiles()StoreFileManager.getStoreFiles()Gets the snapshot of the store files currently in use.StripeStoreFileManager.getStoreFiles()private final ArrayList<HStoreFile>StripeStoreFileManager.CompactionOrFlushMergeCopy.getStripeCopy(int index) ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>>StripeStoreFileManager.getStripes()DefaultStoreFileManager.getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting) StoreFileManager.getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting) StripeStoreFileManager.getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting) StripeStoreFileManager.KeyBeforeConcatenatedLists.iterator()private List<HStoreFile>StoreEngine.openStoreFiles(Collection<StoreFileInfo> files, boolean warmup) DateTieredStoreEngine.DateTieredCompactionContext.preSelect(List<HStoreFile> filesCompacting) DefaultStoreEngine.DefaultCompactionContext.preSelect(List<HStoreFile> filesCompacting) StripeStoreEngine.StripeCompaction.preSelect(List<HStoreFile> filesCompacting) private TreeMap<byte[],HStoreFile> StripeStoreFileManager.CompactionOrFlushMergeCopy.processResults()Process new files, and add them either to the structure of existing stripes, or to the list of new candidate stripes.DefaultStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) StoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) Updates the candidate list for finding row key before.StripeStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) SeeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue)andStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator, KeyValue, Cell)for details on this methods.Methods in org.apache.hadoop.hbase.regionserver with parameters of type HStoreFileModifier and TypeMethodDescriptionStoreFileComparators.GetPathName.apply(HStoreFile sf) longStoreFileComparators.GetBulkTime.applyAsLong(HStoreFile sf) longStoreFileComparators.GetFileSize.applyAsLong(HStoreFile sf) longStoreFileComparators.GetMaxTimestamp.applyAsLong(HStoreFile sf) private voidHStore.bulkLoadHFile(HStoreFile sf) private byte[]StripeStoreFileManager.endOf(HStoreFile sf) private voidStripeStoreFileManager.ensureLevel0Metadata(HStoreFile sf) (package private) static Optional<byte[]>StoreUtils.getFileSplitPoint(HStoreFile file, CellComparator comparator) Gets the approximate mid-point of the given file that is optimal for use in splitting it.static longStoreUtils.getStorefileFieldSize(HStoreFile file, ToLongFunction<StoreFileReader> f) (package private) longHStore.getStoreFileSize(HStoreFile file) Computes the length of a store file without succumbing to any errors along the way.longCellTSTiering.getTimestamp(HStoreFile hStoreFile) longCustomTiering.getTimestamp(HStoreFile hStoreFile) longDataTiering.getTimestamp(HStoreFile hFile) private static voidStripeStoreFileManager.insertFileIntoStripe(ArrayList<HStoreFile> stripe, HStoreFile sf) Inserts a file in the correct place (by seqnum) in a stripe copy.org.apache.hadoop.fs.PathHRegionFileSystem.mergeStoreFile(RegionInfo mergingRegion, String familyName, HStoreFile f, StoreFileTracker tracker) Write out a merge reference under the given merges directory.voidRegionCoprocessorHost.postCompact(HStore store, HStoreFile resultFile, CompactionLifeCycleTracker tracker, CompactionRequest request, User user) Called after the store compaction has completed.voidRegionCoprocessorHost.postFlush(HStore store, HStoreFile storeFile, FlushLifeCycleTracker tracker) Invoked after a memstore flushorg.apache.hadoop.fs.PathHRegionFileSystem.splitStoreFile(RegionInfo hri, String familyName, HStoreFile f, byte[] splitRow, boolean top, RegionSplitPolicy splitPolicy, StoreFileTracker tracker) Write out a split reference.private byte[]StripeStoreFileManager.startOf(HStoreFile sf) Method parameters in org.apache.hadoop.hbase.regionserver with type arguments of type HStoreFileModifier and TypeMethodDescriptionvoidDefaultStoreFileManager.addCompactionResults(Collection<HStoreFile> newCompactedfiles, Collection<HStoreFile> results) voidStoreFileManager.addCompactionResults(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> results) Adds only the new compaction results into the structure.voidStripeStoreFileManager.addCompactionResults(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> results) voidStoreEngine.addStoreFiles(Collection<HStoreFile> storeFiles, IOExceptionRunnable actionAfterAdding) Add the store files to store file manager, and also record it in the store file tracker.private voidHStore.addToCompactingFiles(Collection<HStoreFile> filesToAdd) Adds the files to compacting files.voidStoreFileWriter.appendMetadata(long maxSequenceId, boolean majorCompaction, Collection<HStoreFile> storeFiles) Writes meta data.private voidStoreFileWriter.SingleStoreFileWriter.appendMetadata(long maxSequenceId, boolean majorCompaction, Collection<HStoreFile> storeFiles) Writes meta data.voidHRegionWALFileSystem.archiveRecoveredEdits(String familyName, Collection<HStoreFile> storeFiles) Closes and archives the specified store files from the specified family.private voidHStore.clearCompactedfiles(List<HStoreFile> filesToRemove) List<org.apache.hadoop.fs.Path>AbstractMultiFileWriter.commitWriters(long maxSeqId, boolean majorCompaction, Collection<HStoreFile> storeFiles) List<org.apache.hadoop.fs.Path>CustomTieringMultiFileWriter.commitWriters(long maxSeqId, boolean majorCompaction, Collection<HStoreFile> storeFiles) private booleanHStore.completeFlush(List<HStoreFile> sfs, long snapshotId) (package private) static voidHStoreFile.decreaseStoreFilesRefeCount(Collection<HStoreFile> storeFiles) private voidStripeStoreFileManager.CompactionOrFlushMergeCopy.deleteResults(Collection<HStoreFile> compactedFiles) protected List<HStoreFile>HStore.doCompaction(CompactionRequestImpl cr, Collection<HStoreFile> filesToCompact, User user, long compactionStartTime, List<org.apache.hadoop.fs.Path> newFiles) private voidStripeStoreFileManager.ensureEdgeStripeMetadata(ArrayList<HStoreFile> stripe, boolean isFirst) private Collection<HStoreFile>StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) private Collection<HStoreFile>StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) private Collection<HStoreFile>StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe, long maxTs, List<HStoreFile> filesCompacting, Collection<HStoreFile> expiredStoreFiles) static OptionalIntStoreUtils.getDeterministicRandomSeed(Collection<HStoreFile> files) Creates a deterministic hash code for store file collection.(package private) static Optional<HStoreFile>StoreUtils.getLargestFile(Collection<HStoreFile> candidates) Gets the largest file (with reader) out of the list of files.private List<HStoreFile>DefaultStoreFileManager.getLiveFiles(Collection<HStoreFile> storeFiles) static longStoreUtils.getLowestTimestamp(Collection<HStoreFile> candidates) Gets lowest timestamp from candidate StoreFilesstatic OptionalLongStoreUtils.getMaxMemStoreTSInList(Collection<HStoreFile> sfs) Return the largest memstoreTS found across all storefiles in the given list.static OptionalLongStoreUtils.getMaxSequenceIdInList(Collection<HStoreFile> sfs) Return the highest sequence ID found across all storefiles in the given list.HStore.getScanners(List<HStoreFile> files, boolean cacheBlocks, boolean isGet, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, byte[] stopRow, long readPt, boolean includeMemstoreScanner, boolean onlyLatestVersion) Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).HStore.getScanners(List<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, long readPt, boolean includeMemstoreScanner, boolean onlyLatestVersion) Create scanners on the given files and if needed on the memstore with no filtering based on TTL (that happens further down the line).static List<StoreFileScanner>StoreFileScanner.getScannersForCompaction(Collection<HStoreFile> files, boolean canUseDropBehind, long readPt) Get scanners for compaction.static List<StoreFileScanner>StoreFileScanner.getScannersForStoreFiles(Collection<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, boolean useDropBehind, long readPt) Return an array of scanners corresponding to the given set of store files.static List<StoreFileScanner>StoreFileScanner.getScannersForStoreFiles(Collection<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, boolean canUseDrop, ScanQueryMatcher matcher, long readPt) Return an array of scanners corresponding to the given set of store files, And set the ScanQueryMatcher for each store file scanner for further optimization(package private) static Optional<byte[]>StoreUtils.getSplitPoint(Collection<HStoreFile> storefiles, CellComparator comparator) Gets the mid point of the largest file passed in as split point.static longStoreUtils.getStorefilesSize(Collection<HStoreFile> files, Predicate<HStoreFile> predicate) static longStoreUtils.getStorefilesSize(Collection<HStoreFile> files, Predicate<HStoreFile> predicate) private longHStore.getTotalSize(Collection<HStoreFile> sfs) static longStoreUtils.getTotalUncompressedBytes(List<HStoreFile> files) DefaultStoreFileManager.getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting) StoreFileManager.getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting) StripeStoreFileManager.getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting) static booleanStoreUtils.hasReferences(Collection<HStoreFile> files) Determines whether any files in the collection are references.(package private) static voidHStoreFile.increaseStoreFilesRefeCount(Collection<HStoreFile> storeFiles) private static voidStripeStoreFileManager.insertFileIntoStripe(ArrayList<HStoreFile> stripe, HStoreFile sf) Inserts a file in the correct place (by seqnum) in a stripe copy.voidDefaultStoreFileManager.insertNewFiles(Collection<HStoreFile> sfs) voidStoreFileManager.insertNewFiles(Collection<HStoreFile> sfs) Adds new files, either for from MemStore flush or bulk insert, into the structure.voidStripeStoreFileManager.insertNewFiles(Collection<HStoreFile> sfs) voidDefaultStoreFileManager.loadFiles(List<HStoreFile> storeFiles) voidStoreFileManager.loadFiles(List<HStoreFile> storeFiles) Loads the initial store files into empty StoreFileManager.voidStripeStoreFileManager.loadFiles(List<HStoreFile> storeFiles) private voidStripeStoreFileManager.loadUnclassifiedStoreFiles(List<HStoreFile> storeFiles) Loads initial store files that were picked up from some physical location pertaining to this store (presumably).private voidHStore.logCompactionEndMessage(CompactionRequestImpl cr, List<HStoreFile> sfs, long now, long compactionStartTime) Log a very elaborate compaction completion message.private voidStripeStoreFileManager.markCompactedAway(Collection<HStoreFile> compactedFiles) private voidStripeStoreFileManager.CompactionOrFlushMergeCopy.mergeResults(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> results) booleanDateTieredStoreEngine.needsCompaction(List<HStoreFile> filesCompacting) booleanDefaultStoreEngine.needsCompaction(List<HStoreFile> filesCompacting) abstract booleanStoreEngine.needsCompaction(List<HStoreFile> filesCompacting) booleanStripeStoreEngine.needsCompaction(List<HStoreFile> filesCompacting) private voidHStore.notifyChangedReadersObservers(List<HStoreFile> sfs) Notify all observers that set of Readers has changed.voidRegionCoprocessorHost.postCompactSelection(HStore store, List<HStoreFile> selected, CompactionLifeCycleTracker tracker, CompactionRequest request, User user) Called after theHStoreFiles to be compacted have been selected from the available candidates.booleanRegionCoprocessorHost.preCompactSelection(HStore store, List<HStoreFile> candidates, CompactionLifeCycleTracker tracker, User user) Called prior to selecting theHStoreFiles for compaction from the list of currently available candidates.DateTieredStoreEngine.DateTieredCompactionContext.preSelect(List<HStoreFile> filesCompacting) DefaultStoreEngine.DefaultCompactionContext.preSelect(List<HStoreFile> filesCompacting) StripeStoreEngine.StripeCompaction.preSelect(List<HStoreFile> filesCompacting) private voidStripeStoreFileManager.CompactionOrFlushMergeCopy.processNewCandidateStripes(TreeMap<byte[], HStoreFile> newStripes) SeeStripeStoreFileManager.addCompactionResults(Collection, Collection)- updates the stripe list with new candidate stripes/removes old stripes; produces new set of stripe end rows.private voidHStore.removeCompactedfiles(Collection<HStoreFile> compactedfiles, boolean evictOnClose) Archives and removes the compacted filesvoidDefaultStoreFileManager.removeCompactedFiles(Collection<HStoreFile> removedCompactedfiles) voidStoreEngine.removeCompactedFiles(Collection<HStoreFile> compactedFiles) voidStoreFileManager.removeCompactedFiles(Collection<HStoreFile> compactedFiles) Remove the compacted filesvoidStripeStoreFileManager.removeCompactedFiles(Collection<HStoreFile> compactedFiles) (package private) voidHStore.replaceStoreFiles(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> result, boolean writeCompactionMarker) voidStoreEngine.replaceStoreFiles(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> newFiles, IOExceptionRunnable walMarkerWriter, Runnable actionUnderLock) booleanDateTieredStoreEngine.DateTieredCompactionContext.select(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) booleanDefaultStoreEngine.DefaultCompactionContext.select(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) booleanStripeStoreEngine.StripeCompaction.select(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) private byte[]StoreFileWriter.SingleStoreFileWriter.toCompactionEventTrackerBytes(Collection<HStoreFile> storeFiles) Used when writeCOMPACTION_EVENT_KEYto new file's file info.static List<StoreFileInfo>StoreUtils.toStoreFileInfo(Collection<HStoreFile> storefiles) DefaultStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) StoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) Updates the candidate list for finding row key before.StripeStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) SeeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue)andStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator, KeyValue, Cell)for details on this methods.voidChangedReadersObserver.updateReaders(List<HStoreFile> sfs, List<KeyValueScanner> memStoreScanners) Notify observers.voidStoreScanner.updateReaders(List<HStoreFile> sfs, List<KeyValueScanner> memStoreScanners) (package private) voidHStore.updateSpaceQuotaAfterFileReplacement(RegionSizeStore sizeStore, RegionInfo regionInfo, Collection<HStoreFile> oldFiles, Collection<HStoreFile> newFiles) Updates the space quota usage for this region, removing the size for files compacted away and adding in the size for new files.StoreContext.Builder.withCompactedFilesSupplier(Supplier<Collection<HStoreFile>> compactedFilesSupplier) StoreFileWriter.Builder.withCompactedFilesSupplier(Supplier<Collection<HStoreFile>> compactedFilesSupplier) private voidHStore.writeCompactionWalRecord(Collection<HStoreFile> filesCompacted, Collection<HStoreFile> newFiles) Writes the compaction WAL record.Constructor parameters in org.apache.hadoop.hbase.regionserver with type arguments of type HStoreFileModifierConstructorDescriptionDefaultStoreFileManager(CellComparator cellComparator, Comparator<HStoreFile> storeFileComparator, org.apache.hadoop.conf.Configuration conf, CompactionConfiguration comConf) privateSingleStoreFileWriter(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType bloomType, long maxKeys, InetSocketAddress[] favoredNodes, HFileContext fileContext, boolean shouldDropCacheBehind, Supplier<Collection<HStoreFile>> compactedFilesSupplier) Creates an HFile.Writer that also write helpful meta data.(package private)StoreFileList(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> storeFiles, org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> liveStoreFiles) privateStoreFileWriter(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path liveFilePath, org.apache.hadoop.fs.Path historicalFilePath, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType bloomType, long maxKeys, InetSocketAddress[] favoredNodes, HFileContext fileContext, boolean shouldDropCacheBehind, Supplier<Collection<HStoreFile>> compactedFilesSupplier, CellComparator comparator, int maxVersions, boolean newVersionBehavior) Creates an HFile.Writer that also write helpful meta data. -
Uses of HStoreFile in org.apache.hadoop.hbase.regionserver.compactions
Fields in org.apache.hadoop.hbase.regionserver.compactions with type parameters of type HStoreFileModifier and TypeFieldDescriptionprivate Collection<HStoreFile>CompactionRequestImpl.filesToCompactMethods in org.apache.hadoop.hbase.regionserver.compactions that return types with arguments of type HStoreFileModifier and TypeMethodDescriptionprotected final ArrayList<HStoreFile>ExploringCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mightBeStuck) ExploringCompactionPolicy.applyCompactionPolicy(List<HStoreFile> candidates, boolean mightBeStuck, boolean mayUseOffPeak, int minFiles, int maxFiles) protected ArrayList<HStoreFile>RatioBasedCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mayBeStuck) -- Default minor compaction selection algorithm: choose CompactSelection from candidates -- First exclude bulk-load files if indicated in configuration.protected ArrayList<HStoreFile>SortedCompactionPolicy.checkMinFilesCriteria(ArrayList<HStoreFile> candidates, int minFiles) protected ArrayList<HStoreFile>SortedCompactionPolicy.getCurrentEligibleFiles(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) private Collection<HStoreFile>FIFOCompactionPolicy.getExpiredStores(Collection<HStoreFile> files, Collection<HStoreFile> filesCompacting) CompactionRequestImpl.getFiles()StripeCompactionPolicy.StripeInformationProvider.getLevel0Files()Returns Level 0 files.StripeCompactionPolicy.StripeInformationProvider.getStoreFiles()ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>>StripeCompactionPolicy.StripeInformationProvider.getStripes()Returns The stripes.abstract List<HStoreFile>CompactionContext.preSelect(List<HStoreFile> filesCompacting) Called before coprocessor preCompactSelection and should filter the candidates for coprocessor; i.e.SortedCompactionPolicy.preSelectCompactionForCoprocessor(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) StripeCompactionPolicy.preSelectFilesForCoprocessor(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting) ExploringCompactionPolicy.selectCompactFiles(List<HStoreFile> candidates, int maxFiles, boolean isOffpeak) Select at least one file in the candidates list to compact, through choosing files from the head to the index that the accumulation length larger the max compaction size.private List<HStoreFile>StripeCompactionPolicy.selectSimpleCompaction(List<HStoreFile> sfs, boolean allFilesOnly, boolean isOffpeak, boolean forceCompact) Selects the compaction of a single stripe using default policy.protected ArrayList<HStoreFile>SortedCompactionPolicy.skipLargeFiles(ArrayList<HStoreFile> candidates, boolean mayUseOffpeak) Methods in org.apache.hadoop.hbase.regionserver.compactions with parameters of type HStoreFileModifier and TypeMethodDescriptionprotected booleanDateTieredCompactionPolicy.checkForTtl(long ttl, HStoreFile file) private booleanFIFOCompactionPolicy.isEmptyStoreFile(HStoreFile sf) The FIFOCompactionPolicy only choose the TTL expired store files as the compaction candidates.protected booleanDateTieredCompactionPolicy.isMajorOrBulkloadResult(HStoreFile file, long timeDiff) Method parameters in org.apache.hadoop.hbase.regionserver.compactions with type arguments of type HStoreFileModifier and TypeMethodDescriptionprivate booleanStripeCompactionPolicy.allFilesExpired(List<HStoreFile> storeFiles) protected final ArrayList<HStoreFile>ExploringCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mightBeStuck) ExploringCompactionPolicy.applyCompactionPolicy(List<HStoreFile> candidates, boolean mightBeStuck, boolean mayUseOffPeak, int minFiles, int maxFiles) protected ArrayList<HStoreFile>RatioBasedCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mayBeStuck) -- Default minor compaction selection algorithm: choose CompactSelection from candidates -- First exclude bulk-load files if indicated in configuration.protected ArrayList<HStoreFile>SortedCompactionPolicy.checkMinFilesCriteria(ArrayList<HStoreFile> candidates, int minFiles) protected CompactionRequestImplDateTieredCompactionPolicy.createCompactionRequest(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) protected CompactionRequestImplRatioBasedCompactionPolicy.createCompactionRequest(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) protected abstract CompactionRequestImplSortedCompactionPolicy.createCompactionRequest(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) private List<StoreFileScanner>Compactor.createFileScanners(Collection<HStoreFile> filesToCompact, long smallestReadPoint, boolean useDropBehind) Creates file scanners for compaction.StripeCompactionPolicy.estimateTargetKvs(Collection<HStoreFile> files, double splitCount) private booleanExploringCompactionPolicy.filesInRatio(List<HStoreFile> files, double currentRatio) Check that all files satisfy the constraintprotected voidSortedCompactionPolicy.filterBulk(ArrayList<HStoreFile> candidates) private DateTieredCompactionRequestDateTieredCompactionPolicy.generateCompactionRequest(ArrayList<HStoreFile> storeFiles, CompactionWindow window, boolean mayUseOffPeak, boolean mayBeStuck, int minThreshold, long now) CustomDateTieredCompactionPolicy.getCompactBoundariesForMajor(Collection<HStoreFile> filesToCompact, long now) DateTieredCompactionPolicy.getCompactBoundariesForMajor(Collection<HStoreFile> filesToCompact, long now) Return a list of boundaries for multiple compaction output in ascending order.protected ArrayList<HStoreFile>SortedCompactionPolicy.getCurrentEligibleFiles(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) protected ArrayList<HStoreFile>SortedCompactionPolicy.getCurrentEligibleFiles(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) private Collection<HStoreFile>FIFOCompactionPolicy.getExpiredStores(Collection<HStoreFile> files, Collection<HStoreFile> filesCompacting) private Compactor.FileDetailsCompactor.getFileDetails(Collection<HStoreFile> filesToCompact, boolean allFiles, boolean major) Extracts some details about the files to compact that are commonly needed by compactors.longSortedCompactionPolicy.getNextMajorCompactTime(Collection<HStoreFile> filesToCompact) Returns When to run next major compactionstatic longStripeCompactionPolicy.getTotalFileSize(Collection<HStoreFile> candidates) private static longStripeCompactionPolicy.getTotalKvCount(Collection<HStoreFile> candidates) private longExploringCompactionPolicy.getTotalStoreSize(List<HStoreFile> potentialMatchFiles) Find the total size of a list of store files.private booleanFIFOCompactionPolicy.hasExpiredStores(Collection<HStoreFile> files) private booleanExploringCompactionPolicy.isBetterSelection(List<HStoreFile> bestSelection, long bestSize, List<HStoreFile> selection, long size, boolean mightBeStuck) protected booleanDateTieredCompactionPolicy.isMajorCompactionTime(Collection<HStoreFile> filesToCompact, long now, long lowestModificationTime) booleanDateTieredCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) Heuristics for guessing whether we need minor compaction.booleanDateTieredCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) Heuristics for guessing whether we need minor compaction.booleanFIFOCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) booleanFIFOCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) booleanRatioBasedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) A heuristic method to decide whether to schedule a compaction requestbooleanRatioBasedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) A heuristic method to decide whether to schedule a compaction requestabstract booleanSortedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) abstract booleanSortedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) booleanStripeCompactionPolicy.needsCompactions(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting) abstract List<HStoreFile>CompactionContext.preSelect(List<HStoreFile> filesCompacting) Called before coprocessor preCompactSelection and should filter the candidates for coprocessor; i.e.SortedCompactionPolicy.preSelectCompactionForCoprocessor(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) SortedCompactionPolicy.preSelectCompactionForCoprocessor(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) StripeCompactionPolicy.preSelectFilesForCoprocessor(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting) protected voidSortedCompactionPolicy.removeExcessFiles(ArrayList<HStoreFile> candidates, boolean isUserCompaction, boolean isMajorCompaction) abstract booleanCompactionContext.select(List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) Called to select files for compaction.ExploringCompactionPolicy.selectCompactFiles(List<HStoreFile> candidates, int maxFiles, boolean isOffpeak) Select at least one file in the candidates list to compact, through choosing files from the head to the index that the accumulation length larger the max compaction size.FIFOCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) FIFOCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) SortedCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) SortedCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) StripeCompactionPolicy.selectCompaction(StripeCompactionPolicy.StripeInformationProvider si, List<HStoreFile> filesCompacting, boolean isOffpeak) DateTieredCompactionPolicy.selectMajorCompaction(ArrayList<HStoreFile> candidateSelection) CustomDateTieredCompactionPolicy.selectMinorCompaction(ArrayList<HStoreFile> candidateSelection, boolean mayUseOffPeak, boolean mayBeStuck) DateTieredCompactionPolicy.selectMinorCompaction(ArrayList<HStoreFile> candidateSelection, boolean mayUseOffPeak, boolean mayBeStuck) We receive store files sorted in ascending order by seqId then scan the list of files.private List<HStoreFile>StripeCompactionPolicy.selectSimpleCompaction(List<HStoreFile> sfs, boolean allFilesOnly, boolean isOffpeak, boolean forceCompact) Selects the compaction of a single stripe using default policy.abstract booleanCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) booleanCustomDateTieredCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) booleanDateTieredCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) booleanFIFOCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) booleanRatioBasedCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) abstract booleanSortedCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) booleanStripeCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) protected ArrayList<HStoreFile>SortedCompactionPolicy.skipLargeFiles(ArrayList<HStoreFile> candidates, boolean mayUseOffpeak) voidCompactionRequestImpl.updateFiles(Collection<HStoreFile> files) Constructor parameters in org.apache.hadoop.hbase.regionserver.compactions with type arguments of type HStoreFileModifierConstructorDescriptionBoundaryStripeCompactionRequest(Collection<HStoreFile> files, List<byte[]> targetBoundaries) DateTieredCompactionRequest(Collection<HStoreFile> files, List<Long> boundaryList, Map<Long, String> boundaryPolicyMap) SplitStripeCompactionRequest(Collection<HStoreFile> files, byte[] startRow, byte[] endRow, int targetCount, long targetKvs) SplitStripeCompactionRequest(Collection<HStoreFile> files, byte[] startRow, byte[] endRow, long targetKvs) -
Uses of HStoreFile in org.apache.hadoop.hbase.regionserver.storefiletracker
Method parameters in org.apache.hadoop.hbase.regionserver.storefiletracker with type arguments of type HStoreFileModifier and TypeMethodDescriptionprotected voidStoreFileTrackerBase.archiveStoreFiles(List<HStoreFile> storeFiles) voidMigrationStoreFileTracker.removeStoreFiles(List<HStoreFile> storeFiles) voidStoreFileTracker.removeStoreFiles(List<HStoreFile> storeFiles) Closes and archives the specified store files from the specified family.voidStoreFileTrackerBase.removeStoreFiles(List<HStoreFile> storeFiles)