Modifier and Type | Field and Description |
---|---|
(package private) HStoreFile |
HFileArchiver.FileableStoreFile.file |
Modifier and Type | Method and Description |
---|---|
HFileArchiver.File |
HFileArchiver.StoreToFile.apply(HStoreFile input) |
Modifier and Type | Method and Description |
---|---|
static void |
HFileArchiver.archiveStoreFiles(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
RegionInfo regionInfo,
org.apache.hadoop.fs.Path tableDir,
byte[] family,
Collection<HStoreFile> compactedFiles)
Remove the store files, either by archiving them or outright deletion
|
private static void |
HFileArchiver.deleteStoreFilesWithoutArchiving(Collection<HStoreFile> compactedFiles)
Just do a simple delete of the given store files
|
Constructor and Description |
---|
FileableStoreFile(org.apache.hadoop.fs.FileSystem fs,
HStoreFile store) |
Modifier and Type | Field and Description |
---|---|
private HStoreFile |
SplitTableRegionProcedure.StoreFileSplitter.sf |
Modifier and Type | Method and Description |
---|---|
private Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path> |
SplitTableRegionProcedure.splitStoreFile(HRegionFileSystem regionFs,
byte[] family,
HStoreFile sf) |
Constructor and Description |
---|
StoreFileSplitter(HRegionFileSystem regionFs,
byte[] family,
HStoreFile sf)
Constructor that takes what it needs to split
|
Modifier and Type | Field and Description |
---|---|
private HStoreFile |
MobFile.sf |
Modifier and Type | Method and Description |
---|---|
static void |
MobUtils.removeMobFiles(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
TableName tableName,
org.apache.hadoop.fs.Path tableDir,
byte[] family,
Collection<HStoreFile> storeFiles)
Archives the mob files.
|
Constructor and Description |
---|
CachedMobFile(HStoreFile sf) |
MobFile(HStoreFile sf) |
Modifier and Type | Field and Description |
---|---|
private List<HStoreFile> |
PartitionedMobCompactionRequest.CompactionDelPartition.storeFiles |
Modifier and Type | Method and Description |
---|---|
(package private) List<HStoreFile> |
PartitionedMobCompactor.getListOfDelFilesForPartition(PartitionedMobCompactionRequest.CompactionPartition partition,
List<PartitionedMobCompactionRequest.CompactionDelPartition> delPartitions) |
List<HStoreFile> |
PartitionedMobCompactionRequest.CompactionDelPartition.getStoreFiles() |
Modifier and Type | Method and Description |
---|---|
void |
PartitionedMobCompactionRequest.CompactionDelPartition.addStoreFile(HStoreFile file) |
Modifier and Type | Method and Description |
---|---|
private void |
PartitionedMobCompactor.closeStoreFileReaders(List<HStoreFile> storeFiles)
Closes the readers of store files.
|
private org.apache.hadoop.fs.Path |
PartitionedMobCompactor.compactDelFilesInBatch(PartitionedMobCompactionRequest request,
List<HStoreFile> delFiles)
Compacts the del file in a batch.
|
private List<org.apache.hadoop.fs.Path> |
PartitionedMobCompactor.compactMobFilePartition(PartitionedMobCompactionRequest request,
PartitionedMobCompactionRequest.CompactionPartition partition,
List<HStoreFile> delFiles,
Connection connection,
Table table)
Compacts a partition of selected small mob files and all the del files.
|
private void |
PartitionedMobCompactor.compactMobFilesInBatch(PartitionedMobCompactionRequest request,
PartitionedMobCompactionRequest.CompactionPartition partition,
Connection connection,
Table table,
List<HStoreFile> filesToCompact,
int batch,
org.apache.hadoop.fs.Path bulkloadPathOfPartition,
org.apache.hadoop.fs.Path bulkloadColumnPath,
List<org.apache.hadoop.fs.Path> newFiles)
Compacts a partition of selected small mob files and all the del files in a batch.
|
private StoreScanner |
PartitionedMobCompactor.createScanner(List<HStoreFile> filesToCompact,
ScanType scanType)
Creates a store scanner.
|
private Pair<Long,Long> |
PartitionedMobCompactor.getFileInfo(List<HStoreFile> storeFiles)
Gets the max seqId and number of cells of the store files.
|
Modifier and Type | Field and Description |
---|---|
private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> |
StripeStoreFileManager.State.allCompactedFilesCached |
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> |
StripeStoreFileManager.State.allFilesCached
Cached list of all files in the structure, to return from some calls
|
private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> |
DefaultStoreFileManager.compactedfiles
List of compacted files inside this store that needs to be excluded in reads
because further new reads will be using only the newly created files out of compaction.
|
private Collection<HStoreFile> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.compactedFiles |
private HashMap<HStoreFile,byte[]> |
StripeStoreFileManager.fileEnds |
private List<HStoreFile> |
HStore.filesCompacting |
private HashMap<HStoreFile,byte[]> |
StripeStoreFileManager.fileStarts
Cached file metadata (or overrides as the case may be)
|
private List<HStoreFile> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.l0Results |
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> |
StripeStoreFileManager.State.level0Files
Level 0.
|
private ArrayList<HStoreFile> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.level0Files |
private Collection<HStoreFile> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.results |
static Comparator<HStoreFile> |
StoreFileComparators.SEQ_ID
Comparator that compares based on the Sequence Ids of the the store files.
|
static Comparator<HStoreFile> |
StoreFileComparators.SEQ_ID_MAX_TIMESTAMP
Comparator for time-aware compaction.
|
private Comparator<HStoreFile> |
DefaultStoreFileManager.storeFileComparator |
private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> |
DefaultStoreFileManager.storefiles
List of store files inside this store.
|
ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>> |
StripeStoreFileManager.State.stripeFiles
Files by stripe.
|
private ArrayList<List<HStoreFile>> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.stripeFiles |
Modifier and Type | Method and Description |
---|---|
private HStoreFile |
HStore.commitFile(org.apache.hadoop.fs.Path path,
long logCacheFlushId,
MonitoredTask status) |
protected HStoreFile |
HStore.createStoreFileAndReader(org.apache.hadoop.fs.Path p) |
private HStoreFile |
HStore.createStoreFileAndReader(StoreFileInfo info) |
(package private) HStoreFile |
HStore.moveFileIntoPlace(org.apache.hadoop.fs.Path newFile) |
Modifier and Type | Method and Description |
---|---|
Collection<HStoreFile> |
StoreFileManager.clearCompactedFiles()
Clears all the compacted files and returns them.
|
Collection<HStoreFile> |
DefaultStoreFileManager.clearCompactedFiles() |
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> |
StripeStoreFileManager.clearCompactedFiles() |
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> |
StoreFileManager.clearFiles()
Clears all the files currently in use and returns them.
|
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> |
DefaultStoreFileManager.clearFiles() |
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> |
StripeStoreFileManager.clearFiles() |
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> |
HStore.close()
Close all the readers We don't need to worry about subsequent requests because the Region holds
a write lock that will prevent any more reads or writes.
|
Map<byte[],List<HStoreFile>> |
HRegion.close()
Close down this HRegion.
|
Map<byte[],List<HStoreFile>> |
HRegion.close(boolean abort)
Close down this HRegion.
|
List<HStoreFile> |
HStore.compact(CompactionContext compaction,
ThroughputController throughputController,
User user)
Compact the StoreFiles.
|
private Map<byte[],List<HStoreFile>> |
HRegion.doClose(boolean abort,
MonitoredTask status) |
protected List<HStoreFile> |
HStore.doCompaction(CompactionRequestImpl cr,
Collection<HStoreFile> filesToCompact,
User user,
long compactionStartTime,
List<org.apache.hadoop.fs.Path> newFiles) |
private Collection<HStoreFile> |
StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe,
long maxTs,
List<HStoreFile> filesCompacting,
Collection<HStoreFile> expiredStoreFiles) |
Iterator<HStoreFile> |
StoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue targetKey)
Gets initial, full list of candidate store files to check for row-key-before.
|
Iterator<HStoreFile> |
DefaultStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue targetKey) |
Iterator<HStoreFile> |
StripeStoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue targetKey)
See
StoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue)
for details on this methods. |
Collection<HStoreFile> |
StoreFileManager.getCompactedfiles()
List of compacted files inside this store that needs to be excluded in reads
because further new reads will be using only the newly created files out of compaction.
|
Collection<HStoreFile> |
DefaultStoreFileManager.getCompactedfiles() |
Collection<HStoreFile> |
StripeStoreFileManager.getCompactedfiles() |
Collection<HStoreFile> |
HStore.getCompactedFiles() |
ArrayList<List<HStoreFile>> |
StripeStoreFileManager.KeyBeforeConcatenatedLists.Iterator.getComponents() |
Collection<HStoreFile> |
StoreFileManager.getFilesForScan(byte[] startRow,
boolean includeStartRow,
byte[] stopRow,
boolean includeStopRow)
Gets the store files to scan for a Scan or Get request.
|
Collection<HStoreFile> |
DefaultStoreFileManager.getFilesForScan(byte[] startRow,
boolean includeStartRow,
byte[] stopRow,
boolean includeStopRow) |
Collection<HStoreFile> |
StripeStoreFileManager.getFilesForScan(byte[] startRow,
boolean includeStartRow,
byte[] stopRow,
boolean includeStopRow) |
(package private) static Optional<HStoreFile> |
StoreUtils.getLargestFile(Collection<HStoreFile> candidates)
Gets the largest file (with reader) out of the list of files.
|
private ArrayList<HStoreFile> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.getLevel0Copy() |
List<HStoreFile> |
StripeStoreFileManager.getLevel0Files() |
Comparator<HStoreFile> |
StoreFileManager.getStoreFileComparator() |
Comparator<HStoreFile> |
DefaultStoreFileManager.getStoreFileComparator() |
Comparator<HStoreFile> |
StripeStoreFileManager.getStoreFileComparator() |
Collection<HStoreFile> |
StoreFileManager.getStorefiles()
Gets the snapshot of the store files currently in use.
|
Collection<HStoreFile> |
HStore.getStorefiles() |
Collection<HStoreFile> |
DefaultStoreFileManager.getStorefiles() |
Collection<HStoreFile> |
StripeStoreFileManager.getStorefiles() |
private ArrayList<HStoreFile> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.getStripeCopy(int index) |
ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>> |
StripeStoreFileManager.getStripes() |
Collection<HStoreFile> |
StoreFileManager.getUnneededFiles(long maxTs,
List<HStoreFile> filesCompacting) |
Collection<HStoreFile> |
DefaultStoreFileManager.getUnneededFiles(long maxTs,
List<HStoreFile> filesCompacting) |
Collection<HStoreFile> |
StripeStoreFileManager.getUnneededFiles(long maxTs,
List<HStoreFile> filesCompacting) |
Iterator<HStoreFile> |
StripeStoreFileManager.KeyBeforeConcatenatedLists.iterator() |
private List<HStoreFile> |
HStore.loadStoreFiles()
Creates an unsorted list of StoreFile loaded in parallel
from the given directory.
|
private List<HStoreFile> |
HStore.moveCompactedFilesIntoPlace(CompactionRequestImpl cr,
List<org.apache.hadoop.fs.Path> newFiles,
User user) |
private List<HStoreFile> |
HStore.openStoreFiles(Collection<StoreFileInfo> files) |
List<HStoreFile> |
StripeStoreEngine.StripeCompaction.preSelect(List<HStoreFile> filesCompacting) |
List<HStoreFile> |
DefaultStoreEngine.DefaultCompactionContext.preSelect(List<HStoreFile> filesCompacting) |
List<HStoreFile> |
DateTieredStoreEngine.DateTieredCompactionContext.preSelect(List<HStoreFile> filesCompacting) |
private TreeMap<byte[],HStoreFile> |
StripeStoreFileManager.CompactionOrFlushMergeCopy.processResults()
Process new files, and add them either to the structure of existing stripes,
or to the list of new candidate stripes.
|
Iterator<HStoreFile> |
StoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
KeyValue targetKey,
Cell candidate)
Updates the candidate list for finding row key before.
|
Iterator<HStoreFile> |
DefaultStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
KeyValue targetKey,
Cell candidate) |
Iterator<HStoreFile> |
StripeStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
KeyValue targetKey,
Cell candidate)
See
StoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue) and
StoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator, KeyValue, Cell)
for details on this methods. |
Modifier and Type | Method and Description |
---|---|
Long |
StoreFileComparators.GetSeqId.apply(HStoreFile sf) |
Long |
StoreFileComparators.GetFileSize.apply(HStoreFile sf) |
Long |
StoreFileComparators.GetBulkTime.apply(HStoreFile sf) |
String |
StoreFileComparators.GetPathName.apply(HStoreFile sf) |
Long |
StoreFileComparators.GetMaxTimestamp.apply(HStoreFile sf) |
private void |
HStore.bulkLoadHFile(HStoreFile sf) |
private byte[] |
StripeStoreFileManager.endOf(HStoreFile sf) |
private void |
StripeStoreFileManager.ensureLevel0Metadata(HStoreFile sf) |
(package private) static Optional<byte[]> |
StoreUtils.getFileSplitPoint(HStoreFile file,
CellComparator comparator)
Gets the approximate mid-point of the given file that is optimal for use in splitting it.
|
private static void |
StripeStoreFileManager.insertFileIntoStripe(ArrayList<HStoreFile> stripe,
HStoreFile sf)
Inserts a file in the correct place (by seqnum) in a stripe copy.
|
org.apache.hadoop.fs.Path |
HRegionFileSystem.mergeStoreFile(RegionInfo mergedRegion,
String familyName,
HStoreFile f,
org.apache.hadoop.fs.Path mergedDir)
Write out a merge reference under the given merges directory.
|
void |
RegionCoprocessorHost.postCompact(HStore store,
HStoreFile resultFile,
CompactionLifeCycleTracker tracker,
CompactionRequest request,
User user)
Called after the store compaction has completed.
|
void |
RegionCoprocessorHost.postFlush(HStore store,
HStoreFile storeFile,
FlushLifeCycleTracker tracker)
Invoked after a memstore flush
|
org.apache.hadoop.fs.Path |
HRegionFileSystem.splitStoreFile(RegionInfo hri,
String familyName,
HStoreFile f,
byte[] splitRow,
boolean top,
RegionSplitPolicy splitPolicy)
Write out a split reference.
|
private byte[] |
StripeStoreFileManager.startOf(HStoreFile sf) |
Modifier and Type | Method and Description |
---|---|
void |
StoreFileManager.addCompactionResults(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> results)
Adds only the new compaction results into the structure.
|
void |
StoreFileManager.addCompactionResults(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> results)
Adds only the new compaction results into the structure.
|
void |
DefaultStoreFileManager.addCompactionResults(Collection<HStoreFile> newCompactedfiles,
Collection<HStoreFile> results) |
void |
DefaultStoreFileManager.addCompactionResults(Collection<HStoreFile> newCompactedfiles,
Collection<HStoreFile> results) |
void |
StripeStoreFileManager.addCompactionResults(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> results) |
void |
StripeStoreFileManager.addCompactionResults(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> results) |
private void |
HStore.addToCompactingFiles(Collection<HStoreFile> filesToAdd)
Adds the files to compacting files.
|
private void |
HStore.clearCompactedfiles(List<HStoreFile> filesToRemove) |
protected void |
HStore.completeCompaction(Collection<HStoreFile> compactedFiles)
Update counts.
|
private void |
StripeStoreFileManager.CompactionOrFlushMergeCopy.deleteResults(Collection<HStoreFile> compactedFiles) |
protected List<HStoreFile> |
HStore.doCompaction(CompactionRequestImpl cr,
Collection<HStoreFile> filesToCompact,
User user,
long compactionStartTime,
List<org.apache.hadoop.fs.Path> newFiles) |
private void |
StripeStoreFileManager.ensureEdgeStripeMetadata(ArrayList<HStoreFile> stripe,
boolean isFirst) |
private Collection<HStoreFile> |
StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe,
long maxTs,
List<HStoreFile> filesCompacting,
Collection<HStoreFile> expiredStoreFiles) |
private Collection<HStoreFile> |
StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe,
long maxTs,
List<HStoreFile> filesCompacting,
Collection<HStoreFile> expiredStoreFiles) |
private Collection<HStoreFile> |
StripeStoreFileManager.findExpiredFiles(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> stripe,
long maxTs,
List<HStoreFile> filesCompacting,
Collection<HStoreFile> expiredStoreFiles) |
static OptionalInt |
StoreUtils.getDeterministicRandomSeed(Collection<HStoreFile> files)
Creates a deterministic hash code for store file collection.
|
(package private) static Optional<HStoreFile> |
StoreUtils.getLargestFile(Collection<HStoreFile> candidates)
Gets the largest file (with reader) out of the list of files.
|
static long |
StoreUtils.getLowestTimestamp(Collection<HStoreFile> candidates)
Gets lowest timestamp from candidate StoreFiles
|
static OptionalLong |
StoreUtils.getMaxMemStoreTSInList(Collection<HStoreFile> sfs)
Return the largest memstoreTS found across all storefiles in the given list.
|
static OptionalLong |
StoreUtils.getMaxSequenceIdInList(Collection<HStoreFile> sfs)
Return the highest sequence ID found across all storefiles in the given list.
|
List<KeyValueScanner> |
HStore.getScanners(List<HStoreFile> files,
boolean cacheBlocks,
boolean isGet,
boolean usePread,
boolean isCompaction,
ScanQueryMatcher matcher,
byte[] startRow,
byte[] stopRow,
long readPt,
boolean includeMemstoreScanner)
Create scanners on the given files and if needed on the memstore with no filtering based on TTL
(that happens further down the line).
|
List<KeyValueScanner> |
HStore.getScanners(List<HStoreFile> files,
boolean cacheBlocks,
boolean usePread,
boolean isCompaction,
ScanQueryMatcher matcher,
byte[] startRow,
boolean includeStartRow,
byte[] stopRow,
boolean includeStopRow,
long readPt,
boolean includeMemstoreScanner)
Create scanners on the given files and if needed on the memstore with no filtering based on TTL
(that happens further down the line).
|
static List<StoreFileScanner> |
StoreFileScanner.getScannersForCompaction(Collection<HStoreFile> files,
boolean canUseDropBehind,
long readPt)
Get scanners for compaction.
|
static List<StoreFileScanner> |
StoreFileScanner.getScannersForStoreFiles(Collection<HStoreFile> files,
boolean cacheBlocks,
boolean usePread,
boolean isCompaction,
boolean useDropBehind,
long readPt)
Return an array of scanners corresponding to the given set of store files.
|
static List<StoreFileScanner> |
StoreFileScanner.getScannersForStoreFiles(Collection<HStoreFile> files,
boolean cacheBlocks,
boolean usePread,
boolean isCompaction,
boolean canUseDrop,
ScanQueryMatcher matcher,
long readPt)
Return an array of scanners corresponding to the given set of store files, And set the
ScanQueryMatcher for each store file scanner for further optimization
|
static List<StoreFileScanner> |
StoreFileScanner.getScannersForStoreFiles(Collection<HStoreFile> files,
boolean cacheBlocks,
boolean usePread,
long readPt)
Return an array of scanners corresponding to the given set of store files.
|
(package private) static Optional<byte[]> |
StoreUtils.getSplitPoint(Collection<HStoreFile> storefiles,
CellComparator comparator)
Gets the mid point of the largest file passed in as split point.
|
private long |
HStore.getStorefilesSize(Collection<HStoreFile> files,
Predicate<HStoreFile> predicate) |
private long |
HStore.getStorefilesSize(Collection<HStoreFile> files,
Predicate<HStoreFile> predicate) |
private long |
HStore.getTotalSize(Collection<HStoreFile> sfs) |
private long |
HStore.getTotalUmcompressedBytes(List<HStoreFile> files) |
Collection<HStoreFile> |
StoreFileManager.getUnneededFiles(long maxTs,
List<HStoreFile> filesCompacting) |
Collection<HStoreFile> |
DefaultStoreFileManager.getUnneededFiles(long maxTs,
List<HStoreFile> filesCompacting) |
Collection<HStoreFile> |
StripeStoreFileManager.getUnneededFiles(long maxTs,
List<HStoreFile> filesCompacting) |
static boolean |
StoreUtils.hasReferences(Collection<HStoreFile> files)
Determines whether any files in the collection are references.
|
private static void |
StripeStoreFileManager.insertFileIntoStripe(ArrayList<HStoreFile> stripe,
HStoreFile sf)
Inserts a file in the correct place (by seqnum) in a stripe copy.
|
void |
StoreFileManager.insertNewFiles(Collection<HStoreFile> sfs)
Adds new files, either for from MemStore flush or bulk insert, into the structure.
|
void |
DefaultStoreFileManager.insertNewFiles(Collection<HStoreFile> sfs) |
void |
StripeStoreFileManager.insertNewFiles(Collection<HStoreFile> sfs) |
void |
StoreFileManager.loadFiles(List<HStoreFile> storeFiles)
Loads the initial store files into empty StoreFileManager.
|
void |
DefaultStoreFileManager.loadFiles(List<HStoreFile> storeFiles) |
void |
StripeStoreFileManager.loadFiles(List<HStoreFile> storeFiles) |
private void |
StripeStoreFileManager.loadUnclassifiedStoreFiles(List<HStoreFile> storeFiles)
Loads initial store files that were picked up from some physical location pertaining to
this store (presumably).
|
private void |
HStore.logCompactionEndMessage(CompactionRequestImpl cr,
List<HStoreFile> sfs,
long now,
long compactionStartTime)
Log a very elaborate compaction completion message.
|
private void |
StripeStoreFileManager.markCompactedAway(Collection<HStoreFile> compactedFiles) |
private void |
StripeStoreFileManager.CompactionOrFlushMergeCopy.mergeResults(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> results) |
private void |
StripeStoreFileManager.CompactionOrFlushMergeCopy.mergeResults(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> results) |
boolean |
StripeStoreEngine.needsCompaction(List<HStoreFile> filesCompacting) |
boolean |
DefaultStoreEngine.needsCompaction(List<HStoreFile> filesCompacting) |
abstract boolean |
StoreEngine.needsCompaction(List<HStoreFile> filesCompacting) |
boolean |
DateTieredStoreEngine.needsCompaction(List<HStoreFile> filesCompacting) |
private void |
HStore.notifyChangedReadersObservers(List<HStoreFile> sfs)
Notify all observers that set of Readers has changed.
|
void |
RegionCoprocessorHost.postCompactSelection(HStore store,
List<HStoreFile> selected,
CompactionLifeCycleTracker tracker,
CompactionRequest request,
User user)
Called after the
HStoreFile s to be compacted have been selected from the available
candidates. |
boolean |
RegionCoprocessorHost.preCompactSelection(HStore store,
List<HStoreFile> candidates,
CompactionLifeCycleTracker tracker,
User user)
Called prior to selecting the
HStoreFile s for compaction from the list of currently
available candidates. |
List<HStoreFile> |
StripeStoreEngine.StripeCompaction.preSelect(List<HStoreFile> filesCompacting) |
List<HStoreFile> |
DefaultStoreEngine.DefaultCompactionContext.preSelect(List<HStoreFile> filesCompacting) |
List<HStoreFile> |
DateTieredStoreEngine.DateTieredCompactionContext.preSelect(List<HStoreFile> filesCompacting) |
private void |
StripeStoreFileManager.CompactionOrFlushMergeCopy.processNewCandidateStripes(TreeMap<byte[],HStoreFile> newStripes)
See
StripeStoreFileManager.addCompactionResults(Collection, Collection) - updates the stripe list with
new candidate stripes/removes old stripes; produces new set of stripe end rows. |
void |
StoreFileManager.removeCompactedFiles(Collection<HStoreFile> compactedFiles)
Remove the compacted files
|
void |
DefaultStoreFileManager.removeCompactedFiles(Collection<HStoreFile> removedCompactedfiles) |
void |
StripeStoreFileManager.removeCompactedFiles(Collection<HStoreFile> compactedFiles) |
private void |
HStore.removeCompactedfiles(Collection<HStoreFile> compactedfiles,
boolean storeClosing)
Archives and removes the compacted files
|
void |
HRegionFileSystem.removeStoreFiles(String familyName,
Collection<HStoreFile> storeFiles)
Closes and archives the specified store files from the specified family.
|
(package private) void |
HStore.replaceStoreFiles(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> result) |
(package private) void |
HStore.replaceStoreFiles(Collection<HStoreFile> compactedFiles,
Collection<HStoreFile> result) |
boolean |
StripeStoreEngine.StripeCompaction.select(List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor) |
boolean |
DefaultStoreEngine.DefaultCompactionContext.select(List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor) |
boolean |
DateTieredStoreEngine.DateTieredCompactionContext.select(List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor) |
Iterator<HStoreFile> |
StoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
KeyValue targetKey,
Cell candidate)
Updates the candidate list for finding row key before.
|
Iterator<HStoreFile> |
DefaultStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
KeyValue targetKey,
Cell candidate) |
Iterator<HStoreFile> |
StripeStoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
KeyValue targetKey,
Cell candidate)
See
StoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue) and
StoreFileManager.updateCandidateFilesForRowKeyBefore(Iterator, KeyValue, Cell)
for details on this methods. |
void |
StoreScanner.updateReaders(List<HStoreFile> sfs,
List<KeyValueScanner> memStoreScanners) |
void |
ChangedReadersObserver.updateReaders(List<HStoreFile> sfs,
List<KeyValueScanner> memStoreScanners)
Notify observers.
|
private boolean |
HStore.updateStorefiles(List<HStoreFile> sfs,
long snapshotId)
Change storeFiles adding into place the Reader produced by this new flush.
|
private void |
HStore.writeCompactionWalRecord(Collection<HStoreFile> filesCompacted,
Collection<HStoreFile> newFiles)
Writes the compaction WAL record.
|
private void |
HStore.writeCompactionWalRecord(Collection<HStoreFile> filesCompacted,
Collection<HStoreFile> newFiles)
Writes the compaction WAL record.
|
Constructor and Description |
---|
DefaultStoreFileManager(CellComparator cellComparator,
Comparator<HStoreFile> storeFileComparator,
org.apache.hadoop.conf.Configuration conf,
CompactionConfiguration comConf) |
Modifier and Type | Field and Description |
---|---|
private Collection<HStoreFile> |
CompactionRequestImpl.filesToCompact |
Modifier and Type | Method and Description |
---|---|
protected ArrayList<HStoreFile> |
ExploringCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates,
boolean mayUseOffPeak,
boolean mightBeStuck) |
protected ArrayList<HStoreFile> |
RatioBasedCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates,
boolean mayUseOffPeak,
boolean mayBeStuck)
-- Default minor compaction selection algorithm:
choose CompactSelection from candidates --
First exclude bulk-load files if indicated in configuration.
|
List<HStoreFile> |
ExploringCompactionPolicy.applyCompactionPolicy(List<HStoreFile> candidates,
boolean mightBeStuck,
boolean mayUseOffPeak,
int minFiles,
int maxFiles) |
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.checkMinFilesCriteria(ArrayList<HStoreFile> candidates,
int minFiles) |
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.filterBulk(ArrayList<HStoreFile> candidates) |
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.getCurrentEligibleFiles(ArrayList<HStoreFile> candidateFiles,
List<HStoreFile> filesCompacting) |
private Collection<HStoreFile> |
FIFOCompactionPolicy.getExpiredStores(Collection<HStoreFile> files,
Collection<HStoreFile> filesCompacting) |
Collection<HStoreFile> |
CompactionRequestImpl.getFiles() |
List<HStoreFile> |
StripeCompactionPolicy.StripeInformationProvider.getLevel0Files() |
Collection<HStoreFile> |
StripeCompactionPolicy.StripeInformationProvider.getStorefiles() |
ArrayList<org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>> |
StripeCompactionPolicy.StripeInformationProvider.getStripes() |
abstract List<HStoreFile> |
CompactionContext.preSelect(List<HStoreFile> filesCompacting)
Called before coprocessor preCompactSelection and should filter the candidates
for coprocessor; i.e.
|
List<HStoreFile> |
SortedCompactionPolicy.preSelectCompactionForCoprocessor(Collection<HStoreFile> candidates,
List<HStoreFile> filesCompacting) |
List<HStoreFile> |
StripeCompactionPolicy.preSelectFilesForCoprocessor(StripeCompactionPolicy.StripeInformationProvider si,
List<HStoreFile> filesCompacting) |
private List<HStoreFile> |
StripeCompactionPolicy.selectSimpleCompaction(List<HStoreFile> sfs,
boolean allFilesOnly,
boolean isOffpeak)
Selects the compaction of a single stripe using default policy.
|
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.skipLargeFiles(ArrayList<HStoreFile> candidates,
boolean mayUseOffpeak) |
Modifier and Type | Method and Description |
---|---|
private boolean |
FIFOCompactionPolicy.isEmptyStoreFile(HStoreFile sf)
The FIFOCompactionPolicy only choose those TTL expired HFiles as the compaction candidates.
|
Modifier and Type | Method and Description |
---|---|
protected ArrayList<HStoreFile> |
ExploringCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates,
boolean mayUseOffPeak,
boolean mightBeStuck) |
protected ArrayList<HStoreFile> |
RatioBasedCompactionPolicy.applyCompactionPolicy(ArrayList<HStoreFile> candidates,
boolean mayUseOffPeak,
boolean mayBeStuck)
-- Default minor compaction selection algorithm:
choose CompactSelection from candidates --
First exclude bulk-load files if indicated in configuration.
|
List<HStoreFile> |
ExploringCompactionPolicy.applyCompactionPolicy(List<HStoreFile> candidates,
boolean mightBeStuck,
boolean mayUseOffPeak,
int minFiles,
int maxFiles) |
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.checkMinFilesCriteria(ArrayList<HStoreFile> candidates,
int minFiles) |
List<org.apache.hadoop.fs.Path> |
DefaultCompactor.compactForTesting(Collection<HStoreFile> filesToCompact,
boolean isMajor)
Compact a list of files for testing.
|
protected CompactionRequestImpl |
DateTieredCompactionPolicy.createCompactionRequest(ArrayList<HStoreFile> candidateSelection,
boolean tryingMajor,
boolean mayUseOffPeak,
boolean mayBeStuck) |
protected CompactionRequestImpl |
RatioBasedCompactionPolicy.createCompactionRequest(ArrayList<HStoreFile> candidateSelection,
boolean tryingMajor,
boolean mayUseOffPeak,
boolean mayBeStuck) |
protected abstract CompactionRequestImpl |
SortedCompactionPolicy.createCompactionRequest(ArrayList<HStoreFile> candidateSelection,
boolean tryingMajor,
boolean mayUseOffPeak,
boolean mayBeStuck) |
private List<StoreFileScanner> |
Compactor.createFileScanners(Collection<HStoreFile> filesToCompact,
long smallestReadPoint,
boolean useDropBehind)
Creates file scanners for compaction.
|
private Pair<Long,Integer> |
StripeCompactionPolicy.estimateTargetKvs(Collection<HStoreFile> files,
double splitCount) |
private boolean |
ExploringCompactionPolicy.filesInRatio(List<HStoreFile> files,
double currentRatio)
Check that all files satisfy the constraint
FileSize(i) <= ( Sum(0,N,FileSize(_)) - FileSize(i) ) * Ratio.
|
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.filterBulk(ArrayList<HStoreFile> candidates) |
private DateTieredCompactionRequest |
DateTieredCompactionPolicy.generateCompactionRequest(ArrayList<HStoreFile> storeFiles,
CompactionWindow window,
boolean mayUseOffPeak,
boolean mayBeStuck,
int minThreshold) |
private List<Long> |
DateTieredCompactionPolicy.getCompactBoundariesForMajor(Collection<HStoreFile> filesToCompact,
long now)
Return a list of boundaries for multiple compaction output in ascending order.
|
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.getCurrentEligibleFiles(ArrayList<HStoreFile> candidateFiles,
List<HStoreFile> filesCompacting) |
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.getCurrentEligibleFiles(ArrayList<HStoreFile> candidateFiles,
List<HStoreFile> filesCompacting) |
private Collection<HStoreFile> |
FIFOCompactionPolicy.getExpiredStores(Collection<HStoreFile> files,
Collection<HStoreFile> filesCompacting) |
private Collection<HStoreFile> |
FIFOCompactionPolicy.getExpiredStores(Collection<HStoreFile> files,
Collection<HStoreFile> filesCompacting) |
private Compactor.FileDetails |
Compactor.getFileDetails(Collection<HStoreFile> filesToCompact,
boolean allFiles)
Extracts some details about the files to compact that are commonly needed by compactors.
|
long |
SortedCompactionPolicy.getNextMajorCompactTime(Collection<HStoreFile> filesToCompact) |
static long |
StripeCompactionPolicy.getTotalFileSize(Collection<HStoreFile> candidates) |
private static long |
StripeCompactionPolicy.getTotalKvCount(Collection<HStoreFile> candidates) |
private long |
ExploringCompactionPolicy.getTotalStoreSize(List<HStoreFile> potentialMatchFiles)
Find the total size of a list of store files.
|
private boolean |
FIFOCompactionPolicy.hasExpiredStores(Collection<HStoreFile> files) |
private boolean |
ExploringCompactionPolicy.isBetterSelection(List<HStoreFile> bestSelection,
long bestSize,
List<HStoreFile> selection,
long size,
boolean mightBeStuck) |
private boolean |
ExploringCompactionPolicy.isBetterSelection(List<HStoreFile> bestSelection,
long bestSize,
List<HStoreFile> selection,
long size,
boolean mightBeStuck) |
boolean |
DateTieredCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting)
Heuristics for guessing whether we need minor compaction.
|
boolean |
DateTieredCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting)
Heuristics for guessing whether we need minor compaction.
|
boolean |
RatioBasedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting)
A heuristic method to decide whether to schedule a compaction request
|
boolean |
RatioBasedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting)
A heuristic method to decide whether to schedule a compaction request
|
abstract boolean |
SortedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting) |
abstract boolean |
SortedCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting) |
boolean |
FIFOCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting) |
boolean |
FIFOCompactionPolicy.needsCompaction(Collection<HStoreFile> storeFiles,
List<HStoreFile> filesCompacting) |
boolean |
StripeCompactionPolicy.needsCompactions(StripeCompactionPolicy.StripeInformationProvider si,
List<HStoreFile> filesCompacting) |
abstract List<HStoreFile> |
CompactionContext.preSelect(List<HStoreFile> filesCompacting)
Called before coprocessor preCompactSelection and should filter the candidates
for coprocessor; i.e.
|
List<HStoreFile> |
SortedCompactionPolicy.preSelectCompactionForCoprocessor(Collection<HStoreFile> candidates,
List<HStoreFile> filesCompacting) |
List<HStoreFile> |
SortedCompactionPolicy.preSelectCompactionForCoprocessor(Collection<HStoreFile> candidates,
List<HStoreFile> filesCompacting) |
List<HStoreFile> |
StripeCompactionPolicy.preSelectFilesForCoprocessor(StripeCompactionPolicy.StripeInformationProvider si,
List<HStoreFile> filesCompacting) |
protected void |
SortedCompactionPolicy.removeExcessFiles(ArrayList<HStoreFile> candidates,
boolean isUserCompaction,
boolean isMajorCompaction) |
abstract boolean |
CompactionContext.select(List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor)
Called to select files for compaction.
|
CompactionRequestImpl |
SortedCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles,
List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor) |
CompactionRequestImpl |
SortedCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles,
List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor) |
CompactionRequestImpl |
FIFOCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles,
List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor) |
CompactionRequestImpl |
FIFOCompactionPolicy.selectCompaction(Collection<HStoreFile> candidateFiles,
List<HStoreFile> filesCompacting,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean forceMajor) |
StripeCompactionPolicy.StripeCompactionRequest |
StripeCompactionPolicy.selectCompaction(StripeCompactionPolicy.StripeInformationProvider si,
List<HStoreFile> filesCompacting,
boolean isOffpeak) |
CompactionRequestImpl |
DateTieredCompactionPolicy.selectMajorCompaction(ArrayList<HStoreFile> candidateSelection) |
CompactionRequestImpl |
DateTieredCompactionPolicy.selectMinorCompaction(ArrayList<HStoreFile> candidateSelection,
boolean mayUseOffPeak,
boolean mayBeStuck)
We receive store files sorted in ascending order by seqId then scan the list of files.
|
private List<HStoreFile> |
StripeCompactionPolicy.selectSimpleCompaction(List<HStoreFile> sfs,
boolean allFilesOnly,
boolean isOffpeak)
Selects the compaction of a single stripe using default policy.
|
abstract boolean |
CompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) |
boolean |
DateTieredCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) |
boolean |
RatioBasedCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) |
boolean |
StripeCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) |
abstract boolean |
SortedCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) |
boolean |
FIFOCompactionPolicy.shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) |
protected ArrayList<HStoreFile> |
SortedCompactionPolicy.skipLargeFiles(ArrayList<HStoreFile> candidates,
boolean mayUseOffpeak) |
void |
CompactionRequestImpl.updateFiles(Collection<HStoreFile> files) |
Constructor and Description |
---|
BoundaryStripeCompactionRequest(Collection<HStoreFile> files,
List<byte[]> targetBoundaries) |
CompactionRequestImpl(Collection<HStoreFile> files) |
DateTieredCompactionRequest(Collection<HStoreFile> files,
List<Long> boundaryList) |
SplitStripeCompactionRequest(Collection<HStoreFile> files,
byte[] startRow,
byte[] endRow,
int targetCount,
long targetKvs) |
SplitStripeCompactionRequest(Collection<HStoreFile> files,
byte[] startRow,
byte[] endRow,
long targetKvs) |
Copyright © 2007–2019 The Apache Software Foundation. All rights reserved.