Class RatioBasedCompactionPolicy
java.lang.Object
org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy
- Direct Known Subclasses:
ExploringCompactionPolicy
The default algorithm for selecting files for compaction. Combines the compaction configuration
and the provisional file selection that it's given to produce the list of suitable candidates for
compaction.
-
Field Summary
Fields inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
comConf, storeConfigInfo
-
Constructor Summary
ConstructorDescriptionRatioBasedCompactionPolicy
(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo) -
Method Summary
Modifier and TypeMethodDescriptionprotected ArrayList<HStoreFile>
applyCompactionPolicy
(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mayBeStuck) -- Default minor compaction selection algorithm: choose CompactSelection from candidates -- First exclude bulk-load files if indicated in configuration.protected CompactionRequestImpl
createCompactionRequest
(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) boolean
needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) A heuristic method to decide whether to schedule a compaction requestvoid
setMinThreshold
(int minThreshold) Overwrite min threshold for compactionboolean
shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
checkMinFilesCriteria, filterBulk, getCurrentEligibleFiles, getNextMajorCompactTime, preSelectCompactionForCoprocessor, removeExcessFiles, selectCompaction, skipLargeFiles, throttleCompaction
Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
getConf, setConf
-
Field Details
-
LOG
-
-
Constructor Details
-
RatioBasedCompactionPolicy
public RatioBasedCompactionPolicy(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo)
-
-
Method Details
-
shouldPerformMajorCompaction
public boolean shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) throws IOException - Specified by:
shouldPerformMajorCompaction
in classSortedCompactionPolicy
- Parameters:
filesToCompact
- Files to compact. Can be null.- Returns:
- True if we should run a major compaction.
- Throws:
IOException
-
createCompactionRequest
protected CompactionRequestImpl createCompactionRequest(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) throws IOException - Specified by:
createCompactionRequest
in classSortedCompactionPolicy
- Throws:
IOException
-
applyCompactionPolicy
protected ArrayList<HStoreFile> applyCompactionPolicy(ArrayList<HStoreFile> candidates, boolean mayUseOffPeak, boolean mayBeStuck) throws IOException -- Default minor compaction selection algorithm: choose CompactSelection from candidates -- First exclude bulk-load files if indicated in configuration. Start at the oldest file and stop when you find the first file that meets compaction criteria: (1) a recently-flushed, small file (i.e. <= minCompactSize) OR (2) within the compactRatio of sum(newer_files) Given normal skew, any newer files will also meet this criteria Additional Note: If fileSizes.size() >> maxFilesToCompact, we will recurse on compact(). Consider the oldest files first to avoid a situation where we always compact [end-threshold,end). Then, the last file becomes an aggregate of the previous compactions. normal skew: older ----> newer (increasing seqID) _ | | _ | | | | _ --|-|- |-|- |-|---_-------_------- minCompactSize | | | | | | | | _ | | | | | | | | | | | | | | | | | | | | | | | | | |- Parameters:
candidates
- pre-filtrate- Returns:
- filtered subset
- Throws:
IOException
-
needsCompaction
A heuristic method to decide whether to schedule a compaction request- Specified by:
needsCompaction
in classSortedCompactionPolicy
- Parameters:
storeFiles
- files in the store.filesCompacting
- files being scheduled to compact.- Returns:
- true to schedule a request.
-
setMinThreshold
Overwrite min threshold for compaction
-