Class SortedCompactionPolicy
java.lang.Object
org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
- Direct Known Subclasses:
DateTieredCompactionPolicy
,RatioBasedCompactionPolicy
An abstract compaction policy that select files on seq id order.
-
Field Summary
Modifier and TypeFieldDescriptionprivate static final org.slf4j.Logger
private static final Random
Fields inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
comConf, storeConfigInfo
-
Constructor Summary
ConstructorDescriptionSortedCompactionPolicy
(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo) -
Method Summary
Modifier and TypeMethodDescriptionprotected ArrayList<HStoreFile>
checkMinFilesCriteria
(ArrayList<HStoreFile> candidates, int minFiles) protected abstract CompactionRequestImpl
createCompactionRequest
(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) protected void
filterBulk
(ArrayList<HStoreFile> candidates) protected ArrayList<HStoreFile>
getCurrentEligibleFiles
(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) long
getNextMajorCompactTime
(Collection<HStoreFile> filesToCompact) Returns When to run next major compactionabstract boolean
needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) preSelectCompactionForCoprocessor
(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) protected void
removeExcessFiles
(ArrayList<HStoreFile> candidates, boolean isUserCompaction, boolean isMajorCompaction) selectCompaction
(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) abstract boolean
shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) protected ArrayList<HStoreFile>
skipLargeFiles
(ArrayList<HStoreFile> candidates, boolean mayUseOffpeak) boolean
throttleCompaction
(long compactionSize) Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
getConf, setConf
-
Field Details
-
LOG
-
RNG
-
-
Constructor Details
-
SortedCompactionPolicy
public SortedCompactionPolicy(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo)
-
-
Method Details
-
preSelectCompactionForCoprocessor
public List<HStoreFile> preSelectCompactionForCoprocessor(Collection<HStoreFile> candidates, List<HStoreFile> filesCompacting) -
selectCompaction
public CompactionRequestImpl selectCompaction(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) throws IOException - Parameters:
candidateFiles
- candidate files, ordered from oldest to newest by seqId. We rely on DefaultStoreFileManager to sort the files by seqId to guarantee contiguous compaction based on seqId for data consistency.- Returns:
- subset copy of candidate list that meets compaction criteria
- Throws:
IOException
-
createCompactionRequest
protected abstract CompactionRequestImpl createCompactionRequest(ArrayList<HStoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck) throws IOException - Throws:
IOException
-
shouldPerformMajorCompaction
public abstract boolean shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) throws IOException - Specified by:
shouldPerformMajorCompaction
in classCompactionPolicy
- Parameters:
filesToCompact
- Files to compact. Can be null.- Returns:
- True if we should run a major compaction.
- Throws:
IOException
-
getNextMajorCompactTime
Returns When to run next major compaction -
throttleCompaction
- Specified by:
throttleCompaction
in classCompactionPolicy
- Parameters:
compactionSize
- Total size of some compaction- Returns:
- whether this should be a large or small compaction
-
needsCompaction
public abstract boolean needsCompaction(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) -
getCurrentEligibleFiles
protected ArrayList<HStoreFile> getCurrentEligibleFiles(ArrayList<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting) -
skipLargeFiles
protected ArrayList<HStoreFile> skipLargeFiles(ArrayList<HStoreFile> candidates, boolean mayUseOffpeak) - Parameters:
candidates
- pre-filtrate- Returns:
- filtered subset exclude all files above maxCompactSize Also save all references. We MUST compact them
-
filterBulk
- Parameters:
candidates
- pre-filtrate
-
removeExcessFiles
protected void removeExcessFiles(ArrayList<HStoreFile> candidates, boolean isUserCompaction, boolean isMajorCompaction) - Parameters:
candidates
- pre-filtrate
-
checkMinFilesCriteria
protected ArrayList<HStoreFile> checkMinFilesCriteria(ArrayList<HStoreFile> candidates, int minFiles) - Parameters:
candidates
- pre-filtrate- Returns:
- filtered subset forget the compactionSelection if we don't have enough files
-