Class FIFOCompactionPolicy
java.lang.Object
org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy
FIFO compaction policy selects only files which have all cells expired. The column family MUST
have non-default TTL. One of the use cases for this policy is when we need to store raw data
which will be post-processed later and discarded completely after quite short period of time. Raw
time-series vs. time-based roll up aggregates and compacted time-series. We collect raw
time-series and store them into CF with FIFO compaction policy, periodically we run task which
creates roll up aggregates and compacts time-series, the original raw data can be discarded after
that.
-
Field Summary
Fields inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
comConf, storeConfigInfo
-
Constructor Summary
ConstructorDescriptionFIFOCompactionPolicy
(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo) -
Method Summary
Modifier and TypeMethodDescriptionprivate Collection<HStoreFile>
getExpiredStores
(Collection<HStoreFile> files, Collection<HStoreFile> filesCompacting) private boolean
hasExpiredStores
(Collection<HStoreFile> files) private boolean
The FIFOCompactionPolicy only choose the TTL expired store files as the compaction candidates.boolean
needsCompaction
(Collection<HStoreFile> storeFiles, List<HStoreFile> filesCompacting) A heuristic method to decide whether to schedule a compaction requestselectCompaction
(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) boolean
shouldPerformMajorCompaction
(Collection<HStoreFile> filesToCompact) Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy
applyCompactionPolicy, applyCompactionPolicy, selectCompactFiles
Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy
createCompactionRequest, setMinThreshold
Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
checkMinFilesCriteria, filterBulk, getCurrentEligibleFiles, getNextMajorCompactTime, preSelectCompactionForCoprocessor, removeExcessFiles, skipLargeFiles, throttleCompaction
Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
getConf, setConf
-
Field Details
-
LOG
-
-
Constructor Details
-
FIFOCompactionPolicy
public FIFOCompactionPolicy(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo)
-
-
Method Details
-
selectCompaction
public CompactionRequestImpl selectCompaction(Collection<HStoreFile> candidateFiles, List<HStoreFile> filesCompacting, boolean isUserCompaction, boolean mayUseOffPeak, boolean forceMajor) throws IOException - Overrides:
selectCompaction
in classSortedCompactionPolicy
- Parameters:
candidateFiles
- candidate files, ordered from oldest to newest by seqId. We rely on DefaultStoreFileManager to sort the files by seqId to guarantee contiguous compaction based on seqId for data consistency.- Returns:
- subset copy of candidate list that meets compaction criteria
- Throws:
IOException
-
shouldPerformMajorCompaction
public boolean shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact) throws IOException - Overrides:
shouldPerformMajorCompaction
in classRatioBasedCompactionPolicy
- Parameters:
filesToCompact
- Files to compact. Can be null.- Returns:
- True if we should run a major compaction.
- Throws:
IOException
-
needsCompaction
Description copied from class:RatioBasedCompactionPolicy
A heuristic method to decide whether to schedule a compaction request- Overrides:
needsCompaction
in classRatioBasedCompactionPolicy
- Parameters:
storeFiles
- files in the store.filesCompacting
- files being scheduled to compact.- Returns:
- true to schedule a request.
-
isEmptyStoreFile
The FIFOCompactionPolicy only choose the TTL expired store files as the compaction candidates. If all the store files are TTL expired, then the compaction will generate a new empty file. While its max timestamp will be Long.MAX_VALUE. If not considered separately, the store file will never be archived because its TTL will be never expired. So we'll check the empty store file separately (See HBASE-21504). -
hasExpiredStores
-
getExpiredStores
private Collection<HStoreFile> getExpiredStores(Collection<HStoreFile> files, Collection<HStoreFile> filesCompacting)
-