DateTieredCompactionPolicy (Apache HBase 1.4.11 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
- - org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
  - - org.apache.hadoop.hbase.regionserver.compactions.DateTieredCompactionPolicy

```
@InterfaceAudience.LimitedPrivate(value="Configuration")
public class DateTieredCompactionPolicy
extends SortedCompactionPolicy
```
HBASE-15181 This is a simple implementation of date-based tiered compaction similar to Cassandra's for the following benefits:
1. Improve date-range-based scan by structuring store files in date-based tiered layout.
2. Reduce compaction overhead.
3. Improve TTL efficiency.
Perfect fit for the use cases that:
1. has mostly date-based data write and scan and a focus on the most recent data.
Out-of-order writes are handled gracefully. Time range overlapping among store files is tolerated and the performance impact is minimized. Configuration can be set at hbase-site or overridden at per-table or per-column-family level by hbase shell. Design spec is at https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/

Field Summary
- Fields inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
  comConf, storeConfigInfo

Constructor Summary

Constructors
Constructor and Description

DateTieredCompactionPolicy(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo)

Constructors
Constructor and Description
`DateTieredCompactionPolicy(org.apache.hadoop.conf.Configuration conf, StoreConfigInformation storeConfigInfo)`

Method Summary

Methods
Modifier and Type	Method and Description
`protected CompactionRequest`	`createCompactionRequest(ArrayList<StoreFile> candidateSelection, boolean tryingMajor, boolean mayUseOffPeak, boolean mayBeStuck)`
`boolean`	`needsCompaction(Collection<StoreFile> storeFiles, List<StoreFile> filesCompacting)` Heuristics for guessing whether we need minor compaction.
`CompactionRequest`	`selectMajorCompaction(ArrayList<StoreFile> candidateSelection)`
`CompactionRequest`	`selectMinorCompaction(ArrayList<StoreFile> candidateSelection, boolean mayUseOffPeak, boolean mayBeStuck)` We receive store files sorted in ascending order by seqId then scan the list of files.
`boolean`	`shouldPerformMajorCompaction(Collection<StoreFile> filesToCompact)`

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
checkMinFilesCriteria, filterBulk, getCurrentEligibleFiles, getNextMajorCompactTime, preSelectCompactionForCoprocessor, removeExcessFiles, selectCompaction, skipLargeFiles, throttleCompaction

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
getConf, setConf

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DateTieredCompactionPolicy

public DateTieredCompactionPolicy(org.apache.hadoop.conf.Configuration conf,
                          StoreConfigInformation storeConfigInfo)
                           throws IOException

Throws:: IOException

Method Detail
- needsCompaction
```
public boolean needsCompaction(Collection<StoreFile> storeFiles,
                      List<StoreFile> filesCompacting)
```
  Heuristics for guessing whether we need minor compaction.
  
  Specified by:
  
  needsCompaction in class SortedCompactionPolicy
- shouldPerformMajorCompaction
```
public boolean shouldPerformMajorCompaction(Collection<StoreFile> filesToCompact)
                                     throws IOException
```
  Specified by:
  
  shouldPerformMajorCompaction in class SortedCompactionPolicy
  
  Parameters:
  filesToCompact - Files to compact. Can be null.
  
  Returns:
  True if we should run a major compaction.
  
  Throws:
  
  IOException
- createCompactionRequest
```
protected CompactionRequest createCompactionRequest(ArrayList<StoreFile> candidateSelection,
                                        boolean tryingMajor,
                                        boolean mayUseOffPeak,
                                        boolean mayBeStuck)
                                             throws IOException
```
  Specified by:
  
  createCompactionRequest in class SortedCompactionPolicy
  
  Throws:
  
  IOException
- selectMajorCompaction
```
public CompactionRequest selectMajorCompaction(ArrayList<StoreFile> candidateSelection)
```
- selectMinorCompaction
```
public CompactionRequest selectMinorCompaction(ArrayList<StoreFile> candidateSelection,
                                      boolean mayUseOffPeak,
                                      boolean mayBeStuck)
                                        throws IOException
```
  We receive store files sorted in ascending order by seqId then scan the list of files. If the current file has a maxTimestamp older than last known maximum, treat this file as it carries the last known maximum. This way both seqId and timestamp are in the same order. If files carry the same maxTimestamps, they are ordered by seqId. We then reverse the list so they are ordered by seqId and maxTimestamp in descending order and build the time windows. All the out-of-order data into the same compaction windows, guaranteeing contiguous compaction based on sequence id.
  
  Throws:
  
  IOException

Class DateTieredCompactionPolicy

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy

Methods inherited from class java.lang.Object

Constructor Detail

DateTieredCompactionPolicy

Method Detail

needsCompaction

shouldPerformMajorCompaction

createCompactionRequest

selectMajorCompaction

selectMinorCompaction