Constructor Detail

• SortedCompactionPolicy

  public SortedCompactionPolicy(org.apache.hadoop.conf.Configuration conf,
                        StoreConfigInformation storeConfigInfo)

Method Detail

• preSelectCompactionForCoprocessor

  public List<StoreFile> preSelectCompactionForCoprocessor(Collection<StoreFile> candidates,
                                                  List<StoreFile> filesCompacting)

• selectCompaction

  public CompactionRequest selectCompaction(Collection<StoreFile> candidateFiles,
                                   List<StoreFile> filesCompacting,
                                   boolean isUserCompaction,
                                   boolean mayUseOffPeak,
                                   boolean forceMajor)
                                     throws IOException

  Parameters:

  candidateFiles - candidate files, ordered from oldest to newest by seqId. We rely on DefaultStoreFileManager to sort the files by seqId to guarantee contiguous compaction based on seqId for data consistency.

  Returns:

  subset copy of candidate list that meets compaction criteria

  Throws:

  IOException

• createCompactionRequest

  protected abstract CompactionRequest createCompactionRequest(ArrayList<StoreFile> candidateSelection,
                                          boolean tryingMajor,
                                          boolean mayUseOffPeak,
                                          boolean mayBeStuck)
                                                        throws IOException

  Throws:

  IOException

• shouldPerformMajorCompaction

  public abstract boolean shouldPerformMajorCompaction(Collection<StoreFile> filesToCompact)
                                                throws IOException

  Specified by:

  shouldPerformMajorCompaction in class CompactionPolicy

  Parameters:

  filesToCompact - Files to compact. Can be null.

  Returns:

  True if we should run a major compaction.

  Throws:

  IOException

• getNextMajorCompactTime

  public long getNextMajorCompactTime(Collection<StoreFile> filesToCompact)

  Parameters:

  filesToCompact -

  Returns:

  When to run next major compaction

• throttleCompaction

  public boolean throttleCompaction(long compactionSize)

  Specified by:

  throttleCompaction in class CompactionPolicy

  Parameters:

  compactionSize - Total size of some compaction

  Returns:

  whether this should be a large or small compaction

• needsCompaction

  public abstract boolean needsCompaction(Collection<StoreFile> storeFiles,
                        List<StoreFile> filesCompacting)

• getCurrentEligibleFiles

  protected ArrayList<StoreFile> getCurrentEligibleFiles(ArrayList<StoreFile> candidateFiles,
                                             List<StoreFile> filesCompacting)

• skipLargeFiles

  protected ArrayList<StoreFile> skipLargeFiles(ArrayList<StoreFile> candidates,
                                    boolean mayUseOffpeak)

  Parameters:

  candidates - pre-filtrate

  Returns:

  filtered subset exclude all files above maxCompactSize Also save all references. We MUST compact them

• filterBulk

  protected ArrayList<StoreFile> filterBulk(ArrayList<StoreFile> candidates)

  Parameters:

  candidates - pre-filtrate

  Returns:

  filtered subset exclude all bulk load files if configured

• removeExcessFiles

  protected void removeExcessFiles(ArrayList<StoreFile> candidates,
                       boolean isUserCompaction,
                       boolean isMajorCompaction)

  Parameters:

  candidates - pre-filtrate

• checkMinFilesCriteria

  protected ArrayList<StoreFile> checkMinFilesCriteria(ArrayList<StoreFile> candidates,
                                           int minFiles)

  Parameters:

  candidates - pre-filtrate

  Returns:

  filtered subset forget the compactionSelection if we don't have enough files