Field Detail

• LOG

  private static final org.apache.commons.logging.Log LOG

Constructor Detail

• ExploringCompactionPolicy

  public ExploringCompactionPolicy(org.apache.hadoop.conf.Configuration conf,
                           StoreConfigInformation storeConfigInfo)

  Constructor for ExploringCompactionPolicy.

  Parameters:

  conf - The configuration object

  storeConfigInfo - An object to provide info about the store.

Method Detail

• applyCompactionPolicy

  final ArrayList<StoreFile> applyCompactionPolicy(ArrayList<StoreFile> candidates,
                                           boolean mayUseOffPeak,
                                           boolean mightBeStuck)
                                            throws IOException

  Overrides:

  applyCompactionPolicy in class RatioBasedCompactionPolicy

  Parameters:

  candidates - pre-filtrate

  Returns:

  filtered subset -- Default minor compaction selection algorithm: choose CompactSelection from candidates -- First exclude bulk-load files if indicated in configuration. Start at the oldest file and stop when you find the first file that meets compaction criteria: (1) a recently-flushed, small file (i.e. <= minCompactSize) OR (2) within the compactRatio of sum(newer_files) Given normal skew, any newer files will also meet this criteria

  Additional Note: If fileSizes.size() >> maxFilesToCompact, we will recurse on compact(). Consider the oldest files first to avoid a situation where we always compact [end-threshold,end). Then, the last file becomes an aggregate of the previous compactions. normal skew: older ----> newer (increasing seqID) _ | | _ | | | | _ --|-|- |-|- |-|---_-------_------- minCompactSize | | | | | | | | _ | | | | | | | | | | | | | | | | | | | | | | | | | |

  Throws:

  IOException

• applyCompactionPolicy

  public List<StoreFile> applyCompactionPolicy(List<StoreFile> candidates,
                                      boolean mightBeStuck,
                                      boolean mayUseOffPeak,
                                      int minFiles,
                                      int maxFiles)

• isBetterSelection

  private boolean isBetterSelection(List<StoreFile> bestSelection,
                          long bestSize,
                          List<StoreFile> selection,
                          long size,
                          boolean mightBeStuck)

• getTotalStoreSize

  private long getTotalStoreSize(List<StoreFile> potentialMatchFiles)

  Find the total size of a list of store files.

  Parameters:

  potentialMatchFiles - StoreFile list.

  Returns:

  Sum of StoreFile.getReader().length();

• filesInRatio

  private boolean filesInRatio(List<StoreFile> files,
                     double currentRatio)

  Check that all files satisfy the constraint FileSize(i) <= ( Sum(0,N,FileSize(_)) - FileSize(i) ) * Ratio.

  Parameters:

  files - List of store files to consider as a compaction candidate.

  currentRatio - The ratio to use.

  Returns:

  a boolean if these files satisfy the ratio constraints.