Field Detail

• LOG

  private static final org.apache.commons.logging.Log LOG

• progress

  protected CompactionProgress progress

• conf

  protected org.apache.hadoop.conf.Configuration conf

• store

  protected Store store

• compactionKVMax

  private int compactionKVMax

• compactionCompression

  protected Compression.Algorithm compactionCompression

• keepSeqIdPeriod

  protected int keepSeqIdPeriod

  specify how many days to keep MVCC values during major compaction

• NAME_COUNTER

  private static final AtomicInteger NAME_COUNTER

  Used to prevent compaction name conflict when multiple compactions running parallel on the same store.

Constructor Detail

• Compactor

  Compactor(org.apache.hadoop.conf.Configuration conf,
           Store store)

Method Detail

• getProgress

  public CompactionProgress getProgress()

• getFileDetails

  protected Compactor.FileDetails getFileDetails(Collection<StoreFile> filesToCompact,
                                     boolean allFiles)
                                          throws IOException

  Extracts some details about the files to compact that are commonly needed by compactors.

  Parameters:

  filesToCompact - Files.

  allFiles - Whether all files are included for compaction

  Returns:

  The result.

  Throws:

  IOException

• createFileScanners

  protected List<StoreFileScanner> createFileScanners(Collection<StoreFile> filesToCompact,
                                          long smallestReadPoint,
                                          boolean useDropBehind)
                                               throws IOException

  Creates file scanners for compaction.

  Parameters:

  filesToCompact - Files.

  Returns:

  Scanners.

  Throws:

  IOException

• getSmallestReadPoint

  protected long getSmallestReadPoint()

• preCreateCoprocScanner

  protected InternalScanner preCreateCoprocScanner(CompactionRequest request,
                                       ScanType scanType,
                                       long earliestPutTs,
                                       List<StoreFileScanner> scanners)
                                            throws IOException

  Calls coprocessor, if any, to create compaction scanner - before normal scanner creation.

  Parameters:

  request - Compaction request.

  scanType - Scan type.

  earliestPutTs - Earliest put ts.

  scanners - File scanners for compaction files.

  Returns:

  Scanner override by coprocessor; null if not overriding.

  Throws:

  IOException

• preCreateCoprocScanner

  protected InternalScanner preCreateCoprocScanner(CompactionRequest request,
                                       ScanType scanType,
                                       long earliestPutTs,
                                       List<StoreFileScanner> scanners,
                                       User user)
                                            throws IOException

  Throws:

  IOException

• postCreateCoprocScanner

  protected InternalScanner postCreateCoprocScanner(CompactionRequest request,
                                        ScanType scanType,
                                        InternalScanner scanner,
                                        User user)
                                             throws IOException

  Calls coprocessor, if any, to create scanners - after normal scanner creation.

  Parameters:

  request - Compaction request.

  scanType - Scan type.

  scanner - The default scanner created for compaction.

  Returns:

  Scanner scanner to use (usually the default); null if compaction should not proceed.

  Throws:

  IOException

• generateCompactionName

  private String generateCompactionName()

• performCompaction

  protected boolean performCompaction(InternalScanner scanner,
                          Compactor.CellSink writer,
                          long smallestReadPoint,
                          boolean cleanSeqId,
                          CompactionThroughputController throughputController)
                               throws IOException

  Performs the compaction.

  Parameters:

  scanner - Where to read from.

  writer - Where to write to.

  smallestReadPoint - Smallest read point.

  cleanSeqId - When true, remove seqId(used to be mvcc) value which is <= smallestReadPoint

  Returns:

  Whether compaction ended; false if it was interrupted for some reason.

  Throws:

  IOException

• createScanner

  protected InternalScanner createScanner(Store store,
                              List<StoreFileScanner> scanners,
                              ScanType scanType,
                              long smallestReadPoint,
                              long earliestPutTs)
                                   throws IOException

  Parameters:

  store - store

  scanners - Store file scanners.

  scanType - Scan type.

  smallestReadPoint - Smallest MVCC read point.

  earliestPutTs - Earliest put across all files.

  Returns:

  A compaction scanner.

  Throws:

  IOException

• createScanner

  protected InternalScanner createScanner(Store store,
                              List<StoreFileScanner> scanners,
                              long smallestReadPoint,
                              long earliestPutTs,
                              byte[] dropDeletesFromRow,
                              byte[] dropDeletesToRow)
                                   throws IOException

  Parameters:

  store - The store.

  scanners - Store file scanners.

  smallestReadPoint - Smallest MVCC read point.

  earliestPutTs - Earliest put across all files.

  dropDeletesFromRow - Drop deletes starting with this row, inclusive. Can be null.

  dropDeletesToRow - Drop deletes ending with this row, exclusive. Can be null.

  Returns:

  A compaction scanner.

  Throws:

  IOException