Field Detail

• controller

  protected WALSplitter.PipelineController controller

• entryBuffers

  protected WALSplitter.EntryBuffers entryBuffers

• writers

  protected ConcurrentHashMap<String,WALSplitter.SinkWriter> writers

• regionMaximumEditLogSeqNum

  protected ConcurrentHashMap<String,Long> regionMaximumEditLogSeqNum

• writerThreads

  protected final List<WALSplitter.WriterThread> writerThreads

• blacklistedRegions

  protected final Set<byte[]> blacklistedRegions

• closeAndCleanCompleted

  protected boolean closeAndCleanCompleted

• writersClosed

  protected boolean writersClosed

• numThreads

  protected final int numThreads

• reporter

  protected CancelableProgressable reporter

• skippedEdits

  protected AtomicLong skippedEdits

• splits

  protected List<org.apache.hadoop.fs.Path> splits

Constructor Detail

• WALSplitter.OutputSink

  public WALSplitter.OutputSink(WALSplitter.PipelineController controller,
                        WALSplitter.EntryBuffers entryBuffers,
                        int numWriters)

Method Detail

• startWriterThreads

  public void startWriterThreads()

  Start the threads that will pump data from the entryBuffers to the output files.

• finishWriting

  protected boolean finishWriting(boolean interrupt)
                           throws IOException

  Wait for writer threads to dump all info to the sink

  Returns:

  true when there is no error

  Throws:

  IOException

• finishWritingAndClose

  public abstract List<org.apache.hadoop.fs.Path> finishWritingAndClose()
                                                                 throws IOException

  Throws:

  IOException

• getOutputCounts

  public abstract Map<byte[],Long> getOutputCounts()

  Returns:

  a map from encoded region ID to the number of edits written out for that region.

• getNumberOfRecoveredRegions

  public abstract int getNumberOfRecoveredRegions()

  Returns:

  number of regions we've recovered

• append

  public abstract void append(WALSplitter.RegionEntryBuffer buffer)
                       throws IOException

  Parameters:

  buffer - A WAL Edit Entry

  Throws:

  IOException

• flush

  public boolean flush()
                throws IOException

  WriterThread call this function to help flush internal remaining edits in buffer before close

  Returns:

  true when underlying sink has something to flush

  Throws:

  IOException

• keepRegionEvent

  public abstract boolean keepRegionEvent(WAL.Entry entry)

  Some WALEdit's contain only KV's for account on what happened to a region. Not all sinks will want to get all of those edits.

  Returns:

  Return true if this sink wants to accept this region-level WALEdit.