Field Detail

• BUFFER_THRESHOLD

  private static final double BUFFER_THRESHOLD

  See Also:

  Constant Field Values

• KEY_DELIMITER

  private static final String KEY_DELIMITER

  See Also:

  Constant Field Values

• waitRegionOnlineTimeOut

  private long waitRegionOnlineTimeOut

• recoveredRegions

  private final Set<String> recoveredRegions

• writers

  private final Map<String,WALSplitter.RegionServerWriter> writers

• onlineRegions

  private final Map<String,HRegionLocation> onlineRegions

• tableNameToHConnectionMap

  private Map<TableName,HConnection> tableNameToHConnectionMap

• serverToBufferQueueMap

  private Map<String,List<Pair<HRegionLocation,WAL.Entry>>> serverToBufferQueueMap

  Map key -> value layout : -> Queue

  • thrown

    private List<Throwable> thrown

  • logRecoveredEditsOutputSink

    private WALSplitter.LogRecoveredEditsOutputSink logRecoveredEditsOutputSink

  • hasEditsInDisablingOrDisabledTables

    private boolean hasEditsInDisablingOrDisabledTables

  • Constructor Detail

    • WALSplitter.LogReplayOutputSink

      public WALSplitter.LogReplayOutputSink(WALSplitter.PipelineController controller,
                                     WALSplitter.EntryBuffers entryBuffers,
                                     int numWriters)

  • Method Detail

    • append

      public void append(WALSplitter.RegionEntryBuffer buffer)
                  throws IOException

      Specified by:

      append in class WALSplitter.OutputSink

      Parameters:

      buffer - A WAL Edit Entry

      Throws:

      IOException

    • addToRecoveredRegions

      private void addToRecoveredRegions(String encodedRegionName)

    • groupEditsByServer

      private void groupEditsByServer(List<WAL.Entry> entries)
                               throws IOException

      Helper function to group WALEntries to individual region servers

      Throws:

      IOException

    • locateRegionAndRefreshLastFlushedSequenceId

      private HRegionLocation locateRegionAndRefreshLastFlushedSequenceId(HConnection hconn,
                                                                TableName table,
                                                                byte[] row,
                                                                String originalEncodedRegionName)
                                                                   throws IOException

      Locate destination region based on table name & row. This function also makes sure the destination region is online for replay.

      Throws:

      IOException

    • processWorkItems

      private void processWorkItems(String key,
                          List<Pair<HRegionLocation,WAL.Entry>> actions)
                             throws IOException

      Throws:

      IOException

    • waitUntilRegionOnline

      private HRegionLocation waitUntilRegionOnline(HRegionLocation loc,
                                          byte[] row,
                                          long timeout,
                                          AtomicBoolean isRecovering)
                                             throws IOException

      Wait until region is online on the destination region server

      Parameters:

      loc -

      row -

      timeout - How long to wait

      isRecovering - Recovering state of the region interested on destination region server.

      Returns:

      True when region is online on the destination region server

      Throws:

      InterruptedException

      IOException

    • flush

      public boolean flush()
                    throws IOException

      Description copied from class: WALSplitter.OutputSink
      WriterThread call this function to help flush internal remaining edits in buffer before close

      Overrides:

      flush in class WALSplitter.OutputSink

      Returns:

      true when underlying sink has something to flush

      Throws:

      IOException

    • keepRegionEvent

      public boolean keepRegionEvent(WAL.Entry entry)

      Description copied from class: WALSplitter.OutputSink
      Some WALEdit's contain only KV's for account on what happened to a region. Not all sinks will want to get all of those edits.

      Specified by:

      keepRegionEvent in class WALSplitter.OutputSink

      Returns:

      Return true if this sink wants to accept this region-level WALEdit.

    • addWriterError

      void addWriterError(Throwable t)

    • finishWritingAndClose

      public List<org.apache.hadoop.fs.Path> finishWritingAndClose()
                                                            throws IOException

      Specified by:

      finishWritingAndClose in class WALSplitter.OutputSink

      Throws:

      IOException

    • getNumOpenWriters

      int getNumOpenWriters()

      Overrides:

      getNumOpenWriters in class WALSplitter.OutputSink

      Returns:

      the number of currently opened writers

    • closeRegionServerWriters

      private List<IOException> closeRegionServerWriters()
                                                  throws IOException

      Throws:

      IOException

    • getOutputCounts

      public Map<byte[],Long> getOutputCounts()

      Specified by:

      getOutputCounts in class WALSplitter.OutputSink

      Returns:

      a map from encoded region ID to the number of edits written out for that region.

    • getNumberOfRecoveredRegions

      public int getNumberOfRecoveredRegions()

      Specified by:

      getNumberOfRecoveredRegions in class WALSplitter.OutputSink

      Returns:

      number of regions we've recovered

    • getRegionServerWriter

      private WALSplitter.RegionServerWriter getRegionServerWriter(String loc)
                                                            throws IOException

      Get a writer and path for a log starting at the given entry. This function is threadsafe so long as multiple threads are always acting on different regions.

      Returns:

      null if this region shouldn't output any logs

      Throws:

      IOException

    • getConnectionByTableName

      private HConnection getConnectionByTableName(TableName tableName)
                                            throws IOException

      Throws:

      IOException

    • getTableFromLocationStr

      private TableName getTableFromLocationStr(String loc)

  • Overview
  • Package
  • Class
  • Use
  • Tree
  • Deprecated
  • Index
  • Help

  • Prev Class
  • Next Class

  • Frames
  • No Frames

  • All Classes

  • Summary: 
  • Nested | 
  • Field | 
  • Constr | 
  • Method

  • Detail: 
  • Field | 
  • Constr | 
  • Method

  Copyright © 2007–2019 The Apache Software Foundation. All rights reserved.