org.apache.hadoop.hbase.wal

Class RecoveredEditsOutputSink

java.lang.Object

org.apache.hadoop.hbase.wal.OutputSink

org.apache.hadoop.hbase.wal.AbstractRecoveredEditsOutputSink

org.apache.hadoop.hbase.wal.RecoveredEditsOutputSink

@InterfaceAudience.Private
class RecoveredEditsOutputSink
extends AbstractRecoveredEditsOutputSink

Class that manages the output streams from the log splitting process. Every region only has one recovered edits file PER split WAL (if we split multiple WALs during a log-splitting session, on open, a Region may have multiple recovered.edits files to replay -- one per split WAL).

See Also:

which is like this class but imposes upper bound on the number of writers active at one time (makes for better throughput).

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.wal.AbstractRecoveredEditsOutputSink

AbstractRecoveredEditsOutputSink.RecoveredEditsWriter

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.wal.OutputSink

OutputSink.WriterThread

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOG
private ConcurrentMap<String,AbstractRecoveredEditsOutputSink.RecoveredEditsWriter>	writers

Fields inherited from class org.apache.hadoop.hbase.wal.OutputSink

closeCompletionService, closeThreadPool, entryBuffers, numThreads, reporter, splits, status, totalSkippedEdits

Constructor Summary

Constructors

Constructor and Description
RecoveredEditsOutputSink(WALSplitter walSplitter, WALSplitter.PipelineController controller, EntryBuffers entryBuffers, int numWriters)

Method Summary

Modifier and Type	Method and Description
void	append(EntryBuffers.RegionEntryBuffer buffer)
List<org.apache.hadoop.fs.Path>	close()
private boolean	closeWriters() Close all of the output streams.
int	getNumberOfRecoveredRegions() Returns number of regions we've recovered
int	getNumOpenWriters() Returns the number of currently opened writers
Map<String,Long>	getOutputCounts() Returns a map from encoded region ID to the number of edits written out for that region.
private AbstractRecoveredEditsOutputSink.RecoveredEditsWriter	getRecoveredEditsWriter(TableName tableName, byte[] region, long seqId) Get a writer and path for a log starting at the given entry.

Methods inherited from class org.apache.hadoop.hbase.wal.AbstractRecoveredEditsOutputSink

closeRecoveredEditsWriter, createRecoveredEditsWriter, keepRegionEvent, updateRegionMaximumEditLogSeqNum

Methods inherited from class org.apache.hadoop.hbase.wal.OutputSink

finishWriterThreads, getTotalSkippedEdits, restartWriterThreadsIfNeeded, setReporter, setStatus, startWriterThreads, updateStatusWithMsg

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

writers

private ConcurrentMap<String,AbstractRecoveredEditsOutputSink.RecoveredEditsWriter> writers

Constructor Detail

RecoveredEditsOutputSink

public RecoveredEditsOutputSink(WALSplitter walSplitter,
                                WALSplitter.PipelineController controller,
                                EntryBuffers entryBuffers,
                                int numWriters)

Method Detail

append

public void append(EntryBuffers.RegionEntryBuffer buffer)
            throws IOException

Specified by:

append in class OutputSink

Parameters:

buffer - A buffer of some number of edits for a given region.

Throws:

IOException - For any IO errors

getRecoveredEditsWriter

private AbstractRecoveredEditsOutputSink.RecoveredEditsWriter getRecoveredEditsWriter(TableName tableName,
                                                                                      byte[] region,
                                                                                      long seqId)
                                                                               throws IOException

Get a writer and path for a log starting at the given entry. This function is threadsafe so long as multiple threads are always acting on different regions.

Returns:

null if this region shouldn't output any logs

Throws:

IOException

close

public List<org.apache.hadoop.fs.Path> close()
                                      throws IOException

Specified by:

close in class OutputSink

Throws:

IOException

closeWriters

private boolean closeWriters()
                      throws IOException

Close all of the output streams.

Returns:

true when there is no error.

Throws:

IOException

getOutputCounts

public Map<String,Long> getOutputCounts()

Description copied from class: OutputSink

Returns a map from encoded region ID to the number of edits written out for that region.

Specified by:

getOutputCounts in class OutputSink

getNumberOfRecoveredRegions

public int getNumberOfRecoveredRegions()

Description copied from class: OutputSink

Returns number of regions we've recovered

Specified by:

getNumberOfRecoveredRegions in class OutputSink

getNumOpenWriters

public int getNumOpenWriters()

Description copied from class: OutputSink

Returns the number of currently opened writers

Specified by:

getNumOpenWriters in class OutputSink