org.apache.hadoop.hbase.wal

Class LogRecoveredEditsOutputSink

java.lang.Object

org.apache.hadoop.hbase.wal.OutputSink

org.apache.hadoop.hbase.wal.LogRecoveredEditsOutputSink

Direct Known Subclasses:

BoundedLogWriterCreationOutputSink

@InterfaceAudience.Private
public class LogRecoveredEditsOutputSink
extends OutputSink

Class that manages the output streams from the log splitting process.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.wal.OutputSink

OutputSink.WriterThread

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf
private static org.slf4j.Logger	LOG
private org.apache.hadoop.fs.FileSystem	walFS
private WALSplitter	walSplitter

Fields inherited from class org.apache.hadoop.hbase.wal.OutputSink

blacklistedRegions, closeAndCleanCompleted, controller, entryBuffers, numThreads, regionMaximumEditLogSeqNum, reporter, skippedEdits, splits, writers, writersClosed, writerThreads

Constructor Summary

Constructors

Constructor and Description
LogRecoveredEditsOutputSink(WALSplitter walSplitter, WALSplitter.PipelineController controller, EntryBuffers entryBuffers, int numWriters)

Method Summary

Modifier and Type	Method and Description
void	append(WALSplitter.RegionEntryBuffer buffer)
(package private) WALSplitter.WriterAndPath	appendBuffer(WALSplitter.RegionEntryBuffer buffer, boolean reusable)
(package private) List<org.apache.hadoop.fs.Path>	close() Close all of the output streams.
private List<IOException>	closeLogWriters(List<IOException> thrown)
(package private) org.apache.hadoop.fs.Path	closeWriter(String encodedRegionName, WALSplitter.WriterAndPath wap, List<IOException> thrown)
(package private) WALSplitter.WriterAndPath	createWAP(byte[] region, WAL.Entry entry)
private void	deleteOneWithFewerEntries(WALSplitter.WriterAndPath wap, org.apache.hadoop.fs.Path dst)
(package private) boolean	executeCloseTask(CompletionService<Void> completionService, List<IOException> thrown, List<org.apache.hadoop.fs.Path> paths)
(package private) void	filterCellByStore(WAL.Entry logEntry)
List<org.apache.hadoop.fs.Path>	finishWritingAndClose()
int	getNumberOfRecoveredRegions()
Map<byte[],Long>	getOutputCounts()
(package private) WALSplitter.WriterAndPath	getWriterAndPath(WAL.Entry entry, boolean reusable) Get a writer and path for a log starting at the given entry.
boolean	keepRegionEvent(WAL.Entry entry) Some WALEdit's contain only KV's for account on what happened to a region.

Methods inherited from class org.apache.hadoop.hbase.wal.OutputSink

finishWriting, flush, getNumOpenWriters, getSkippedEdits, setReporter, startWriterThreads, updateRegionMaximumEditLogSeqNum

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

walSplitter

private WALSplitter walSplitter

walFS

private org.apache.hadoop.fs.FileSystem walFS

conf

private org.apache.hadoop.conf.Configuration conf

Constructor Detail

LogRecoveredEditsOutputSink

public LogRecoveredEditsOutputSink(WALSplitter walSplitter,
                                   WALSplitter.PipelineController controller,
                                   EntryBuffers entryBuffers,
                                   int numWriters)

Method Detail

finishWritingAndClose

public List<org.apache.hadoop.fs.Path> finishWritingAndClose()
                                                      throws IOException

Specified by:

finishWritingAndClose in class OutputSink

Returns:

null if failed to report progress

Throws:

IOException

deleteOneWithFewerEntries

private void deleteOneWithFewerEntries(WALSplitter.WriterAndPath wap,
                                       org.apache.hadoop.fs.Path dst)
                                throws IOException

Throws:

IOException

close

List<org.apache.hadoop.fs.Path> close()
                               throws IOException

Close all of the output streams.

Returns:

the list of paths written.

Throws:

IOException

executeCloseTask

boolean executeCloseTask(CompletionService<Void> completionService,
                         List<IOException> thrown,
                         List<org.apache.hadoop.fs.Path> paths)
                  throws InterruptedException,
                         ExecutionException

Parameters:

completionService - threadPool to execute the closing tasks

thrown - store the exceptions

paths - arrayList to store the paths written

Returns:

if close tasks executed successful

Throws:

InterruptedException

ExecutionException

closeWriter

org.apache.hadoop.fs.Path closeWriter(String encodedRegionName,
                                      WALSplitter.WriterAndPath wap,
                                      List<IOException> thrown)
                               throws IOException

Throws:

IOException

closeLogWriters

private List<IOException> closeLogWriters(List<IOException> thrown)
                                   throws IOException

Throws:

IOException

getWriterAndPath

WALSplitter.WriterAndPath getWriterAndPath(WAL.Entry entry,
                                           boolean reusable)
                                    throws IOException

Get a writer and path for a log starting at the given entry. This function is threadsafe so long as multiple threads are always acting on different regions.

Returns:

null if this region shouldn't output any logs

Throws:

IOException

createWAP

WALSplitter.WriterAndPath createWAP(byte[] region,
                                    WAL.Entry entry)
                             throws IOException

Returns:

a path with a write for that path. caller should close.

Throws:

IOException

filterCellByStore

void filterCellByStore(WAL.Entry logEntry)

append

public void append(WALSplitter.RegionEntryBuffer buffer)
            throws IOException

Specified by:

append in class OutputSink

Parameters:

buffer - A WAL Edit Entry

Throws:

IOException

appendBuffer

WALSplitter.WriterAndPath appendBuffer(WALSplitter.RegionEntryBuffer buffer,
                                       boolean reusable)
                                throws IOException

Throws:

IOException

keepRegionEvent

public boolean keepRegionEvent(WAL.Entry entry)

Description copied from class: OutputSink

Some WALEdit's contain only KV's for account on what happened to a region. Not all sinks will want to get all of those edits.

Specified by:

keepRegionEvent in class OutputSink

Returns:

Return true if this sink wants to accept this region-level WALEdit.

getOutputCounts

public Map<byte[],Long> getOutputCounts()

Specified by:

getOutputCounts in class OutputSink

Returns:

a map from encoded region ID to the number of edits written out for that region.

getNumberOfRecoveredRegions

public int getNumberOfRecoveredRegions()

Specified by:

getNumberOfRecoveredRegions in class OutputSink

Returns:

number of regions we've recovered