org.apache.hadoop.hbase.wal

Class BoundedLogWriterCreationOutputSink

java.lang.Object

org.apache.hadoop.hbase.wal.OutputSink

org.apache.hadoop.hbase.wal.LogRecoveredEditsOutputSink

org.apache.hadoop.hbase.wal.BoundedLogWriterCreationOutputSink

@InterfaceAudience.Private
public class BoundedLogWriterCreationOutputSink
extends LogRecoveredEditsOutputSink

Class that manages the output streams from the log splitting process. Bounded means the output streams will be no more than the size of threadpool

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.wal.OutputSink

OutputSink.WriterThread

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOG
private ConcurrentHashMap<String,Long>	regionRecoverStatMap

Fields inherited from class org.apache.hadoop.hbase.wal.OutputSink

blacklistedRegions, closeAndCleanCompleted, controller, entryBuffers, numThreads, regionMaximumEditLogSeqNum, reporter, skippedEdits, splits, writers, writersClosed, writerThreads

Constructor Summary

Constructors

Constructor and Description
BoundedLogWriterCreationOutputSink(WALSplitter walSplitter, WALSplitter.PipelineController controller, EntryBuffers entryBuffers, int numWriters)

Method Summary

Modifier and Type	Method and Description
void	append(WALSplitter.RegionEntryBuffer buffer) Append the buffer to a new recovered edits file, then close it after all done
(package private) boolean	executeCloseTask(CompletionService<Void> completionService, List<IOException> thrown, List<org.apache.hadoop.fs.Path> paths)
List<org.apache.hadoop.fs.Path>	finishWritingAndClose()
int	getNumberOfRecoveredRegions()
Map<byte[],Long>	getOutputCounts() since the splitting process may create multiple output files, we need a map regionRecoverStatMap to track the output count of each region.
private org.apache.hadoop.fs.Path	writeThenClose(WALSplitter.RegionEntryBuffer buffer)

Methods inherited from class org.apache.hadoop.hbase.wal.LogRecoveredEditsOutputSink

appendBuffer, close, closeWriter, createWAP, filterCellByStore, getWriterAndPath, keepRegionEvent

Methods inherited from class org.apache.hadoop.hbase.wal.OutputSink

finishWriting, flush, getNumOpenWriters, getSkippedEdits, setReporter, startWriterThreads, updateRegionMaximumEditLogSeqNum

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

regionRecoverStatMap

private ConcurrentHashMap<String,Long> regionRecoverStatMap

Constructor Detail

BoundedLogWriterCreationOutputSink

public BoundedLogWriterCreationOutputSink(WALSplitter walSplitter,
                                          WALSplitter.PipelineController controller,
                                          EntryBuffers entryBuffers,
                                          int numWriters)

Method Detail

finishWritingAndClose

public List<org.apache.hadoop.fs.Path> finishWritingAndClose()
                                                      throws IOException

Overrides:

finishWritingAndClose in class LogRecoveredEditsOutputSink

Returns:

null if failed to report progress

Throws:

IOException

executeCloseTask

boolean executeCloseTask(CompletionService<Void> completionService,
                         List<IOException> thrown,
                         List<org.apache.hadoop.fs.Path> paths)
                  throws InterruptedException,
                         ExecutionException

Overrides:

executeCloseTask in class LogRecoveredEditsOutputSink

Parameters:

completionService - threadPool to execute the closing tasks

thrown - store the exceptions

paths - arrayList to store the paths written

Returns:

if close tasks executed successful

Throws:

InterruptedException

ExecutionException

getOutputCounts

public Map<byte[],Long> getOutputCounts()

since the splitting process may create multiple output files, we need a map regionRecoverStatMap to track the output count of each region.

Overrides:

getOutputCounts in class LogRecoveredEditsOutputSink

Returns:

a map from encoded region ID to the number of edits written out for that region.

getNumberOfRecoveredRegions

public int getNumberOfRecoveredRegions()

Overrides:

getNumberOfRecoveredRegions in class LogRecoveredEditsOutputSink

Returns:

the number of recovered regions

append

public void append(WALSplitter.RegionEntryBuffer buffer)
            throws IOException

Append the buffer to a new recovered edits file, then close it after all done

Overrides:

append in class LogRecoveredEditsOutputSink

Parameters:

buffer - contain all entries of a certain region

Throws:

IOException - when closeWriter failed

writeThenClose

private org.apache.hadoop.fs.Path writeThenClose(WALSplitter.RegionEntryBuffer buffer)
                                          throws IOException

Throws:

IOException