org.apache.hadoop.hbase.wal

Class WALSplitter.BoundedLogWriterCreationOutputSink

java.lang.Object

org.apache.hadoop.hbase.wal.WALSplitter.OutputSink

org.apache.hadoop.hbase.wal.WALSplitter.LogRecoveredEditsOutputSink

org.apache.hadoop.hbase.wal.WALSplitter.BoundedLogWriterCreationOutputSink

Enclosing class:

WALSplitter

class WALSplitter.BoundedLogWriterCreationOutputSink
extends WALSplitter.LogRecoveredEditsOutputSink

Field Summary

Fields

Modifier and Type	Field and Description
private ConcurrentHashMap<String,Long>	regionRecoverStatMap

Fields inherited from class org.apache.hadoop.hbase.wal.WALSplitter.OutputSink

blacklistedRegions, closeAndCleanCompleted, controller, entryBuffers, numThreads, regionMaximumEditLogSeqNum, reporter, skippedEdits, splits, writers, writersClosed, writerThreads

Constructor Summary

Constructors

Constructor and Description
BoundedLogWriterCreationOutputSink(WALSplitter.PipelineController controller, WALSplitter.EntryBuffers entryBuffers, int numWriters)

Method Summary

Modifier and Type	Method and Description
void	append(WALSplitter.RegionEntryBuffer buffer) Append the buffer to a new recovered edits file, then close it after all done
(package private) boolean	executeCloseTask(CompletionService<Void> completionService, List<IOException> thrown, List<org.apache.hadoop.fs.Path> paths)
List<org.apache.hadoop.fs.Path>	finishWritingAndClose()
int	getNumberOfRecoveredRegions()
Map<byte[],Long>	getOutputCounts() since the splitting process may create multiple output files, we need a map regionRecoverStatMap to track the output count of each region.
private org.apache.hadoop.fs.Path	writeThenClose(WALSplitter.RegionEntryBuffer buffer)

Methods inherited from class org.apache.hadoop.hbase.wal.WALSplitter.LogRecoveredEditsOutputSink

appendBuffer, close, closeWriter, createWAP, filterCellByStore, getWriterAndPath, keepRegionEvent

Methods inherited from class org.apache.hadoop.hbase.wal.WALSplitter.OutputSink

finishWriting, flush, getNumOpenWriters, getSkippedEdits, setReporter, startWriterThreads, updateRegionMaximumEditLogSeqNum

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

regionRecoverStatMap

private ConcurrentHashMap<String,Long> regionRecoverStatMap

Constructor Detail

BoundedLogWriterCreationOutputSink

public BoundedLogWriterCreationOutputSink(WALSplitter.PipelineController controller,
                                          WALSplitter.EntryBuffers entryBuffers,
                                          int numWriters)

Method Detail

finishWritingAndClose

public List<org.apache.hadoop.fs.Path> finishWritingAndClose()
                                                      throws IOException

Overrides:

finishWritingAndClose in class WALSplitter.LogRecoveredEditsOutputSink

Returns:

null if failed to report progress

Throws:

IOException

executeCloseTask

boolean executeCloseTask(CompletionService<Void> completionService,
                         List<IOException> thrown,
                         List<org.apache.hadoop.fs.Path> paths)
                  throws InterruptedException,
                         ExecutionException

Overrides:

executeCloseTask in class WALSplitter.LogRecoveredEditsOutputSink

Parameters:

completionService - threadPool to execute the closing tasks

thrown - store the exceptions

paths - arrayList to store the paths written

Returns:

if close tasks executed successful

Throws:

InterruptedException

ExecutionException

getOutputCounts

public Map<byte[],Long> getOutputCounts()

since the splitting process may create multiple output files, we need a map regionRecoverStatMap to track the output count of each region.

Overrides:

getOutputCounts in class WALSplitter.LogRecoveredEditsOutputSink

Returns:

a map from encoded region ID to the number of edits written out for that region.

getNumberOfRecoveredRegions

public int getNumberOfRecoveredRegions()

Overrides:

getNumberOfRecoveredRegions in class WALSplitter.LogRecoveredEditsOutputSink

Returns:

the number of recovered regions

append

public void append(WALSplitter.RegionEntryBuffer buffer)
            throws IOException

Append the buffer to a new recovered edits file, then close it after all done

Overrides:

append in class WALSplitter.LogRecoveredEditsOutputSink

Parameters:

buffer - contain all entries of a certain region

Throws:

IOException - when closeWriter failed

writeThenClose

private org.apache.hadoop.fs.Path writeThenClose(WALSplitter.RegionEntryBuffer buffer)
                                          throws IOException

Throws:

IOException