org.apache.hadoop.hbase.util

Class MultiThreadedReader

java.lang.Object

org.apache.hadoop.hbase.util.MultiThreadedAction

org.apache.hadoop.hbase.util.MultiThreadedReader

Direct Known Subclasses:

IntegrationTestTimeBoundedRequestsWithRegionReplicas.TimeBoundedMultiThreadedReader, MultiThreadedReaderWithACL

public class MultiThreadedReader
extends MultiThreadedAction

Creates multiple threads that read and verify previously written data

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	MultiThreadedReader.HBaseReaderThread

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.MultiThreadedAction

MultiThreadedAction.DefaultDataGenerator

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	aborted
private int	batchSize
static int	DEFAULT_BATCH_SIZE Default batch size for multigets
static int	DEFAULT_KEY_WINDOW Default "window" size between the last key written by the writer and the key that we attempt to read.
static int	DEFAULT_MAX_ERRORS Default maximum number of read errors to tolerate before shutting down all readers.
private int	keyWindow
private static org.slf4j.Logger	LOG
private int	maxErrors
protected AtomicLong	nullResult
protected AtomicLong	numKeysVerified
protected AtomicLong	numReadErrors
protected AtomicLong	numReadFailures
private AtomicLong	numUniqueKeysVerified The number of keys verified in a sequence.
protected Set<MultiThreadedReader.HBaseReaderThread>	readers
private int	regionReplicaId
private double	verifyPercent
protected MultiThreadedWriterBase	writer

Fields inherited from class org.apache.hadoop.hbase.util.MultiThreadedAction

conf, connection, dataGenerator, endKey, numCols, numKeys, numThreads, numThreadsWorking, REPORTING_INTERVAL_MS, startKey, tableName, totalOpTimeMs, verbose

Constructor Summary

Constructors

Constructor and Description
MultiThreadedReader(LoadTestDataGenerator dataGen, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.TableName tableName, double verifyPercent)

Method Summary

Modifier and Type	Method and Description
protected void	addReaderThreads(int numThreads)
protected MultiThreadedReader.HBaseReaderThread	createReaderThread(int readerId)
long	getNullResultsCount()
long	getNumKeysVerified()
long	getNumReadErrors()
long	getNumReadFailures()
long	getNumUniqueKeysVerified()
void	linkToWriter(MultiThreadedWriterBase writer)
protected String	progressInfo() Returns a task-specific progress string
void	setKeyWindow(int keyWindow)
void	setMaxErrors(int maxErrors)
void	setMultiGetBatchSize(int batchSize)
void	setRegionReplicaId(int regionReplicaId)
void	start(long startKey, long endKey, int numThreads)

Methods inherited from class org.apache.hadoop.hbase.util.MultiThreadedAction

appendToStatus, appendToStatus, close, getEndKey, isDone, startThreads, verifyResultAgainstDataGenerator, verifyResultAgainstDataGenerator, waitForFinish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

readers

protected Set<MultiThreadedReader.HBaseReaderThread> readers

verifyPercent

private final double verifyPercent

aborted

protected volatile boolean aborted

writer

protected MultiThreadedWriterBase writer

numUniqueKeysVerified

private final AtomicLong numUniqueKeysVerified

The number of keys verified in a sequence. This will never be larger than the total number of keys in the range. The reader might also verify random keys when it catches up with the writer.

DEFAULT_MAX_ERRORS

public static final int DEFAULT_MAX_ERRORS

Default maximum number of read errors to tolerate before shutting down all readers.

See Also:

Constant Field Values

DEFAULT_KEY_WINDOW

public static final int DEFAULT_KEY_WINDOW

Default "window" size between the last key written by the writer and the key that we attempt to read. The lower this number, the stricter our testing is. If this is zero, we always attempt to read the highest key in the contiguous sequence of keys written by the writers.

See Also:

Constant Field Values

DEFAULT_BATCH_SIZE

public static final int DEFAULT_BATCH_SIZE

Default batch size for multigets

See Also:

Constant Field Values

numKeysVerified

protected AtomicLong numKeysVerified

numReadErrors

protected AtomicLong numReadErrors

numReadFailures

protected AtomicLong numReadFailures

nullResult

protected AtomicLong nullResult

maxErrors

private int maxErrors

keyWindow

private int keyWindow

batchSize

private int batchSize

regionReplicaId

private int regionReplicaId

Constructor Detail

MultiThreadedReader

public MultiThreadedReader(LoadTestDataGenerator dataGen,
                           org.apache.hadoop.conf.Configuration conf,
                           org.apache.hadoop.hbase.TableName tableName,
                           double verifyPercent)
                    throws IOException

Throws:

IOException

Method Detail

linkToWriter

public void linkToWriter(MultiThreadedWriterBase writer)

setMaxErrors

public void setMaxErrors(int maxErrors)

setKeyWindow

public void setKeyWindow(int keyWindow)

setMultiGetBatchSize

public void setMultiGetBatchSize(int batchSize)

setRegionReplicaId

public void setRegionReplicaId(int regionReplicaId)

start

public void start(long startKey,
                  long endKey,
                  int numThreads)
           throws IOException

Overrides:

start in class MultiThreadedAction

Throws:

IOException

addReaderThreads

protected void addReaderThreads(int numThreads)
                         throws IOException

Throws:

IOException

createReaderThread

protected MultiThreadedReader.HBaseReaderThread createReaderThread(int readerId)
                                                            throws IOException

Throws:

IOException

getNumReadFailures

public long getNumReadFailures()

getNumReadErrors

public long getNumReadErrors()

getNumKeysVerified

public long getNumKeysVerified()

getNumUniqueKeysVerified

public long getNumUniqueKeysVerified()

getNullResultsCount

public long getNullResultsCount()

progressInfo

protected String progressInfo()

Description copied from class: MultiThreadedAction

Returns a task-specific progress string

Specified by:

progressInfo in class MultiThreadedAction