org.apache.hadoop.hbase.client

Class ClientAsyncPrefetchScanner

java.lang.Object

org.apache.hadoop.hbase.client.AbstractClientScanner

org.apache.hadoop.hbase.client.ClientScanner

org.apache.hadoop.hbase.client.ClientSimpleScanner

org.apache.hadoop.hbase.client.ClientAsyncPrefetchScanner

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<Result>, ResultScanner

@InterfaceAudience.Private
public class ClientAsyncPrefetchScanner
extends ClientSimpleScanner

ClientAsyncPrefetchScanner implements async scanner behaviour. Specifically, the cache used by this scanner is a concurrent queue which allows both the producer (hbase client) and consumer (application) to access the queue in parallel. The number of rows returned in a prefetch is defined by the caching factor and the result size factor. This class allocates a buffer cache, whose size is a function of both factors. The prefetch is invoked when the cache is half­filled, instead of waiting for it to be empty. This is defined in the method prefetchCondition().

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	ClientAsyncPrefetchScanner.PrefetchRunnable

Field Summary

Fields

Modifier and Type	Field and Description
private AtomicLong	cacheSizeInBytes
private Queue<Exception>	exceptionsQueue
private Lock	lock
private long	maxCacheSize
private Condition	notEmpty
private Condition	notFull
private Thread	prefetcher
private Consumer<Boolean>	prefetchListener

Fields inherited from class org.apache.hadoop.hbase.client.ClientScanner

cache, caching, callable, caller, closed, conf, currentRegion, lastNext, lastResult, maxScannerResultSize, pool, primaryOperationTimeout, rpcControllerFactory, scan, scanMetricsPublished, scannerTimeout, tableName

Fields inherited from class org.apache.hadoop.hbase.client.AbstractClientScanner

scanMetrics

Constructor Summary

Constructors

Constructor and Description
ClientAsyncPrefetchScanner(org.apache.hadoop.conf.Configuration configuration, Scan scan, TableName name, ClusterConnection connection, RpcRetryingCallerFactory rpcCallerFactory, RpcControllerFactory rpcControllerFactory, ExecutorService pool, int replicaCallTimeoutMicroSecondScan)

Method Summary

Modifier and Type	Method and Description
protected void	addEstimatedSize(long estimatedSize)
void	close() Closes the scanner and releases any resources it has allocated
private void	handleException()
protected void	initCache()
Result	next() Grab the next row's worth of values.
private Result	pollCache()
private boolean	prefetchCondition()
private long	resultSize2CacheSize(long maxResultSize)
(package private) void	setPrefetchListener(Consumer<Boolean> prefetchListener)

Methods inherited from class org.apache.hadoop.hbase.client.ClientSimpleScanner

createScannerCallable, setNewStartKey

Methods inherited from class org.apache.hadoop.hbase.client.ClientScanner

getCacheCount, getCacheSize, getCaching, getConf, getConnection, getMaxResultSize, getPool, getPrimaryOperationTimeout, getRetries, getScan, getScannerTimeout, getTable, getTimestamp, initSyncCache, isAnyRPCcancelled, loadCache, moveToNextRegion, nextWithSyncCache, renewLease, writeScanMetrics

Methods inherited from class org.apache.hadoop.hbase.client.AbstractClientScanner

getScanMetrics, initScanMetrics

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.client.ResultScanner

iterator, next

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

maxCacheSize

private long maxCacheSize

cacheSizeInBytes

private AtomicLong cacheSizeInBytes

exceptionsQueue

private Queue<Exception> exceptionsQueue

prefetcher

private Thread prefetcher

prefetchListener

private Consumer<Boolean> prefetchListener

lock

private final Lock lock

notEmpty

private final Condition notEmpty

notFull

private final Condition notFull

Constructor Detail

ClientAsyncPrefetchScanner

public ClientAsyncPrefetchScanner(org.apache.hadoop.conf.Configuration configuration,
                                  Scan scan,
                                  TableName name,
                                  ClusterConnection connection,
                                  RpcRetryingCallerFactory rpcCallerFactory,
                                  RpcControllerFactory rpcControllerFactory,
                                  ExecutorService pool,
                                  int replicaCallTimeoutMicroSecondScan)
                           throws IOException

Throws:

IOException

Method Detail

setPrefetchListener

void setPrefetchListener(Consumer<Boolean> prefetchListener)

initCache

protected void initCache()

Overrides:

initCache in class ClientScanner

resultSize2CacheSize

private long resultSize2CacheSize(long maxResultSize)

next

public Result next()
            throws IOException

Description copied from interface: ResultScanner

Grab the next row's worth of values. The scanner will return a Result.

Specified by:

next in interface ResultScanner

Overrides:

next in class ClientScanner

Returns:

Result object if there is another row, null if the scanner is exhausted.

Throws:

IOException - e

close

public void close()

Description copied from interface: ResultScanner

Closes the scanner and releases any resources it has allocated

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface ResultScanner

Overrides:

close in class ClientScanner

addEstimatedSize

protected void addEstimatedSize(long estimatedSize)

Overrides:

addEstimatedSize in class ClientScanner

handleException

private void handleException()
                      throws IOException

Throws:

IOException

prefetchCondition

private boolean prefetchCondition()

pollCache

private Result pollCache()