Package org.apache.hadoop.hbase.client

Class ClientAsyncPrefetchScanner

java.lang.Object

org.apache.hadoop.hbase.client.AbstractClientScanner

org.apache.hadoop.hbase.client.ClientScanner

org.apache.hadoop.hbase.client.ClientSimpleScanner

org.apache.hadoop.hbase.client.ClientAsyncPrefetchScanner

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<Result>, ResultScanner

@Private
public class ClientAsyncPrefetchScanner
extends ClientSimpleScanner

ClientAsyncPrefetchScanner implements async scanner behaviour. Specifically, the cache used by this scanner is a concurrent queue which allows both the producer (hbase client) and consumer (application) to access the queue in parallel. The number of rows returned in a prefetch is defined by the caching factor and the result size factor. This class allocates a buffer cache, whose size is a function of both factors. The prefetch is invoked when the cache is half-filled, instead of waiting for it to be empty. This is defined in the method prefetchCondition().

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private class	ClientAsyncPrefetchScanner.PrefetchRunnable

Field Summary

Fields

Modifier and Type	Field	Description
private AtomicLong	cacheSizeInBytes
private final Queue<Exception>	exceptionsQueue
private final Lock	lock
private long	maxCacheSize
private final Condition	notEmpty
private final Condition	notFull
private Consumer<Boolean>	prefetchListener

Fields inherited from class org.apache.hadoop.hbase.client.ClientScanner

cache, caching, callable, caller, closed, conf, currentRegion, lastNext, lastResult, maxScannerResultSize, pool, primaryOperationTimeout, readRpcTimeout, requestAttributes, rpcControllerFactory, scan, scanMetricsPublished, scannerTimeout, span, tableName

Fields inherited from class org.apache.hadoop.hbase.client.AbstractClientScanner

scanMetrics

Constructor Summary

Constructors

Constructor

Description

ClientAsyncPrefetchScanner(org.apache.hadoop.conf.Configuration configuration, Scan scan, Scan scanForMetrics, TableName name, ClusterConnection connection, RpcRetryingCallerFactory rpcCallerFactory, RpcControllerFactory rpcControllerFactory, ExecutorService pool, int scanReadRpcTimeout, int scannerTimeout, int replicaCallTimeoutMicroSecondScan, ConnectionConfiguration connectionConfiguration, Map<String,byte[]> requestAttributes)

Method Summary

Modifier and Type	Method	Description
protected void	addEstimatedSize(long estimatedSize)
void	close()	Closes the scanner and releases any resources it has allocated
private void	handleException()
protected void	initCache()
Result	next()	Grab the next row's worth of values.
private Result	pollCache()
private boolean	prefetchCondition()
private long	resultSize2CacheSize(long maxResultSize)
(package private) void	setPrefetchListener(Consumer<Boolean> prefetchListener)

Methods inherited from class org.apache.hadoop.hbase.client.ClientSimpleScanner

createScannerCallable, setNewStartKey

Methods inherited from class org.apache.hadoop.hbase.client.ClientScanner

getCacheCount, getCacheSize, getCaching, getConf, getConnection, getMaxResultSize, getPool, getPrimaryOperationTimeout, getRetries, getScan, getScannerTimeout, getScanReplicaId, getTable, getTimestamp, initSyncCache, isAnyRPCcancelled, loadCache, moveToNextRegion, nextWithSyncCache, renewLease, writeScanMetrics

Methods inherited from class org.apache.hadoop.hbase.client.AbstractClientScanner

getScanMetrics, initScanMetrics

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Methods inherited from interface org.apache.hadoop.hbase.client.ResultScanner

iterator, next

Field Details

maxCacheSize

private long maxCacheSize

cacheSizeInBytes

private AtomicLong cacheSizeInBytes

exceptionsQueue

private final Queue<Exception> exceptionsQueue

prefetchListener

private Consumer<Boolean> prefetchListener

lock

private final Lock lock

notEmpty

private final Condition notEmpty

notFull

private final Condition notFull

Constructor Details

ClientAsyncPrefetchScanner

public ClientAsyncPrefetchScanner(org.apache.hadoop.conf.Configuration configuration,
 Scan scan,
 Scan scanForMetrics,
 TableName name,
 ClusterConnection connection,
 RpcRetryingCallerFactory rpcCallerFactory,
 RpcControllerFactory rpcControllerFactory,
 ExecutorService pool,
 int scanReadRpcTimeout,
 int scannerTimeout,
 int replicaCallTimeoutMicroSecondScan,
 ConnectionConfiguration connectionConfiguration,
 Map<String,byte[]> requestAttributes)
                           throws IOException

Throws:

IOException

Method Details

setPrefetchListener

void setPrefetchListener(Consumer<Boolean> prefetchListener)

initCache

protected void initCache()

Overrides:

initCache in class ClientScanner

resultSize2CacheSize

private long resultSize2CacheSize(long maxResultSize)

next

public Result next()
            throws IOException

Description copied from interface: ResultScanner

Grab the next row's worth of values. The scanner will return a Result.

Specified by:

next in interface ResultScanner

Overrides:

next in class ClientScanner

Returns:

Result object if there is another row, null if the scanner is exhausted.

Throws:

IOException - e

close

public void close()

Description copied from interface: ResultScanner

Closes the scanner and releases any resources it has allocated

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface ResultScanner

Overrides:

close in class ClientScanner

addEstimatedSize

protected void addEstimatedSize(long estimatedSize)

Overrides:

addEstimatedSize in class ClientScanner

handleException

private void handleException()
                      throws IOException

Throws:

IOException

prefetchCondition

private boolean prefetchCondition()

pollCache

private Result pollCache()