org.apache.hadoop.hbase.client

Class SimpleRequestController

java.lang.Object

org.apache.hadoop.hbase.client.SimpleRequestController

All Implemented Interfaces:

RequestController

@InterfaceAudience.Private
 @InterfaceStability.Evolving
class SimpleRequestController
extends Object
implements RequestController

Holds back the requests if they reach any thresholds.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	SimpleRequestController.RequestHeapSizeChecker limit the heap size for each request.
(package private) static class	SimpleRequestController.RequestRowsChecker limit the number of rows for each request.
(package private) static interface	SimpleRequestController.RowChecker Provide a way to control the flow of rows iteration.
(package private) static class	SimpleRequestController.SubmittedSizeChecker limit the heapsize of total submitted data.
(package private) static class	SimpleRequestController.TaskCountChecker limit the max number of tasks in an AsyncProcess.

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.client.RequestController

RequestController.Checker, RequestController.ReturnCode

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static long	DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE Default value of HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE.
(package private) static long	DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_ROWS Default value of HBASE_CLIENT_MAX_PERREQUEST_ROWS.
(package private) static long	DEFAULT_HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE Default value of HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE.
private static int	DEFAULT_THRESHOLD_TO_LOG_REGION_DETAILS
private static int	DEFAULT_THRESHOLD_TO_LOG_UNDONE_TASK_DETAILS
static String	HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE The maximum heap size for each request.
static String	HBASE_CLIENT_MAX_PERREQUEST_ROWS The maximum number of rows for each request.
static String	HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE The maximum size of submit.
private static org.slf4j.Logger	LOG
(package private) int	maxConcurrentTasksPerRegion The number of tasks we run in parallel on a single region.
(package private) int	maxConcurrentTasksPerServer The number of task simultaneously executed on a single region server.
private long	maxHeapSizePerRequest The maximum heap size for each request.
private long	maxHeapSizeSubmit
private long	maxRowsPerRequest The maximum number of rows for each request.
private int	maxTotalConcurrentTasks The number of tasks simultaneously executed on the cluster.
(package private) ConcurrentMap<byte[],AtomicInteger>	taskCounterPerRegion
(package private) ConcurrentMap<ServerName,AtomicInteger>	taskCounterPerServer
(package private) AtomicLong	tasksInProgress
static String	THRESHOLD_TO_LOG_REGION_DETAILS
static String	THRESHOLD_TO_LOG_UNDONE_TASK_DETAILS
private int	thresholdToLogRegionDetails
private int	thresholdToLogUndoneTaskDetails

Constructor Summary

Constructors

Constructor and Description
SimpleRequestController(org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
private static int	checkAndGet(org.apache.hadoop.conf.Configuration conf, String key, int defaultValue)
private static long	checkAndGet(org.apache.hadoop.conf.Configuration conf, String key, long defaultValue)
void	decTaskCounters(Collection<byte[]> regions, ServerName sn) Decrement the counter if a task is accomplished.
long	getNumberOfTasksInProgress()
void	incTaskCounters(Collection<byte[]> regions, ServerName sn) Increment the counter if we build a valid task.
private void	logDetailsOfUndoneTasks(long taskInProgress)
RequestController.Checker	newChecker()
(package private) static RequestController.Checker	newChecker(List<SimpleRequestController.RowChecker> checkers)
void	waitForFreeSlot(long id, int periodToTrigger, Consumer<Long> trigger) Wait until there is at least one slot for a new task.
void	waitForMaximumCurrentTasks(long max, long id, int periodToTrigger, Consumer<Long> trigger) Waits for the running tasks to complete.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE

public static final String HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE

The maximum heap size for each request.

See Also:

Constant Field Values

DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE

static final long DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE

Default value of HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE.

See Also:

Constant Field Values

HBASE_CLIENT_MAX_PERREQUEST_ROWS

public static final String HBASE_CLIENT_MAX_PERREQUEST_ROWS

The maximum number of rows for each request.

See Also:

Constant Field Values

DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_ROWS

static final long DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_ROWS

Default value of HBASE_CLIENT_MAX_PERREQUEST_ROWS.

See Also:

Constant Field Values

HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE

public static final String HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE

The maximum size of submit.

See Also:

Constant Field Values

DEFAULT_HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE

static final long DEFAULT_HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE

Default value of HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE.

See Also:

Constant Field Values

tasksInProgress

final AtomicLong tasksInProgress

taskCounterPerRegion

final ConcurrentMap<byte[],AtomicInteger> taskCounterPerRegion

taskCounterPerServer

final ConcurrentMap<ServerName,AtomicInteger> taskCounterPerServer

maxTotalConcurrentTasks

private final int maxTotalConcurrentTasks

The number of tasks simultaneously executed on the cluster.

maxHeapSizePerRequest

private final long maxHeapSizePerRequest

The maximum heap size for each request.

maxRowsPerRequest

private final long maxRowsPerRequest

The maximum number of rows for each request.

maxHeapSizeSubmit

private final long maxHeapSizeSubmit

maxConcurrentTasksPerRegion

final int maxConcurrentTasksPerRegion

The number of tasks we run in parallel on a single region. With 1 (the default) , we ensure that the ordering of the queries is respected: we don't start a set of operations on a region before the previous one is done. As well, this limits the pressure we put on the region server.

maxConcurrentTasksPerServer

final int maxConcurrentTasksPerServer

The number of task simultaneously executed on a single region server.

thresholdToLogUndoneTaskDetails

private final int thresholdToLogUndoneTaskDetails

THRESHOLD_TO_LOG_UNDONE_TASK_DETAILS

public static final String THRESHOLD_TO_LOG_UNDONE_TASK_DETAILS

See Also:

Constant Field Values

DEFAULT_THRESHOLD_TO_LOG_UNDONE_TASK_DETAILS

private static final int DEFAULT_THRESHOLD_TO_LOG_UNDONE_TASK_DETAILS

See Also:

Constant Field Values

THRESHOLD_TO_LOG_REGION_DETAILS

public static final String THRESHOLD_TO_LOG_REGION_DETAILS

See Also:

Constant Field Values

DEFAULT_THRESHOLD_TO_LOG_REGION_DETAILS

private static final int DEFAULT_THRESHOLD_TO_LOG_REGION_DETAILS

See Also:

Constant Field Values

thresholdToLogRegionDetails

private final int thresholdToLogRegionDetails

Constructor Detail

SimpleRequestController

SimpleRequestController(org.apache.hadoop.conf.Configuration conf)

Method Detail

checkAndGet

private static int checkAndGet(org.apache.hadoop.conf.Configuration conf,
                               String key,
                               int defaultValue)

checkAndGet

private static long checkAndGet(org.apache.hadoop.conf.Configuration conf,
                                String key,
                                long defaultValue)

newChecker

static RequestController.Checker newChecker(List<SimpleRequestController.RowChecker> checkers)

newChecker

public RequestController.Checker newChecker()

Specified by:

newChecker in interface RequestController

Returns:

A new checker for evaluating a batch rows.

incTaskCounters

public void incTaskCounters(Collection<byte[]> regions,
                            ServerName sn)

Description copied from interface: RequestController

Increment the counter if we build a valid task.

Specified by:

incTaskCounters in interface RequestController

Parameters:

regions - The destination of task

sn - The target server

decTaskCounters

public void decTaskCounters(Collection<byte[]> regions,
                            ServerName sn)

Description copied from interface: RequestController

Decrement the counter if a task is accomplished.

Specified by:

decTaskCounters in interface RequestController

Parameters:

regions - The destination of task

sn - The target server

getNumberOfTasksInProgress

public long getNumberOfTasksInProgress()

Specified by:

getNumberOfTasksInProgress in interface RequestController

Returns:

The number of running task.

waitForMaximumCurrentTasks

public void waitForMaximumCurrentTasks(long max,
                                       long id,
                                       int periodToTrigger,
                                       Consumer<Long> trigger)
                                throws InterruptedIOException

Description copied from interface: RequestController

Waits for the running tasks to complete. If there are specified threshold and trigger, the implementation should wake up once in a while for checking the threshold and calling trigger.

Specified by:

waitForMaximumCurrentTasks in interface RequestController

Parameters:

max - This method will return if the number of running tasks is less than or equal to max.

id - the caller's id

periodToTrigger - The period to invoke the trigger. This value is a hint. The real period depends on the implementation.

trigger - The object to call periodically.

Throws:

InterruptedIOException - If the waiting is interrupted

logDetailsOfUndoneTasks

private void logDetailsOfUndoneTasks(long taskInProgress)

waitForFreeSlot

public void waitForFreeSlot(long id,
                            int periodToTrigger,
                            Consumer<Long> trigger)
                     throws InterruptedIOException

Description copied from interface: RequestController

Wait until there is at least one slot for a new task.

Specified by:

waitForFreeSlot in interface RequestController

Parameters:

id - the caller's id

periodToTrigger - The period to invoke the trigger. This value is a hint. The real period depends on the implementation.

trigger - The object to call periodically.

Throws:

InterruptedIOException - If the waiting is interrupted