Package org.apache.hadoop.hbase.quotas

Class DefaultOperationQuota

java.lang.Object

org.apache.hadoop.hbase.quotas.DefaultOperationQuota

All Implemented Interfaces:

OperationQuota

Direct Known Subclasses:

ExceedOperationQuota

@Private
@Evolving
public class DefaultOperationQuota
extends Object
implements OperationQuota

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.quotas.OperationQuota

OperationQuota.OperationType

Field Summary

Fields

Modifier and Type	Field	Description
private long	blockSizeBytes
protected final List<QuotaLimiter>	limiters
private static final double	MAX_SCAN_ESTIMATE_PROPORTIONAL_LIMIT_CONSUMPTION
private long	maxScanEstimate
private final long[]	operationSize
protected long	readAvailable
private final long	readCapacityUnit
protected long	readCapacityUnitConsumed
protected long	readCapacityUnitDiff
protected long	readConsumed
protected long	readDiff
private boolean	useResultSizeBytes
private final long	writeCapacityUnit
protected long	writeCapacityUnitConsumed
protected long	writeCapacityUnitDiff
protected long	writeConsumed
protected long	writeDiff

Fields inherited from interface org.apache.hadoop.hbase.quotas.OperationQuota

USE_RESULT_SIZE_BYTES, USE_RESULT_SIZE_BYTES_DEFAULT

Constructor Summary

Constructors

Constructor	Description
DefaultOperationQuota(org.apache.hadoop.conf.Configuration conf, int blockSizeBytes, QuotaLimiter... limiters)
DefaultOperationQuota(org.apache.hadoop.conf.Configuration conf, List<QuotaLimiter> limiters)	NOTE: The order matters.

Method Summary

Modifier and Type	Method	Description
void	addGetResult(Result result)	Add a get result.
void	addMutation(Mutation mutation)	Add a mutation result.
void	addScanResult(List<Result> results)	Add a scan result.
void	addScanResultCells(List<Cell> cells)	Add a scan result in the form of cells.
private long	calculateReadCapacityUnit(long size)
private long	calculateReadCapacityUnitDiff(long actualSize, long estimateSize)
private long	calculateWriteCapacityUnit(long size)
private long	calculateWriteCapacityUnitDiff(long actualSize, long estimateSize)
void	checkBatchQuota(int numWrites, int numReads)	Checks if it is possible to execute the specified operation.
private void	checkQuota(long numWrites, long numReads)
void	checkScanQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest scanRequest, long maxScannerResultSize, long maxBlockBytesScanned, long prevBlockBytesScannedDifference)	Checks if it is possible to execute the scan.
void	close()	Cleanup method on operation completion
private long	estimateConsume(OperationQuota.OperationType type, int numReqs, long avgSize)
long	getReadAvailable()	Returns the number of bytes available to read to avoid exceeding the quota
long	getReadConsumed()	Returns the number of bytes consumed from the quota by the operation
protected static long	getScanReadConsumeEstimate(long blockSizeBytes, long nextCallSeq, long maxScannerResultSize, long maxBlockBytesScanned, long prevBlockBytesScannedDifference)
protected void	updateEstimateConsumeBatchQuota(int numWrites, int numReads)	Update estimate quota(read/write size/capacityUnits) which will be consumed
protected void	updateEstimateConsumeScanQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest scanRequest, long maxScannerResultSize, long maxBlockBytesScanned, long prevBlockBytesScannedDifference)	Update estimate quota(read/write size/capacityUnits) which will be consumed

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.quotas.OperationQuota

getMaxResultSize

Field Details

MAX_SCAN_ESTIMATE_PROPORTIONAL_LIMIT_CONSUMPTION

private static final double MAX_SCAN_ESTIMATE_PROPORTIONAL_LIMIT_CONSUMPTION

See Also:

• Constant Field Values

limiters

protected final List<QuotaLimiter> limiters

writeCapacityUnit

private final long writeCapacityUnit

readCapacityUnit

private final long readCapacityUnit

readAvailable

protected long readAvailable

writeConsumed

protected long writeConsumed

readConsumed

protected long readConsumed

writeCapacityUnitConsumed

protected long writeCapacityUnitConsumed

readCapacityUnitConsumed

protected long readCapacityUnitConsumed

operationSize

private final long[] operationSize

writeDiff

protected long writeDiff

readDiff

protected long readDiff

writeCapacityUnitDiff

protected long writeCapacityUnitDiff

readCapacityUnitDiff

protected long readCapacityUnitDiff

useResultSizeBytes

private boolean useResultSizeBytes

blockSizeBytes

private long blockSizeBytes

maxScanEstimate

private long maxScanEstimate

Constructor Details

DefaultOperationQuota

public DefaultOperationQuota(org.apache.hadoop.conf.Configuration conf,
 int blockSizeBytes,
 QuotaLimiter... limiters)

DefaultOperationQuota

public DefaultOperationQuota(org.apache.hadoop.conf.Configuration conf,
 List<QuotaLimiter> limiters)

NOTE: The order matters. It should be something like [user, table, namespace, global]

Method Details

checkBatchQuota

public void checkBatchQuota(int numWrites,
 int numReads)
                     throws RpcThrottlingException

Description copied from interface: OperationQuota

Checks if it is possible to execute the specified operation. The quota will be estimated based on the number of operations to perform and the average size accumulated during time.

Specified by:

checkBatchQuota in interface OperationQuota

Parameters:

numWrites - number of write operation that will be performed

numReads - number of small-read operation that will be performed

Throws:

RpcThrottlingException - if the operation cannot be performed because RPC quota is exceeded.

checkScanQuota

public void checkScanQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest scanRequest,
 long maxScannerResultSize,
 long maxBlockBytesScanned,
 long prevBlockBytesScannedDifference)
                    throws RpcThrottlingException

Description copied from interface: OperationQuota

Checks if it is possible to execute the scan. The quota will be estimated based on the composition of the scan.

Specified by:

checkScanQuota in interface OperationQuota

Parameters:

scanRequest - the given scan operation

maxScannerResultSize - the maximum bytes to be returned by the scanner

maxBlockBytesScanned - the maximum bytes scanned in a single RPC call by the scanner

prevBlockBytesScannedDifference - the difference between BBS of the previous two next calls

Throws:

RpcThrottlingException - if the operation cannot be performed because RPC quota is exceeded.

checkQuota

private void checkQuota(long numWrites,
 long numReads)
                 throws RpcThrottlingException

Throws:

RpcThrottlingException

close

public void close()

Description copied from interface: OperationQuota

Cleanup method on operation completion

Specified by:

close in interface OperationQuota

getReadAvailable

public long getReadAvailable()

Description copied from interface: OperationQuota

Returns the number of bytes available to read to avoid exceeding the quota

Specified by:

getReadAvailable in interface OperationQuota

getReadConsumed

public long getReadConsumed()

Description copied from interface: OperationQuota

Returns the number of bytes consumed from the quota by the operation

Specified by:

getReadConsumed in interface OperationQuota

addGetResult

public void addGetResult(Result result)

Description copied from interface: OperationQuota

Add a get result. This will be used to calculate the exact quota and have a better short-read average size for the next time.

Specified by:

addGetResult in interface OperationQuota

addScanResult

public void addScanResult(List<Result> results)

Description copied from interface: OperationQuota

Add a scan result. This will be used to calculate the exact quota and have a better long-read average size for the next time.

Specified by:

addScanResult in interface OperationQuota

addScanResultCells

public void addScanResultCells(List<Cell> cells)

Description copied from interface: OperationQuota

Add a scan result in the form of cells. This will be used to calculate the exact quota and have a better long-read average size for the next time.

Specified by:

addScanResultCells in interface OperationQuota

addMutation

public void addMutation(Mutation mutation)

Description copied from interface: OperationQuota

Add a mutation result. This will be used to calculate the exact quota and have a better mutation average size for the next time.

Specified by:

addMutation in interface OperationQuota

updateEstimateConsumeBatchQuota

protected void updateEstimateConsumeBatchQuota(int numWrites,
 int numReads)

Update estimate quota(read/write size/capacityUnits) which will be consumed

Parameters:

numWrites - the number of write requests

numReads - the number of read requests

updateEstimateConsumeScanQuota

protected void updateEstimateConsumeScanQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ScanRequest scanRequest,
 long maxScannerResultSize,
 long maxBlockBytesScanned,
 long prevBlockBytesScannedDifference)

Update estimate quota(read/write size/capacityUnits) which will be consumed

Parameters:

scanRequest - the scan to be executed

maxScannerResultSize - the maximum bytes to be returned by the scanner

maxBlockBytesScanned - the maximum bytes scanned in a single RPC call by the scanner

prevBlockBytesScannedDifference - the difference between BBS of the previous two next calls

getScanReadConsumeEstimate

protected static long getScanReadConsumeEstimate(long blockSizeBytes,
 long nextCallSeq,
 long maxScannerResultSize,
 long maxBlockBytesScanned,
 long prevBlockBytesScannedDifference)

estimateConsume

private long estimateConsume(OperationQuota.OperationType type,
 int numReqs,
 long avgSize)

calculateWriteCapacityUnit

private long calculateWriteCapacityUnit(long size)

calculateReadCapacityUnit

private long calculateReadCapacityUnit(long size)

calculateWriteCapacityUnitDiff

private long calculateWriteCapacityUnitDiff(long actualSize,
 long estimateSize)

calculateReadCapacityUnitDiff

private long calculateReadCapacityUnitDiff(long actualSize,
 long estimateSize)