AggregationClient (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.client.coprocessor.AggregationClient

All Implemented Interfaces:

Closeable, AutoCloseable
```
@InterfaceAudience.Private
public class AggregationClient
extends Object
implements Closeable
```
This client class is for invoking the aggregate functions deployed on the Region Server side via the AggregateService. This class will implement the supporting functionality for summing/processing the individual results obtained from the AggregateService for each region.
This will serve as the client side handler for invoking the aggregate functions. For all aggregate functions,
- start row < end row is an essential condition (if they are not HConstants.EMPTY_BYTE_ARRAY)
- Column family can't be null. In case where multiple families are provided, an IOException will be thrown. An optional column qualifier can also be defined.
- For methods to find maximum, minimum, sum, rowcount, it returns the parameter type. For average and std, it returns a double value. For row count, it returns a long value.
Call close() when done.

Field Summary

Fields
Modifier and Type Field and Description

private Connection connection

private static org.apache.commons.logging.Log log

Fields
Modifier and Type	Field and Description
`private Connection`	`connection`
`private static org.apache.commons.logging.Log`	`log`

Constructor Summary

Constructors
Constructor and Description

AggregationClient(org.apache.hadoop.conf.Configuration cfg)
Constructor with Conf object

Constructors
Constructor and Description
`AggregationClient(org.apache.hadoop.conf.Configuration cfg)` Constructor with Conf object

Method Summary

Methods
Modifier and Type	Method and Description
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double`	`avg(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` This is the client side interface/handle for calling the average method for a given cf-cq combination.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double`	`avg(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` This is the client side interface/handle for calling the average method for a given cf-cq combination.
`void`	`close()`
`private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<S,Long>`	`getAvgArgs(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It computes average while fetching sum and row count from all the corresponding regions.
`private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<S,Long>`	`getAvgArgs(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It computes average while fetching sum and row count from all the corresponding regions.
`(package private) byte[]`	`getBytesFromResponse(com.google.protobuf.ByteString response)`
`private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<NavigableMap<byte[],List<S>>,List<S>>`	`getMedianArgs(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It helps locate the region with median for a given column whose weight is specified in an optional column.
`private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<List<S>,Long>`	`getStdArgs(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It computes a global standard deviation for a given column and its value.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R`	`max(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It gives the maximum value of a column for a given column family for the given range.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R`	`max(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It gives the maximum value of a column for a given column family for the given range.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R`	`median(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` This is the client side interface/handler for calling the median method for a given cf-cq combination.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R`	`median(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` This is the client side interface/handler for calling the median method for a given cf-cq combination.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R`	`min(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It gives the minimum value of a column for a given column family for the given range.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R`	`min(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It gives the minimum value of a column for a given column family for the given range.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> long`	`rowCount(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It gives the row count, by summing up the individual results obtained from regions.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> long`	`rowCount(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It gives the row count, by summing up the individual results obtained from regions.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double`	`std(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` This is the client side interface/handle for calling the std method for a given cf-cq combination.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double`	`std(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` This is the client side interface/handle for calling the std method for a given cf-cq combination.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> S`	`sum(Table table, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It sums up the value returned from various regions.
`<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> S`	`sum(TableName tableName, ColumnInterpreter<R,S,P,Q,T> ci, Scan scan)` It sums up the value returned from various regions.
`(package private) <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> org.apache.hadoop.hbase.protobuf.generated.AggregateProtos.AggregateRequest`	`validateArgAndGetPB(Scan scan, ColumnInterpreter<R,S,P,Q,T> ci, boolean canFamilyBeAbsent)`
`private void`	`validateParameters(Scan scan, boolean canFamilyBeAbsent)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

private static final org.apache.commons.logging.Log log

connection
```
private final Connection connection
```

Constructor Detail
- AggregationClient
```
public AggregationClient(org.apache.hadoop.conf.Configuration cfg)
```
  Constructor with Conf object
  
  Parameters:
  cfg -

Method Detail

close
```
public void close()
           throws IOException
```
Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

max

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R max(TableName tableName,
                                                                                                                            ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                            Scan scan)
      throws Throwable

It gives the maximum value of a column for a given column family for the given range. In case qualifier is null, a max of all values for the given family is returned.

Parameters:: tableName -; ci -; scan -
Returns:: max val <R>
Throws:: Throwable - The caller is supposed to handle the exception as they are thrown & propagated to it.

max

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R max(Table table,
                                                                                                                            ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                            Scan scan)
      throws Throwable

It gives the maximum value of a column for a given column family for the given range. In case qualifier is null, a max of all values for the given family is returned.

Parameters:: table -; ci -; scan -
Returns:: max val <>
Throws:: Throwable - The caller is supposed to handle the exception as they are thrown & propagated to it.

validateParameters

private void validateParameters(Scan scan,
                      boolean canFamilyBeAbsent)
                         throws IOException

Throws:: IOException

min

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R min(TableName tableName,
                                                                                                                            ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                            Scan scan)
      throws Throwable

It gives the minimum value of a column for a given column family for the given range. In case qualifier is null, a min of all values for the given family is returned.

Parameters:: tableName -; ci -; scan -
Returns:: min val <R>
Throws:: Throwable

min

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R min(Table table,
                                                                                                                            ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                            Scan scan)
      throws Throwable

It gives the minimum value of a column for a given column family for the given range. In case qualifier is null, a min of all values for the given family is returned.

Parameters:: table -; ci -; scan -
Returns:: min val <R>
Throws:: Throwable

rowCount

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> long rowCount(TableName tableName,
                                                                                                                                    ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                    Scan scan)
              throws Throwable

It gives the row count, by summing up the individual results obtained from regions. In case the qualifier is null, FirstKeyValueFilter is used to optimised the operation. In case qualifier is provided, I can't use the filter as it may set the flag to skip to next row, but the value read is not of the given filter: in this case, this particular row will not be counted ==> an error.

Parameters:: tableName -; ci -; scan -
Returns:: <R, S>
Throws:: Throwable

rowCount

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> long rowCount(Table table,
                                                                                                                                    ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                    Scan scan)
              throws Throwable

Parameters:: table -; ci -; scan -
Returns:: <R, S>
Throws:: Throwable

sum

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> S sum(TableName tableName,
                                                                                                                            ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                            Scan scan)
      throws Throwable

It sums up the value returned from various regions. In case qualifier is null, summation of all the column qualifiers in the given family is done.

Parameters:: tableName -; ci -; scan -
Returns:: sum <S>
Throws:: Throwable

sum

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> S sum(Table table,
                                                                                                                            ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                            Scan scan)
      throws Throwable

It sums up the value returned from various regions. In case qualifier is null, summation of all the column qualifiers in the given family is done.

Parameters:: table -; ci -; scan -
Returns:: sum <S>
Throws:: Throwable

getAvgArgs

private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<S,Long> getAvgArgs(TableName tableName,
                                                                                                                                              ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                              Scan scan)
                         throws Throwable

It computes average while fetching sum and row count from all the corresponding regions. Approach is to compute a global sum of region level sum and rowcount and then compute the average.

Parameters:: tableName -; scan -
Throws:: Throwable

getAvgArgs

private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<S,Long> getAvgArgs(Table table,
                                                                                                                                              ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                              Scan scan)
                         throws Throwable

It computes average while fetching sum and row count from all the corresponding regions. Approach is to compute a global sum of region level sum and rowcount and then compute the average.

Parameters:: table -; scan -
Throws:: Throwable

avg

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double avg(TableName tableName,
                                                                                                                                 ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                 Scan scan)
           throws Throwable

This is the client side interface/handle for calling the average method for a given cf-cq combination. It was necessary to add one more call stack as its return type should be a decimal value, irrespective of what columninterpreter says. So, this methods collects the necessary parameters to compute the average and returs the double value.

Parameters:: tableName -; ci -; scan -
Returns:: <R, S>
Throws:: Throwable

avg

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double avg(Table table,
                                                                                                                                 ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                 Scan scan)
           throws Throwable

Parameters:: table -; ci -; scan -
Returns:: <R, S>
Throws:: Throwable

getStdArgs

private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<List<S>,Long> getStdArgs(Table table,
                                                                                                                                                    ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                                    Scan scan)
                               throws Throwable

It computes a global standard deviation for a given column and its value. Standard deviation is square root of (average of squares - average*average). From individual regions, it obtains sum, square sum and number of rows. With these, the above values are computed to get the global std.

Parameters:: table -; scan -
Returns:: standard deviations
Throws:: Throwable

std

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double std(TableName tableName,
                                                                                                                                 ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                 Scan scan)
           throws Throwable

This is the client side interface/handle for calling the std method for a given cf-cq combination. It was necessary to add one more call stack as its return type should be a decimal value, irrespective of what columninterpreter says. So, this methods collects the necessary parameters to compute the std and returns the double value.

Parameters:: tableName -; ci -; scan -
Returns:: <R, S>
Throws:: Throwable

std

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> double std(Table table,
                                                                                                                                 ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                 Scan scan)
           throws Throwable

Parameters:: table -; ci -; scan -
Returns:: <R, S>
Throws:: Throwable

getMedianArgs

private <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> Pair<NavigableMap<byte[],List<S>>,List<S>> getMedianArgs(Table table,
                                                                                                                                                                               ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                                                               Scan scan)
                                                          throws Throwable

It helps locate the region with median for a given column whose weight is specified in an optional column. From individual regions, it obtains sum of values and sum of weights.

Parameters:: table -; ci -; scan -
Returns:: pair whose first element is a map between start row of the region and (sum of values, sum of weights) for the region, the second element is (sum of values, sum of weights) for all the regions chosen
Throws:: Throwable

median

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R median(TableName tableName,
                                                                                                                               ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                               Scan scan)
         throws Throwable

This is the client side interface/handler for calling the median method for a given cf-cq combination. This method collects the necessary parameters to compute the median and returns the median.

Parameters:: tableName -; ci -; scan -
Returns:: R the median
Throws:: Throwable

median

public <R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> R median(Table table,
                                                                                                                               ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                               Scan scan)
         throws Throwable

This is the client side interface/handler for calling the median method for a given cf-cq combination. This method collects the necessary parameters to compute the median and returns the median.

Parameters:: table -; ci -; scan -
Returns:: R the median
Throws:: Throwable

validateArgAndGetPB

<R,S,P extends com.google.protobuf.Message,Q extends com.google.protobuf.Message,T extends com.google.protobuf.Message> org.apache.hadoop.hbase.protobuf.generated.AggregateProtos.AggregateRequest validateArgAndGetPB(Scan scan,
                                                                                                                                                                                                                      ColumnInterpreter<R,S,P,Q,T> ci,
                                                                                                                                                                                                                      boolean canFamilyBeAbsent)
                                                                                          throws IOException

Throws:: IOException

getBytesFromResponse

byte[] getBytesFromResponse(com.google.protobuf.ByteString response)

Class AggregationClient

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

log

connection

Constructor Detail

AggregationClient

Method Detail

close

max

max

validateParameters

min

min

rowCount

rowCount

sum

sum

getAvgArgs

getAvgArgs

avg

avg

getStdArgs

std

std

getMedianArgs

median

median

validateArgAndGetPB

getBytesFromResponse