org.apache.hadoop.hbase.util

Class LossyCounting

java.lang.Object

org.apache.hadoop.hbase.util.LossyCounting

@InterfaceAudience.Private
public class LossyCounting
extends Object

LossyCounting utility, bounded data structure that maintains approximate high frequency elements in data stream. Bucket size is 1 / error rate. (Error rate is 0.02 by default) Lemma If element
does not appear in set, then is frequency is less than e * N (N is total element counts until now.) Based on paper: http://www.vldb.org/conf/2002/S10P03.pdf

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	LossyCounting.LossyCountingListener

Field Summary

Fields

Modifier and Type	Field and Description
private long	bucketSize
private int	currentTerm
private Map<String,Integer>	data
private double	errorRate
private LossyCounting.LossyCountingListener	listener
private static org.slf4j.Logger	LOG
private String	name
private long	totalDataCount

Constructor Summary

Constructors

Constructor and Description
LossyCounting(double errorRate, String name, LossyCounting.LossyCountingListener listener)
LossyCounting(String name, LossyCounting.LossyCountingListener listener)

Method Summary

Modifier and Type	Method and Description
void	add(String key)
private void	addByOne(String key)
private void	calculateCurrentTerm() Calculate and set current term
boolean	contains(String key)
long	getBucketSize()
long	getCurrentTerm()
long	getDataSize()
Set<String>	getElements()
private void	sweep() sweep low frequency data

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

bucketSize

private long bucketSize

currentTerm

private int currentTerm

errorRate

private double errorRate

data

private Map<String,Integer> data

totalDataCount

private long totalDataCount

name

private String name

listener

private LossyCounting.LossyCountingListener listener

Constructor Detail

LossyCounting

public LossyCounting(double errorRate,
                     String name,
                     LossyCounting.LossyCountingListener listener)

LossyCounting

public LossyCounting(String name,
                     LossyCounting.LossyCountingListener listener)

Method Detail

addByOne

private void addByOne(String key)

add

public void add(String key)

sweep

private void sweep()

sweep low frequency data

calculateCurrentTerm

private void calculateCurrentTerm()

Calculate and set current term

getBucketSize

public long getBucketSize()

getDataSize

public long getDataSize()

contains

public boolean contains(String key)

getElements

public Set<String> getElements()

getCurrentTerm

public long getCurrentTerm()