org.apache.hadoop.hbase.regionserver

Class AdaptiveMemStoreCompactionStrategy

java.lang.Object

org.apache.hadoop.hbase.regionserver.MemStoreCompactionStrategy

org.apache.hadoop.hbase.regionserver.AdaptiveMemStoreCompactionStrategy

@InterfaceAudience.Private
public class AdaptiveMemStoreCompactionStrategy
extends MemStoreCompactionStrategy

Adaptive is a heuristic that chooses whether to apply data compaction or not based on the level of redundancy in the data. Adaptive triggers redundancy elimination only for those stores where positive impact is expected. Adaptive uses two parameters to determine whether to perform redundancy elimination. The first parameter, u, estimates the ratio of unique keys in the memory store based on the fraction of unique keys encountered during the previous merge of segment indices. The second is the perceived probability (compactionProbability) that the store can benefit from redundancy elimination. Initially, compactionProbability=0.5; it then grows exponentially by 2% whenever a compaction is successful and decreased by 2% whenever a compaction did not meet the expectation. It is reset back to the default value (namely 0.5) upon disk flush. Adaptive triggers redundancy elimination with probability compactionProbability if the fraction of redundant keys 1-u exceeds a parameter threshold compactionThreshold.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.regionserver.MemStoreCompactionStrategy

MemStoreCompactionStrategy.Action

Field Summary

Fields

Modifier and Type	Field and Description
private static double	ADAPTIVE_COMPACTION_THRESHOLD_DEFAULT
static String	ADAPTIVE_COMPACTION_THRESHOLD_KEY
private static double	ADAPTIVE_INITIAL_COMPACTION_PROBABILITY_DEFAULT
static String	ADAPTIVE_INITIAL_COMPACTION_PROBABILITY_KEY
private static double	ADAPTIVE_PROBABILITY_FACTOR
private boolean	compacted
private double	compactionProbability
private double	compactionThreshold
private double	initialCompactionProbability
private static String	NAME
private double	numCellsInVersionedList
private Random	rand

Fields inherited from class org.apache.hadoop.hbase.regionserver.MemStoreCompactionStrategy

cfName, COMPACTING_MEMSTORE_THRESHOLD_DEFAULT, COMPACTING_MEMSTORE_THRESHOLD_KEY, LOG, pipelineThreshold

Constructor Summary

Constructors

Constructor and Description
AdaptiveMemStoreCompactionStrategy(org.apache.hadoop.conf.Configuration conf, String cfName)

Method Summary

Modifier and Type	Method and Description
MemStoreCompactionStrategy.Action	getAction(VersionedSegmentsList versionedList)
protected MemStoreCompactionStrategy.Action	getFlattenAction()
protected MemStoreCompactionStrategy.Action	getMergingAction()
protected String	getName()
void	resetStats()
void	updateStats(Segment replacement)

Methods inherited from class org.apache.hadoop.hbase.regionserver.MemStoreCompactionStrategy

compact, simpleMergeOrFlatten, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

NAME

private static final String NAME

See Also:

Constant Field Values

ADAPTIVE_COMPACTION_THRESHOLD_KEY

public static final String ADAPTIVE_COMPACTION_THRESHOLD_KEY

See Also:

Constant Field Values

ADAPTIVE_COMPACTION_THRESHOLD_DEFAULT

private static final double ADAPTIVE_COMPACTION_THRESHOLD_DEFAULT

See Also:

Constant Field Values

ADAPTIVE_INITIAL_COMPACTION_PROBABILITY_KEY

public static final String ADAPTIVE_INITIAL_COMPACTION_PROBABILITY_KEY

See Also:

Constant Field Values

ADAPTIVE_INITIAL_COMPACTION_PROBABILITY_DEFAULT

private static final double ADAPTIVE_INITIAL_COMPACTION_PROBABILITY_DEFAULT

See Also:

Constant Field Values

ADAPTIVE_PROBABILITY_FACTOR

private static final double ADAPTIVE_PROBABILITY_FACTOR

See Also:

Constant Field Values

compactionThreshold

private double compactionThreshold

initialCompactionProbability

private double initialCompactionProbability

compactionProbability

private double compactionProbability

rand

private Random rand

numCellsInVersionedList

private double numCellsInVersionedList

compacted

private boolean compacted

Constructor Detail

AdaptiveMemStoreCompactionStrategy

public AdaptiveMemStoreCompactionStrategy(org.apache.hadoop.conf.Configuration conf,
                                          String cfName)

Method Detail

getAction

public MemStoreCompactionStrategy.Action getAction(VersionedSegmentsList versionedList)

Specified by:

getAction in class MemStoreCompactionStrategy

updateStats

public void updateStats(Segment replacement)

Overrides:

updateStats in class MemStoreCompactionStrategy

resetStats

public void resetStats()

Overrides:

resetStats in class MemStoreCompactionStrategy

getMergingAction

protected MemStoreCompactionStrategy.Action getMergingAction()

Overrides:

getMergingAction in class MemStoreCompactionStrategy

getFlattenAction

protected MemStoreCompactionStrategy.Action getFlattenAction()

Overrides:

getFlattenAction in class MemStoreCompactionStrategy

getName

protected String getName()

Specified by:

getName in class MemStoreCompactionStrategy