org.apache.hadoop.hbase.regionserver

Class DefaultMemStore

java.lang.Object

org.apache.hadoop.hbase.regionserver.AbstractMemStore

org.apache.hadoop.hbase.regionserver.DefaultMemStore

All Implemented Interfaces:

MemStore

@InterfaceAudience.Private
public class DefaultMemStore
extends AbstractMemStore

The MemStore holds in-memory modifications to the Store. Modifications are Cells. When asked to flush, current memstore is moved to snapshot and is cleared. We continue to serve edits out of new memstore and backing snapshot until flusher reports in that the flush succeeded. At this point we let the snapshot go.

The MemStore functions should not be called in parallel. Callers should hold write and read locks. This is done in HStore.

TODO: Adjust size of the memstore when we remove items because they have been deleted. TODO: With new KVSLS, need to make sure we update HeapSize with difference in KV size.

Field Summary

Fields

Modifier and Type	Field and Description
static long	DEEP_OVERHEAD
static long	FIXED_OVERHEAD
private static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

active, regionServices, snapshot, snapshotId

Constructor Summary

Constructors

Constructor and Description
DefaultMemStore() Default constructor.
DefaultMemStore(org.apache.hadoop.conf.Configuration conf, CellComparator c) Constructor.
DefaultMemStore(org.apache.hadoop.conf.Configuration conf, CellComparator c, RegionServicesForStores regionServices) Constructor.

Method Summary

Modifier and Type	Method and Description
protected void	checkActiveSize() Check whether anything need to be done based on the current active set size Nothing need to be done for the DefaultMemStore
MemStoreSize	getFlushableSize() Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set).
(package private) Cell	getNextRow(Cell cell)
List<KeyValueScanner>	getScanners(long readPt)
protected List<Segment>	getSegments()
protected long	heapSize()
boolean	isSloppy()
protected long	keySize()
static void	main(String[] args) Code to help figure if our approximation of object heap sizes is close enough.
long	preFlushSeqIDEstimation() This method is called before the flush is executed.
MemStoreSize	size()
MemStoreSnapshot	snapshot() Creates a snapshot of the current memstore.
void	updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent) Updates the wal with the lowest sequence id (oldest entry) that is still in memory

Methods inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

add, add, addToScanners, addToScanners, clearSnapshot, dump, getActive, getComparator, getConfiguration, getLowest, getNextRow, getSnapshot, getSnapshotSize, resetActive, timeOfOldestEdit, toString, upsert

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.regionserver.MemStore

startReplayingFromWAL, stopReplayingFromWAL

Field Detail

LOG

private static final org.slf4j.Logger LOG

DEEP_OVERHEAD

public static final long DEEP_OVERHEAD

FIXED_OVERHEAD

public static final long FIXED_OVERHEAD

Constructor Detail

DefaultMemStore

public DefaultMemStore()

Default constructor. Used for tests.

DefaultMemStore

public DefaultMemStore(org.apache.hadoop.conf.Configuration conf,
                       CellComparator c)

Constructor.

Parameters:

c - Comparator

DefaultMemStore

public DefaultMemStore(org.apache.hadoop.conf.Configuration conf,
                       CellComparator c,
                       RegionServicesForStores regionServices)

Constructor.

Parameters:

c - Comparator

Method Detail

snapshot

public MemStoreSnapshot snapshot()

Creates a snapshot of the current memstore. Snapshot must be cleared by call to AbstractMemStore.clearSnapshot(long)

Returns:

MemStoreSnapshot

getFlushableSize

public MemStoreSize getFlushableSize()

Description copied from interface: MemStore

Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set). If snapshot is empty, current Cell set will be flushed.

Returns:

On flush, how much memory we will clear.

keySize

protected long keySize()

Specified by:

keySize in class AbstractMemStore

Returns:

The total size of cells in this memstore. We will not consider cells in the snapshot

heapSize

protected long heapSize()

Specified by:

heapSize in class AbstractMemStore

Returns:

The total heap size of cells in this memstore. We will not consider cells in the snapshot

getScanners

public List<KeyValueScanner> getScanners(long readPt)
                                  throws IOException

Returns:

scanner over the memstore. This might include scanner over the snapshot when one is present.

Throws:

IOException

getSegments

protected List<Segment> getSegments()
                             throws IOException

Specified by:

getSegments in class AbstractMemStore

Returns:

an ordered list of segments from most recent to oldest in memstore

Throws:

IOException

getNextRow

Cell getNextRow(Cell cell)

Parameters:

cell - Find the row that comes after this one. If null, we return the first.

Returns:

Next row or null if none found.

updateLowestUnflushedSequenceIdInWAL

public void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent)

Description copied from class: AbstractMemStore

Updates the wal with the lowest sequence id (oldest entry) that is still in memory

Specified by:

updateLowestUnflushedSequenceIdInWAL in class AbstractMemStore

Parameters:

onlyIfMoreRecent - a flag that marks whether to update the sequence id no matter what or only if it is greater than the previous sequence id

size

public MemStoreSize size()

Returns:

Total memory occupied by this MemStore. This won't include any size occupied by the snapshot. We assume the snapshot will get cleared soon. This is not thread safe and the memstore may be changed while computing its size. It is the responsibility of the caller to make sure this doesn't happen.

checkActiveSize

protected void checkActiveSize()

Check whether anything need to be done based on the current active set size Nothing need to be done for the DefaultMemStore

Specified by:

checkActiveSize in class AbstractMemStore

preFlushSeqIDEstimation

public long preFlushSeqIDEstimation()

Description copied from interface: MemStore

This method is called before the flush is executed.

Returns:

an estimation (lower bound) of the unflushed sequence id in memstore after the flush is executed. if memstore will be cleared returns HConstants.NO_SEQNUM.

isSloppy

public boolean isSloppy()

main

public static void main(String[] args)

Code to help figure if our approximation of object heap sizes is close enough. See hbase-900. Fills memstores then waits so user can heap dump and bring up resultant hprof in something like jprofiler which allows you get 'deep size' on objects.

Parameters:

args - main args