org.apache.hadoop.hbase.regionserver

Class DefaultMemStore

java.lang.Object

org.apache.hadoop.hbase.regionserver.AbstractMemStore

org.apache.hadoop.hbase.regionserver.DefaultMemStore

All Implemented Interfaces:

MemStore

@InterfaceAudience.Private
public class DefaultMemStore
extends AbstractMemStore

The MemStore holds in-memory modifications to the Store. Modifications are Cells. When asked to flush, current memstore is moved to snapshot and is cleared. We continue to serve edits out of new memstore and backing snapshot until flusher reports in that the flush succeeded. At this point we let the snapshot go.

The MemStore functions should not be called in parallel. Callers should hold write and read locks. This is done in HStore.

TODO: Adjust size of the memstore when we remove items because they have been deleted. TODO: With new KVSLS, need to make sure we update HeapSize with difference in KV size.

Field Summary

Fields

Modifier and Type	Field and Description
static long	DEEP_OVERHEAD
static long	FIXED_OVERHEAD
private static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

regionServices, snapshot, snapshotId

Constructor Summary

Constructors

Constructor and Description
DefaultMemStore() Default constructor.
DefaultMemStore(org.apache.hadoop.conf.Configuration conf, CellComparator c) Constructor.
DefaultMemStore(org.apache.hadoop.conf.Configuration conf, CellComparator c, RegionServicesForStores regionServices) Constructor.

Method Summary

Modifier and Type	Method and Description
MemStoreSize	getFlushableSize() Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set).
(package private) Cell	getNextRow(Cell cell)
List<KeyValueScanner>	getScanners(long readPt)
protected List<Segment>	getSegments()
protected long	heapSize()
boolean	isSloppy()
protected long	keySize()
static void	main(String[] args) Code to help figure if our approximation of object heap sizes is close enough.
protected void	postUpdate(MutableSegment currentActive) Issue any post update synchronization and tests
long	preFlushSeqIDEstimation() This method is called before the flush is executed.
protected boolean	preUpdate(MutableSegment currentActive, Cell cell, MemStoreSizing memstoreSizing) Issue any synchronization and test needed before applying the update
MemStoreSize	size()
protected boolean	sizeAddedPreOperation()
MemStoreSnapshot	snapshot() Creates a snapshot of the current memstore.
void	updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent) Updates the wal with the lowest sequence id (oldest entry) that is still in memory

Methods inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

add, add, addToScanners, addToScanners, clearSnapshot, dump, getActive, getComparator, getConfiguration, getLowest, getNextRow, getSnapshot, getSnapshotSize, resetActive, timeOfOldestEdit, toString, upsert

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.regionserver.MemStore

startReplayingFromWAL, stopReplayingFromWAL

Field Detail

LOG

private static final org.slf4j.Logger LOG

DEEP_OVERHEAD

public static final long DEEP_OVERHEAD

FIXED_OVERHEAD

public static final long FIXED_OVERHEAD

Constructor Detail

DefaultMemStore

public DefaultMemStore()

Default constructor. Used for tests.

DefaultMemStore

public DefaultMemStore(org.apache.hadoop.conf.Configuration conf,
                       CellComparator c)

Constructor.

Parameters:

c - Comparator

DefaultMemStore

public DefaultMemStore(org.apache.hadoop.conf.Configuration conf,
                       CellComparator c,
                       RegionServicesForStores regionServices)

Constructor.

Parameters:

c - Comparator

Method Detail

snapshot

public MemStoreSnapshot snapshot()

Creates a snapshot of the current memstore. Snapshot must be cleared by call to AbstractMemStore.clearSnapshot(long)

Returns:

MemStoreSnapshot

getFlushableSize

public MemStoreSize getFlushableSize()

Description copied from interface: MemStore

Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set). If snapshot is empty, current Cell set will be flushed.

Returns:

On flush, how much memory we will clear.

keySize

protected long keySize()

Specified by:

keySize in class AbstractMemStore

Returns:

The total size of cells in this memstore. We will not consider cells in the snapshot

heapSize

protected long heapSize()

Specified by:

heapSize in class AbstractMemStore

Returns:

The total heap size of cells in this memstore. We will not consider cells in the snapshot

getScanners

public List<KeyValueScanner> getScanners(long readPt)
                                  throws IOException

Returns:

scanner over the memstore. This might include scanner over the snapshot when one is present.

Throws:

IOException

getSegments

protected List<Segment> getSegments()
                             throws IOException

Specified by:

getSegments in class AbstractMemStore

Returns:

an ordered list of segments from most recent to oldest in memstore

Throws:

IOException

getNextRow

Cell getNextRow(Cell cell)

Parameters:

cell - Find the row that comes after this one. If null, we return the first.

Returns:

Next row or null if none found.

updateLowestUnflushedSequenceIdInWAL

public void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent)

Description copied from class: AbstractMemStore

Updates the wal with the lowest sequence id (oldest entry) that is still in memory

Specified by:

updateLowestUnflushedSequenceIdInWAL in class AbstractMemStore

Parameters:

onlyIfMoreRecent - a flag that marks whether to update the sequence id no matter what or only if it is greater than the previous sequence id

preUpdate

protected boolean preUpdate(MutableSegment currentActive,
                            Cell cell,
                            MemStoreSizing memstoreSizing)

Description copied from class: AbstractMemStore

Issue any synchronization and test needed before applying the update

Specified by:

preUpdate in class AbstractMemStore

Parameters:

currentActive - the segment to be updated

cell - the cell to be added

memstoreSizing - object to accumulate region size changes

Returns:

true iff can proceed with applying the update

postUpdate

protected void postUpdate(MutableSegment currentActive)

Description copied from class: AbstractMemStore

Issue any post update synchronization and tests

Specified by:

postUpdate in class AbstractMemStore

Parameters:

currentActive - updated segment

sizeAddedPreOperation

protected boolean sizeAddedPreOperation()

Specified by:

sizeAddedPreOperation in class AbstractMemStore

size

public MemStoreSize size()

Returns:

Total memory occupied by this MemStore. This won't include any size occupied by the snapshot. We assume the snapshot will get cleared soon. This is not thread safe and the memstore may be changed while computing its size. It is the responsibility of the caller to make sure this doesn't happen.

preFlushSeqIDEstimation

public long preFlushSeqIDEstimation()

Description copied from interface: MemStore

This method is called before the flush is executed.

Returns:

an estimation (lower bound) of the unflushed sequence id in memstore after the flush is executed. if memstore will be cleared returns HConstants.NO_SEQNUM.

isSloppy

public boolean isSloppy()

main

public static void main(String[] args)

Code to help figure if our approximation of object heap sizes is close enough. See hbase-900. Fills memstores then waits so user can heap dump and bring up resultant hprof in something like jprofiler which allows you get 'deep size' on objects.

Parameters:

args - main args