org.apache.hadoop.hbase.regionserver

Class DefaultMemStore

java.lang.Object

org.apache.hadoop.hbase.regionserver.DefaultMemStore

All Implemented Interfaces:

HeapSize, MemStore

@InterfaceAudience.Private
public class DefaultMemStore
extends Object
implements MemStore

The MemStore holds in-memory modifications to the Store. Modifications are Cells. When asked to flush, current memstore is moved to snapshot and is cleared. We continue to serve edits out of new memstore and backing snapshot until flusher reports in that the flush succeeded. At this point we let the snapshot go.

The MemStore functions should not be called in parallel. Callers should hold write and read locks. This is done in HStore.

TODO: Adjust size of the memstore when we remove items because they have been deleted. TODO: With new KVSLS, need to make sure we update HeapSize with difference in KV size.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	DefaultMemStore.MemStoreScanner

Field Summary

Fields

Modifier and Type	Field and Description
static long	DEEP_OVERHEAD
static long	FIXED_OVERHEAD

Constructor Summary

Constructors

Constructor and Description
DefaultMemStore() Default constructor.
DefaultMemStore(org.apache.hadoop.conf.Configuration conf, KeyValue.KVComparator c) Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
Pair<Long,Cell>	add(Cell cell) Write an update
void	clearSnapshot(long id) The passed snapshot was successfully persisted; it can be let go.
long	delete(Cell deleteCell) Write a delete
long	getFlushableSize() On flush, how much memory we will clear.
void	getRowKeyAtOrBefore(org.apache.hadoop.hbase.regionserver.GetClosestRowBeforeTracker state) Find the key that matches row exactly, or the one that immediately precedes it.
List<KeyValueScanner>	getScanners(long readPt)
long	getSnapshotSize() Return the size of the snapshot(s) if any
long	heapSize() Get the entire heap usage for this MemStore not including keys in the snapshot.
static void	main(String[] args) Code to help figure if our approximation of object heap sizes is close enough.
void	rollback(Cell cell) Remove n key from the memstore.
boolean	shouldSeek(Scan scan, long oldestUnexpiredTS) Check if this memstore may contain the required keys
long	size()
MemStoreSnapshot	snapshot() Creates a snapshot of the current memstore.
long	timeOfOldestEdit()
long	updateColumnValue(byte[] row, byte[] family, byte[] qualifier, long newValue, long now) Only used by tests.
long	upsert(Iterable<Cell> cells, long readpoint) Update or insert the specified KeyValues.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

FIXED_OVERHEAD

public static final long FIXED_OVERHEAD

DEEP_OVERHEAD

public static final long DEEP_OVERHEAD

Constructor Detail

DefaultMemStore

public DefaultMemStore()

Default constructor. Used for tests.

DefaultMemStore

public DefaultMemStore(org.apache.hadoop.conf.Configuration conf,
               KeyValue.KVComparator c)

Constructor.

Parameters:

c - Comparator

Method Detail

snapshot

public MemStoreSnapshot snapshot()

Creates a snapshot of the current memstore. Snapshot must be cleared by call to clearSnapshot(long)

Specified by:

snapshot in interface MemStore

Returns:

MemStoreSnapshot

clearSnapshot

public void clearSnapshot(long id)
                   throws UnexpectedStateException

The passed snapshot was successfully persisted; it can be let go.

Specified by:

clearSnapshot in interface MemStore

Parameters:

id - Id of the snapshot to clean out.

Throws:

UnexpectedStateException

See Also:

snapshot()

getFlushableSize

public long getFlushableSize()

Description copied from interface: MemStore

On flush, how much memory we will clear. Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set). If snapshot is empty, current Cell set will be flushed.

Specified by:

getFlushableSize in interface MemStore

Returns:

size of data that is going to be flushed

getSnapshotSize

public long getSnapshotSize()

Description copied from interface: MemStore

Return the size of the snapshot(s) if any

Specified by:

getSnapshotSize in interface MemStore

Returns:

size of the memstore snapshot

add

public Pair<Long,Cell> add(Cell cell)

Write an update

Specified by:

add in interface MemStore

Parameters:

cell -

Returns:

approximate size of the passed KV & newly added KV which maybe different than the passed-in KV

timeOfOldestEdit

public long timeOfOldestEdit()

Specified by:

timeOfOldestEdit in interface MemStore

Returns:

Oldest timestamp of all the Cells in the MemStore

rollback

public void rollback(Cell cell)

Remove n key from the memstore. Only cells that have the same key and the same memstoreTS are removed. It is ok to not update timeRangeTracker in this call. It is possible that we can optimize this method by using tailMap/iterator, but since this method is called rarely (only for error recovery), we can leave those optimization for the future.

Specified by:

rollback in interface MemStore

Parameters:

cell -

delete

public long delete(Cell deleteCell)

Write a delete

Specified by:

delete in interface MemStore

Parameters:

deleteCell -

Returns:

approximate size of the passed key and value.

getRowKeyAtOrBefore

public void getRowKeyAtOrBefore(org.apache.hadoop.hbase.regionserver.GetClosestRowBeforeTracker state)

Description copied from interface: MemStore

Find the key that matches row exactly, or the one that immediately precedes it. The target row key is set in state.

Specified by:

getRowKeyAtOrBefore in interface MemStore

Parameters:

state - column/delete tracking state

updateColumnValue

public long updateColumnValue(byte[] row,
                     byte[] family,
                     byte[] qualifier,
                     long newValue,
                     long now)

Only used by tests. TODO: Remove Given the specs of a column, update it, first by inserting a new record, then removing the old one. Since there is only 1 KeyValue involved, the memstoreTS will be set to 0, thus ensuring that they instantly appear to anyone. The underlying store will ensure that the insert/delete each are atomic. A scanner/reader will either get the new value, or the old value and all readers will eventually only see the new value after the old was removed.

Specified by:

updateColumnValue in interface MemStore

Parameters:

row -

family -

qualifier -

newValue -

now -

Returns:

Timestamp

upsert

public long upsert(Iterable<Cell> cells,
          long readpoint)

Update or insert the specified KeyValues.

For each KeyValue, insert into MemStore. This will atomically upsert the value for that row/family/qualifier. If a KeyValue did already exist, it will then be removed.

Currently the memstoreTS is kept at 0 so as each insert happens, it will be immediately visible. May want to change this so it is atomic across all KeyValues.

This is called under row lock, so Get operations will still see updates atomically. Scans will only see each KeyValue update as atomic.

Specified by:

upsert in interface MemStore

Parameters:

cells -

readpoint - readpoint below which we can safely remove duplicate KVs

Returns:

change in memstore size

getScanners

public List<KeyValueScanner> getScanners(long readPt)

Specified by:

getScanners in interface MemStore

Returns:

scanner on memstore and snapshot in this order.

shouldSeek

public boolean shouldSeek(Scan scan,
                 long oldestUnexpiredTS)

Check if this memstore may contain the required keys

Parameters:

scan -

Returns:

False if the key definitely does not exist in this Memstore

heapSize

public long heapSize()

Get the entire heap usage for this MemStore not including keys in the snapshot.

Specified by:

heapSize in interface HeapSize

Returns:

Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.

size

public long size()

Specified by:

size in interface MemStore

Returns:

Total memory occupied by this MemStore.

main

public static void main(String[] args)

Code to help figure if our approximation of object heap sizes is close enough. See hbase-900. Fills memstores then waits so user can heap dump and bring up resultant hprof in something like jprofiler which allows you get 'deep size' on objects.

Parameters:

args - main args