org.apache.hadoop.hbase.regionserver

Class AbstractMemStore

java.lang.Object

org.apache.hadoop.hbase.regionserver.AbstractMemStore

All Implemented Interfaces:

MemStore

Direct Known Subclasses:

CompactingMemStore, DefaultMemStore

@InterfaceAudience.Private
public abstract class AbstractMemStore
extends Object
implements MemStore

An abstract class, which implements the behaviour shared by all concrete memstore instances.

Field Summary

Fields

Modifier and Type	Field and Description
private MutableSegment	active
private CellComparator	comparator
private org.apache.hadoop.conf.Configuration	conf
static long	DEEP_OVERHEAD
static long	FIXED_OVERHEAD
private static long	NO_SNAPSHOT_ID
protected RegionServicesForStores	regionServices
protected ImmutableSegment	snapshot
protected long	snapshotId
private long	timeOfOldestEdit

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractMemStore(org.apache.hadoop.conf.Configuration conf, CellComparator c, RegionServicesForStores regionServices)

Method Summary

Modifier and Type	Method and Description
void	add(Cell cell, MemStoreSizing memstoreSizing) Write an update n * @param memstoreSizing The delta in memstore size will be passed back via this.
void	add(Iterable<Cell> cells, MemStoreSizing memstoreSizing) Write the updates n * @param memstoreSizing The delta in memstore size will be passed back via this.
static void	addToScanners(List<? extends Segment> segments, long readPt, List<KeyValueScanner> scanners)
protected static void	addToScanners(Segment segment, long readPt, List<KeyValueScanner> scanners)
void	clearSnapshot(long id) This method is protected under HStore write lock. The passed snapshot was successfully persisted; it can be let go.
private static Cell	deepCopyIfNeeded(Cell cell)
protected void	doAdd(MutableSegment currentActive, Cell cell, MemStoreSizing memstoreSizing)
private void	doAddOrUpsert(Cell cell, long readpoint, MemStoreSizing memstoreSizing, boolean doAdd)
protected void	doClearSnapShot()
private void	doUpsert(MutableSegment currentActive, Cell cell, long readpoint, MemStoreSizing memstoreSizing)
protected void	dump(org.slf4j.Logger log)
(package private) MutableSegment	getActive()
protected CellComparator	getComparator()
protected org.apache.hadoop.conf.Configuration	getConfiguration()
protected Cell	getLowest(Cell a, Cell b)
protected Cell	getNextRow(Cell key, NavigableSet<Cell> set)
protected abstract List<Segment>	getSegments() Returns an ordered list of segments from most recent to oldest in memstore
(package private) ImmutableSegment	getSnapshot()
MemStoreSize	getSnapshotSize() Return the size of the snapshot(s) if any
protected abstract long	heapSize()
private void	internalAdd(MutableSegment currentActive, Cell toAdd, boolean mslabUsed, MemStoreSizing memstoreSizing)
protected abstract long	keySize() Returns The total size of cells in this memstore.
private Cell	maybeCloneWithAllocator(MutableSegment currentActive, Cell cell, boolean forceCloneOfBigCell) If the segment has a memory allocator the cell is being cloned to this space, and returned; Otherwise the given cell is returned When a cell's size is too big (bigger than maxAlloc), it is not allocated on MSLAB.
protected abstract void	postUpdate(MutableSegment currentActive) Issue any post update synchronization and tests
protected abstract boolean	preUpdate(MutableSegment currentActive, Cell cell, MemStoreSizing memstoreSizing) Issue any synchronization and test needed before applying the update
protected void	resetActive()
protected void	resetTimeOfOldestEdit()
private void	setOldestEditTimeToNow()
protected abstract boolean	sizeAddedPreOperation()
long	timeOfOldestEdit() Returns Oldest timestamp of all the Cells in the MemStore
String	toString()
abstract void	updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent) Updates the wal with the lowest sequence id (oldest entry) that is still in memory
private void	upsert(Cell cell, long readpoint, MemStoreSizing memstoreSizing)
void	upsert(Iterable<Cell> cells, long readpoint, MemStoreSizing memstoreSizing) Update or insert the specified cells.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.regionserver.MemStore

getFlushableSize, getScanners, isSloppy, preFlushSeqIDEstimation, size, snapshot, startReplayingFromWAL, stopReplayingFromWAL

Field Detail

NO_SNAPSHOT_ID

private static final long NO_SNAPSHOT_ID

See Also:

Constant Field Values

conf

private final org.apache.hadoop.conf.Configuration conf

comparator

private final CellComparator comparator

active

private volatile MutableSegment active

snapshot

protected volatile ImmutableSegment snapshot

snapshotId

protected volatile long snapshotId

timeOfOldestEdit

private volatile long timeOfOldestEdit

regionServices

protected RegionServicesForStores regionServices

FIXED_OVERHEAD

public static final long FIXED_OVERHEAD

DEEP_OVERHEAD

public static final long DEEP_OVERHEAD

Constructor Detail

AbstractMemStore

protected AbstractMemStore(org.apache.hadoop.conf.Configuration conf,
                           CellComparator c,
                           RegionServicesForStores regionServices)

Method Detail

addToScanners

public static void addToScanners(List<? extends Segment> segments,
                                 long readPt,
                                 List<KeyValueScanner> scanners)

addToScanners

protected static void addToScanners(Segment segment,
                                    long readPt,
                                    List<KeyValueScanner> scanners)

resetActive

protected void resetActive()

resetTimeOfOldestEdit

protected void resetTimeOfOldestEdit()

updateLowestUnflushedSequenceIdInWAL

public abstract void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent)

Updates the wal with the lowest sequence id (oldest entry) that is still in memory

Parameters:

onlyIfMoreRecent - a flag that marks whether to update the sequence id no matter what or only if it is greater than the previous sequence id

add

public void add(Iterable<Cell> cells,
                MemStoreSizing memstoreSizing)

Description copied from interface: MemStore

Write the updates n * @param memstoreSizing The delta in memstore size will be passed back via this. This will include both data size and heap overhead delta.

Specified by:

add in interface MemStore

add

public void add(Cell cell,
                MemStoreSizing memstoreSizing)

Description copied from interface: MemStore

Write an update n * @param memstoreSizing The delta in memstore size will be passed back via this. This will include both data size and heap overhead delta.

Specified by:

add in interface MemStore

upsert

private void upsert(Cell cell,
                    long readpoint,
                    MemStoreSizing memstoreSizing)

doAddOrUpsert

private void doAddOrUpsert(Cell cell,
                           long readpoint,
                           MemStoreSizing memstoreSizing,
                           boolean doAdd)

doAdd

protected void doAdd(MutableSegment currentActive,
                     Cell cell,
                     MemStoreSizing memstoreSizing)

doUpsert

private void doUpsert(MutableSegment currentActive,
                      Cell cell,
                      long readpoint,
                      MemStoreSizing memstoreSizing)

preUpdate

protected abstract boolean preUpdate(MutableSegment currentActive,
                                     Cell cell,
                                     MemStoreSizing memstoreSizing)

Issue any synchronization and test needed before applying the update

Parameters:

currentActive - the segment to be updated

cell - the cell to be added

memstoreSizing - object to accumulate region size changes

Returns:

true iff can proceed with applying the update

postUpdate

protected abstract void postUpdate(MutableSegment currentActive)

Issue any post update synchronization and tests

Parameters:

currentActive - updated segment

deepCopyIfNeeded

private static Cell deepCopyIfNeeded(Cell cell)

upsert

public void upsert(Iterable<Cell> cells,
                   long readpoint,
                   MemStoreSizing memstoreSizing)

Description copied from interface: MemStore

Update or insert the specified cells.

For each Cell, insert into MemStore. This will atomically upsert the value for that row/family/qualifier. If a Cell did already exist, it will then be removed.

Currently the memstoreTS is kept at 0 so as each insert happens, it will be immediately visible. May want to change this so it is atomic across all KeyValues.

This is called under row lock, so Get operations will still see updates atomically. Scans will only see each KeyValue update as atomic. n * @param readpoint readpoint below which we can safely remove duplicate Cells.

Specified by:

upsert in interface MemStore

memstoreSizing - The delta in memstore size will be passed back via this. This will include both data size and heap overhead delta.

timeOfOldestEdit

public long timeOfOldestEdit()

Returns Oldest timestamp of all the Cells in the MemStore

Specified by:

timeOfOldestEdit in interface MemStore

clearSnapshot

public void clearSnapshot(long id)
                   throws UnexpectedStateException

This method is protected under HStore write lock.
The passed snapshot was successfully persisted; it can be let go.

Specified by:

clearSnapshot in interface MemStore

Parameters:

id - Id of the snapshot to clean out.

Throws:

UnexpectedStateException

See Also:

MemStore.snapshot()

doClearSnapShot

protected void doClearSnapShot()

getSnapshotSize

public MemStoreSize getSnapshotSize()

Description copied from interface: MemStore

Return the size of the snapshot(s) if any

Specified by:

getSnapshotSize in interface MemStore

Returns:

size of the memstore snapshot

toString

public String toString()

Overrides:

toString in class Object

getConfiguration

protected org.apache.hadoop.conf.Configuration getConfiguration()

dump

protected void dump(org.slf4j.Logger log)

getLowest

protected Cell getLowest(Cell a,
                         Cell b)

getNextRow

protected Cell getNextRow(Cell key,
                          NavigableSet<Cell> set)

maybeCloneWithAllocator

private Cell maybeCloneWithAllocator(MutableSegment currentActive,
                                     Cell cell,
                                     boolean forceCloneOfBigCell)

If the segment has a memory allocator the cell is being cloned to this space, and returned; Otherwise the given cell is returned When a cell's size is too big (bigger than maxAlloc), it is not allocated on MSLAB. Since the process of flattening to CellChunkMap assumes that all cells are allocated on MSLAB, during this process, the input parameter forceCloneOfBigCell is set to 'true' and the cell is copied into MSLAB.

Parameters:

cell - the cell to clone

forceCloneOfBigCell - true only during the process of flattening to CellChunkMap.

Returns:

either the given cell or its clone

internalAdd

private void internalAdd(MutableSegment currentActive,
                         Cell toAdd,
                         boolean mslabUsed,
                         MemStoreSizing memstoreSizing)

sizeAddedPreOperation

protected abstract boolean sizeAddedPreOperation()

setOldestEditTimeToNow

private void setOldestEditTimeToNow()

keySize

protected abstract long keySize()

Returns The total size of cells in this memstore. We will not consider cells in the snapshot

heapSize

protected abstract long heapSize()

Returns:

The total heap size of cells in this memstore. We will not consider cells in the snapshot

getComparator

protected CellComparator getComparator()

getActive

MutableSegment getActive()

getSnapshot

ImmutableSegment getSnapshot()

getSegments

protected abstract List<Segment> getSegments()
                                      throws IOException

Returns an ordered list of segments from most recent to oldest in memstore

Throws:

IOException