Package org.apache.hadoop.hbase.regionserver

Class DefaultMemStore

java.lang.Object
org.apache.hadoop.hbase.regionserver.AbstractMemStore
org.apache.hadoop.hbase.regionserver.DefaultMemStore
All Implemented Interfaces:
Closeable, AutoCloseable, MemStore
@Private public class DefaultMemStore extends AbstractMemStore
The MemStore holds in-memory modifications to the Store. Modifications are Cells. When asked to flush, current memstore is moved to snapshot and is cleared. We continue to serve edits out of new memstore and backing snapshot until flusher reports in that the flush succeeded. At this point we let the snapshot go.

The MemStore functions should not be called in parallel. Callers should hold write and read locks. This is done in HStore.

TODO: Adjust size of the memstore when we remove items because they have been deleted. TODO: With new KVSLS, need to make sure we update HeapSize with difference in KV size.

  • Field Details

    • LOG

      private static final org.slf4j.Logger LOG

    • DEEP_OVERHEAD

      public static final long DEEP_OVERHEAD

    • FIXED_OVERHEAD

      public static final long FIXED_OVERHEAD

  • Constructor Details

  • Method Details

    • snapshot

      public MemStoreSnapshot snapshot()
      Creates a snapshot of the current memstore. Snapshot must be cleared by call to AbstractMemStore.clearSnapshot(long)
      Returns:
      MemStoreSnapshot

    • getFlushableSize

      public MemStoreSize getFlushableSize()
      Description copied from interface: MemStore
      Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set). If snapshot is empty, current Cell set will be flushed.
      Returns:
      On flush, how much memory we will clear.

    • keySize

      protected long keySize()
      Description copied from class: AbstractMemStore
      Returns The total size of cells in this memstore. We will not consider cells in the snapshot
      Specified by:
      keySize in class AbstractMemStore

    • heapSize

      protected long heapSize()
      Specified by:
      heapSize in class AbstractMemStore
      Returns:
      The total heap size of cells in this memstore. We will not consider cells in the snapshot

    • getScanners

      public List<KeyValueScanner> getScanners(long readPt) throws IOException
      Returns:
      scanner over the memstore. This might include scanner over the snapshot when one is present.
      Throws:
      IOException

    • getSnapshotSegments

      protected List<Segment> getSnapshotSegments()

    • getSegments

      protected List<Segment> getSegments() throws IOException
      Description copied from class: AbstractMemStore
      Returns an ordered list of segments from most recent to oldest in memstore
      Specified by:
      getSegments in class AbstractMemStore
      Throws:
      IOException

    • getNextRow

      ExtendedCell getNextRow(ExtendedCell cell)
      Parameters:
      cell - Find the row that comes after this one. If null, we return the first.
      Returns:
      Next row or null if none found.

    • updateLowestUnflushedSequenceIdInWAL

      public void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent)
      Description copied from class: AbstractMemStore
      Updates the wal with the lowest sequence id (oldest entry) that is still in memory
      Specified by:
      updateLowestUnflushedSequenceIdInWAL in class AbstractMemStore
      Parameters:
      onlyIfMoreRecent - a flag that marks whether to update the sequence id no matter what or only if it is greater than the previous sequence id

    • preUpdate

      protected boolean preUpdate(MutableSegment currentActive, ExtendedCell cell, MemStoreSizing memstoreSizing)
      Description copied from class: AbstractMemStore
      Issue any synchronization and test needed before applying the update
      Specified by:
      preUpdate in class AbstractMemStore
      Parameters:
      currentActive - the segment to be updated
      cell - the cell to be added
      memstoreSizing - object to accumulate region size changes
      Returns:
      true iff can proceed with applying the update

    • postUpdate

      protected void postUpdate(MutableSegment currentActive)
      Description copied from class: AbstractMemStore
      Issue any post update synchronization and tests
      Specified by:
      postUpdate in class AbstractMemStore
      Parameters:
      currentActive - updated segment

    • sizeAddedPreOperation

      protected boolean sizeAddedPreOperation()
      Specified by:
      sizeAddedPreOperation in class AbstractMemStore

    • size

      public MemStoreSize size()
      Returns:
      Total memory occupied by this MemStore. This won't include any size occupied by the snapshot. We assume the snapshot will get cleared soon. This is not thread safe and the memstore may be changed while computing its size. It is the responsibility of the caller to make sure this doesn't happen.

    • preFlushSeqIDEstimation

      public long preFlushSeqIDEstimation()
      Description copied from interface: MemStore
      This method is called before the flush is executed.
      Returns:
      an estimation (lower bound) of the unflushed sequence id in memstore after the flush is executed. if memstore will be cleared returns HConstants.NO_SEQNUM.

    • isSloppy

      public boolean isSloppy()

    • main

      public static void main(String[] args)
      Code to help figure if our approximation of object heap sizes is close enough. See hbase-900. Fills memstores then waits so user can heap dump and bring up resultant hprof in something like jprofiler which allows you get 'deep size' on objects.
      Parameters:
      args - main args