Package org.apache.hadoop.hbase.regionserver

Class AbstractMemStore

java.lang.Object
org.apache.hadoop.hbase.regionserver.AbstractMemStore
All Implemented Interfaces:
Closeable, AutoCloseable, MemStore
Direct Known Subclasses:
CompactingMemStore, DefaultMemStore
@Private public abstract class AbstractMemStore extends Object implements MemStore
An abstract class, which implements the behaviour shared by all concrete memstore instances.

  • Field Details

  • Constructor Details

  • Method Details

    • addToScanners

      public static void addToScanners(List<? extends Segment> segments, long readPt, List<KeyValueScanner> scanners)

    • addToScanners

      protected static void addToScanners(Segment segment, long readPt, List<KeyValueScanner> scanners)

    • resetActive

      protected void resetActive()

    • resetTimeOfOldestEdit

      protected void resetTimeOfOldestEdit()

    • updateLowestUnflushedSequenceIdInWAL

      public abstract void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfMoreRecent)
      Updates the wal with the lowest sequence id (oldest entry) that is still in memory
      Parameters:
      onlyIfMoreRecent - a flag that marks whether to update the sequence id no matter what or only if it is greater than the previous sequence id

    • add

      public void add(Iterable<ExtendedCell> cells, MemStoreSizing memstoreSizing)
      Description copied from interface: MemStore
      Write the updates
      Specified by:
      add in interface MemStore
      memstoreSizing - The delta in memstore size will be passed back via this. This will include both data size and heap overhead delta.

    • add

      public void add(ExtendedCell cell, MemStoreSizing memstoreSizing)
      Description copied from interface: MemStore
      Write an update
      Specified by:
      add in interface MemStore
      memstoreSizing - The delta in memstore size will be passed back via this. This will include both data size and heap overhead delta.

    • upsert

      private void upsert(ExtendedCell cell, long readpoint, MemStoreSizing memstoreSizing)

    • doAddOrUpsert

      private void doAddOrUpsert(ExtendedCell cell, long readpoint, MemStoreSizing memstoreSizing, boolean doAdd)

    • doAdd

      protected void doAdd(MutableSegment currentActive, ExtendedCell cell, MemStoreSizing memstoreSizing)

    • doUpsert

      private void doUpsert(MutableSegment currentActive, ExtendedCell cell, long readpoint, MemStoreSizing memstoreSizing)

    • preUpdate

      protected abstract boolean preUpdate(MutableSegment currentActive, ExtendedCell cell, MemStoreSizing memstoreSizing)
      Issue any synchronization and test needed before applying the update
      Parameters:
      currentActive - the segment to be updated
      cell - the cell to be added
      memstoreSizing - object to accumulate region size changes
      Returns:
      true iff can proceed with applying the update

    • postUpdate

      protected abstract void postUpdate(MutableSegment currentActive)
      Issue any post update synchronization and tests
      Parameters:
      currentActive - updated segment

    • deepCopyIfNeeded

      private static ExtendedCell deepCopyIfNeeded(ExtendedCell cell)

    • upsert

      public void upsert(Iterable<ExtendedCell> cells, long readpoint, MemStoreSizing memstoreSizing)
      Description copied from interface: MemStore
      Update or insert the specified cells.

      For each Cell, insert into MemStore. This will atomically upsert the value for that row/family/qualifier. If a Cell did already exist, it will then be removed.

      Currently the memstoreTS is kept at 0 so as each insert happens, it will be immediately visible. May want to change this so it is atomic across all KeyValues.

      This is called under row lock, so Get operations will still see updates atomically. Scans will only see each KeyValue update as atomic.

      Specified by:
      upsert in interface MemStore
      readpoint - readpoint below which we can safely remove duplicate Cells.
      memstoreSizing - The delta in memstore size will be passed back via this. This will include both data size and heap overhead delta.

    • timeOfOldestEdit

      public long timeOfOldestEdit()
      Returns Oldest timestamp of all the Cells in the MemStore
      Specified by:
      timeOfOldestEdit in interface MemStore

    • clearSnapshot

      public void clearSnapshot(long id) throws UnexpectedStateException
      This method is protected under HStore#lock write lock,
      and this method is used by HStore#updateStorefiles after flushing is completed.
      The passed snapshot was successfully persisted; it can be let go.
      Specified by:
      clearSnapshot in interface MemStore
      Parameters:
      id - Id of the snapshot to clean out.
      Throws:
      UnexpectedStateException
      See Also:

    • doClearSnapShot

      protected void doClearSnapShot()

    • getSnapshotSize

      public MemStoreSize getSnapshotSize()
      Description copied from interface: MemStore
      Return the size of the snapshot(s) if any
      Specified by:
      getSnapshotSize in interface MemStore
      Returns:
      size of the memstore snapshot

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getConfiguration

      protected org.apache.hadoop.conf.Configuration getConfiguration()

    • dump

      protected void dump(org.slf4j.Logger log)

    • getLowest

      protected ExtendedCell getLowest(ExtendedCell a, ExtendedCell b)
      Returns Return lowest of a or b or null if both a and b are null

    • getNextRow

      protected ExtendedCell getNextRow(ExtendedCell key, NavigableSet<ExtendedCell> set)
      Parameters:
      key - Find row that follows this one. If null, return first.
      set - Set to look in for a row beyond row.
      Returns:
      Next row or null if none found. If one found, will be a new KeyValue -- can be destroyed by subsequent calls to this method.

    • maybeCloneWithAllocator

      private ExtendedCell maybeCloneWithAllocator(MutableSegment currentActive, ExtendedCell cell, boolean forceCloneOfBigCell)
      If the segment has a memory allocator the cell is being cloned to this space, and returned; Otherwise the given cell is returned When a cell's size is too big (bigger than maxAlloc), it is not allocated on MSLAB. Since the process of flattening to CellChunkMap assumes that all cells are allocated on MSLAB, during this process, the input parameter forceCloneOfBigCell is set to 'true' and the cell is copied into MSLAB.
      Parameters:
      cell - the cell to clone
      forceCloneOfBigCell - true only during the process of flattening to CellChunkMap.
      Returns:
      either the given cell or its clone

    • internalAdd

      private void internalAdd(MutableSegment currentActive, ExtendedCell toAdd, boolean mslabUsed, MemStoreSizing memstoreSizing)
      Internal version of add() that doesn't clone Cells with the allocator, and doesn't take the lock. Callers should ensure they already have the read lock taken
      Parameters:
      toAdd - the cell to add
      mslabUsed - whether using MSLAB
      memstoreSizing - object to accumulate changed size

    • sizeAddedPreOperation

      protected abstract boolean sizeAddedPreOperation()

    • setOldestEditTimeToNow

      private void setOldestEditTimeToNow()

    • keySize

      protected abstract long keySize()
      Returns The total size of cells in this memstore. We will not consider cells in the snapshot

    • heapSize

      protected abstract long heapSize()
      Returns:
      The total heap size of cells in this memstore. We will not consider cells in the snapshot

    • getComparator

      protected CellComparator getComparator()

    • getActive

      MutableSegment getActive()

    • getSnapshot

      ImmutableSegment getSnapshot()

    • close

      public void close()
      Description copied from interface: MemStore
      Close the memstore.

      Usually this should only be called when there is nothing in the memstore, unless we are going to abort ourselves.

      For normal cases, this method is only used to fix the reference counting, see HBASE-27941.

      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Specified by:
      close in interface MemStore

    • getSegments

      protected abstract List<Segment> getSegments() throws IOException
      Returns an ordered list of segments from most recent to oldest in memstore
      Throws:
      IOException