MemStoreLABImpl (Apache HBase 2.5.0 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.MemStoreLABImpl

All Implemented Interfaces:

MemStoreLAB
```
@InterfaceAudience.Private
public class MemStoreLABImpl
extends Object
implements MemStoreLAB
```
A memstore-local allocation buffer.
The MemStoreLAB is basically a bump-the-pointer allocator that allocates big (2MB) byte[] chunks from and then doles it out to threads that request slices into the array.
The purpose of this class is to combat heap fragmentation in the regionserver. By ensuring that all Cells in a given memstore refer only to large chunks of contiguous memory, we ensure that large blocks get freed up when the memstore is flushed.
Without the MSLAB, the byte array allocated during insertion end up interleaved throughout the heap, and the old generation gets progressively more fragmented until a stop-the-world compacting collection occurs.
TODO: we should probably benchmark whether word-aligning the allocations would provide a performance improvement - probably would speed up the Bytes.toLong/Bytes.toInt calls in KeyValue, but some of those are cached anyway. The chunks created by this MemStoreLAB can get pooled at ChunkCreator. When the Chunk comes from pool, it can be either an on heap or an off heap backed chunk. The chunks, which this MemStoreLAB creates on its own (when no chunk available from pool), those will be always on heap backed.
NOTE:if user requested to work with MSLABs (whether on- or off-heap), in CompactingMemStore ctor, the CompactingMemStore.indexType could only be CompactingMemStore.IndexType.CHUNK_MAP,that is to say the immutable segments using MSLABs are going to use CellChunkMap as their index.

Field Summary

Fields
Modifier and Type	Field and Description
`private ChunkCreator`	`chunkCreator`
`(package private) Set<Integer>`	`chunks`
`private AtomicBoolean`	`closed`
`private AtomicReference<Chunk>`	`currChunk`
`private int`	`dataChunkSize`
`private ReentrantLock`	`lock`
`(package private) static org.slf4j.Logger`	`LOG`
`private int`	`maxAlloc`
`private AtomicBoolean`	`reclaimed`
`private RefCnt`	`refCnt` Its initial value is 1, so it is one bigger than the current count of open scanners which reading data from this MemStoreLAB.

Fields inherited from interface org.apache.hadoop.hbase.regionserver.MemStoreLAB
CHUNK_POOL_INITIALSIZE_KEY, CHUNK_POOL_MAXSIZE_KEY, CHUNK_SIZE_DEFAULT, CHUNK_SIZE_KEY, INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT, INDEX_CHUNK_SIZE_PERCENTAGE_KEY, MAX_ALLOC_DEFAULT, MAX_ALLOC_KEY, MSLAB_CLASS_NAME, POOL_INITIAL_SIZE_DEFAULT, POOL_MAX_SIZE_DEFAULT, USEMSLAB_DEFAULT, USEMSLAB_KEY

Constructor Summary

Constructors
Constructor and Description

MemStoreLABImpl()

MemStoreLABImpl(org.apache.hadoop.conf.Configuration conf)

Constructors
Constructor and Description
`MemStoreLABImpl()`
`MemStoreLABImpl(org.apache.hadoop.conf.Configuration conf)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`close()` Close this instance since it won't be used any more, try to put the chunks back to pool
`private Cell`	`copyBBECellInto(ByteBufferExtendedCell cell, int maxAlloc)` Mostly a duplicate of `copyCellInto(Cell, int)`} done for perf sake.
`private static Cell`	`copyBBECToChunkCell(ByteBufferExtendedCell cell, ByteBuffer buf, int offset, int len)` Clone the passed cell by copying its data into the passed buf and create a cell with a chunkid out of it
`Cell`	`copyCellInto(Cell cell)` Allocates slice in this LAB and copy the passed Cell into this area.
`private Cell`	`copyCellInto(Cell cell, int maxAlloc)`
`private static Cell`	`copyToChunkCell(Cell cell, ByteBuffer buf, int offset, int len)` Clone the passed cell by copying its data into the passed buf and create a cell with a chunkid out of it
`private static Cell`	`createChunkCell(ByteBuffer buf, int offset, int len, int tagsLen, long sequenceId)`
`void`	`decScannerCount()` Called when closing a scanner on the data of this MemStoreLAB
`Cell`	`forceCopyOfBigCellInto(Cell cell)` When a cell's size is too big (bigger than maxAlloc), copyCellInto does not allocate it on MSLAB.
`(package private) Chunk`	`getCurrentChunk()`
`Chunk`	`getNewExternalChunk(ChunkCreator.ChunkType chunkType)`
`Chunk`	`getNewExternalChunk(int size)`
`(package private) Integer`	`getNumOfChunksReturnedToPool(Set<Integer> chunksId)`
`private Chunk`	`getOrMakeChunk()` Get the current chunk, or, if there is no current chunk, allocate a new one from the JVM.
`(package private) BlockingQueue<Chunk>`	`getPooledChunks()`
`(package private) int`	`getRefCntValue()`
`void`	`incScannerCount()` Called when opening a scanner on the data of this MemStoreLAB
`(package private) boolean`	`isClosed()`
`boolean`	`isOffHeap()`
`boolean`	`isOnHeap()`
`(package private) boolean`	`isReclaimed()`
`private void`	`recycleChunks()`
`private void`	`tryRetireChunk(Chunk c)` Try to retire the current chunk if it is still `c`.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.regionserver.MemStoreLAB
isEnabled, newInstance

- Field Detail
  - LOG
```
static final org.slf4j.Logger LOG
```
  - currChunk
```
private AtomicReference<Chunk> currChunk
```
  - lock
```
private ReentrantLock lock
```
  - chunks
```
Set<Integer> chunks
```
  - dataChunkSize
```
private final int dataChunkSize
```
  - maxAlloc
```
private final int maxAlloc
```
  - chunkCreator
```
private final ChunkCreator chunkCreator
```
  - closed
```
private final AtomicBoolean closed
```
  - reclaimed
```
private final AtomicBoolean reclaimed
```
  - refCnt
```
private final RefCnt refCnt
```
    Its initial value is 1, so it is one bigger than the current count of open scanners which reading data from this MemStoreLAB.
- Constructor Detail
  - MemStoreLABImpl
```
public MemStoreLABImpl()
```
  - MemStoreLABImpl
```
public MemStoreLABImpl(org.apache.hadoop.conf.Configuration conf)
```
- Method Detail
  - copyCellInto
```
public Cell copyCellInto(Cell cell)
```
    Description copied from interface: MemStoreLAB
    
    Allocates slice in this LAB and copy the passed Cell into this area. Returns new Cell instance over the copied the data. When this MemStoreLAB can not copy this Cell, it returns null.
    
    Specified by:
    
    copyCellInto in interface MemStoreLAB
  - forceCopyOfBigCellInto
```
public Cell forceCopyOfBigCellInto(Cell cell)
```
    When a cell's size is too big (bigger than maxAlloc), copyCellInto does not allocate it on MSLAB. Since the process of flattening to CellChunkMap assumes that all cells are allocated on MSLAB, during this process, the big cells are copied into MSLAB using this method.
    
    Specified by:
    
    forceCopyOfBigCellInto in interface MemStoreLAB
  - copyBBECellInto
```
private Cell copyBBECellInto(ByteBufferExtendedCell cell,
                             int maxAlloc)
```
    Mostly a duplicate of copyCellInto(Cell, int)} done for perf sake. It presumes ByteBufferExtendedCell instead of Cell so we deal with a specific type rather than the super generic Cell. Removes instanceof checks. Shrinkage is enough to make this inline where before it was too big. Uses less CPU. See HBASE-20875 for evidence.
    
    See Also:
    
    copyCellInto(Cell, int)
  - copyCellInto
```
private Cell copyCellInto(Cell cell,
                          int maxAlloc)
```
    See Also:
    
    copyBBECellInto(ByteBufferExtendedCell, int)
  - copyToChunkCell
```
private static Cell copyToChunkCell(Cell cell,
                                    ByteBuffer buf,
                                    int offset,
                                    int len)
```
    Clone the passed cell by copying its data into the passed buf and create a cell with a chunkid out of it
    
    See Also:
    
    copyBBECToChunkCell(ByteBufferExtendedCell, ByteBuffer, int, int)
  - copyBBECToChunkCell
```
private static Cell copyBBECToChunkCell(ByteBufferExtendedCell cell,
                                        ByteBuffer buf,
                                        int offset,
                                        int len)
```
    Clone the passed cell by copying its data into the passed buf and create a cell with a chunkid out of it
    
    See Also:
    
    copyToChunkCell(Cell, ByteBuffer, int, int)
  - createChunkCell
```
private static Cell createChunkCell(ByteBuffer buf,
                                    int offset,
                                    int len,
                                    int tagsLen,
                                    long sequenceId)
```
  - close
```
public void close()
```
    Close this instance since it won't be used any more, try to put the chunks back to pool
    
    Specified by:
    
    close in interface MemStoreLAB
  - getRefCntValue
```
int getRefCntValue()
```
  - incScannerCount
```
public void incScannerCount()
```
    Called when opening a scanner on the data of this MemStoreLAB
    
    Specified by:
    
    incScannerCount in interface MemStoreLAB
  - decScannerCount
```
public void decScannerCount()
```
    Called when closing a scanner on the data of this MemStoreLAB
    
    Specified by:
    
    decScannerCount in interface MemStoreLAB
  - recycleChunks
```
private void recycleChunks()
```
  - tryRetireChunk
```
private void tryRetireChunk(Chunk c)
```
    Try to retire the current chunk if it is still c. Postcondition is that curChunk.get() != c
    
    Parameters:
    
    c - the chunk to retire
  - getOrMakeChunk
```
private Chunk getOrMakeChunk()
```
    Get the current chunk, or, if there is no current chunk, allocate a new one from the JVM.
  - getNewExternalChunk
```
public Chunk getNewExternalChunk(ChunkCreator.ChunkType chunkType)
```
    Specified by:
    
    getNewExternalChunk in interface MemStoreLAB
  - getNewExternalChunk
```
public Chunk getNewExternalChunk(int size)
```
    Specified by:
    
    getNewExternalChunk in interface MemStoreLAB
  - isOnHeap
```
public boolean isOnHeap()
```
    Specified by:
    
    isOnHeap in interface MemStoreLAB
  - isOffHeap
```
public boolean isOffHeap()
```
    Specified by:
    
    isOffHeap in interface MemStoreLAB
  - getCurrentChunk
```
Chunk getCurrentChunk()
```
  - getPooledChunks
```
BlockingQueue<Chunk> getPooledChunks()
```
  - getNumOfChunksReturnedToPool
```
Integer getNumOfChunksReturnedToPool(Set<Integer> chunksId)
```
  - isReclaimed
```
boolean isReclaimed()
```
  - isClosed
```
boolean isClosed()
```

Class MemStoreLABImpl

Field Summary

Fields inherited from interface org.apache.hadoop.hbase.regionserver.MemStoreLAB

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.hadoop.hbase.regionserver.MemStoreLAB

Field Detail

LOG

currChunk

lock

chunks

dataChunkSize

maxAlloc

chunkCreator

closed

reclaimed

refCnt

Constructor Detail

MemStoreLABImpl

MemStoreLABImpl

Method Detail

copyCellInto

forceCopyOfBigCellInto

copyBBECellInto

copyCellInto

copyToChunkCell

copyBBECToChunkCell

createChunkCell

close

getRefCntValue

incScannerCount

decScannerCount

recycleChunks

tryRetireChunk

getOrMakeChunk

getNewExternalChunk

getNewExternalChunk

isOnHeap

isOffHeap

getCurrentChunk

getPooledChunks

getNumOfChunksReturnedToPool

isReclaimed

isClosed