Interface MemStoreLAB
- All Known Implementing Classes:
ImmutableMemStoreLAB
,MemStoreLABImpl
The MemStoreLAB is basically a bump-the-pointer allocator that allocates big (2MB) chunks from
and then doles it out to threads that request slices into the array. These chunks can get pooled
as well. See ChunkCreator
.
The purpose of this is to combat heap fragmentation in the regionserver. By ensuring that all Cells in a given memstore refer only to large chunks of contiguous memory, we ensure that large blocks get freed up when the memstore is flushed.
Without the MSLAB, the byte array allocated during insertion end up interleaved throughout the heap, and the old generation gets progressively more fragmented until a stop-the-world compacting collection occurs.
This manages the large sized chunks. When Cells are to be added to Memstore, MemStoreLAB's
copyCellInto(ExtendedCell)
gets called. This allocates enough size in the chunk to hold
this cell's data and copies into this area and then recreate a Cell over this copied data.
- See Also:
-
Field Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
Close instance since it won't be used any more, try to put the chunks back to poolcopyCellInto
(ExtendedCell cell) Allocates slice in this LAB and copy the passed Cell into this area.void
Called when closing a scanner on the data of this MemStoreLABAllocates slice in this LAB and copy the passed Cell into this area.getNewExternalChunk
(int size) getNewExternalChunk
(ChunkCreator.ChunkType chunkType) void
Called when opening a scanner on the data of this MemStoreLABstatic boolean
isEnabled
(org.apache.hadoop.conf.Configuration conf) boolean
boolean
isOnHeap()
static MemStoreLAB
newInstance
(org.apache.hadoop.conf.Configuration conf)
-
Field Details
-
USEMSLAB_KEY
- See Also:
-
USEMSLAB_DEFAULT
- See Also:
-
MSLAB_CLASS_NAME
- See Also:
-
CHUNK_SIZE_KEY
- See Also:
-
CHUNK_SIZE_DEFAULT
- See Also:
-
INDEX_CHUNK_SIZE_PERCENTAGE_KEY
- See Also:
-
INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT
- See Also:
-
MAX_ALLOC_KEY
- See Also:
-
MAX_ALLOC_DEFAULT
- See Also:
-
CHUNK_POOL_MAXSIZE_KEY
- See Also:
-
CHUNK_POOL_INITIALSIZE_KEY
- See Also:
-
POOL_MAX_SIZE_DEFAULT
- See Also:
-
POOL_INITIAL_SIZE_DEFAULT
- See Also:
-
-
Method Details
-
copyCellInto
Allocates slice in this LAB and copy the passed Cell into this area. Returns new Cell instance over the copied the data. When this MemStoreLAB can not copy this Cell, it returns null. -
forceCopyOfBigCellInto
Allocates slice in this LAB and copy the passed Cell into this area. Returns new Cell instance over the copied the data. When this MemStoreLAB can not copy this Cell, it returns null. Since the process of flattening to CellChunkMap assumes all cells are allocated on MSLAB, and since copyCellInto does not copy big cells (for whom size > maxAlloc) into MSLAB, this method is called while the process of flattening to CellChunkMap is running, for forcing the allocation of big cells on this MSLAB. -
close
void close()Close instance since it won't be used any more, try to put the chunks back to pool -
incScannerCount
void incScannerCount()Called when opening a scanner on the data of this MemStoreLAB -
decScannerCount
void decScannerCount()Called when closing a scanner on the data of this MemStoreLAB -
getNewExternalChunk
-
getNewExternalChunk
-
newInstance
-
isEnabled
-
isOnHeap
boolean isOnHeap() -
isOffHeap
boolean isOffHeap()
-