@InterfaceAudience.Private public interface MemStoreLAB
The MemStoreLAB is basically a bump-the-pointer allocator that allocates big (2MB) chunks from
and then doles it out to threads that request slices into the array. These chunks can get pooled
as well. See ChunkCreator
.
The purpose of this is to combat heap fragmentation in the regionserver. By ensuring that all Cells in a given memstore refer only to large chunks of contiguous memory, we ensure that large blocks get freed up when the memstore is flushed.
Without the MSLAB, the byte array allocated during insertion end up interleaved throughout the heap, and the old generation gets progressively more fragmented until a stop-the-world compacting collection occurs.
This manages the large sized chunks. When Cells are to be added to Memstore, MemStoreLAB's
copyCellInto(Cell)
gets called. This allocates enough size in the chunk to hold this
cell's data and copies into this area and then recreate a Cell over this copied data.
ChunkCreator
Modifier and Type | Field and Description |
---|---|
static String |
CHUNK_POOL_INITIALSIZE_KEY |
static String |
CHUNK_POOL_MAXSIZE_KEY |
static int |
CHUNK_SIZE_DEFAULT |
static String |
CHUNK_SIZE_KEY |
static float |
INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT |
static String |
INDEX_CHUNK_SIZE_PERCENTAGE_KEY |
static int |
MAX_ALLOC_DEFAULT |
static String |
MAX_ALLOC_KEY |
static String |
MSLAB_CLASS_NAME |
static float |
POOL_INITIAL_SIZE_DEFAULT |
static float |
POOL_MAX_SIZE_DEFAULT |
static boolean |
USEMSLAB_DEFAULT |
static String |
USEMSLAB_KEY |
Modifier and Type | Method and Description |
---|---|
void |
close()
Close instance since it won't be used any more, try to put the chunks back to pool
|
Cell |
copyCellInto(Cell cell)
Allocates slice in this LAB and copy the passed Cell into this area.
|
void |
decScannerCount()
Called when closing a scanner on the data of this MemStoreLAB
|
Cell |
forceCopyOfBigCellInto(Cell cell)
Allocates slice in this LAB and copy the passed Cell into this area.
|
Chunk |
getNewExternalChunk(ChunkCreator.ChunkType chunkType) |
Chunk |
getNewExternalChunk(int size) |
void |
incScannerCount()
Called when opening a scanner on the data of this MemStoreLAB
|
static boolean |
isEnabled(org.apache.hadoop.conf.Configuration conf) |
boolean |
isOffHeap() |
boolean |
isOnHeap() |
static MemStoreLAB |
newInstance(org.apache.hadoop.conf.Configuration conf) |
static final String USEMSLAB_KEY
static final boolean USEMSLAB_DEFAULT
static final String MSLAB_CLASS_NAME
static final String CHUNK_SIZE_KEY
static final int CHUNK_SIZE_DEFAULT
static final String INDEX_CHUNK_SIZE_PERCENTAGE_KEY
static final float INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT
static final String MAX_ALLOC_KEY
static final int MAX_ALLOC_DEFAULT
static final String CHUNK_POOL_MAXSIZE_KEY
static final String CHUNK_POOL_INITIALSIZE_KEY
static final float POOL_MAX_SIZE_DEFAULT
static final float POOL_INITIAL_SIZE_DEFAULT
Cell copyCellInto(Cell cell)
Cell forceCopyOfBigCellInto(Cell cell)
void close()
void incScannerCount()
void decScannerCount()
Chunk getNewExternalChunk(ChunkCreator.ChunkType chunkType)
Chunk getNewExternalChunk(int size)
static MemStoreLAB newInstance(org.apache.hadoop.conf.Configuration conf)
static boolean isEnabled(org.apache.hadoop.conf.Configuration conf)
boolean isOnHeap()
boolean isOffHeap()
Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.