@InterfaceAudience.Private public class ByteBufferIOEngine extends Object implements IOEngine
ByteBufferArray.
ByteBufferArray and how it gives a view across multiple
ByteBuffers managed by it internally. This class does the physical BB create and the write and
read to the underlying BBs. So we will create N BBs based on the total BC capacity specified on
create of the ByteBufferArray. So say we have 10 GB of off heap BucketCache, we will create 2560
such BBs inside our ByteBufferArray.
Now the way BucketCache works is that the entire 10 GB is split into diff sized buckets: by
default from 5 KB to 513 KB. Within each bucket of a particular size, there are usually more than
one bucket 'block'. The way it is calculate in bucketcache is that the total bucketcache size is
divided by 4 (hard-coded currently) * max size option. So using defaults, buckets will be is 4 *
513kb (the biggest default value) = 2052kb. A bucket of 2052kb at offset zero will serve out
bucket 'blocks' of 5kb, the next bucket will do the next size up and so on up to the maximum
(default) of 513kb).
When we write blocks to the bucketcache, we will see which bucket size group it best fits. So a 4
KB block size goes to the 5 KB size group. Each of the block writes, writes within its
appropriate bucket. Though the bucket is '4kb' in size, it will occupy one of the 5 KB bucket
'blocks' (even if actual size of the bucket is less). Bucket 'blocks' will not span buckets.
But you can see the physical memory under the bucket 'blocks' can be split across the underlying
backing BBs from ByteBufferArray. All is split into 4 MB sized BBs.
Each Bucket knows its offset in the entire space of BC and when block is written the offset
arrives at ByteBufferArray and it figures which BB to write to. It may so happen that the entire
block to be written does not fit a particular backing ByteBufferArray so the remainder goes to
another BB. See ByteBufferArray.write(long, ByteBuff).
So said all these, when we read a block it may be possible that the bytes of that blocks is
physically placed in 2 adjucent BBs. In such case also, we avoid any copy need by having the
MBB...
| Modifier and Type | Field and Description |
|---|---|
private ByteBufferArray |
bufferArray |
private long |
capacity |
| Constructor and Description |
|---|
ByteBufferIOEngine(long capacity)
Construct the ByteBufferIOEngine with the given capacity
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
isPersistent()
Memory IO engine is always unable to support persistent storage for the
cache
|
Cacheable |
read(BucketEntry be)
Transfers data from IOEngine to a Cacheable object.
|
void |
shutdown()
No operation for the shutdown in the memory IO engine
|
void |
sync()
No operation for the sync in the memory IO engine
|
String |
toString() |
boolean |
usesSharedMemory()
IOEngine uses shared memory means, when reading Cacheable from it, those refers to the same
memory area as used by the Engine for caching it.
|
void |
write(ByteBuffer src,
long offset)
Transfers data from the given
ByteBuffer to the buffer array. |
void |
write(ByteBuff src,
long offset)
Transfers data from the given
ByteBuff to the buffer array. |
private ByteBufferArray bufferArray
private final long capacity
public ByteBufferIOEngine(long capacity) throws IOException
capacity - IOException - ideally here no exception to be thrown from the allocatorpublic boolean isPersistent()
isPersistent in interface IOEnginepublic boolean usesSharedMemory()
IOEngineusesSharedMemory in interface IOEnginepublic Cacheable read(BucketEntry be) throws IOException
IOEngineread in interface IOEnginebe - maintains an (offset,len,refCnt) inside.IOException - when any IO error happenpublic void write(ByteBuffer src, long offset) throws IOException
ByteBuffer to the buffer array. Position of source will
be advanced by the Buffer.remaining().write in interface IOEnginesrc - the given byte buffer from which bytes are to be read.offset - The offset in the ByteBufferArray of the first byte to be writtenIOException - throws IOException if writing to the array throws exceptionpublic void write(ByteBuff src, long offset) throws IOException
ByteBuff to the buffer array. Position of source will be
advanced by the Buffer.remaining().write in interface IOEnginesrc - the given byte buffer from which bytes are to be read.offset - The offset in the ByteBufferArray of the first byte to be writtenIOException - throws IOException if writing to the array throws exceptionpublic void sync()
Copyright © 2007–2021 The Apache Software Foundation. All rights reserved.