org.apache.hadoop.hbase.io.hfile.bucket

Class BucketAllocator

java.lang.Object

org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator

@InterfaceAudience.Private
public final class BucketAllocator
extends Object

This class is used to allocate a block with specified size and free the block when evicting. It manages an array of buckets, each bucket is associated with a size and caches elements up to this size. For a completely empty bucket, this size could be re-specified dynamically.

This class is not thread safe.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BucketAllocator.Bucket
(package private) class	BucketAllocator.BucketSizeInfo
(package private) static class	BucketAllocator.IndexStatistics

Field Summary

Fields

Modifier and Type	Field and Description
private int	bigItemSize
private long	bucketCapacity
private BucketAllocator.Bucket[]	buckets
private BucketAllocator.BucketSizeInfo[]	bucketSizeInfos
private int[]	bucketSizes
private static int[]	DEFAULT_BUCKET_SIZES
static int	FEWEST_ITEMS_IN_BUCKET So, what is the minimum amount of items we'll tolerate in a single bucket?
private static org.slf4j.Logger	LOG
private long	totalSize
private long	usedSize

Constructor Summary

Constructors

Constructor and Description
BucketAllocator(long availableSpace, int[] bucketSizes)
BucketAllocator(long availableSpace, int[] bucketSizes, Map<BlockCacheKey,BucketCache.BucketEntry> map, LongAdder realCacheSize) Rebuild the allocator's data structures from a persisted map.

Method Summary

Modifier and Type	Method and Description
long	allocateBlock(int blockSize) Allocate a block with specified size.
int	freeBlock(long offset) Free a block with the offset
long	freeBlock(long[] freeList)
int	getBucketIndex(long offset)
BucketAllocator.Bucket[]	getBuckets()
long	getFreeSize()
(package private) BucketAllocator.IndexStatistics[]	getIndexStatistics()
(package private) BucketAllocator.IndexStatistics[]	getIndexStatistics(BucketAllocator.IndexStatistics grandTotal)
Set<Integer>	getLeastFilledBuckets(Set<Integer> excludedBuckets, int bucketCount) Returns a set of indices of the buckets that are least filled excluding the offsets, we also the fully free buckets for the BucketSizes where everything is empty and they only have one completely free bucket as a reserved
long	getTotalSize()
long	getUsedSize()
private BucketAllocator.Bucket	grabGlobalCompletelyFreeBucket()
(package private) void	logStatistics()
BucketAllocator.BucketSizeInfo	roundUpToBucketSizeInfo(int blockSize) Round up the given block size to bucket size, and get the corresponding BucketSizeInfo
int	sizeIndexOfAllocation(long offset)
int	sizeOfAllocation(long offset)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

DEFAULT_BUCKET_SIZES

private static final int[] DEFAULT_BUCKET_SIZES

FEWEST_ITEMS_IN_BUCKET

public static final int FEWEST_ITEMS_IN_BUCKET

So, what is the minimum amount of items we'll tolerate in a single bucket?

See Also:

Constant Field Values

bucketSizes

private final int[] bucketSizes

bigItemSize

private final int bigItemSize

bucketCapacity

private final long bucketCapacity

buckets

private BucketAllocator.Bucket[] buckets

bucketSizeInfos

private BucketAllocator.BucketSizeInfo[] bucketSizeInfos

totalSize

private final long totalSize

usedSize

private transient long usedSize

Constructor Detail

BucketAllocator

BucketAllocator(long availableSpace,
                int[] bucketSizes)
         throws BucketAllocatorException

Throws:

BucketAllocatorException

BucketAllocator

BucketAllocator(long availableSpace,
                int[] bucketSizes,
                Map<BlockCacheKey,BucketCache.BucketEntry> map,
                LongAdder realCacheSize)
         throws BucketAllocatorException

Rebuild the allocator's data structures from a persisted map.

Parameters:

availableSpace - capacity of cache

map - A map stores the block key and BucketEntry(block's meta data like offset, length)

realCacheSize - cached data size statistics for bucket cache

Throws:

BucketAllocatorException

Method Detail

roundUpToBucketSizeInfo

public BucketAllocator.BucketSizeInfo roundUpToBucketSizeInfo(int blockSize)

Round up the given block size to bucket size, and get the corresponding BucketSizeInfo

toString

public String toString()

Overrides:

toString in class Object

getUsedSize

public long getUsedSize()

getFreeSize

public long getFreeSize()

getTotalSize

public long getTotalSize()

allocateBlock

public long allocateBlock(int blockSize)
                   throws CacheFullException,
                          BucketAllocatorException

Allocate a block with specified size. Return the offset

Parameters:

blockSize - size of block

Returns:

the offset in the IOEngine

Throws:

BucketAllocatorException

CacheFullException

grabGlobalCompletelyFreeBucket

private BucketAllocator.Bucket grabGlobalCompletelyFreeBucket()

freeBlock

public int freeBlock(long offset)

Free a block with the offset

Parameters:

offset - block's offset

Returns:

size freed

sizeIndexOfAllocation

public int sizeIndexOfAllocation(long offset)

sizeOfAllocation

public int sizeOfAllocation(long offset)

getBuckets

public BucketAllocator.Bucket[] getBuckets()

logStatistics

void logStatistics()

getIndexStatistics

BucketAllocator.IndexStatistics[] getIndexStatistics(BucketAllocator.IndexStatistics grandTotal)

getIndexStatistics

BucketAllocator.IndexStatistics[] getIndexStatistics()

freeBlock

public long freeBlock(long[] freeList)

getBucketIndex

public int getBucketIndex(long offset)

getLeastFilledBuckets

public Set<Integer> getLeastFilledBuckets(Set<Integer> excludedBuckets,
                                          int bucketCount)

Returns a set of indices of the buckets that are least filled excluding the offsets, we also the fully free buckets for the BucketSizes where everything is empty and they only have one completely free bucket as a reserved

Parameters:

excludedBuckets - the buckets that need to be excluded due to currently being in used

bucketCount - max Number of buckets to return

Returns:

set of bucket indices which could be used for eviction