Package org.apache.hadoop.hbase.io.compress

Class CodecPool

java.lang.Object

org.apache.hadoop.hbase.io.compress.CodecPool

@Private
public class CodecPool
extends Object

A global compressor/decompressor pool used to save and reuse (possibly native) compression/decompression codecs. Copied from the class of the same name in hadoop-common and augmented to improve borrow/return performance.

Field Summary

Fields

Modifier and Type	Field	Description
private static final ConcurrentMap<Class<ByteBuffDecompressor>,NavigableSet<ByteBuffDecompressor>>	BYTE_BUFF_DECOMPRESSOR_POOL
private static final ConcurrentMap<Class<org.apache.hadoop.io.compress.Compressor>,NavigableSet<org.apache.hadoop.io.compress.Compressor>>	COMPRESSOR_POOL
private static com.github.benmanes.caffeine.cache.LoadingCache<Class<org.apache.hadoop.io.compress.Compressor>,AtomicInteger>	compressorCounts	Map to track the number of leased compressors.
private static final ConcurrentMap<Class<org.apache.hadoop.io.compress.Decompressor>,NavigableSet<org.apache.hadoop.io.compress.Decompressor>>	DECOMPRESSOR_POOL
private static com.github.benmanes.caffeine.cache.LoadingCache<Class<org.apache.hadoop.io.compress.Decompressor>,AtomicInteger>	decompressorCounts	Map to tracks the number of leased decompressors.
private static final org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
CodecPool()

Method Summary

Modifier and Type	Method	Description
private static <T> T	borrow(ConcurrentMap<Class<T>,NavigableSet<T>> pool, Class<? extends T> codecClass)
private static <T> com.github.benmanes.caffeine.cache.LoadingCache<Class<T>,AtomicInteger>	createCache()
static ByteBuffDecompressor	getByteBuffDecompressor(ByteBuffDecompressionCodec codec)
static org.apache.hadoop.io.compress.Compressor	getCompressor(org.apache.hadoop.io.compress.CompressionCodec codec)
static org.apache.hadoop.io.compress.Compressor	getCompressor(org.apache.hadoop.io.compress.CompressionCodec codec, org.apache.hadoop.conf.Configuration conf)	Get a Compressor for the given CompressionCodec from the pool, or get a new one if the pool is empty.
static org.apache.hadoop.io.compress.Decompressor	getDecompressor(org.apache.hadoop.io.compress.CompressionCodec codec)	Get a Decompressor for the given CompressionCodec from the pool, or get a new one if the pool is empty.
private static <T> int	getLeaseCount(com.github.benmanes.caffeine.cache.LoadingCache<Class<T>,AtomicInteger> usageCounts, Class<? extends T> codecClass)	Copied from hadoop-common without significant modification.
(package private) static int	getLeasedCompressorsCount(org.apache.hadoop.io.compress.CompressionCodec codec)	Returns the number of leased Compressors for this CompressionCodec.
(package private) static int	getLeasedDecompressorsCount(org.apache.hadoop.io.compress.CompressionCodec codec)	Returns the number of leased Decompressors for this CompressionCodec.
(package private) static void	initLeaseCounting()	Call if you want lease counting to be enabled.
private static <T> boolean	payback(ConcurrentMap<Class<T>,NavigableSet<T>> pool, T codec)
static void	returnByteBuffDecompressor(ByteBuffDecompressor decompressor)
static void	returnCompressor(org.apache.hadoop.io.compress.Compressor compressor)	Return the Compressor to the pool.
static void	returnDecompressor(org.apache.hadoop.io.compress.Decompressor decompressor)	Return the Decompressor to the pool.
private static <T> void	updateLeaseCount(com.github.benmanes.caffeine.cache.LoadingCache<Class<T>,AtomicInteger> usageCounts, T codec, int delta)	Copied from hadoop-common without significant modification.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

COMPRESSOR_POOL

private static final ConcurrentMap<Class<org.apache.hadoop.io.compress.Compressor>,NavigableSet<org.apache.hadoop.io.compress.Compressor>> COMPRESSOR_POOL

DECOMPRESSOR_POOL

private static final ConcurrentMap<Class<org.apache.hadoop.io.compress.Decompressor>,NavigableSet<org.apache.hadoop.io.compress.Decompressor>> DECOMPRESSOR_POOL

BYTE_BUFF_DECOMPRESSOR_POOL

private static final ConcurrentMap<Class<ByteBuffDecompressor>,NavigableSet<ByteBuffDecompressor>> BYTE_BUFF_DECOMPRESSOR_POOL

compressorCounts

@Nullable
private static com.github.benmanes.caffeine.cache.LoadingCache<Class<org.apache.hadoop.io.compress.Compressor>,AtomicInteger> compressorCounts

Map to track the number of leased compressors. Only used in unit tests, kept null otherwise.

decompressorCounts

@Nullable
private static com.github.benmanes.caffeine.cache.LoadingCache<Class<org.apache.hadoop.io.compress.Decompressor>,AtomicInteger> decompressorCounts

Map to tracks the number of leased decompressors. Only used in unit tests, kept null otherwise.

Constructor Details

CodecPool

public CodecPool()

Method Details

createCache

private static <T>
com.github.benmanes.caffeine.cache.LoadingCache<Class<T>,AtomicInteger> createCache()

initLeaseCounting

static void initLeaseCounting()

Call if you want lease counting to be enabled. Only used in unit tests.

borrow

private static <T> T borrow(ConcurrentMap<Class<T>,NavigableSet<T>> pool,
 Class<? extends T> codecClass)

payback

private static <T> boolean payback(ConcurrentMap<Class<T>,NavigableSet<T>> pool,
 T codec)

getLeaseCount

private static <T> int getLeaseCount(com.github.benmanes.caffeine.cache.LoadingCache<Class<T>,AtomicInteger> usageCounts,
 Class<? extends T> codecClass)

Copied from hadoop-common without significant modification.

updateLeaseCount

private static <T> void updateLeaseCount(com.github.benmanes.caffeine.cache.LoadingCache<Class<T>,AtomicInteger> usageCounts,
 T codec,
 int delta)

Copied from hadoop-common without significant modification.

getCompressor

public static org.apache.hadoop.io.compress.Compressor getCompressor(org.apache.hadoop.io.compress.CompressionCodec codec,
 org.apache.hadoop.conf.Configuration conf)

Get a Compressor for the given CompressionCodec from the pool, or get a new one if the pool is empty. Copied from hadoop-common without significant modification.

getCompressor

public static org.apache.hadoop.io.compress.Compressor getCompressor(org.apache.hadoop.io.compress.CompressionCodec codec)

getDecompressor

public static org.apache.hadoop.io.compress.Decompressor getDecompressor(org.apache.hadoop.io.compress.CompressionCodec codec)

Get a Decompressor for the given CompressionCodec from the pool, or get a new one if the pool is empty. Copied from hadoop-common without significant modification.

getByteBuffDecompressor

public static ByteBuffDecompressor getByteBuffDecompressor(ByteBuffDecompressionCodec codec)

returnCompressor

public static void returnCompressor(org.apache.hadoop.io.compress.Compressor compressor)

Return the Compressor to the pool. Copied from hadoop-common without significant modification.

returnDecompressor

public static void returnDecompressor(org.apache.hadoop.io.compress.Decompressor decompressor)

Return the Decompressor to the pool. Copied from hadoop-common without significant modification.

returnByteBuffDecompressor

public static void returnByteBuffDecompressor(ByteBuffDecompressor decompressor)

getLeasedCompressorsCount

static int getLeasedCompressorsCount(@Nullable
 org.apache.hadoop.io.compress.CompressionCodec codec)

Returns the number of leased Compressors for this CompressionCodec. Copied from hadoop-common without significant modification.

getLeasedDecompressorsCount

static int getLeasedDecompressorsCount(@Nullable
 org.apache.hadoop.io.compress.CompressionCodec codec)

Returns the number of leased Decompressors for this CompressionCodec. Copied from hadoop-common without significant modification.