@InterfaceAudience.Private public class CacheConfig extends Object
Modifier and Type | Class and Description |
---|---|
private static class |
CacheConfig.ExternalBlockCaches
Enum of all built in external block caches.
|
Modifier and Type | Field and Description |
---|---|
private BlockCache |
blockCache
Local reference to the block cache, null if completely disabled
|
static String |
BLOCKCACHE_BLOCKSIZE_KEY
The target block size used by blockcache instances.
|
(package private) static boolean |
blockCacheDisabled
Boolean whether we have disabled the block cache entirely.
|
static String |
BUCKET_CACHE_BUCKETS_KEY
A comma-delimited array of values for use as bucket sizes.
|
static String |
BUCKET_CACHE_COMBINED_KEY
If the bucket cache is used in league with the lru on-heap block cache (meta blocks such
as indices and blooms are kept in the lru blockcache and the data blocks in the
bucket cache).
|
static String |
BUCKET_CACHE_PERSISTENT_PATH_KEY
If the chosen ioengine can persist its state across restarts, the path to the file to
persist to.
|
static String |
BUCKET_CACHE_WRITER_QUEUE_KEY |
static String |
BUCKET_CACHE_WRITER_THREADS_KEY |
static String |
CACHE_BLOCKS_ON_WRITE_KEY
Configuration key to cache data blocks on write.
|
static String |
CACHE_BLOOM_BLOCKS_ON_WRITE_KEY
Configuration key to cache compound bloom filter blocks on write.
|
static String |
CACHE_DATA_BLOCKS_COMPRESSED_KEY
Configuration key to cache data blocks in compressed and/or encrypted format.
|
static String |
CACHE_INDEX_BLOCKS_ON_WRITE_KEY
Configuration key to cache leaf and intermediate-level index blocks on
write.
|
private boolean |
cacheBloomsOnWrite
Whether compound bloom filter blocks should be cached on write
|
private boolean |
cacheDataCompressed
Whether data blocks should be stored in compressed and/or encrypted form in the cache
|
private boolean |
cacheDataInL1
If true and if more than one tier in this cache deploy -- e.g.
|
private boolean |
cacheDataOnRead
Whether blocks should be cached on read (default is on if there is a
cache but this can be turned off on a per-family or per-request basis).
|
private boolean |
cacheDataOnWrite
Whether data blocks should be cached when new files are written
|
private boolean |
cacheIndexesOnWrite
Whether index blocks should be cached when new files are written
|
static boolean |
DEFAULT_BUCKET_CACHE_COMBINED
Defaults for Bucket cache
|
static int |
DEFAULT_BUCKET_CACHE_WRITER_QUEUE |
static int |
DEFAULT_BUCKET_CACHE_WRITER_THREADS |
static boolean |
DEFAULT_CACHE_BLOOMS_ON_WRITE |
static boolean |
DEFAULT_CACHE_DATA_COMPRESSED |
static boolean |
DEFAULT_CACHE_DATA_ON_READ |
static boolean |
DEFAULT_CACHE_DATA_ON_WRITE |
static boolean |
DEFAULT_CACHE_INDEXES_ON_WRITE |
static boolean |
DEFAULT_EVICT_ON_CLOSE |
static boolean |
DEFAULT_IN_MEMORY |
static boolean |
DEFAULT_PREFETCH_ON_OPEN |
private static boolean |
DROP_BEHIND_CACHE_COMPACTION_DEFAULT |
private static String |
DROP_BEHIND_CACHE_COMPACTION_KEY |
private boolean |
dropBehindCompaction |
static String |
EVICT_BLOCKS_ON_CLOSE_KEY
Configuration key to evict all blocks of a given file from the block cache
when the file is closed.
|
private boolean |
evictOnClose
Whether blocks of a file should be evicted when the file is closed
|
private static String |
EXTERNAL_BLOCKCACHE_CLASS_KEY |
private static boolean |
EXTERNAL_BLOCKCACHE_DEFAULT |
private static String |
EXTERNAL_BLOCKCACHE_KEY |
(package private) static BlockCache |
GLOBAL_BLOCK_CACHE_INSTANCE
Static reference to the block cache, or null if no caching should be used
at all.
|
private boolean |
inMemory
Whether blocks should be flagged as in-memory when being cached
|
private static org.apache.commons.logging.Log |
LOG |
static String |
PREFETCH_BLOCKS_ON_OPEN_KEY
Configuration key to prefetch all blocks of a given file into the block cache
when the file is opened.
|
private boolean |
prefetchOnOpen
Whether data blocks should be prefetched into the cache
|
Constructor and Description |
---|
CacheConfig(BlockCache blockCache,
boolean cacheDataOnRead,
boolean inMemory,
boolean cacheDataOnWrite,
boolean cacheIndexesOnWrite,
boolean cacheBloomsOnWrite,
boolean evictOnClose,
boolean cacheDataCompressed,
boolean prefetchOnOpen,
boolean cacheDataInL1,
boolean dropBehindCompaction)
Create a block cache configuration with the specified cache and
configuration parameters.
|
CacheConfig(CacheConfig cacheConf)
Constructs a cache configuration copied from the specified configuration.
|
CacheConfig(org.apache.hadoop.conf.Configuration conf)
Create a cache configuration using the specified configuration object and
defaults for family level settings.
|
CacheConfig(org.apache.hadoop.conf.Configuration conf,
HColumnDescriptor family)
Create a cache configuration using the specified configuration object and
family descriptor.
|
Modifier and Type | Method and Description |
---|---|
BlockCache |
getBlockCache()
Returns the block cache.
|
private static BlockCache |
getBucketCache(org.apache.hadoop.conf.Configuration c) |
private static BlockCache |
getExternalBlockcache(org.apache.hadoop.conf.Configuration c) |
private static LruBlockCache |
getL1(org.apache.hadoop.conf.Configuration c) |
private static BlockCache |
getL2(org.apache.hadoop.conf.Configuration c) |
static BlockCache |
instantiateBlockCache(org.apache.hadoop.conf.Configuration conf)
Returns the block cache or
null in case none should be used. |
boolean |
isBlockCacheEnabled()
Checks whether the block cache is enabled.
|
boolean |
isCacheDataInL1() |
boolean |
isInMemory() |
void |
setCacheDataInL1(boolean cacheDataInL1)
Only used for testing.
|
void |
setCacheDataOnWrite(boolean cacheDataOnWrite)
Only used for testing.
|
void |
setEvictOnClose(boolean evictOnClose)
Only used for testing.
|
boolean |
shouldCacheBlockOnRead(BlockType.BlockCategory category)
Should we cache a block of a particular category? We always cache
important blocks such as index blocks, as long as the block cache is
available.
|
boolean |
shouldCacheBloomsOnWrite() |
boolean |
shouldCacheCompressed(BlockType.BlockCategory category) |
boolean |
shouldCacheDataCompressed() |
boolean |
shouldCacheDataOnRead()
Returns whether the DATA blocks of this HFile should be cached on read or not (we always
cache the meta blocks, the INDEX and BLOOM blocks).
|
boolean |
shouldCacheDataOnWrite() |
boolean |
shouldCacheIndexesOnWrite() |
boolean |
shouldDropBehindCompaction() |
boolean |
shouldEvictOnClose() |
boolean |
shouldLockOnCacheMiss(BlockType blockType)
If we make sure the block could not be cached, we will not acquire the lock
otherwise we will acquire lock
|
boolean |
shouldPrefetchOnOpen() |
boolean |
shouldReadBlockFromCache(BlockType blockType)
Return true if we may find this type of block in block cache.
|
String |
toString() |
private static final org.apache.commons.logging.Log LOG
public static final String CACHE_BLOCKS_ON_WRITE_KEY
public static final String CACHE_INDEX_BLOCKS_ON_WRITE_KEY
public static final String CACHE_BLOOM_BLOCKS_ON_WRITE_KEY
public static final String CACHE_DATA_BLOCKS_COMPRESSED_KEY
public static final String EVICT_BLOCKS_ON_CLOSE_KEY
public static final String BUCKET_CACHE_PERSISTENT_PATH_KEY
public static final String BUCKET_CACHE_COMBINED_KEY
public static final String BUCKET_CACHE_WRITER_THREADS_KEY
public static final String BUCKET_CACHE_WRITER_QUEUE_KEY
public static final String BUCKET_CACHE_BUCKETS_KEY
public static final boolean DEFAULT_BUCKET_CACHE_COMBINED
public static final int DEFAULT_BUCKET_CACHE_WRITER_THREADS
public static final int DEFAULT_BUCKET_CACHE_WRITER_QUEUE
public static final String PREFETCH_BLOCKS_ON_OPEN_KEY
public static final String BLOCKCACHE_BLOCKSIZE_KEY
HConstants.DEFAULT_BLOCKSIZE
.
TODO: this config point is completely wrong, as it's used to determine the
target block size of BlockCache instances. Rename.private static final String EXTERNAL_BLOCKCACHE_KEY
private static final boolean EXTERNAL_BLOCKCACHE_DEFAULT
private static final String EXTERNAL_BLOCKCACHE_CLASS_KEY
private static final String DROP_BEHIND_CACHE_COMPACTION_KEY
private static final boolean DROP_BEHIND_CACHE_COMPACTION_DEFAULT
public static final boolean DEFAULT_CACHE_DATA_ON_READ
public static final boolean DEFAULT_CACHE_DATA_ON_WRITE
public static final boolean DEFAULT_IN_MEMORY
public static final boolean DEFAULT_CACHE_INDEXES_ON_WRITE
public static final boolean DEFAULT_CACHE_BLOOMS_ON_WRITE
public static final boolean DEFAULT_EVICT_ON_CLOSE
public static final boolean DEFAULT_CACHE_DATA_COMPRESSED
public static final boolean DEFAULT_PREFETCH_ON_OPEN
private final BlockCache blockCache
private boolean cacheDataOnRead
private final boolean inMemory
private boolean cacheDataOnWrite
private final boolean cacheIndexesOnWrite
private final boolean cacheBloomsOnWrite
private boolean evictOnClose
private final boolean cacheDataCompressed
private final boolean prefetchOnOpen
private boolean cacheDataInL1
private final boolean dropBehindCompaction
static BlockCache GLOBAL_BLOCK_CACHE_INSTANCE
static boolean blockCacheDisabled
public CacheConfig(org.apache.hadoop.conf.Configuration conf, HColumnDescriptor family)
conf
- hbase configurationfamily
- column family configurationpublic CacheConfig(org.apache.hadoop.conf.Configuration conf)
CacheConfig(Configuration, HColumnDescriptor)
conf
- hbase configurationCacheConfig(Configuration, HColumnDescriptor)
CacheConfig(BlockCache blockCache, boolean cacheDataOnRead, boolean inMemory, boolean cacheDataOnWrite, boolean cacheIndexesOnWrite, boolean cacheBloomsOnWrite, boolean evictOnClose, boolean cacheDataCompressed, boolean prefetchOnOpen, boolean cacheDataInL1, boolean dropBehindCompaction)
blockCache
- reference to block cache, null if completely disabledcacheDataOnRead
- whether DATA blocks should be cached on read (we always cache INDEX
blocks and BLOOM blocks; this cannot be disabled).inMemory
- whether blocks should be flagged as in-memorycacheDataOnWrite
- whether data blocks should be cached on writecacheIndexesOnWrite
- whether index blocks should be cached on writecacheBloomsOnWrite
- whether blooms should be cached on writeevictOnClose
- whether blocks should be evicted when HFile is closedcacheDataCompressed
- whether to store blocks as compressed in the cacheprefetchOnOpen
- whether to prefetch blocks upon opencacheDataInL1
- If more than one cache tier deployed, if true, cache this column families
data blocks up in the L1 tier.public CacheConfig(CacheConfig cacheConf)
cacheConf
- public boolean isBlockCacheEnabled()
public BlockCache getBlockCache()
public boolean shouldCacheDataOnRead()
public boolean shouldDropBehindCompaction()
public boolean shouldCacheBlockOnRead(BlockType.BlockCategory category)
public boolean isInMemory()
public boolean isCacheDataInL1()
public boolean shouldCacheDataOnWrite()
public void setCacheDataOnWrite(boolean cacheDataOnWrite)
cacheDataOnWrite
- whether data blocks should be written to the cache
when an HFile is writtenpublic void setCacheDataInL1(boolean cacheDataInL1)
cacheDataInL1
- Whether to cache data blocks up in l1 (if a multi-tier cache
implementation).public boolean shouldCacheIndexesOnWrite()
public boolean shouldCacheBloomsOnWrite()
public boolean shouldEvictOnClose()
public void setEvictOnClose(boolean evictOnClose)
evictOnClose
- whether blocks should be evicted from the cache when an
HFile reader is closedpublic boolean shouldCacheDataCompressed()
public boolean shouldCacheCompressed(BlockType.BlockCategory category)
BlockType.BlockCategory
should be compressed in blockcache, false otherwisepublic boolean shouldPrefetchOnOpen()
public boolean shouldReadBlockFromCache(BlockType blockType)
family.isBlockCacheEnabled()
only means cacheDataOnRead
, so here we
consider lots of other configurations such as cacheDataOnWrite
. We should fix this in
the future, cacheDataOnWrite
should honor the CF level isBlockCacheEnabled
configuration.public boolean shouldLockOnCacheMiss(BlockType blockType)
private static LruBlockCache getL1(org.apache.hadoop.conf.Configuration c)
c
- Configuration to use.private static BlockCache getL2(org.apache.hadoop.conf.Configuration c)
c
- Configuration to use.private static BlockCache getExternalBlockcache(org.apache.hadoop.conf.Configuration c)
private static BlockCache getBucketCache(org.apache.hadoop.conf.Configuration c)
public static BlockCache instantiateBlockCache(org.apache.hadoop.conf.Configuration conf)
null
in case none should be used.
Sets GLOBAL_BLOCK_CACHE_INSTANCEconf
- The current configuration.null
.Copyright © 2007–2019 The Apache Software Foundation. All rights reserved.