Implements pretty-printing functionality for
BlockCache. Caches are configured (and instantiated) by
CacheConfig. See head of the
CacheConfigclass for constants that define cache options and configuration keys to use setting cache options. Cache implementations include the default, native on-heap
SlabCachethat can serve as an L2 for
LruBlockCache(hosted inside the class
DoubleBlockCachethat caches blocks in BOTH L1 and L2, and on evict, moves from L1 to L2, etc), and a
BucketCachethat has a bunch of deploy formats including acting as a L2 for LruBlockCache -- when a block is evicted from LruBlockCache, it goes to the BucketCache and when we search a block, we look in both places -- or using
CombinedBlockCache, as a host for data blocks with meta blocks in the LRUBlockCache as well as onheap, offheap, and file options.
DoubleBlockCachecurrently works. It is tough doing an apples to apples compare since their hosting classes,
CombinedBlockCachefor BucketCache vs
DoubleBlockCacheoperate so differently. See Nick Dimiduk's BlockCache 101 for some numbers. See also the description of HBASE-7404 where Chunhui Shen lists issues he found with BlockCache (inefficent use of memory, doesn't help w/ GC).
SlabCacheis the original offheap block cache but unfortunately has seen little use. It is originally described in Caching in Apache HBase: SlabCache.To enable it, set the float
CacheConfig.SLAB_CACHE_OFFHEAP_PERCENTAGE_KEY) to some value between 0 and 1 in your
hbase-site.xmlfile. This enables
DoubleBlockCache, a facade over
SlabCache. DoubleBlockCache works as follows. When caching, it "...attempts to cache the block in both caches, while readblock reads first from the faster onheap cache before looking for the block in the off heap cache. Metrics are the combined size and hits and misses of both caches." The value set in
hbase.offheapcache.percentagewill be multiplied by whatever the setting for
-XX:MaxDirectMemorySizeis in your
hbase-env.shconfiguration file and this is what will be used by
SlabCacheas its offheap store. Onheap store will be whatever the float
HConstants.HFILE_BLOCK_CACHE_SIZE_KEYsetting is (some value between 0 and 1) times the size of the allocated java heap.
Restart (or rolling restart) your cluster for the configs to take effect. Check logs to ensure your configurations came out as expected.
BucketCacheEnsure the SlabCache config
hbase.offheapcache.percentageis not set (or set to 0). At this point, it is probably best to read the code to learn the list of bucket cache options and how they combine (to be fixed). Read the options and defaults for BucketCache in the head of the
Here is a simple example of how to enable a
offheap bucket cache with 1G onheap cache.
The onheap/offheap caches
are managed by
CombinedBlockCache by default. For the
CombinedBlockCache (from the class comment), "The smaller lruCache is used
to cache bloom blocks and index blocks, the larger bucketCache is used to
cache data blocks. getBlock reads first from the smaller lruCache before
looking for the block in the bucketCache. Metrics are the combined size and
hits and misses of both caches." To disable CombinedBlockCache and have the BucketCache act
as a strict L2 cache to the L1 LruBlockCache (i.e. on eviction from L1, blocks go to L2), set
CacheConfig.BUCKET_CACHE_COMBINED_KEY to false.
Also by default, unless you change it,
the top of the CacheConfig in the BucketCache defaults section). This means that whatever
size you set for the bucket cache with
90% will be used for offheap and
10% of the size will be used
by the onheap
Back to the example of setting an onheap cache of 1G and ofheap of 4G, in
hbase-env.sh ensure the java option
enabled and 5G in size: e.g.
-XX:MaxDirectMemorySize=5G. Then in
hbase-site.xml add the following configurations:
<property> <name>hbase.bucketcache.ioengine</name> <value>offheap</value> </property> <property> <name>hbase.bucketcache.percentage.in.combinedcache</name> <value>0.8</value> </property> <property> <name>hbase.bucketcache.size</name> <value>5120</value> </property>. Above we set a cache of 5G, 80% of which will be offheap (4G) and 1G onheap. Restart (or rolling restart) your cluster for the configs to take effect. Check logs to ensure your configurations came out as expected.
Copyright © 2014 The Apache Software Foundation. All rights reserved.