Package org.apache.hadoop.hbase.io.hfile

Class CombinedBlockCache

java.lang.Object
org.apache.hadoop.hbase.io.hfile.CombinedBlockCache
All Implemented Interfaces:
Iterable<CachedBlock>, ConfigurationObserver, HeapSize, BlockCache, ResizableBlockCache
Direct Known Subclasses:
InclusiveCombinedBlockCache
@Private public class CombinedBlockCache extends Object implements ResizableBlockCache, HeapSize
CombinedBlockCache is an abstraction layer that combines FirstLevelBlockCache and BucketCache. The smaller lruCache is used to cache bloom blocks and index blocks. The larger Cache is used to cache data blocks. getBlock(BlockCacheKey, boolean, boolean, boolean) reads first from the smaller l1Cache before looking for the block in the l2Cache. Blocks evicted from l1Cache are put into the bucket cache. Metrics are the combined size and hits and misses of both caches.

  • Field Details

  • Constructor Details

  • Method Details

    • heapSize

      public long heapSize()
      Description copied from interface: HeapSize
      Return the approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.
      Specified by:
      heapSize in interface HeapSize

    • cacheBlock

      public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory)
      Description copied from interface: BlockCache
      Add block to cache.
      Specified by:
      cacheBlock in interface BlockCache
      Parameters:
      cacheKey - The block's cache key.
      buf - The block contents wrapped in a ByteBuffer.
      inMemory - Whether block should be treated as in-memory

    • cacheBlock

      public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory, boolean waitWhenCache)
      Description copied from interface: BlockCache
      Add block to cache.
      Specified by:
      cacheBlock in interface BlockCache
      Parameters:
      cacheKey - The block's cache key.
      buf - The block contents wrapped in a ByteBuffer.
      inMemory - Whether block should be treated as in-memory
      waitWhenCache - Whether to wait for the cache to be flushed mainly when BucketCache is configured.

    • cacheBlock

      public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf)
      Description copied from interface: BlockCache
      Add block to cache (defaults to not in-memory).
      Specified by:
      cacheBlock in interface BlockCache
      Parameters:
      cacheKey - The block's cache key.
      buf - The object to cache.

    • getBlock

      public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean repeat, boolean updateCacheMetrics)
      Description copied from interface: BlockCache
      Fetch block from cache.
      Specified by:
      getBlock in interface BlockCache
      Parameters:
      cacheKey - Block to fetch.
      caching - Whether this request has caching enabled (used for stats)
      repeat - Whether this is a repeat lookup for the same block (used to avoid double counting cache misses when doing double-check locking)
      updateCacheMetrics - Whether to update cache metrics or not
      Returns:
      Block or null if block is not in 2 cache.

    • updateBlockMetrics

      private void updateBlockMetrics(Cacheable block, BlockCacheKey key, BlockCache cache, boolean caching)

    • getBlock

      public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean repeat, boolean updateCacheMetrics, BlockType blockType)
      Description copied from interface: BlockCache
      Fetch block from cache.
      Specified by:
      getBlock in interface BlockCache
      Parameters:
      cacheKey - Block to fetch.
      caching - Whether this request has caching enabled (used for stats)
      repeat - Whether this is a repeat lookup for the same block (used to avoid double counting cache misses when doing double-check locking)
      updateCacheMetrics - Whether to update cache metrics or not
      blockType - BlockType
      Returns:
      Block or null if block is not in 2 cache.

    • getBlockWithType

      private Cacheable getBlockWithType(BlockCacheKey cacheKey, boolean caching, boolean repeat, boolean updateCacheMetrics)

    • evictBlock

      public boolean evictBlock(BlockCacheKey cacheKey)
      Description copied from interface: BlockCache
      Evict block from cache.
      Specified by:
      evictBlock in interface BlockCache
      Parameters:
      cacheKey - Block to evict
      Returns:
      true if block existed and was evicted, false if not

    • evictBlocksByHfileName

      public int evictBlocksByHfileName(String hfileName)
      Description copied from interface: BlockCache
      Evicts all blocks for the given HFile.
      Specified by:
      evictBlocksByHfileName in interface BlockCache
      Returns:
      the number of blocks evicted

    • getStats

      public CacheStats getStats()
      Description copied from interface: BlockCache
      Get the statistics for this block cache.
      Specified by:
      getStats in interface BlockCache

    • shutdown

      public void shutdown()
      Description copied from interface: BlockCache
      Shutdown the cache.
      Specified by:
      shutdown in interface BlockCache

    • size

      public long size()
      Description copied from interface: BlockCache
      Returns the total size of the block cache, in bytes.
      Specified by:
      size in interface BlockCache
      Returns:
      size of cache, in bytes

    • getMaxSize

      public long getMaxSize()
      Description copied from interface: BlockCache
      Returns the Max size of the block cache, in bytes.
      Specified by:
      getMaxSize in interface BlockCache
      Returns:
      size of cache, in bytes

    • getCurrentDataSize

      public long getCurrentDataSize()
      Description copied from interface: BlockCache
      Returns the occupied size of data blocks, in bytes.
      Specified by:
      getCurrentDataSize in interface BlockCache
      Returns:
      occupied space in cache, in bytes

    • getFreeSize

      public long getFreeSize()
      Description copied from interface: BlockCache
      Returns the free size of the block cache, in bytes.
      Specified by:
      getFreeSize in interface BlockCache
      Returns:
      free space in cache, in bytes

    • getCurrentSize

      public long getCurrentSize()
      Description copied from interface: BlockCache
      Returns the occupied size of the block cache, in bytes.
      Specified by:
      getCurrentSize in interface BlockCache
      Returns:
      occupied space in cache, in bytes

    • getBlockCount

      public long getBlockCount()
      Description copied from interface: BlockCache
      Returns the number of blocks currently cached in the block cache.
      Specified by:
      getBlockCount in interface BlockCache
      Returns:
      number of blocks in the cache

    • getDataBlockCount

      public long getDataBlockCount()
      Description copied from interface: BlockCache
      Returns the number of data blocks currently cached in the block cache.
      Specified by:
      getDataBlockCount in interface BlockCache
      Returns:
      number of blocks in the cache

    • iterator

      public Iterator<CachedBlock> iterator()
      Description copied from interface: BlockCache
      Returns Iterator over the blocks in the cache.
      Specified by:
      iterator in interface BlockCache
      Specified by:
      iterator in interface Iterable<CachedBlock>

    • getBlockCaches

      public BlockCache[] getBlockCaches()
      Description copied from interface: BlockCache
      Returns The list of sub blockcaches that make up this one; returns null if no sub caches.
      Specified by:
      getBlockCaches in interface BlockCache

    • getFullyCachedFiles

      public Optional<Map<String,Pair<String,Long>>> getFullyCachedFiles()
      Returns the list of fully cached files
      Specified by:
      getFullyCachedFiles in interface BlockCache
      Returns:
      empty optional if this method is not supported, otherwise the returned optional contains a map of all files that have been fully cached.

    • getRegionCachedInfo

      public Optional<Map<String,Long>> getRegionCachedInfo()
      Description copied from interface: BlockCache
      Returns an Optional containing a map of regions and the percentage of how much of it has been cached so far.
      Specified by:
      getRegionCachedInfo in interface BlockCache
      Returns:
      empty optional if this method is not supported, otherwise the returned optional contains a map of current regions caching percentage.

    • setMaxSize

      public void setMaxSize(long size)
      Description copied from interface: ResizableBlockCache
      Sets the max heap size that can be used by the BlockCache.
      Specified by:
      setMaxSize in interface ResizableBlockCache
      Parameters:
      size - The max heap size.

    • getRpcRefCount

      public int getRpcRefCount(BlockCacheKey cacheKey)

    • getFirstLevelCache

      public FirstLevelBlockCache getFirstLevelCache()

    • getSecondLevelCache

      public BlockCache getSecondLevelCache()

    • notifyFileCachingCompleted

      public void notifyFileCachingCompleted(org.apache.hadoop.fs.Path fileName, int totalBlockCount, int dataBlockCount, long size)
      Description copied from interface: BlockCache
      Notifies the cache implementation that the given file has been fully cached (all its blocks made into the cache).
      Specified by:
      notifyFileCachingCompleted in interface BlockCache
      Parameters:
      fileName - the file that has been completely cached.
      totalBlockCount - the total of blocks cached for this file.
      dataBlockCount - number of DATA block type cached.
      size - the size, in bytes, cached.

    • onConfigurationChange

      public void onConfigurationChange(org.apache.hadoop.conf.Configuration config)
      Description copied from interface: BlockCache
      Allows for BlockCache implementations to provide a mean to refresh their configurations. Since HBASE-29249, CacheConfig implements PropagatingConfigurationObserver and registers itself together with the used BlockCache implementation for notifications of dynamic configuration changes. The default is a noop.
      Specified by:
      onConfigurationChange in interface BlockCache
      Specified by:
      onConfigurationChange in interface ConfigurationObserver
      Parameters:
      config - the new configuration to be updated.

    • blockFitsIntoTheCache

      public Optional<Boolean> blockFitsIntoTheCache(HFileBlock block)
      Description copied from interface: BlockCache
      Checks whether there's enough space left in the cache to accommodate the passed block. This method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty. For subclasses implementing this logic, the returned Optional would contain the boolean value reflecting if the passed block fits into the remaining cache space available.
      Specified by:
      blockFitsIntoTheCache in interface BlockCache
      Parameters:
      block - the block we want to check if fits into the cache.
      Returns:
      empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the block fits into the cache available space.

    • shouldCacheFile

      public Optional<Boolean> shouldCacheFile(HFileInfo hFileInfo, org.apache.hadoop.conf.Configuration conf)
      Description copied from interface: BlockCache
      Checks whether blocks for the passed file should be cached or not. This method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty. For subclasses implementing this logic, the returned Optional would contain the boolean value reflecting if the passed file should indeed be cached.
      Specified by:
      shouldCacheFile in interface BlockCache
      Parameters:
      hFileInfo - Information about the file to check if it should be cached.
      conf - The configuration object to use for determining caching behavior.
      Returns:
      empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the file should be cached.

    • shouldCacheBlock

      public Optional<Boolean> shouldCacheBlock(BlockCacheKey key, long maxTimeStamp, org.apache.hadoop.conf.Configuration conf)
      Description copied from interface: BlockCache
      Checks whether the block represented by the given key should be cached or not. This method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty. For subclasses implementing this logic, the returned Optional would contain the boolean value reflecting if the passed block should indeed be cached.
      Specified by:
      shouldCacheBlock in interface BlockCache
      Parameters:
      key - The key representing the block to check if it should be cached.
      maxTimeStamp - The maximum timestamp for the block to check if it should be cached.
      conf - The configuration object to use for determining caching behavior.
      Returns:
      An empty Optional if this method is not supported; otherwise, the returned Optional contains the boolean value indicating if the block should be cached.

    • combineCacheResults

      private Optional<Boolean> combineCacheResults(Optional<Boolean> result1, Optional<Boolean> result2)

    • isAlreadyCached

      public Optional<Boolean> isAlreadyCached(BlockCacheKey key)
      Description copied from interface: BlockCache
      Checks whether the block for the passed key is already cached. This method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty. For subclasses implementing this logic, the returned Optional would contain the boolean value reflecting if the block for the passed key is already cached or not.
      Specified by:
      isAlreadyCached in interface BlockCache
      Parameters:
      key - for the block we want to check if it's already in the cache.
      Returns:
      empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the block is already cached.

    • getBlockSize

      public Optional<Integer> getBlockSize(BlockCacheKey key)
      Description copied from interface: BlockCache
      Returns an Optional containing the size of the block related to the passed key. If the block is not in the cache, returned optional will be empty. Also, this method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty.
      Specified by:
      getBlockSize in interface BlockCache
      Parameters:
      key - for the block we want to check if it's already in the cache.
      Returns:
      empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the block is already cached.

    • evictBlocksRangeByHfileName

      public int evictBlocksRangeByHfileName(String hfileName, long initOffset, long endOffset)
      Description copied from interface: BlockCache
      Evict all blocks for the given file name between the passed offset values.
      Specified by:
      evictBlocksRangeByHfileName in interface BlockCache
      Parameters:
      hfileName - The file for which blocks should be evicted.
      initOffset - the initial offset for the range of blocks to be evicted.
      endOffset - the end offset for the range of blocks to be evicted.
      Returns:
      number of blocks evicted.

    • waitForCacheInitialization

      public boolean waitForCacheInitialization(long timeout)
      Description copied from interface: BlockCache
      Wait for the block cache implementation to be completely enabled. Some block cache implementations may take longer to initialise, and this initialisation may be asynchronous.
      Specified by:
      waitForCacheInitialization in interface BlockCache
      Parameters:
      timeout - time to wait for the cache to become enabled.
      Returns:
      boolean true if the cache is enabled, false otherwise.

    • isCacheEnabled

      public boolean isCacheEnabled()
      Description copied from interface: BlockCache
      API to check whether or not, the cache is enabled.
      Specified by:
      isCacheEnabled in interface BlockCache
      Returns:
      returns true if the cache is enabled, false otherwise.