Package org.apache.hadoop.hbase.io.util

Class MemorySizeUtil

java.lang.Object
org.apache.hadoop.hbase.io.util.MemorySizeUtil
@Private public class MemorySizeUtil extends Object
Util class to calculate memory size for memstore(on heap, off heap), block cache(L1, L2) of RS.

  • Field Details

  • Constructor Details

  • Method Details

    • safeGetHeapMemoryUsage

      public static MemoryUsage safeGetHeapMemoryUsage()
      Return JVM memory statistics while properly handling runtime exceptions from the JVM.
      Returns:
      a memory usage object, null if there was a runtime exception. (n.b. you could also get -1 values back from the JVM)
      See Also:

    • validateRegionServerHeapMemoryAllocation

      public static void validateRegionServerHeapMemoryAllocation(org.apache.hadoop.conf.Configuration conf)
      Validates that heap allocations for MemStore and block cache do not exceed the allowed limit, ensuring enough free heap remains for other RegionServer tasks.
      Parameters:
      conf - the configuration to validate
      Throws:
      RuntimeException - if the combined allocation exceeds the threshold

    • getRegionServerMinFreeHeapInBytes

      private static long getRegionServerMinFreeHeapInBytes(org.apache.hadoop.conf.Configuration conf)
      Retrieve an explicit minimum required free heap size in bytes in the configuration.
      Parameters:
      conf - used to read configs
      Returns:
      the minimum required free heap size in bytes, or a negative value if not configured.
      Throws:
      IllegalArgumentException - if HBASE_REGION_SERVER_FREE_HEAP_MIN_MEMORY_SIZE_KEY format is invalid

    • getRegionServerMinFreeHeapFraction

      public static float getRegionServerMinFreeHeapFraction(org.apache.hadoop.conf.Configuration conf)
      Returns the minimum required free heap as a fraction of total heap.

    • getGlobalMemStoreHeapPercent

      public static float getGlobalMemStoreHeapPercent(org.apache.hadoop.conf.Configuration c, boolean logInvalid)
      Retrieve global memstore configured size as percentage of total heap.

    • getGlobalMemStoreHeapLowerMark

      public static float getGlobalMemStoreHeapLowerMark(org.apache.hadoop.conf.Configuration conf, boolean honorOldConfig)
      Retrieve configured size for global memstore lower water mark as fraction of global memstore size.

    • getGlobalMemStoreSize

      public static Pair<Long,MemoryType> getGlobalMemStoreSize(org.apache.hadoop.conf.Configuration conf)
      Returns Pair of global memstore size and memory type(ie. on heap or off heap).

    • getOnheapGlobalMemStoreSize

      public static long getOnheapGlobalMemStoreSize(org.apache.hadoop.conf.Configuration conf)
      Returns the onheap global memstore limit based on the config 'hbase.regionserver.global.memstore.size'.
      Returns:
      the onheap global memstore limt

    • getBlockCacheHeapPercent

      public static float getBlockCacheHeapPercent(org.apache.hadoop.conf.Configuration conf)
      Retrieve configured size for on heap block cache as percentage of total heap.

    • getBlockCacheSizeInBytes

      public static long getBlockCacheSizeInBytes(org.apache.hadoop.conf.Configuration conf)
      Retrieve an explicit block cache size in bytes in the configuration.
      Parameters:
      conf - used to read cache configs
      Returns:
      the number of bytes to use for LRU, negative if disabled.
      Throws:
      IllegalArgumentException - if HFILE_BLOCK_CACHE_MEMORY_SIZE_KEY format is invalid

    • getOnHeapCacheSize

      public static long getOnHeapCacheSize(org.apache.hadoop.conf.Configuration conf)
      Parameters:
      conf - used to read cache configs
      Returns:
      the number of bytes to use for LRU, negative if disabled.
      Throws:
      IllegalArgumentException - if HFILE_BLOCK_CACHE_SIZE_KEY is > 1.0

    • getBucketCacheSize

      public static long getBucketCacheSize(org.apache.hadoop.conf.Configuration conf)
      Parameters:
      conf - used to read config for bucket cache size.
      Returns:
      the number of bytes to use for bucket cache, negative if disabled.