org.apache.hadoop.hbase.io.util

Class MemorySizeUtil

java.lang.Object

org.apache.hadoop.hbase.io.util.MemorySizeUtil

@InterfaceAudience.Private
public class MemorySizeUtil
extends Object

Util class to calculate memory size for memstore, block cache(L1, L2) of RS.

Field Summary

Fields

Modifier and Type	Field and Description
private static int	CONVERT_TO_PERCENTAGE
static float	DEFAULT_MEMSTORE_SIZE
static float	DEFAULT_MEMSTORE_SIZE_LOWER_LIMIT
private static String	JVM_HEAP_EXCEPTION
private static org.slf4j.Logger	LOG
static String	MEMSTORE_SIZE_KEY
static String	MEMSTORE_SIZE_LOWER_LIMIT_KEY
static String	MEMSTORE_SIZE_LOWER_LIMIT_OLD_KEY
static String	MEMSTORE_SIZE_OLD_KEY
static String	OFFHEAP_MEMSTORE_SIZE_KEY

Constructor Summary

Constructors

Constructor and Description
MemorySizeUtil()

Method Summary

Modifier and Type	Method and Description
static void	checkForClusterFreeHeapMemoryLimit(org.apache.hadoop.conf.Configuration conf) Checks whether we have enough heap memory left out after portion for Memstore and Block cache.
static float	getBlockCacheHeapPercent(org.apache.hadoop.conf.Configuration conf) Retrieve configured size for on heap block cache as percentage of total heap.
static long	getBucketCacheSize(org.apache.hadoop.conf.Configuration conf)
static float	getGlobalMemStoreHeapLowerMark(org.apache.hadoop.conf.Configuration conf, boolean honorOldConfig) Retrieve configured size for global memstore lower water mark as fraction of global memstore size.
static float	getGlobalMemStoreHeapPercent(org.apache.hadoop.conf.Configuration c, boolean logInvalid) Retrieve global memstore configured size as percentage of total heap.
static Pair<Long,MemoryType>	getGlobalMemStoreSize(org.apache.hadoop.conf.Configuration conf)
static long	getOnHeapCacheSize(org.apache.hadoop.conf.Configuration conf)
static long	getOnheapGlobalMemStoreSize(org.apache.hadoop.conf.Configuration conf) Returns the onheap global memstore limit based on the config 'hbase.regionserver.global.memstore.size'.
static MemoryUsage	safeGetHeapMemoryUsage() Return JVM memory statistics while properly handling runtime exceptions from the JVM.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MEMSTORE_SIZE_KEY

public static final String MEMSTORE_SIZE_KEY

See Also:

Constant Field Values

MEMSTORE_SIZE_OLD_KEY

public static final String MEMSTORE_SIZE_OLD_KEY

See Also:

Constant Field Values

MEMSTORE_SIZE_LOWER_LIMIT_KEY

public static final String MEMSTORE_SIZE_LOWER_LIMIT_KEY

See Also:

Constant Field Values

MEMSTORE_SIZE_LOWER_LIMIT_OLD_KEY

public static final String MEMSTORE_SIZE_LOWER_LIMIT_OLD_KEY

See Also:

Constant Field Values

OFFHEAP_MEMSTORE_SIZE_KEY

public static final String OFFHEAP_MEMSTORE_SIZE_KEY

See Also:

Constant Field Values

DEFAULT_MEMSTORE_SIZE

public static final float DEFAULT_MEMSTORE_SIZE

See Also:

Constant Field Values

DEFAULT_MEMSTORE_SIZE_LOWER_LIMIT

public static final float DEFAULT_MEMSTORE_SIZE_LOWER_LIMIT

See Also:

Constant Field Values

LOG

private static final org.slf4j.Logger LOG

CONVERT_TO_PERCENTAGE

private static final int CONVERT_TO_PERCENTAGE

See Also:

Constant Field Values

JVM_HEAP_EXCEPTION

private static final String JVM_HEAP_EXCEPTION

See Also:

Constant Field Values

Constructor Detail

MemorySizeUtil

public MemorySizeUtil()

Method Detail

safeGetHeapMemoryUsage

public static MemoryUsage safeGetHeapMemoryUsage()

Return JVM memory statistics while properly handling runtime exceptions from the JVM.

Returns:

a memory usage object, null if there was a runtime exception. (n.b. you could also get -1 values back from the JVM)

See Also:

MemoryUsage

checkForClusterFreeHeapMemoryLimit

public static void checkForClusterFreeHeapMemoryLimit(org.apache.hadoop.conf.Configuration conf)

Checks whether we have enough heap memory left out after portion for Memstore and Block cache. We need atleast 20% of heap left out for other RS functions.

Parameters:

conf -

getGlobalMemStoreHeapPercent

public static float getGlobalMemStoreHeapPercent(org.apache.hadoop.conf.Configuration c,
                                                 boolean logInvalid)

Retrieve global memstore configured size as percentage of total heap.

Parameters:

c -

logInvalid -

getGlobalMemStoreHeapLowerMark

public static float getGlobalMemStoreHeapLowerMark(org.apache.hadoop.conf.Configuration conf,
                                                   boolean honorOldConfig)

Retrieve configured size for global memstore lower water mark as fraction of global memstore size.

getGlobalMemStoreSize

public static Pair<Long,MemoryType> getGlobalMemStoreSize(org.apache.hadoop.conf.Configuration conf)

Returns:

Pair of global memstore size and memory type(ie. on heap or off heap).

getOnheapGlobalMemStoreSize

public static long getOnheapGlobalMemStoreSize(org.apache.hadoop.conf.Configuration conf)

Returns the onheap global memstore limit based on the config 'hbase.regionserver.global.memstore.size'.

Parameters:

conf -

Returns:

the onheap global memstore limt

getBlockCacheHeapPercent

public static float getBlockCacheHeapPercent(org.apache.hadoop.conf.Configuration conf)

Retrieve configured size for on heap block cache as percentage of total heap.

Parameters:

conf -

getOnHeapCacheSize

public static long getOnHeapCacheSize(org.apache.hadoop.conf.Configuration conf)

Parameters:

conf - used to read cache configs

Returns:

the number of bytes to use for LRU, negative if disabled.

Throws:

IllegalArgumentException - if HFILE_BLOCK_CACHE_SIZE_KEY is > 1.0

getBucketCacheSize

public static long getBucketCacheSize(org.apache.hadoop.conf.Configuration conf)

Parameters:

conf - used to read config for bucket cache size. (< 1 is treated as % and > is treated as MiB)

Returns:

the number of bytes to use for bucket cache, negative if disabled.