org.apache.hadoop.hbase.master.normalizer

Class SimpleRegionNormalizer

java.lang.Object

org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, ConfigurationObserver, RegionNormalizer

@InterfaceAudience.LimitedPrivate(value="Configuration")
class SimpleRegionNormalizer
extends Object
implements RegionNormalizer, ConfigurationObserver

Simple implementation of region normalizer. Logic in use:

1. Get all regions of a given table
2. Get avg size S of the regions in the table (by total size of store files reported in RegionMetrics)
3. For each region R0, if R0 is bigger than S * 2, it is kindly requested to split.
4. Otherwise, for the next region in the chain R1, if R0 + R1 is smaller then S, R0 and R1 are kindly requested to merge.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	SimpleRegionNormalizer.NormalizeContext Inner class caries the state necessary to perform a single invocation of computePlansForTable(TableName).
private static class	SimpleRegionNormalizer.NormalizerConfiguration Holds the configuration values read from Configuration.

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static boolean	DEFAULT_MERGE_ENABLED
(package private) static int	DEFAULT_MERGE_MIN_REGION_AGE_DAYS
(package private) static int	DEFAULT_MERGE_MIN_REGION_SIZE_MB
(package private) static int	DEFAULT_MIN_REGION_COUNT
(package private) static boolean	DEFAULT_SPLIT_ENABLED
private static org.slf4j.Logger	LOG
private MasterServices	masterServices
(package private) static String	MERGE_ENABLED_KEY
(package private) static String	MERGE_MIN_REGION_AGE_DAYS_KEY
(package private) static String	MERGE_MIN_REGION_SIZE_MB_KEY
(package private) static String	MIN_REGION_COUNT_KEY
private SimpleRegionNormalizer.NormalizerConfiguration	normalizerConfiguration
(package private) static String	SPLIT_ENABLED_KEY

Constructor Summary

Constructors

Constructor and Description
SimpleRegionNormalizer()

Method Summary

Modifier and Type	Method and Description
private List<NormalizationPlan>	computeMergeNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx) Computes the merge plans that should be executed for this table to converge average region towards target average or target region count.
List<NormalizationPlan>	computePlansForTable(TableName table) Computes a list of normalizer actions to perform on the target table.
private List<NormalizationPlan>	computeSplitNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx) Computes the split plans that should be executed for this table to converge average region size towards target average or target region count.
private double	getAverageRegionSizeMb(List<RegionInfo> tableRegions)
org.apache.hadoop.conf.Configuration	getConf()
Period	getMergeMinRegionAge() Return this instance's configured value for "hbase.normalizer.merge.min_region_age.days".
long	getMergeMinRegionSizeMb() Return this instance's configured value for "hbase.normalizer.merge.min_region_size.mb".
int	getMinRegionCount() Return this instance's configured value for "hbase.normalizer.min.region.count".
private long	getRegionSizeMB(RegionInfo hri)
private boolean	isLargeEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration, RegionInfo regionInfo) Return true when regionInfo has a size that is sufficient to be considered for a merge operation, false otherwise.
private boolean	isMasterSwitchEnabled(MasterSwitchType masterSwitchType)
boolean	isMergeEnabled() Return this instance's configured value for "hbase.normalizer.merge.enabled".
private static boolean	isOldEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration, RegionInfo regionInfo) Return true when regionInfo has a creation date that is old enough to be considered for a merge operation, false otherwise.
boolean	isSplitEnabled() Return this instance's configured value for "hbase.normalizer.split.enabled".
private static <T> void	logConfigurationUpdated(String key, T oldValue, T newValue)
private static boolean	logTraceReason(BooleanSupplier predicate, String fmtWhenTrue, Object... args)
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf) This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
private static Period	parseMergeMinRegionAge(org.apache.hadoop.conf.Configuration conf)
private static long	parseMergeMinRegionSizeMb(org.apache.hadoop.conf.Configuration conf)
private static int	parseMinRegionCount(org.apache.hadoop.conf.Configuration conf)
private boolean	proceedWithMergePlanning()
private boolean	proceedWithSplitPlanning()
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setMasterServices(MasterServices masterServices) Set the master service.
private boolean	skipForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration, RegionStates regionStates, RegionInfo regionInfo) Determine if a RegionInfo should be considered for a merge operation.
private static boolean	skipForSplit(RegionState state, RegionInfo regionInfo) Determine if a region in RegionState should be considered for a split operation.
private static <T> void	warnInvalidValue(String key, T parsedValue, T settledValue)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

SPLIT_ENABLED_KEY

static final String SPLIT_ENABLED_KEY

See Also:

Constant Field Values

DEFAULT_SPLIT_ENABLED

static final boolean DEFAULT_SPLIT_ENABLED

See Also:

Constant Field Values

MERGE_ENABLED_KEY

static final String MERGE_ENABLED_KEY

See Also:

Constant Field Values

DEFAULT_MERGE_ENABLED

static final boolean DEFAULT_MERGE_ENABLED

See Also:

Constant Field Values

MIN_REGION_COUNT_KEY

static final String MIN_REGION_COUNT_KEY

See Also:

Constant Field Values

DEFAULT_MIN_REGION_COUNT

static final int DEFAULT_MIN_REGION_COUNT

See Also:

Constant Field Values

MERGE_MIN_REGION_AGE_DAYS_KEY

static final String MERGE_MIN_REGION_AGE_DAYS_KEY

See Also:

Constant Field Values

DEFAULT_MERGE_MIN_REGION_AGE_DAYS

static final int DEFAULT_MERGE_MIN_REGION_AGE_DAYS

See Also:

Constant Field Values

MERGE_MIN_REGION_SIZE_MB_KEY

static final String MERGE_MIN_REGION_SIZE_MB_KEY

See Also:

Constant Field Values

DEFAULT_MERGE_MIN_REGION_SIZE_MB

static final int DEFAULT_MERGE_MIN_REGION_SIZE_MB

See Also:

Constant Field Values

masterServices

private MasterServices masterServices

normalizerConfiguration

private SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration

Constructor Detail

SimpleRegionNormalizer

public SimpleRegionNormalizer()

Method Detail

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

parseMinRegionCount

private static int parseMinRegionCount(org.apache.hadoop.conf.Configuration conf)

parseMergeMinRegionAge

private static Period parseMergeMinRegionAge(org.apache.hadoop.conf.Configuration conf)

parseMergeMinRegionSizeMb

private static long parseMergeMinRegionSizeMb(org.apache.hadoop.conf.Configuration conf)

warnInvalidValue

private static <T> void warnInvalidValue(String key,
                                         T parsedValue,
                                         T settledValue)

logConfigurationUpdated

private static <T> void logConfigurationUpdated(String key,
                                                T oldValue,
                                                T newValue)

isSplitEnabled

public boolean isSplitEnabled()

Return this instance's configured value for "hbase.normalizer.split.enabled".

isMergeEnabled

public boolean isMergeEnabled()

Return this instance's configured value for "hbase.normalizer.merge.enabled".

getMinRegionCount

public int getMinRegionCount()

Return this instance's configured value for "hbase.normalizer.min.region.count".

getMergeMinRegionAge

public Period getMergeMinRegionAge()

Return this instance's configured value for "hbase.normalizer.merge.min_region_age.days".

getMergeMinRegionSizeMb

public long getMergeMinRegionSizeMb()

Return this instance's configured value for "hbase.normalizer.merge.min_region_size.mb".

setMasterServices

public void setMasterServices(MasterServices masterServices)

Description copied from interface: RegionNormalizer

Set the master service. Must be called before first call to RegionNormalizer.computePlansForTable(TableName).

Specified by:

setMasterServices in interface RegionNormalizer

Parameters:

masterServices - master services to use

computePlansForTable

public List<NormalizationPlan> computePlansForTable(TableName table)

Description copied from interface: RegionNormalizer

Computes a list of normalizer actions to perform on the target table. This is the primary entry-point from the Master driving a normalization activity.

Specified by:

computePlansForTable in interface RegionNormalizer

Parameters:

table - table to normalize

Returns:

A list of the normalization actions to perform, or an empty list if there's nothing to do.

getRegionSizeMB

private long getRegionSizeMB(RegionInfo hri)

Returns:

size of region in MB and if region is not found than -1

isMasterSwitchEnabled

private boolean isMasterSwitchEnabled(MasterSwitchType masterSwitchType)

proceedWithSplitPlanning

private boolean proceedWithSplitPlanning()

proceedWithMergePlanning

private boolean proceedWithMergePlanning()

getAverageRegionSizeMb

private double getAverageRegionSizeMb(List<RegionInfo> tableRegions)

Parameters:

tableRegions - regions of table to normalize

Returns:

average region size depending on

See Also:

Also make sure tableRegions contains regions of the same table

skipForMerge

private boolean skipForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration,
                             RegionStates regionStates,
                             RegionInfo regionInfo)

Determine if a RegionInfo should be considered for a merge operation.

Callers beware: for safe concurrency, be sure to pass in the local instance of SimpleRegionNormalizer.NormalizerConfiguration, don't use this's instance.

computeMergeNormalizationPlans

private List<NormalizationPlan> computeMergeNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx)

Computes the merge plans that should be executed for this table to converge average region towards target average or target region count.

skipForSplit

private static boolean skipForSplit(RegionState state,
                                    RegionInfo regionInfo)

Determine if a region in RegionState should be considered for a split operation.

computeSplitNormalizationPlans

private List<NormalizationPlan> computeSplitNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx)

Computes the split plans that should be executed for this table to converge average region size towards target average or target region count.
if the region is > 2 times larger than average, we split it. split is more high priority normalization action than merge.

isOldEnoughForMerge

private static boolean isOldEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration,
                                           RegionInfo regionInfo)

Return true when regionInfo has a creation date that is old enough to be considered for a merge operation, false otherwise.

isLargeEnoughForMerge

private boolean isLargeEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration,
                                      RegionInfo regionInfo)

Return true when regionInfo has a size that is sufficient to be considered for a merge operation, false otherwise.

Callers beware: for safe concurrency, be sure to pass in the local instance of SimpleRegionNormalizer.NormalizerConfiguration, don't use this's instance.

logTraceReason

private static boolean logTraceReason(BooleanSupplier predicate,
                                      String fmtWhenTrue,
                                      Object... args)