org.apache.hadoop.hbase.master

Class RegionPlacementMaintainer

java.lang.Object

org.apache.hadoop.hbase.master.RegionPlacementMaintainer

@InterfaceAudience.Private
public class RegionPlacementMaintainer
extends Object

A tool that is used for manipulating and viewing favored nodes information for regions. Run with -h to get a list of the options

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	RegionPlacementMaintainer.RandomizedMatrix Some algorithms for solving the assignment problem may traverse workers or jobs in linear order which may result in skewing the assignments of the first jobs in the matrix toward the last workers in the matrix if the costs are uniform.

Field Summary

Fields

Modifier and Type	Field and Description
private static float	AVOID_COST
private org.apache.hadoop.conf.Configuration	conf
private Connection	connection
private boolean	enforceLocality
private boolean	enforceMinAssignmentMove
private static float	LAST_SLOT_COST_PENALTY
private static org.slf4j.Logger	LOG
private static float	MAX_COST
private static float	NOT_CURRENT_HOST_PENALTY
private RackManager	rackManager
private Set<TableName>	targetTableSet
private static boolean	USE_MUNKRES_FOR_PLACING_SECONDARY_AND_TERTIARY

Constructor Summary

Constructors

Constructor and Description
RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf)
RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf, boolean enforceLocality, boolean enforceMinAssignmentMove)

Method Summary

Modifier and Type	Method and Description
void	checkDifferencesWithOldPlan(Map<TableName,Integer> movesPerTable, Map<String,Map<String,Float>> regionLocalityMap, FavoredNodesPlan newPlan) Compares two plans and check whether the locality dropped or increased (prints the information as a string) also prints the baseline locality
private void	genAssignmentPlan(TableName tableName, SnapshotOfRegionAssignmentFromMeta assignmentSnapshot, Map<String,Map<String,Float>> regionLocalityMap, FavoredNodesPlan plan, boolean munkresForSecondaryAndTertiary) Generate the assignment plan for the existing table
static List<ServerName>	getFavoredNodeList(String favoredNodesStr)
FavoredNodesPlan	getNewAssignmentPlan()
SnapshotOfRegionAssignmentFromMeta	getRegionAssignmentSnapshot()
Map<TableName,Integer>	getRegionsMovement(FavoredNodesPlan newPlan) Return how many regions will move per table since their primary RS will change
static void	main(String[] args)
static void	printAssignmentPlan(FavoredNodesPlan plan) Print the assignment plan to the system output stream
void	printDispersionScores(TableName table, SnapshotOfRegionAssignmentFromMeta snapshot, int numRegions, FavoredNodesPlan newPlan, boolean simplePrint)
private static void	printHelp(org.apache.hbase.thirdparty.org.apache.commons.cli.Options opt)
void	printLocalityAndDispersionForCurrentPlan(Map<String,Map<String,Float>> regionLocalityMap)
void	setTargetTableName(String[] tableNames)
void	updateAssignmentPlan(FavoredNodesPlan plan)
void	updateAssignmentPlanToMeta(FavoredNodesPlan plan) Update the assignment plan into hbase:meta
private void	updateAssignmentPlanToRegionServers(FavoredNodesPlan plan) Update the assignment plan to all the region servers
List<AssignmentVerificationReport>	verifyRegionPlacement(boolean isDetailMode) Verify the region placement is consistent with the assignment plan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

MAX_COST

private static final float MAX_COST

See Also:

Constant Field Values

AVOID_COST

private static final float AVOID_COST

See Also:

Constant Field Values

LAST_SLOT_COST_PENALTY

private static final float LAST_SLOT_COST_PENALTY

See Also:

Constant Field Values

NOT_CURRENT_HOST_PENALTY

private static final float NOT_CURRENT_HOST_PENALTY

See Also:

Constant Field Values

USE_MUNKRES_FOR_PLACING_SECONDARY_AND_TERTIARY

private static boolean USE_MUNKRES_FOR_PLACING_SECONDARY_AND_TERTIARY

conf

private org.apache.hadoop.conf.Configuration conf

enforceLocality

private final boolean enforceLocality

enforceMinAssignmentMove

private final boolean enforceMinAssignmentMove

rackManager

private RackManager rackManager

targetTableSet

private Set<TableName> targetTableSet

connection

private final Connection connection

Constructor Detail

RegionPlacementMaintainer

public RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf)

RegionPlacementMaintainer

public RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf,
                                 boolean enforceLocality,
                                 boolean enforceMinAssignmentMove)

Method Detail

printHelp

private static void printHelp(org.apache.hbase.thirdparty.org.apache.commons.cli.Options opt)

setTargetTableName

public void setTargetTableName(String[] tableNames)

getRegionAssignmentSnapshot

public SnapshotOfRegionAssignmentFromMeta getRegionAssignmentSnapshot()
                                                               throws IOException

Returns:

the new RegionAssignmentSnapshot

Throws:

IOException

verifyRegionPlacement

public List<AssignmentVerificationReport> verifyRegionPlacement(boolean isDetailMode)
                                                         throws IOException

Verify the region placement is consistent with the assignment plan

Parameters:

isDetailMode -

Returns:

reports

Throws:

IOException

genAssignmentPlan

private void genAssignmentPlan(TableName tableName,
                               SnapshotOfRegionAssignmentFromMeta assignmentSnapshot,
                               Map<String,Map<String,Float>> regionLocalityMap,
                               FavoredNodesPlan plan,
                               boolean munkresForSecondaryAndTertiary)
                        throws IOException

Generate the assignment plan for the existing table

Parameters:

tableName -

assignmentSnapshot -

regionLocalityMap -

plan -

munkresForSecondaryAndTertiary - if set on true the assignment plan for the tertiary and secondary will be generated with Munkres algorithm, otherwise will be generated using placeSecondaryAndTertiaryRS

Throws:

IOException

getNewAssignmentPlan

public FavoredNodesPlan getNewAssignmentPlan()
                                      throws IOException

Throws:

IOException

printAssignmentPlan

public static void printAssignmentPlan(FavoredNodesPlan plan)

Print the assignment plan to the system output stream

Parameters:

plan -

updateAssignmentPlanToMeta

public void updateAssignmentPlanToMeta(FavoredNodesPlan plan)
                                throws IOException

Update the assignment plan into hbase:meta

Parameters:

plan - the assignments plan to be updated into hbase:meta

Throws:

IOException - if cannot update assignment plan in hbase:meta

updateAssignmentPlanToRegionServers

private void updateAssignmentPlanToRegionServers(FavoredNodesPlan plan)
                                          throws IOException

Update the assignment plan to all the region servers

Parameters:

plan -

Throws:

IOException

updateAssignmentPlan

public void updateAssignmentPlan(FavoredNodesPlan plan)
                          throws IOException

Throws:

IOException

getRegionsMovement

public Map<TableName,Integer> getRegionsMovement(FavoredNodesPlan newPlan)
                                          throws IOException

Return how many regions will move per table since their primary RS will change

Parameters:

newPlan - - new AssignmentPlan

Returns:

how many primaries will move per table

Throws:

IOException

checkDifferencesWithOldPlan

public void checkDifferencesWithOldPlan(Map<TableName,Integer> movesPerTable,
                                        Map<String,Map<String,Float>> regionLocalityMap,
                                        FavoredNodesPlan newPlan)
                                 throws IOException

Compares two plans and check whether the locality dropped or increased (prints the information as a string) also prints the baseline locality

Parameters:

movesPerTable - - how many primary regions will move per table

regionLocalityMap - - locality map from FS

newPlan - - new assignment plan

Throws:

IOException

printDispersionScores

public void printDispersionScores(TableName table,
                                  SnapshotOfRegionAssignmentFromMeta snapshot,
                                  int numRegions,
                                  FavoredNodesPlan newPlan,
                                  boolean simplePrint)

printLocalityAndDispersionForCurrentPlan

public void printLocalityAndDispersionForCurrentPlan(Map<String,Map<String,Float>> regionLocalityMap)
                                              throws IOException

Throws:

IOException

getFavoredNodeList

public static List<ServerName> getFavoredNodeList(String favoredNodesStr)

Parameters:

favoredNodesStr - The String of favored nodes

Returns:

the list of ServerName for the byte array of favored nodes.

main

public static void main(String[] args)
                 throws IOException

Throws:

IOException