Package org.apache.hadoop.hbase.master

Class RegionPlacementMaintainer

java.lang.Object

org.apache.hadoop.hbase.master.RegionPlacementMaintainer

All Implemented Interfaces:

Closeable, AutoCloseable

@Private
public class RegionPlacementMaintainer
extends Object
implements Closeable

A tool that is used for manipulating and viewing favored nodes information for regions. Run with -h to get a list of the options

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	RegionPlacementMaintainer.RandomizedMatrix	Some algorithms for solving the assignment problem may traverse workers or jobs in linear order which may result in skewing the assignments of the first jobs in the matrix toward the last workers in the matrix if the costs are uniform.

Field Summary

Fields

Modifier and Type	Field	Description
private static final float	AVOID_COST
private org.apache.hadoop.conf.Configuration	conf
private AsyncClusterConnection	connection
private final boolean	enforceLocality
private final boolean	enforceMinAssignmentMove
private static final float	LAST_SLOT_COST_PENALTY
private static final org.slf4j.Logger	LOG
private static final float	MAX_COST
private static final float	NOT_CURRENT_HOST_PENALTY
private RackManager	rackManager
private Set<TableName>	targetTableSet
private static boolean	USE_MUNKRES_FOR_PLACING_SECONDARY_AND_TERTIARY

Constructor Summary

Constructors

Constructor	Description
RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf)
RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf, boolean enforceLocality, boolean enforceMinAssignmentMove)

Method Summary

Modifier and Type	Method	Description
void	checkDifferencesWithOldPlan(Map<TableName,Integer> movesPerTable, Map<String,Map<String,Float>> regionLocalityMap, FavoredNodesPlan newPlan)	Compares two plans and check whether the locality dropped or increased (prints the information as a string) also prints the baseline locality
void	close()
private void	genAssignmentPlan(TableName tableName, SnapshotOfRegionAssignmentFromMeta assignmentSnapshot, Map<String,Map<String,Float>> regionLocalityMap, FavoredNodesPlan plan, boolean munkresForSecondaryAndTertiary)	Generate the assignment plan for the existing table
private AsyncClusterConnection	getConnection()
static List<ServerName>	getFavoredNodeList(String favoredNodesStr)
FavoredNodesPlan	getNewAssignmentPlan()
SnapshotOfRegionAssignmentFromMeta	getRegionAssignmentSnapshot()	Returns the new RegionAssignmentSnapshot
Map<TableName,Integer>	getRegionsMovement(FavoredNodesPlan newPlan)	Return how many regions will move per table since their primary RS will change
static void	main(String[] args)
static void	printAssignmentPlan(FavoredNodesPlan plan)	Print the assignment plan to the system output stream
void	printDispersionScores(TableName table, SnapshotOfRegionAssignmentFromMeta snapshot, int numRegions, FavoredNodesPlan newPlan, boolean simplePrint)
private static void	printHelp(org.apache.hbase.thirdparty.org.apache.commons.cli.Options opt)
void	printLocalityAndDispersionForCurrentPlan(Map<String,Map<String,Float>> regionLocalityMap)
void	setTargetTableName(String[] tableNames)
void	updateAssignmentPlan(FavoredNodesPlan plan)
void	updateAssignmentPlanToMeta(FavoredNodesPlan plan)	Update the assignment plan into hbase:meta
private void	updateAssignmentPlanToRegionServers(FavoredNodesPlan plan)	Update the assignment plan to all the region servers
List<AssignmentVerificationReport>	verifyRegionPlacement(boolean isDetailMode)	Verify the region placement is consistent with the assignment plan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

MAX_COST

private static final float MAX_COST

See Also:

• Constant Field Values

AVOID_COST

private static final float AVOID_COST

See Also:

• Constant Field Values

LAST_SLOT_COST_PENALTY

private static final float LAST_SLOT_COST_PENALTY

See Also:

• Constant Field Values

NOT_CURRENT_HOST_PENALTY

private static final float NOT_CURRENT_HOST_PENALTY

See Also:

• Constant Field Values

USE_MUNKRES_FOR_PLACING_SECONDARY_AND_TERTIARY

private static boolean USE_MUNKRES_FOR_PLACING_SECONDARY_AND_TERTIARY

conf

private org.apache.hadoop.conf.Configuration conf

enforceLocality

private final boolean enforceLocality

enforceMinAssignmentMove

private final boolean enforceMinAssignmentMove

rackManager

private RackManager rackManager

targetTableSet

private Set<TableName> targetTableSet

connection

private AsyncClusterConnection connection

Constructor Details

RegionPlacementMaintainer

public RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf)
                          throws IOException

Throws:

IOException

RegionPlacementMaintainer

public RegionPlacementMaintainer(org.apache.hadoop.conf.Configuration conf,
 boolean enforceLocality,
 boolean enforceMinAssignmentMove)

Method Details

printHelp

private static void printHelp(org.apache.hbase.thirdparty.org.apache.commons.cli.Options opt)

getConnection

private AsyncClusterConnection getConnection()
                                      throws IOException

Throws:

IOException

setTargetTableName

public void setTargetTableName(String[] tableNames)

getRegionAssignmentSnapshot

public SnapshotOfRegionAssignmentFromMeta getRegionAssignmentSnapshot()
                                                               throws IOException

Returns the new RegionAssignmentSnapshot

Throws:

IOException

verifyRegionPlacement

public List<AssignmentVerificationReport> verifyRegionPlacement(boolean isDetailMode)
                                                         throws IOException

Verify the region placement is consistent with the assignment plan

Throws:

IOException

genAssignmentPlan

private void genAssignmentPlan(TableName tableName,
 SnapshotOfRegionAssignmentFromMeta assignmentSnapshot,
 Map<String,Map<String,Float>> regionLocalityMap,
 FavoredNodesPlan plan,
 boolean munkresForSecondaryAndTertiary)
                        throws IOException

Generate the assignment plan for the existing table

Parameters:

munkresForSecondaryAndTertiary - if set on true the assignment plan for the tertiary and secondary will be generated with Munkres algorithm, otherwise will be generated using placeSecondaryAndTertiaryRS

Throws:

IOException

getNewAssignmentPlan

public FavoredNodesPlan getNewAssignmentPlan()
                                      throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

printAssignmentPlan

public static void printAssignmentPlan(FavoredNodesPlan plan)

Print the assignment plan to the system output stream

updateAssignmentPlanToMeta

public void updateAssignmentPlanToMeta(FavoredNodesPlan plan)
                                throws IOException

Update the assignment plan into hbase:meta

Parameters:

plan - the assignments plan to be updated into hbase:meta

Throws:

IOException - if cannot update assignment plan in hbase:meta

updateAssignmentPlanToRegionServers

private void updateAssignmentPlanToRegionServers(FavoredNodesPlan plan)
                                          throws IOException

Update the assignment plan to all the region servers

Throws:

IOException

updateAssignmentPlan

public void updateAssignmentPlan(FavoredNodesPlan plan)
                          throws IOException

Throws:

IOException

getRegionsMovement

public Map<TableName,Integer> getRegionsMovement(FavoredNodesPlan newPlan)
                                          throws IOException

Return how many regions will move per table since their primary RS will change

Parameters:

newPlan - - new AssignmentPlan

Returns:

how many primaries will move per table

Throws:

IOException

checkDifferencesWithOldPlan

public void checkDifferencesWithOldPlan(Map<TableName,Integer> movesPerTable,
 Map<String,Map<String,Float>> regionLocalityMap,
 FavoredNodesPlan newPlan)
                                 throws IOException

Compares two plans and check whether the locality dropped or increased (prints the information as a string) also prints the baseline locality

Parameters:

movesPerTable - - how many primary regions will move per table

regionLocalityMap - - locality map from FS

newPlan - - new assignment plan

Throws:

IOException

printDispersionScores

public void printDispersionScores(TableName table,
 SnapshotOfRegionAssignmentFromMeta snapshot,
 int numRegions,
 FavoredNodesPlan newPlan,
 boolean simplePrint)

printLocalityAndDispersionForCurrentPlan

public void printLocalityAndDispersionForCurrentPlan(Map<String,Map<String,Float>> regionLocalityMap)
                                              throws IOException

Throws:

IOException

getFavoredNodeList

public static List<ServerName> getFavoredNodeList(String favoredNodesStr)

Parameters:

favoredNodesStr - The String of favored nodes

Returns:

the list of ServerName for the byte array of favored nodes.

main

public static void main(String[] args)
                 throws IOException

Throws:

IOException