org.apache.hadoop.hbase.util

Class RegionMover

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.RegionMover

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

@InterfaceAudience.Public
public class RegionMover
extends AbstractHBaseTool
implements Closeable

Tool for loading/unloading regions to/from given regionserver This tool can be run from Command line directly as a utility. Supports Ack/No Ack mode for loading/unloading operations.Ack mode acknowledges if regions are online after movement while noAck mode is best effort mode that improves performance but will still move on if region is stuck/not moved. Motivation behind noAck mode being RS shutdown where even if a Region is stuck, upon shutdown master will move it anyways. This can also be used by constructiong an Object using the builder and then calling load() or unload() methods for the desired operations.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	RegionMover.MoveWithAck Move Regions and make sure that they are up on the target server.If a region movement fails we exit as failure
private class	RegionMover.MoveWithoutAck Move Regions without Acknowledging.Usefule in case of RS shutdown as we might want to shut the RS down anyways and not abort on a stuck region.
static class	RegionMover.RegionMoverBuilder Builder for Region mover.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field and Description
private boolean	ack
private Admin	admin
private Connection	conn
static int	DEFAULT_MOVE_RETRIES_MAX
static int	DEFAULT_MOVE_WAIT_MAX
static int	DEFAULT_SERVERSTART_WAIT_MAX
private String	excludeFile
private String	filename
private String	hostname
private String	loadUnload
private static org.slf4j.Logger	LOG
private int	maxthreads
static String	MOVE_RETRIES_MAX_KEY
static String	MOVE_WAIT_MAX_KEY
private int	port
private RegionMover.RegionMoverBuilder	rmbuilder
static String	SERVERSTART_WAIT_MAX_KEY
private int	timeout

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Modifier	Constructor and Description
private	RegionMover()
private	RegionMover(RegionMover.RegionMoverBuilder builder)

Method Summary

Modifier and Type	Method and Description
protected void	addOptions() Override this to add command-line options using AbstractHBaseTool.addOptWithArg(java.lang.String, java.lang.String) and similar methods.
void	close()
private void	deleteFile(String filename)
protected int	doWork() The "main function" of the tool
private ServerName	getServerNameForRegion(RegionInfo region) Get servername that is up in hbase:meta hosting the given region.
private ServerName	getTargetServer()
private boolean	isSameServer(RegionInfo region, ServerName serverName) Returns true if passed region is still on serverName when we look at hbase:meta.
private void	isSuccessfulScan(RegionInfo region) Tries to scan a row from passed region
boolean	load() Loads the specified hostname with regions listed in the filename RegionMover Object has to be created using RegionMover(RegionMoverBuilder)
private void	loadRegions(List<RegionInfo> regionsToMove)
static void	main(String[] args)
protected void	processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd) This method is called to process the options after they have been parsed.
private List<String>	readExcludes(String excludeFile)
private List<RegionInfo>	readRegionsFromFile(String filename)
private void	stripExcludes(List<ServerName> regionServers) Excludes the servername whose hostname and port portion matches the list given in exclude file
private void	stripMaster(List<ServerName> regionServers) Exclude master from list of RSs to move regions to
private ServerName	stripServer(List<ServerName> regionServers, String hostname, int port) Remove the servername whose hostname and port portion matches from the passed array of servers.
boolean	unload() Unload regions from given hostname using ack/noAck mode and maxthreads.In noAck mode we do not make sure that region is successfully online on the target region server,hence it is best effort.We do not unload regions to hostnames given in excludeFile.
private void	unloadRegions(ServerName server, List<ServerName> regionServers, List<RegionInfo> movedRegions)
private void	waitMoveTasksToFinish(ExecutorService moveRegionsPool, List<Future<Boolean>> taskList, long timeoutInSeconds)
private boolean	waitTaskToFinish(ExecutorService pool, Future<Boolean> task, String operation)
private void	writeFile(String filename, List<RegionInfo> movedRegions) Write the number of regions moved in the first line followed by regions moved in subsequent lines

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getConf, getOptionAsDouble, getOptionAsInt, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MOVE_RETRIES_MAX_KEY

public static final String MOVE_RETRIES_MAX_KEY

See Also:

Constant Field Values

MOVE_WAIT_MAX_KEY

public static final String MOVE_WAIT_MAX_KEY

See Also:

Constant Field Values

SERVERSTART_WAIT_MAX_KEY

public static final String SERVERSTART_WAIT_MAX_KEY

See Also:

Constant Field Values

DEFAULT_MOVE_RETRIES_MAX

public static final int DEFAULT_MOVE_RETRIES_MAX

See Also:

Constant Field Values

DEFAULT_MOVE_WAIT_MAX

public static final int DEFAULT_MOVE_WAIT_MAX

See Also:

Constant Field Values

DEFAULT_SERVERSTART_WAIT_MAX

public static final int DEFAULT_SERVERSTART_WAIT_MAX

See Also:

Constant Field Values

LOG

private static final org.slf4j.Logger LOG

rmbuilder

private RegionMover.RegionMoverBuilder rmbuilder

ack

private boolean ack

maxthreads

private int maxthreads

timeout

private int timeout

loadUnload

private String loadUnload

hostname

private String hostname

filename

private String filename

excludeFile

private String excludeFile

port

private int port

conn

private Connection conn

admin

private Admin admin

Constructor Detail

RegionMover

private RegionMover(RegionMover.RegionMoverBuilder builder)
             throws IOException

Throws:

IOException

RegionMover

private RegionMover()

Method Detail

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

load

public boolean load()
             throws ExecutionException,
                    InterruptedException,
                    TimeoutException

Loads the specified hostname with regions listed in the filename RegionMover Object has to be created using RegionMover(RegionMoverBuilder)

Returns:

true if loading succeeded, false otherwise

Throws:

ExecutionException

InterruptedException

TimeoutException

loadRegions

private void loadRegions(List<RegionInfo> regionsToMove)
                  throws Exception

Throws:

Exception

unload

public boolean unload()
               throws InterruptedException,
                      ExecutionException,
                      TimeoutException

Unload regions from given hostname using ack/noAck mode and maxthreads.In noAck mode we do not make sure that region is successfully online on the target region server,hence it is best effort.We do not unload regions to hostnames given in excludeFile.

Returns:

true if unloading succeeded, false otherwise

Throws:

InterruptedException

ExecutionException

TimeoutException

unloadRegions

private void unloadRegions(ServerName server,
                           List<ServerName> regionServers,
                           List<RegionInfo> movedRegions)
                    throws Exception

Throws:

Exception

waitTaskToFinish

private boolean waitTaskToFinish(ExecutorService pool,
                                 Future<Boolean> task,
                                 String operation)
                          throws TimeoutException,
                                 InterruptedException,
                                 ExecutionException

Throws:

TimeoutException

InterruptedException

ExecutionException

waitMoveTasksToFinish

private void waitMoveTasksToFinish(ExecutorService moveRegionsPool,
                                   List<Future<Boolean>> taskList,
                                   long timeoutInSeconds)
                            throws Exception

Throws:

Exception

getTargetServer

private ServerName getTargetServer()
                            throws Exception

Throws:

Exception

readRegionsFromFile

private List<RegionInfo> readRegionsFromFile(String filename)
                                      throws IOException

Throws:

IOException

writeFile

private void writeFile(String filename,
                       List<RegionInfo> movedRegions)
                throws IOException

Write the number of regions moved in the first line followed by regions moved in subsequent lines

Throws:

IOException

deleteFile

private void deleteFile(String filename)

readExcludes

private List<String> readExcludes(String excludeFile)
                           throws IOException

Returns:

List of servers from the exclude file in format 'hostname:port'.

Throws:

IOException

stripExcludes

private void stripExcludes(List<ServerName> regionServers)
                    throws IOException

Excludes the servername whose hostname and port portion matches the list given in exclude file

Throws:

IOException

stripMaster

private void stripMaster(List<ServerName> regionServers)
                  throws IOException

Exclude master from list of RSs to move regions to

Throws:

IOException

stripServer

private ServerName stripServer(List<ServerName> regionServers,
                               String hostname,
                               int port)

Remove the servername whose hostname and port portion matches from the passed array of servers. Returns as side-effect the servername removed.

Returns:

server removed from list of Region Servers

isSuccessfulScan

private void isSuccessfulScan(RegionInfo region)
                       throws IOException

Tries to scan a row from passed region

Throws:

IOException

isSameServer

private boolean isSameServer(RegionInfo region,
                             ServerName serverName)
                      throws IOException

Returns true if passed region is still on serverName when we look at hbase:meta.

Returns:

true if region is hosted on serverName otherwise false

Throws:

IOException

getServerNameForRegion

private ServerName getServerNameForRegion(RegionInfo region)
                                   throws IOException

Get servername that is up in hbase:meta hosting the given region. this is hostname + port + startcode comma-delimited. Can return null

Returns:

regionServer hosting the given region

Throws:

IOException

addOptions

protected void addOptions()

Description copied from class: AbstractHBaseTool

Override this to add command-line options using AbstractHBaseTool.addOptWithArg(java.lang.String, java.lang.String) and similar methods.

Specified by:

addOptions in class AbstractHBaseTool

processOptions

protected void processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)

Description copied from class: AbstractHBaseTool

This method is called to process the options after they have been parsed.

Specified by:

processOptions in class AbstractHBaseTool

doWork

protected int doWork()
              throws Exception

Description copied from class: AbstractHBaseTool

The "main function" of the tool

Specified by:

doWork in class AbstractHBaseTool

Throws:

Exception

main

public static void main(String[] args)