org.apache.hadoop.hbase.regionserver.snapshot

Class RegionServerSnapshotManager

java.lang.Object

org.apache.hadoop.hbase.procedure.ProcedureManager

org.apache.hadoop.hbase.procedure.RegionServerProcedureManager

org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager

@InterfaceAudience.LimitedPrivate(value="Configuration")
 @InterfaceStability.Unstable
public class RegionServerSnapshotManager
extends RegionServerProcedureManager

This manager class handles the work dealing with snapshots for a HRegionServer.

This provides the mechanism necessary to kick off a online snapshot specific Subprocedure that is responsible for the regions being served by this region server. If any failures occur with the subprocedure, the RegionSeverSnapshotManager's subprocedure handler, ProcedureMember, notifies the master's ProcedureCoordinator to abort all others.

On startup, requires start() to be called.

On shutdown, requires stop(boolean) to be called

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	RegionServerSnapshotManager.SnapshotSubprocedureBuilder Build the actual snapshot runner that will do all the 'hard' work
(package private) static class	RegionServerSnapshotManager.SnapshotSubprocedurePool We use the SnapshotSubprocedurePool, a class specific thread pool instead of ExecutorService.

Field Summary

Fields

Modifier and Type	Field and Description
private static String	CONCURENT_SNAPSHOT_TASKS_KEY Maximum number of snapshot region tasks that can run concurrently
private static int	DEFAULT_CONCURRENT_SNAPSHOT_TASKS
private static org.slf4j.Logger	LOG
private ProcedureMember	member
private ProcedureMemberRpcs	memberRpcs
private RegionServerServices	rss
static int	SNAPSHOT_REQUEST_THREADS_DEFAULT # of threads for snapshotting regions on the rs.
static String	SNAPSHOT_REQUEST_THREADS_KEY Conf key for number of request threads to start snapshots on regionservers
private static long	SNAPSHOT_REQUEST_WAKE_MILLIS_DEFAULT Default amount of time to check for errors while regions finish snapshotting
static String	SNAPSHOT_REQUEST_WAKE_MILLIS_KEY Conf key for millis between checks to see if snapshot completed or if there are errors
static long	SNAPSHOT_TIMEOUT_MILLIS_DEFAULT Keep threads alive in request pool for max of 300 seconds
static String	SNAPSHOT_TIMEOUT_MILLIS_KEY Conf key for max time to keep threads in snapshot request pool waiting

Constructor Summary

Constructors

Constructor and Description
RegionServerSnapshotManager()
RegionServerSnapshotManager(org.apache.hadoop.conf.Configuration conf, HRegionServer parent, ProcedureMemberRpcs memberRpc, ProcedureMember procMember) Exposed for testing.

Method Summary

Modifier and Type	Method and Description
Subprocedure	buildSubprocedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) If in a running state, creates the specified subprocedure for handling an online snapshot.
String	getProcedureSignature() Return the unique signature of the procedure.
private List<HRegion>	getRegionsToSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Determine if the snapshot should be handled on this server NOTE: This is racy -- the master expects a list of regionservers.
void	initialize(RegionServerServices rss) Create a default snapshot handler - uses a zookeeper based member controller.
void	start() Start accepting snapshot requests.
void	stop(boolean force) Close this and all running snapshot tasks

Methods inherited from class org.apache.hadoop.hbase.procedure.ProcedureManager

equals, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

CONCURENT_SNAPSHOT_TASKS_KEY

private static final String CONCURENT_SNAPSHOT_TASKS_KEY

Maximum number of snapshot region tasks that can run concurrently

See Also:

Constant Field Values

DEFAULT_CONCURRENT_SNAPSHOT_TASKS

private static final int DEFAULT_CONCURRENT_SNAPSHOT_TASKS

See Also:

Constant Field Values

SNAPSHOT_REQUEST_THREADS_KEY

public static final String SNAPSHOT_REQUEST_THREADS_KEY

Conf key for number of request threads to start snapshots on regionservers

See Also:

Constant Field Values

SNAPSHOT_REQUEST_THREADS_DEFAULT

public static final int SNAPSHOT_REQUEST_THREADS_DEFAULT

# of threads for snapshotting regions on the rs.

See Also:

Constant Field Values

SNAPSHOT_TIMEOUT_MILLIS_KEY

public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY

Conf key for max time to keep threads in snapshot request pool waiting

See Also:

Constant Field Values

SNAPSHOT_TIMEOUT_MILLIS_DEFAULT

public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT

Keep threads alive in request pool for max of 300 seconds

See Also:

Constant Field Values

SNAPSHOT_REQUEST_WAKE_MILLIS_KEY

public static final String SNAPSHOT_REQUEST_WAKE_MILLIS_KEY

Conf key for millis between checks to see if snapshot completed or if there are errors

See Also:

Constant Field Values

SNAPSHOT_REQUEST_WAKE_MILLIS_DEFAULT

private static final long SNAPSHOT_REQUEST_WAKE_MILLIS_DEFAULT

Default amount of time to check for errors while regions finish snapshotting

See Also:

Constant Field Values

rss

private RegionServerServices rss

memberRpcs

private ProcedureMemberRpcs memberRpcs

member

private ProcedureMember member

Constructor Detail

RegionServerSnapshotManager

RegionServerSnapshotManager(org.apache.hadoop.conf.Configuration conf,
                            HRegionServer parent,
                            ProcedureMemberRpcs memberRpc,
                            ProcedureMember procMember)

Exposed for testing.

Parameters:

conf - HBase configuration.

parent - parent running the snapshot handler

memberRpc - use specified memberRpc instance

procMember - use specified ProcedureMember

RegionServerSnapshotManager

public RegionServerSnapshotManager()

Method Detail

start

public void start()

Start accepting snapshot requests.

Specified by:

start in class RegionServerProcedureManager

stop

public void stop(boolean force)
          throws IOException

Close this and all running snapshot tasks

Specified by:

stop in class RegionServerProcedureManager

Parameters:

force - forcefully stop all running tasks

Throws:

IOException

buildSubprocedure

public Subprocedure buildSubprocedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)

If in a running state, creates the specified subprocedure for handling an online snapshot. Because this gets the local list of regions to snapshot and not the set the master had, there is a possibility of a race where regions may be missed. This detected by the master in the snapshot verification step.

Parameters:

snapshot -

Returns:

Subprocedure to submit to the ProcedureMemeber.

getRegionsToSnapshot

private List<HRegion> getRegionsToSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                                    throws IOException

Determine if the snapshot should be handled on this server NOTE: This is racy -- the master expects a list of regionservers. This means if a region moves somewhere between the calls we'll miss some regions. For example, a region move during a snapshot could result in a region to be skipped or done twice. This is manageable because the MasterSnapshotVerifier will double check the region lists after the online portion of the snapshot completes and will explicitly fail the snapshot.

Parameters:

snapshot -

Returns:

the list of online regions. Empty list is returned if no regions are responsible for the given snapshot.

Throws:

IOException

initialize

public void initialize(RegionServerServices rss)
                throws org.apache.zookeeper.KeeperException

Create a default snapshot handler - uses a zookeeper based member controller.

Specified by:

initialize in class RegionServerProcedureManager

Parameters:

rss - region server running the handler

Throws:

org.apache.zookeeper.KeeperException - if the zookeeper cluster cannot be reached

getProcedureSignature

public String getProcedureSignature()

Description copied from class: ProcedureManager

Return the unique signature of the procedure. This signature uniquely identifies the procedure. By default, this signature is the string used in the procedure controller (i.e., the root ZK node name for the procedure)

Specified by:

getProcedureSignature in class ProcedureManager