org.apache.hadoop.hbase.master.snapshot

Class SnapshotManager

java.lang.Object

org.apache.hadoop.hbase.procedure.ProcedureManager

org.apache.hadoop.hbase.procedure.MasterProcedureManager

org.apache.hadoop.hbase.master.snapshot.SnapshotManager

All Implemented Interfaces:

Stoppable

@InterfaceAudience.LimitedPrivate(value="Configuration")
 @InterfaceStability.Unstable
public class SnapshotManager
extends MasterProcedureManager
implements Stoppable

This class manages the procedure of taking and restoring snapshots. There is only one SnapshotManager for the master.

The class provides methods for monitoring in-progress snapshot actions.

Note: Currently there can only be one snapshot being taken at a time over the cluster. This is a simplification in the current implementation.

Field Summary

Fields

Modifier and Type	Field and Description
private ProcedureCoordinator	coordinator
private ExecutorService	executorService
static String	HBASE_SNAPSHOT_ENABLED Enable or disable snapshot support
static String	HBASE_SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLIS Wait time before removing a finished sentinel from the in-progress map NOTE: This is used as a safety auto cleanup.
private boolean	isSnapshotSupported
private static org.slf4j.Logger	LOG
private MasterServices	master
static String	ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION Name of the operation to use in the controller
private Map<TableName,Long>	restoreTableToProcIdMap
private org.apache.hadoop.fs.Path	rootDir
private ScheduledExecutorService	scheduleThreadPool
static String	SNAPSHOT_MAX_FILE_SIZE_PRESERVE Conf key for preserving original max file size configs
static int	SNAPSHOT_POOL_THREADS_DEFAULT number of current operations running on the master
static String	SNAPSHOT_POOL_THREADS_KEY Conf key for # of threads used by the SnapshotManager thread pool
static long	SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLS_DEFAULT
private static int	SNAPSHOT_WAKE_MILLIS_DEFAULT By default, check to see if the snapshot is complete every WAKE MILLIS (ms)
private static String	SNAPSHOT_WAKE_MILLIS_KEY Conf key for # of ms elapsed between checks for snapshot errors while waiting for completion.
private ScheduledFuture<?>	snapshotHandlerChoreCleanerTask
private Map<TableName,SnapshotSentinel>	snapshotHandlers
private boolean	stopped
private ReentrantReadWriteLock	takingSnapshotLock Read write lock between taking snapshot and snapshot HFile cleaner.

Constructor Summary

Constructors

Constructor and Description
SnapshotManager()
SnapshotManager(MasterServices master, ProcedureCoordinator coordinator, ExecutorService pool, int sentinelCleanInterval) Fully specify all necessary components of a snapshot manager.

Method Summary

Modifier and Type	Method and Description
void	checkPermissions(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc, AccessChecker accessChecker, User user) Check for required permissions before executing the procedure.
void	checkSnapshotSupport() Throws an exception if snapshot operations (take a snapshot, restore, clone) are not supported.
private void	checkSnapshotSupport(org.apache.hadoop.conf.Configuration conf, MasterFileSystem mfs) Called at startup, to verify if snapshot operation is supported, and to avoid starting the master if there're snapshots present but the cleaners needed are missing.
private void	cleanupCompletedRestoreInMap() Remove the procedures that are marked as finished
private void	cleanupSentinels() Removes "abandoned" snapshot/restore requests.
private void	cleanupSentinels(Map<TableName,SnapshotSentinel> sentinels) Remove the sentinels that are marked as finished and the completion time has exceeded the removal timeout.
(package private) long	cloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, TableDescriptor tableDescriptor, NonceKey nonceKey, boolean restoreAcl, String customSFT) Clone the specified snapshot into a new table.
private long	cloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot, TableName tableName, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, TableDescriptor snapshotTableDesc, NonceKey nonceKey, boolean restoreAcl, String customSFT) Clone the specified snapshot.
void	deleteSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Delete the specified snapshot n * @throws SnapshotDoesNotExistException If the specified snapshot does not exist.
void	execProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc) Execute a distributed procedure on cluster
List<org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription>	getCompletedSnapshots() Gets the list of all completed snapshots.
private List<org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription>	getCompletedSnapshots(org.apache.hadoop.fs.Path snapshotDir, boolean withCpCall) Gets the list of all completed snapshots.
(package private) ProcedureCoordinator	getCoordinator() Returns distributed commit coordinator for all running snapshots
String	getProcedureSignature() Return the unique signature of the procedure.
ReadWriteLock	getTakingSnapshotLock()
void	initialize(MasterServices master, MetricsMaster metricsMaster) Initialize a globally barriered procedure for master.
boolean	isProcedureDone(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc) Check if the procedure is finished successfully
private boolean	isRestoringTable(TableName tableName) Verify if the restore of the specified table is in progress.
private boolean	isSnapshotCompleted(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Check to see if the snapshot is one of the currently completed snapshots Returns true if the snapshot exists in the "completed snapshots folder".
boolean	isSnapshotDone(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription expected) Check if the specified snapshot is done n * @return true if snapshot is ready to be restored, false if it is still being taken.
boolean	isStopped() Returns True if Stoppable.stop(String) has been closed.
boolean	isTakingAnySnapshot() The snapshot operation processing as following: 1.
(package private) boolean	isTakingSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Check to see if there is a snapshot in progress with the same name or on the same table.
boolean	isTakingSnapshot(TableName tableName) Check to see if the specified table has a snapshot in progress.
private void	prepareToTakeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Check to make sure that we are OK to run the passed snapshot.
private SnapshotSentinel	removeSentinelIfFinished(Map<TableName,SnapshotSentinel> sentinels, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Return the handler if it is currently live and has the same snapshot target name.
private void	resetTempDir() Cleans up any snapshots in the snapshot/.tmp directory that were left from failed snapshot attempts.
long	restoreOrCloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot, NonceKey nonceKey, boolean restoreAcl, String customSFT) Restore or Clone the specified snapshot n * @param nonceKey unique identifier to prevent duplicated RPC n
private long	restoreSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, TableDescriptor tableDescriptor, NonceKey nonceKey, boolean restoreAcl) Restore the specified snapshot.
private long	restoreSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot, TableName tableName, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, TableDescriptor snapshotTableDesc, NonceKey nonceKey, boolean restoreAcl) Restore the specified snapshot.
void	setSnapshotHandlerForTesting(TableName tableName, SnapshotSentinel handler) Set the handler for the current snapshot
private void	snapshotDisabledTable(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Take a snapshot of a disabled table.
private void	snapshotEnabledTable(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Take a snapshot of an enabled table.
private void	snapshotTable(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, TakeSnapshotHandler handler) Take a snapshot using the specified handler.
void	stop(String why) Stop this service.
void	takeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot) Take a snapshot based on the enabled/disabled state of the table.
private void	takeSnapshotInternal(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription	toSnapshotDescription(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc)

Methods inherited from class org.apache.hadoop.hbase.procedure.MasterProcedureManager

execProcedureWithRet

Methods inherited from class org.apache.hadoop.hbase.procedure.ProcedureManager

equals, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

SNAPSHOT_WAKE_MILLIS_DEFAULT

private static final int SNAPSHOT_WAKE_MILLIS_DEFAULT

By default, check to see if the snapshot is complete every WAKE MILLIS (ms)

See Also:

Constant Field Values

HBASE_SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLIS

public static final String HBASE_SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLIS

Wait time before removing a finished sentinel from the in-progress map NOTE: This is used as a safety auto cleanup. The snapshot and restore handlers map entries are removed when a user asks if a snapshot or restore is completed. This operation is part of the HBaseAdmin snapshot/restore API flow. In case something fails on the client side and the snapshot/restore state is not reclaimed after a default timeout, the entry is removed from the in-progress map. At this point, if the user asks for the snapshot/restore status, the result will be snapshot done if exists or failed if it doesn't exists.

See Also:

Constant Field Values

SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLS_DEFAULT

public static final long SNAPSHOT_SENTINELS_CLEANUP_TIMEOUT_MILLS_DEFAULT

See Also:

Constant Field Values

HBASE_SNAPSHOT_ENABLED

public static final String HBASE_SNAPSHOT_ENABLED

Enable or disable snapshot support

See Also:

Constant Field Values

SNAPSHOT_WAKE_MILLIS_KEY

private static final String SNAPSHOT_WAKE_MILLIS_KEY

Conf key for # of ms elapsed between checks for snapshot errors while waiting for completion.

See Also:

Constant Field Values

ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION

public static final String ONLINE_SNAPSHOT_CONTROLLER_DESCRIPTION

Name of the operation to use in the controller

See Also:

Constant Field Values

SNAPSHOT_POOL_THREADS_KEY

public static final String SNAPSHOT_POOL_THREADS_KEY

Conf key for # of threads used by the SnapshotManager thread pool

See Also:

Constant Field Values

SNAPSHOT_POOL_THREADS_DEFAULT

public static final int SNAPSHOT_POOL_THREADS_DEFAULT

number of current operations running on the master

See Also:

Constant Field Values

SNAPSHOT_MAX_FILE_SIZE_PRESERVE

public static final String SNAPSHOT_MAX_FILE_SIZE_PRESERVE

Conf key for preserving original max file size configs

See Also:

Constant Field Values

stopped

private boolean stopped

master

private MasterServices master

coordinator

private ProcedureCoordinator coordinator

isSnapshotSupported

private boolean isSnapshotSupported

snapshotHandlers

private final Map<TableName,SnapshotSentinel> snapshotHandlers

scheduleThreadPool

private final ScheduledExecutorService scheduleThreadPool

snapshotHandlerChoreCleanerTask

private ScheduledFuture<?> snapshotHandlerChoreCleanerTask

restoreTableToProcIdMap

private Map<TableName,Long> restoreTableToProcIdMap

rootDir

private org.apache.hadoop.fs.Path rootDir

executorService

private ExecutorService executorService

takingSnapshotLock

private ReentrantReadWriteLock takingSnapshotLock

Read write lock between taking snapshot and snapshot HFile cleaner. The cleaner should skip to check the HFiles if any snapshot is in progress, otherwise it may clean a HFile which would belongs to the newly creating snapshot. So we should grab the write lock first when cleaner start to work. (See HBASE-21387)

Constructor Detail

SnapshotManager

public SnapshotManager()

SnapshotManager

@InterfaceAudience.Private
SnapshotManager(MasterServices master,
                                           ProcedureCoordinator coordinator,
                                           ExecutorService pool,
                                           int sentinelCleanInterval)
                                    throws IOException,
                                           UnsupportedOperationException

Fully specify all necessary components of a snapshot manager. Exposed for testing.

Parameters:

master - services for the master where the manager is running

coordinator - procedure coordinator instance. exposed for testing.

pool - HBase ExecutorServcie instance, exposed for testing.

Throws:

IOException

UnsupportedOperationException

Method Detail

getCompletedSnapshots

public List<org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription> getCompletedSnapshots()
                                                                                                                 throws IOException

Gets the list of all completed snapshots.

Returns:

list of SnapshotDescriptions

Throws:

IOException - File system exception

getCompletedSnapshots

private List<org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription> getCompletedSnapshots(org.apache.hadoop.fs.Path snapshotDir,
                                                                                                                         boolean withCpCall)
                                                                                                                  throws IOException

Gets the list of all completed snapshots.

Parameters:

snapshotDir - snapshot directory

withCpCall - Whether to call CP hooks

Returns:

list of SnapshotDescriptions

Throws:

IOException - File system exception

resetTempDir

private void resetTempDir()
                   throws IOException

Cleans up any snapshots in the snapshot/.tmp directory that were left from failed snapshot attempts.

Throws:

IOException - if we can't reach the filesystem

deleteSnapshot

public void deleteSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                    throws IOException

Delete the specified snapshot n * @throws SnapshotDoesNotExistException If the specified snapshot does not exist.

Throws:

IOException - For filesystem IOExceptions

isSnapshotDone

public boolean isSnapshotDone(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription expected)
                       throws IOException

Check if the specified snapshot is done n * @return true if snapshot is ready to be restored, false if it is still being taken.

Throws:

IOException - IOException if error from HDFS or RPC

UnknownSnapshotException - if snapshot is invalid or does not exist.

isTakingSnapshot

boolean isTakingSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)

Check to see if there is a snapshot in progress with the same name or on the same table. Currently we have a limitation only allowing a single snapshot per table at a time. Also we don't allow snapshot with the same name.

Parameters:

snapshot - description of the snapshot being checked.

Returns:

true if there is a snapshot in progress with the same name or on the same table.

isTakingSnapshot

public boolean isTakingSnapshot(TableName tableName)

Check to see if the specified table has a snapshot in progress. Currently we have a limitation only allowing a single snapshot per table at a time.

Parameters:

tableName - name of the table being snapshotted.

Returns:

true if there is a snapshot in progress on the specified table.

prepareToTakeSnapshot

private void prepareToTakeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                            throws HBaseSnapshotException

Check to make sure that we are OK to run the passed snapshot. Checks to make sure that we aren't already running a snapshot or restore on the requested table.

Parameters:

snapshot - description of the snapshot we want to start

Throws:

HBaseSnapshotException - if the filesystem could not be prepared to start the snapshot

snapshotDisabledTable

private void snapshotDisabledTable(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                            throws IOException

Take a snapshot of a disabled table.

Parameters:

snapshot - description of the snapshot to take. Modified to be SnapshotProtos.SnapshotDescription.Type.DISABLED.

Throws:

IOException - if the snapshot could not be started or filesystem for snapshot temporary directory could not be determined

snapshotEnabledTable

private void snapshotEnabledTable(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                           throws IOException

Take a snapshot of an enabled table.

Parameters:

snapshot - description of the snapshot to take.

Throws:

IOException - if the snapshot could not be started or filesystem for snapshot temporary directory could not be determined

snapshotTable

private void snapshotTable(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                           TakeSnapshotHandler handler)
                    throws IOException

Take a snapshot using the specified handler. On failure the snapshot temporary working directory is removed. NOTE: prepareToTakeSnapshot() called before this one takes care of the rejecting the snapshot request if the table is busy with another snapshot/restore operation.

Parameters:

snapshot - the snapshot description

handler - the snapshot handler

Throws:

IOException

getTakingSnapshotLock

public ReadWriteLock getTakingSnapshotLock()

isTakingAnySnapshot

public boolean isTakingAnySnapshot()

The snapshot operation processing as following:
1. Create a Snapshot Handler, and do some initialization;
2. Put the handler into snapshotHandlers
So when we consider if any snapshot is taking, we should consider both the takingSnapshotLock and snapshotHandlers;

Returns:

true to indicate that there're some running snapshots.

takeSnapshot

public void takeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                  throws IOException

Take a snapshot based on the enabled/disabled state of the table. n * @throws HBaseSnapshotException when a snapshot specific exception occurs.

Throws:

IOException - when some sort of generic IO exception occurs.

takeSnapshotInternal

private void takeSnapshotInternal(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                           throws IOException

Throws:

IOException

setSnapshotHandlerForTesting

public void setSnapshotHandlerForTesting(TableName tableName,
                                         SnapshotSentinel handler)

Set the handler for the current snapshot

Exposed for TESTING n * @param handler handler the master should use TODO get rid of this if possible, repackaging, modify tests.

getCoordinator

ProcedureCoordinator getCoordinator()

Returns distributed commit coordinator for all running snapshots

isSnapshotCompleted

private boolean isSnapshotCompleted(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)
                             throws IOException

Check to see if the snapshot is one of the currently completed snapshots Returns true if the snapshot exists in the "completed snapshots folder".

Parameters:

snapshot - expected snapshot to check

Returns:

true if the snapshot is stored on the FileSystem, false if is not stored

Throws:

IOException - if the filesystem throws an unexpected exception,

IllegalArgumentException - if snapshot name is invalid.

cloneSnapshot

private long cloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot,
                           TableName tableName,
                           org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                           TableDescriptor snapshotTableDesc,
                           NonceKey nonceKey,
                           boolean restoreAcl,
                           String customSFT)
                    throws IOException

Clone the specified snapshot. The clone will fail if the destination table has a snapshot or restore in progress.

Parameters:

reqSnapshot - Snapshot Descriptor from request

tableName - table to clone

snapshot - Snapshot Descriptor

snapshotTableDesc - Table Descriptor

nonceKey - unique identifier to prevent duplicated RPC

Returns:

procId the ID of the clone snapshot procedure n

Throws:

IOException

cloneSnapshot

long cloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                   TableDescriptor tableDescriptor,
                   NonceKey nonceKey,
                   boolean restoreAcl,
                   String customSFT)
            throws HBaseSnapshotException

Clone the specified snapshot into a new table. The operation will fail if the destination table has a snapshot or restore in progress.

Parameters:

snapshot - Snapshot Descriptor

tableDescriptor - Table Descriptor of the table to create

nonceKey - unique identifier to prevent duplicated RPC

Returns:

procId the ID of the clone snapshot procedure

Throws:

HBaseSnapshotException

restoreOrCloneSnapshot

public long restoreOrCloneSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot,
                                   NonceKey nonceKey,
                                   boolean restoreAcl,
                                   String customSFT)
                            throws IOException

Restore or Clone the specified snapshot n * @param nonceKey unique identifier to prevent duplicated RPC n

Throws:

IOException

restoreSnapshot

private long restoreSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription reqSnapshot,
                             TableName tableName,
                             org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                             TableDescriptor snapshotTableDesc,
                             NonceKey nonceKey,
                             boolean restoreAcl)
                      throws IOException

Restore the specified snapshot. The restore will fail if the destination table has a snapshot or restore in progress.

Parameters:

reqSnapshot - Snapshot Descriptor from request

tableName - table to restore

snapshot - Snapshot Descriptor

snapshotTableDesc - Table Descriptor

nonceKey - unique identifier to prevent duplicated RPC

restoreAcl - true to restore acl of snapshot

Returns:

procId the ID of the restore snapshot procedure n

Throws:

IOException

restoreSnapshot

private long restoreSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                             TableDescriptor tableDescriptor,
                             NonceKey nonceKey,
                             boolean restoreAcl)
                      throws HBaseSnapshotException

Restore the specified snapshot. The restore will fail if the destination table has a snapshot or restore in progress.

Parameters:

snapshot - Snapshot Descriptor

tableDescriptor - Table Descriptor

nonceKey - unique identifier to prevent duplicated RPC

restoreAcl - true to restore acl of snapshot

Returns:

procId the ID of the restore snapshot procedure

Throws:

HBaseSnapshotException

isRestoringTable

private boolean isRestoringTable(TableName tableName)

Verify if the restore of the specified table is in progress.

Parameters:

tableName - table under restore

Returns:

true if there is a restore in progress of the specified table.

removeSentinelIfFinished

private SnapshotSentinel removeSentinelIfFinished(Map<TableName,SnapshotSentinel> sentinels,
                                                  org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot)

Return the handler if it is currently live and has the same snapshot target name. The handler is removed from the sentinels map if completed.

Parameters:

sentinels - live handlers

snapshot - snapshot description

Returns:

null if doesn't match, else a live handler.

cleanupSentinels

private void cleanupSentinels()

Removes "abandoned" snapshot/restore requests. As part of the HBaseAdmin snapshot/restore API the operation status is checked until completed, and the in-progress maps are cleaned up when the status of a completed task is requested. To avoid having sentinels staying around for long time if something client side is failed, each operation tries to clean up the in-progress maps sentinels finished from a long time.

cleanupSentinels

private void cleanupSentinels(Map<TableName,SnapshotSentinel> sentinels)

Remove the sentinels that are marked as finished and the completion time has exceeded the removal timeout.

Parameters:

sentinels - map of sentinels to clean

cleanupCompletedRestoreInMap

private void cleanupCompletedRestoreInMap()

Remove the procedures that are marked as finished

stop

public void stop(String why)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Specified by:

stop in interface Stoppable

Parameters:

why - Why we're stopping.

isStopped

public boolean isStopped()

Description copied from interface: Stoppable

Returns True if Stoppable.stop(String) has been closed.

Specified by:

isStopped in interface Stoppable

checkSnapshotSupport

public void checkSnapshotSupport()
                          throws UnsupportedOperationException

Throws an exception if snapshot operations (take a snapshot, restore, clone) are not supported. Called at the beginning of snapshot() and restoreSnapshot() methods.

Throws:

UnsupportedOperationException - if snapshot are not supported

checkSnapshotSupport

private void checkSnapshotSupport(org.apache.hadoop.conf.Configuration conf,
                                  MasterFileSystem mfs)
                           throws IOException,
                                  UnsupportedOperationException

Called at startup, to verify if snapshot operation is supported, and to avoid starting the master if there're snapshots present but the cleaners needed are missing. Otherwise we can end up with snapshot data loss.

Parameters:

conf - The Configuration object to use

mfs - The MasterFileSystem to use

Throws:

IOException - in case of file-system operation failure

UnsupportedOperationException - in case cleaners are missing and there're snapshot in the system

initialize

public void initialize(MasterServices master,
                       MetricsMaster metricsMaster)
                throws org.apache.zookeeper.KeeperException,
                       IOException,
                       UnsupportedOperationException

Description copied from class: MasterProcedureManager

Initialize a globally barriered procedure for master.

Specified by:

initialize in class MasterProcedureManager

Parameters:

master - Master service interface nnn

Throws:

org.apache.zookeeper.KeeperException

IOException

UnsupportedOperationException

getProcedureSignature

public String getProcedureSignature()

Description copied from class: ProcedureManager

Return the unique signature of the procedure. This signature uniquely identifies the procedure. By default, this signature is the string used in the procedure controller (i.e., the root ZK node name for the procedure)

Specified by:

getProcedureSignature in class ProcedureManager

execProcedure

public void execProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc)
                   throws IOException

Description copied from class: MasterProcedureManager

Execute a distributed procedure on cluster

Overrides:

execProcedure in class MasterProcedureManager

Parameters:

desc - Procedure description n

Throws:

IOException

checkPermissions

public void checkPermissions(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc,
                             AccessChecker accessChecker,
                             User user)
                      throws IOException

Description copied from class: MasterProcedureManager

Check for required permissions before executing the procedure.

Specified by:

checkPermissions in class MasterProcedureManager

Throws:

IOException - if permissions requirements are not met.

isProcedureDone

public boolean isProcedureDone(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc)
                        throws IOException

Description copied from class: MasterProcedureManager

Check if the procedure is finished successfully

Specified by:

isProcedureDone in class MasterProcedureManager

Parameters:

desc - Procedure description

Returns:

true if the specified procedure is finished successfully n

Throws:

IOException

toSnapshotDescription

private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription toSnapshotDescription(org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ProcedureDescription desc)
                                                                                                            throws IOException

Throws:

IOException