org.apache.hadoop.hbase.snapshot

Class SnapshotDescriptionUtils

java.lang.Object

org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils

@InterfaceAudience.Private
public final class SnapshotDescriptionUtils
extends Object

Utility class to help manage SnapshotDesriptions.

Snapshots are laid out on disk like this:

 /hbase/.snapshots
          /.tmp                <---- working directory
          /[snapshot name]     <----- completed snapshot
 

A completed snapshot named 'completed' then looks like (multiple regions, servers, files, etc. signified by '...' on the same directory depth).

 /hbase/.snapshots/completed
                   .snapshotinfo          <--- Description of the snapshot
                   .tableinfo             <--- Copy of the tableinfo
                    /.logs
                        /[server_name]
                            /... [log files]
                         ...
                   /[region name]           <---- All the region's information
                   .regioninfo              <---- Copy of the HRegionInfo
                      /[column family name]
                          /[hfile name]     <--- name of the hfile in the real region
                          ...
                      ...
                    ...
 

Utility methods in this class are useful for getting the correct locations for different parts of the snapshot, as well as moving completed snapshots into place (see completeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription, org.apache.hadoop.fs.Path, org.apache.hadoop.fs.Path, org.apache.hadoop.fs.FileSystem), and writing the SnapshotProtos.SnapshotDescription to the working snapshot directory.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SnapshotDescriptionUtils.CompletedSnaphotDirectoriesFilter Filter that only accepts completed snapshot directories

Field Summary

Fields

Modifier and Type	Field and Description
static long	DEFAULT_MAX_WAIT_TIME By default, wait 300 seconds for a snapshot to complete
private static org.slf4j.Logger	LOG
static String	MASTER_SNAPSHOT_TIMEOUT_MILLIS
static long	NO_SNAPSHOT_START_TIME_SPECIFIED Default value if no start time is specified
static int	SNAPSHOT_LAYOUT_VERSION Version of the fs layout for a snapshot.
static int	SNAPSHOT_TIMEOUT_MILLIS_DEFAULT Deprecated. Use DEFAULT_MAX_WAIT_TIME instead.
static String	SNAPSHOT_TIMEOUT_MILLIS_KEY Deprecated. Use MASTER_SNAPSHOT_TIMEOUT_MILLIS instead.
static String	SNAPSHOT_TMP_DIR_NAME Temporary directory under the snapshot directory to store in-progress snapshots
static String	SNAPSHOTINFO_FILE The file contains the snapshot basic information and it is under the directory of a snapshot.

Constructor Summary

Constructors

Modifier	Constructor and Description
private	SnapshotDescriptionUtils()

Method Summary

Modifier and Type	Method and Description
static void	completeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path rootdir, org.apache.hadoop.fs.Path workingDir, org.apache.hadoop.fs.FileSystem fs) Move the finished snapshot to its final, publicly visible directory - this marks the snapshot as 'complete'.
private static org.apache.hadoop.fs.Path	getCompletedSnapshotDir(org.apache.hadoop.fs.Path snapshotsDir, String snapshotName) Get the directory to store the snapshot instance
static org.apache.hadoop.fs.Path	getCompletedSnapshotDir(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path rootDir) Get the directory for a specified snapshot.
static org.apache.hadoop.fs.Path	getCompletedSnapshotDir(String snapshotName, org.apache.hadoop.fs.Path rootDir) Get the directory for a completed snapshot.
static long	getMaxMasterTimeout(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription.Type type, long defaultMaxWaitTime)
static org.apache.hadoop.fs.Path	getSnapshotRootDir(org.apache.hadoop.fs.Path rootDir) Get the snapshot root directory.
static org.apache.hadoop.fs.Path	getSnapshotsDir(org.apache.hadoop.fs.Path rootDir)
static org.apache.hadoop.fs.Path	getWorkingSnapshotDir(org.apache.hadoop.fs.Path rootDir) Get the general working directory for snapshots - where they are built, where they are temporarily copied on export, etc.
static org.apache.hadoop.fs.Path	getWorkingSnapshotDir(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path rootDir) Get the directory to build a snapshot, before it is finalized
static org.apache.hadoop.fs.Path	getWorkingSnapshotDir(String snapshotName, org.apache.hadoop.fs.Path rootDir) Get the directory to build a snapshot, before it is finalized
static boolean	isSecurityAvailable(org.apache.hadoop.conf.Configuration conf)
static boolean	isSnapshotOwner(SnapshotDescription snapshot, User user) Check if the user is this table snapshot's owner
static org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription	readSnapshotInfo(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path snapshotDir) Read in the HBaseProtos.SnapshotDescription stored for the snapshot in the passed directory
static org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription	validate(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.conf.Configuration conf) Convert the passed snapshot description into a 'full' snapshot description based on default parameters, if none have been supplied.
private static org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription	writeAclToSnapshotDescription(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.conf.Configuration conf)
static void	writeSnapshotInfo(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path workingDir, org.apache.hadoop.fs.FileSystem fs) Write the snapshot description into the working directory of a snapshot

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

SNAPSHOT_LAYOUT_VERSION

public static final int SNAPSHOT_LAYOUT_VERSION

Version of the fs layout for a snapshot. Future snapshots may have different file layouts, which we may need to read in differently.

See Also:

Constant Field Values

SNAPSHOTINFO_FILE

public static final String SNAPSHOTINFO_FILE

The file contains the snapshot basic information and it is under the directory of a snapshot.

See Also:

Constant Field Values

SNAPSHOT_TMP_DIR_NAME

public static final String SNAPSHOT_TMP_DIR_NAME

Temporary directory under the snapshot directory to store in-progress snapshots

See Also:

Constant Field Values

NO_SNAPSHOT_START_TIME_SPECIFIED

public static final long NO_SNAPSHOT_START_TIME_SPECIFIED

Default value if no start time is specified

See Also:

Constant Field Values

MASTER_SNAPSHOT_TIMEOUT_MILLIS

public static final String MASTER_SNAPSHOT_TIMEOUT_MILLIS

See Also:

Constant Field Values

DEFAULT_MAX_WAIT_TIME

public static final long DEFAULT_MAX_WAIT_TIME

By default, wait 300 seconds for a snapshot to complete

See Also:

Constant Field Values

SNAPSHOT_TIMEOUT_MILLIS_DEFAULT

@Deprecated
public static final int SNAPSHOT_TIMEOUT_MILLIS_DEFAULT

Deprecated. Use DEFAULT_MAX_WAIT_TIME instead.

By default, check to see if the snapshot is complete (ms)

See Also:

Constant Field Values

SNAPSHOT_TIMEOUT_MILLIS_KEY

@Deprecated
public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY

Deprecated. Use MASTER_SNAPSHOT_TIMEOUT_MILLIS instead.

Conf key for # of ms elapsed before injecting a snapshot timeout error when waiting for completion.

See Also:

Constant Field Values

Constructor Detail

SnapshotDescriptionUtils

private SnapshotDescriptionUtils()

Method Detail

getMaxMasterTimeout

public static long getMaxMasterTimeout(org.apache.hadoop.conf.Configuration conf,
                                       org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription.Type type,
                                       long defaultMaxWaitTime)

Parameters:

conf - Configuration from which to check for the timeout

type - type of snapshot being taken

defaultMaxWaitTime - Default amount of time to wait, if none is in the configuration

Returns:

the max amount of time the master should wait for a snapshot to complete

getSnapshotRootDir

public static org.apache.hadoop.fs.Path getSnapshotRootDir(org.apache.hadoop.fs.Path rootDir)

Get the snapshot root directory. All the snapshots are kept under this directory, i.e. ${hbase.rootdir}/.snapshot

Parameters:

rootDir - hbase root directory

Returns:

the base directory in which all snapshots are kept

getCompletedSnapshotDir

public static org.apache.hadoop.fs.Path getCompletedSnapshotDir(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                                                org.apache.hadoop.fs.Path rootDir)

Get the directory for a specified snapshot. This directory is a sub-directory of snapshot root directory and all the data files for a snapshot are kept under this directory.

Parameters:

snapshot - snapshot being taken

rootDir - hbase root directory

Returns:

the final directory for the completed snapshot

getCompletedSnapshotDir

public static org.apache.hadoop.fs.Path getCompletedSnapshotDir(String snapshotName,
                                                                org.apache.hadoop.fs.Path rootDir)

Get the directory for a completed snapshot. This directory is a sub-directory of snapshot root directory and all the data files for a snapshot are kept under this directory.

Parameters:

snapshotName - name of the snapshot being taken

rootDir - hbase root directory

Returns:

the final directory for the completed snapshot

getWorkingSnapshotDir

public static org.apache.hadoop.fs.Path getWorkingSnapshotDir(org.apache.hadoop.fs.Path rootDir)

Get the general working directory for snapshots - where they are built, where they are temporarily copied on export, etc.

Parameters:

rootDir - root directory of the HBase installation

Returns:

Path to the snapshot tmp directory, relative to the passed root directory

getWorkingSnapshotDir

public static org.apache.hadoop.fs.Path getWorkingSnapshotDir(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                                              org.apache.hadoop.fs.Path rootDir)

Get the directory to build a snapshot, before it is finalized

Parameters:

snapshot - snapshot that will be built

rootDir - root directory of the hbase installation

Returns:

Path where one can build a snapshot

getWorkingSnapshotDir

public static org.apache.hadoop.fs.Path getWorkingSnapshotDir(String snapshotName,
                                                              org.apache.hadoop.fs.Path rootDir)

Get the directory to build a snapshot, before it is finalized

Parameters:

snapshotName - name of the snapshot

rootDir - root directory of the hbase installation

Returns:

Path where one can build a snapshot

getCompletedSnapshotDir

private static final org.apache.hadoop.fs.Path getCompletedSnapshotDir(org.apache.hadoop.fs.Path snapshotsDir,
                                                                       String snapshotName)

Get the directory to store the snapshot instance

Parameters:

snapshotsDir - hbase-global directory for storing all snapshots

snapshotName - name of the snapshot to take

Returns:

the final directory for the completed snapshot

getSnapshotsDir

public static final org.apache.hadoop.fs.Path getSnapshotsDir(org.apache.hadoop.fs.Path rootDir)

Parameters:

rootDir - hbase root directory

Returns:

the directory for all completed snapshots;

validate

public static org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription validate(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                                                                                            org.apache.hadoop.conf.Configuration conf)
                                                                                                     throws IllegalArgumentException,
                                                                                                            IOException

Convert the passed snapshot description into a 'full' snapshot description based on default parameters, if none have been supplied. This resolves any 'optional' parameters that aren't supplied to their default values.

Parameters:

snapshot - general snapshot descriptor

conf - Configuration to read configured snapshot defaults if snapshot is not complete

Returns:

a valid snapshot description

Throws:

IllegalArgumentException - if the SnapshotProtos.SnapshotDescription is not a complete SnapshotProtos.SnapshotDescription.

IOException

writeSnapshotInfo

public static void writeSnapshotInfo(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                     org.apache.hadoop.fs.Path workingDir,
                                     org.apache.hadoop.fs.FileSystem fs)
                              throws IOException

Write the snapshot description into the working directory of a snapshot

Parameters:

snapshot - description of the snapshot being taken

workingDir - working directory of the snapshot

fs - FileSystem on which the snapshot should be taken

Throws:

IOException - if we can't reach the filesystem and the file cannot be cleaned up on failure

readSnapshotInfo

public static org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription readSnapshotInfo(org.apache.hadoop.fs.FileSystem fs,
                                                                                                                    org.apache.hadoop.fs.Path snapshotDir)
                                                                                                             throws CorruptedSnapshotException

Read in the HBaseProtos.SnapshotDescription stored for the snapshot in the passed directory

Parameters:

fs - filesystem where the snapshot was taken

snapshotDir - directory where the snapshot was stored

Returns:

the stored snapshot description

Throws:

CorruptedSnapshotException - if the snapshot cannot be read

completeSnapshot

public static void completeSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                    org.apache.hadoop.fs.Path rootdir,
                                    org.apache.hadoop.fs.Path workingDir,
                                    org.apache.hadoop.fs.FileSystem fs)
                             throws SnapshotCreationException,
                                    IOException

Move the finished snapshot to its final, publicly visible directory - this marks the snapshot as 'complete'.

Parameters:

snapshot - description of the snapshot being tabken

rootdir - root directory of the hbase installation

workingDir - directory where the in progress snapshot was built

fs - FileSystem where the snapshot was built

Throws:

SnapshotCreationException - if the snapshot could not be moved

IOException - the filesystem could not be reached

isSnapshotOwner

public static boolean isSnapshotOwner(SnapshotDescription snapshot,
                                      User user)

Check if the user is this table snapshot's owner

Parameters:

snapshot - the table snapshot description

user - the user

Returns:

true if the user is the owner of the snapshot, false otherwise or the snapshot owner field is not present.

isSecurityAvailable

public static boolean isSecurityAvailable(org.apache.hadoop.conf.Configuration conf)
                                   throws IOException

Throws:

IOException

writeAclToSnapshotDescription

private static org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription writeAclToSnapshotDescription(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                                                                                                                                  org.apache.hadoop.conf.Configuration conf)
                                                                                                                           throws IOException

Throws:

IOException