org.apache.hadoop.hbase.mapreduce

Class MultiTableSnapshotInputFormatImpl

java.lang.Object

org.apache.hadoop.hbase.mapreduce.MultiTableSnapshotInputFormatImpl

@InterfaceAudience.LimitedPrivate(value="HBase")
 @InterfaceStability.Evolving
public class MultiTableSnapshotInputFormatImpl
extends Object

Shared implementation of mapreduce code over multiple table snapshots. Utilized by both mapreduce (.MultiTableSnapshotInputFormat and mapred (MultiTableSnapshotInputFormat implementations.

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOG
static String	RESTORE_DIRS_KEY
static String	SNAPSHOT_TO_SCANS_KEY

Constructor Summary

Constructors

Constructor and Description
MultiTableSnapshotInputFormatImpl()

Method Summary

Modifier and Type	Method and Description
private Map<String,org.apache.hadoop.fs.Path>	generateSnapshotToRestoreDirMapping(Collection<String> snapshots, org.apache.hadoop.fs.Path baseRestoreDir) Generate a random path underneath baseRestoreDir for each snapshot in snapshots and return a map from the snapshot to the restore directory.
Map<String,org.apache.hadoop.fs.Path>	getSnapshotDirs(org.apache.hadoop.conf.Configuration conf) Retrieve the directories into which snapshots have been restored from (RESTORE_DIRS_KEY)
Map<String,Collection<Scan>>	getSnapshotsToScans(org.apache.hadoop.conf.Configuration conf) Retrieve the snapshot name -> list<scan> mapping pushed to configuration by setSnapshotToScans(org.apache.hadoop.conf.Configuration, java.util.Map)
List<TableSnapshotInputFormatImpl.InputSplit>	getSplits(org.apache.hadoop.conf.Configuration conf) Return the list of splits extracted from the scans/snapshots pushed to conf by setInput(org.apache.hadoop.conf.Configuration, java.util.Map, org.apache.hadoop.fs.Path)
(package private) void	restoreSnapshot(org.apache.hadoop.conf.Configuration conf, String snapshotName, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path restoreDir, org.apache.hadoop.fs.FileSystem fs)
void	restoreSnapshots(org.apache.hadoop.conf.Configuration conf, Map<String,org.apache.hadoop.fs.Path> snapshotToDir, org.apache.hadoop.fs.FileSystem fs) Restore each (snapshot name, restore directory) pair in snapshotToDir
void	setInput(org.apache.hadoop.conf.Configuration conf, Map<String,Collection<Scan>> snapshotScans, org.apache.hadoop.fs.Path restoreDir) Configure conf to read from snapshotScans, with snapshots restored to a subdirectory of restoreDir.
void	setSnapshotDirs(org.apache.hadoop.conf.Configuration conf, Map<String,org.apache.hadoop.fs.Path> snapshotDirs)
void	setSnapshotToScans(org.apache.hadoop.conf.Configuration conf, Map<String,Collection<Scan>> snapshotScans) Push snapshotScans to conf (under the key SNAPSHOT_TO_SCANS_KEY)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

RESTORE_DIRS_KEY

public static final String RESTORE_DIRS_KEY

See Also:

Constant Field Values

SNAPSHOT_TO_SCANS_KEY

public static final String SNAPSHOT_TO_SCANS_KEY

See Also:

Constant Field Values

Constructor Detail

MultiTableSnapshotInputFormatImpl

public MultiTableSnapshotInputFormatImpl()

Method Detail

setInput

public void setInput(org.apache.hadoop.conf.Configuration conf,
                     Map<String,Collection<Scan>> snapshotScans,
                     org.apache.hadoop.fs.Path restoreDir)
              throws IOException

Configure conf to read from snapshotScans, with snapshots restored to a subdirectory of restoreDir. Sets: RESTORE_DIRS_KEY, SNAPSHOT_TO_SCANS_KEY

Parameters:

conf -

snapshotScans -

restoreDir -

Throws:

IOException

getSplits

public List<TableSnapshotInputFormatImpl.InputSplit> getSplits(org.apache.hadoop.conf.Configuration conf)
                                                        throws IOException

Return the list of splits extracted from the scans/snapshots pushed to conf by setInput(org.apache.hadoop.conf.Configuration, java.util.Map, org.apache.hadoop.fs.Path)

Parameters:

conf - Configuration to determine splits from

Returns:

Return the list of splits extracted from the scans/snapshots pushed to conf

Throws:

IOException

getSnapshotsToScans

public Map<String,Collection<Scan>> getSnapshotsToScans(org.apache.hadoop.conf.Configuration conf)
                                                 throws IOException

Retrieve the snapshot name -> list<scan> mapping pushed to configuration by setSnapshotToScans(org.apache.hadoop.conf.Configuration, java.util.Map)

Parameters:

conf - Configuration to extract name -> list<scan> mappings from.

Returns:

the snapshot name -> list<scan> mapping pushed to configuration

Throws:

IOException

setSnapshotToScans

public void setSnapshotToScans(org.apache.hadoop.conf.Configuration conf,
                               Map<String,Collection<Scan>> snapshotScans)
                        throws IOException

Push snapshotScans to conf (under the key SNAPSHOT_TO_SCANS_KEY)

Parameters:

conf -

snapshotScans -

Throws:

IOException

getSnapshotDirs

public Map<String,org.apache.hadoop.fs.Path> getSnapshotDirs(org.apache.hadoop.conf.Configuration conf)
                                                      throws IOException

Retrieve the directories into which snapshots have been restored from (RESTORE_DIRS_KEY)

Parameters:

conf - Configuration to extract restore directories from

Returns:

the directories into which snapshots have been restored from

Throws:

IOException

setSnapshotDirs

public void setSnapshotDirs(org.apache.hadoop.conf.Configuration conf,
                            Map<String,org.apache.hadoop.fs.Path> snapshotDirs)

generateSnapshotToRestoreDirMapping

private Map<String,org.apache.hadoop.fs.Path> generateSnapshotToRestoreDirMapping(Collection<String> snapshots,
                                                                                  org.apache.hadoop.fs.Path baseRestoreDir)

Generate a random path underneath baseRestoreDir for each snapshot in snapshots and return a map from the snapshot to the restore directory.

Parameters:

snapshots - collection of snapshot names to restore

baseRestoreDir - base directory under which all snapshots in snapshots will be restored

Returns:

a mapping from snapshot name to the directory in which that snapshot has been restored

restoreSnapshots

public void restoreSnapshots(org.apache.hadoop.conf.Configuration conf,
                             Map<String,org.apache.hadoop.fs.Path> snapshotToDir,
                             org.apache.hadoop.fs.FileSystem fs)
                      throws IOException

Restore each (snapshot name, restore directory) pair in snapshotToDir

Parameters:

conf - configuration to restore with

snapshotToDir - mapping from snapshot names to restore directories

fs - filesystem to do snapshot restoration on

Throws:

IOException

restoreSnapshot

void restoreSnapshot(org.apache.hadoop.conf.Configuration conf,
                     String snapshotName,
                     org.apache.hadoop.fs.Path rootDir,
                     org.apache.hadoop.fs.Path restoreDir,
                     org.apache.hadoop.fs.FileSystem fs)
              throws IOException

Throws:

IOException