Package org.apache.hadoop.hbase.backup.util

Class BulkFilesCollector

java.lang.Object

org.apache.hadoop.hbase.backup.util.BulkFilesCollector

@Private
public final class BulkFilesCollector
extends Object

Utility to run BulkLoadCollectorJob over a comma-separated list of WAL directories and return a deduplicated list of discovered bulk-load file paths.

Field Summary

Fields

Modifier and Type	Field	Description
private static final org.slf4j.Logger	LOG

Constructor Summary

Constructors

Modifier	Constructor	Description
private	BulkFilesCollector()

Method Summary

Modifier and Type	Method	Description
static List<org.apache.hadoop.fs.Path>	collectFromWalDirs(org.apache.hadoop.conf.Configuration conf, String walDirsCsv, org.apache.hadoop.fs.Path restoreRootDir, TableName sourceTable, TableName targetTable, long startTime, long endTime)	Convenience overload: collector will create and configure BulkLoadCollectorJob internally.
static List<org.apache.hadoop.fs.Path>	collectFromWalDirs(org.apache.hadoop.conf.Configuration conf, String walDirsCsv, org.apache.hadoop.fs.Path restoreRootDir, TableName sourceTable, TableName targetTable, org.apache.hadoop.util.Tool bulkCollector)	Primary implementation: runs the provided Tool (BulkLoadCollectorJob) with args " " and returns deduped list of Paths.
private static List<org.apache.hadoop.fs.Path>	readBulkFilesListFromOutput(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path bulkFilesOut)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

Constructor Details

BulkFilesCollector

private BulkFilesCollector()

Method Details

collectFromWalDirs

public static List<org.apache.hadoop.fs.Path> collectFromWalDirs(org.apache.hadoop.conf.Configuration conf,
 String walDirsCsv,
 org.apache.hadoop.fs.Path restoreRootDir,
 TableName sourceTable,
 TableName targetTable,
 long startTime,
 long endTime)
                                                          throws IOException

Convenience overload: collector will create and configure BulkLoadCollectorJob internally.

Parameters:

conf - cluster/configuration used to initialize job and access FS

walDirsCsv - comma-separated WAL directories

restoreRootDir - parent path under which temporary output dir will be created

sourceTable - source table name (for args/logging)

targetTable - target table name (for args/logging)

startTime - start time (ms) to set in the job config (WALInputFormat.START_TIME_KEY)

endTime - end time (ms) to set in the job config (WALInputFormat.END_TIME_KEY)

Returns:

deduplicated list of Paths discovered by the collector

Throws:

IOException - on IO or job failure

collectFromWalDirs

public static List<org.apache.hadoop.fs.Path> collectFromWalDirs(org.apache.hadoop.conf.Configuration conf,
 String walDirsCsv,
 org.apache.hadoop.fs.Path restoreRootDir,
 TableName sourceTable,
 TableName targetTable,
 org.apache.hadoop.util.Tool bulkCollector)
                                                          throws IOException

Primary implementation: runs the provided Tool (BulkLoadCollectorJob) with args " " and returns deduped list of Paths.

Throws:

IOException

readBulkFilesListFromOutput

private static List<org.apache.hadoop.fs.Path> readBulkFilesListFromOutput(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path bulkFilesOut)
                                                                    throws IOException

Throws:

IOException