Package org.apache.hadoop.hbase.mapreduce

Class WALPlayer

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.mapreduce.WALPlayer

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

@Public
public class WALPlayer
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

A tool to replay WAL files as a M/R job. The WAL can be replayed for a set of tables or all tables, and a time range can be provided (in milliseconds). The WAL is filtered to the passed set of tables and the output can optionally be mapped to another set of tables. WAL replay can also generate HFiles for later bulk importing, in that case the WAL is replayed for a single table only.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static enum	WALPlayer.Counter	Enum for map metrics.
(package private) static class	WALPlayer.WALKeyValueMapper	A mapper that just writes out KeyValues.
protected static class	WALPlayer.WALMapper	A mapper that writes out Mutation to be directly applied to a running HBase instance.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	BULK_OUTPUT_CONF_KEY
static final boolean	DEFAULT_IGNORE_EMPTY_FILES
static final String	IGNORE_EMPTY_FILES	Configuration flag that controls how the WALPlayer handles empty input WAL files.
static final String	IGNORE_MISSING_FILES
static final String	INPUT_FILES_SEPARATOR_KEY
private static final String	JOB_NAME_CONF_KEY
private static final org.slf4j.Logger	LOG
static final String	MULTI_TABLES_SUPPORT
(package private) static final String	NAME
static final String	TABLE_MAP_KEY
static final String	TABLES_KEY
protected static final String	tableSeparator

Constructor Summary

Constructors

Modifier	Constructor	Description
	WALPlayer()
protected	WALPlayer(org.apache.hadoop.conf.Configuration c)

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.mapreduce.Job	createSubmittableJob(String[] args)	Sets up the actual job.
private static RegionLocator	getRegionLocator(TableName tableName, org.apache.hadoop.conf.Configuration conf, Connection conn)
private List<TableName>	getTableNameList(String[] tables)
static void	main(String[] args)	Main entry point.
int	run(String[] args)
(package private) void	setupTime(org.apache.hadoop.conf.Configuration conf, String option)
private void	usage(String errorMsg)	Print usage

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Details

LOG

private static final org.slf4j.Logger LOG

NAME

static final String NAME

See Also:

• Constant Field Values

BULK_OUTPUT_CONF_KEY

public static final String BULK_OUTPUT_CONF_KEY

See Also:

• Constant Field Values

TABLES_KEY

public static final String TABLES_KEY

See Also:

• Constant Field Values

TABLE_MAP_KEY

public static final String TABLE_MAP_KEY

See Also:

• Constant Field Values

INPUT_FILES_SEPARATOR_KEY

public static final String INPUT_FILES_SEPARATOR_KEY

See Also:

• Constant Field Values

IGNORE_MISSING_FILES

public static final String IGNORE_MISSING_FILES

See Also:

• Constant Field Values

MULTI_TABLES_SUPPORT

public static final String MULTI_TABLES_SUPPORT

See Also:

• Constant Field Values

IGNORE_EMPTY_FILES

public static final String IGNORE_EMPTY_FILES

Configuration flag that controls how the WALPlayer handles empty input WAL files.

If set to true, the WALPlayer will silently ignore empty files that cannot be parsed as valid WAL files. This is useful in scenarios where such files are expected (e.g., due to partial writes or cleanup operations).

If set to false (default), the WALPlayer will throw an exception when it encounters an empty or un-parsable WAL file. This is useful for catching unexpected data issues early.

Default value: DEFAULT_IGNORE_EMPTY_FILES (false)

See Also:

• Constant Field Values

DEFAULT_IGNORE_EMPTY_FILES

public static final boolean DEFAULT_IGNORE_EMPTY_FILES

See Also:

• Constant Field Values

tableSeparator

protected static final String tableSeparator

See Also:

• Constant Field Values

JOB_NAME_CONF_KEY

private static final String JOB_NAME_CONF_KEY

See Also:

• Constant Field Values

Constructor Details

WALPlayer

public WALPlayer()

WALPlayer

protected WALPlayer(org.apache.hadoop.conf.Configuration c)

Method Details

setupTime

void setupTime(org.apache.hadoop.conf.Configuration conf,
 String option)
        throws IOException

Throws:

IOException

createSubmittableJob

public org.apache.hadoop.mapreduce.Job createSubmittableJob(String[] args)
                                                     throws IOException

Sets up the actual job.

Parameters:

args - The command line parameters.

Returns:

The newly created job.

Throws:

IOException - When setting up the job fails.

getTableNameList

private List<TableName> getTableNameList(String[] tables)

usage

private void usage(String errorMsg)

Print usage

Parameters:

errorMsg - Error message. Can be null.

main

public static void main(String[] args)
                 throws Exception

Main entry point.

Parameters:

args - The command line parameters.

Throws:

Exception - When running the job fails.

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

getRegionLocator

private static RegionLocator getRegionLocator(TableName tableName,
 org.apache.hadoop.conf.Configuration conf,
 Connection conn)
                                       throws IOException

Throws:

IOException