org.apache.hadoop.hbase.mapreduce.replication

Class VerifyReplication

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

@InterfaceAudience.Private
public class VerifyReplication
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

This map-only job compares the data from a local table with a remote one. Every cell is compared and must have exactly the same keys (even timestamp) as well as same value. It is possible to restrict the job by time range and families. The peer id that's provided must match the one given when the replication stream was setup.

Two counters are provided, Verifier.Counters.GOODROWS and BADROWS. The reason for a why a row is different is shown in the map's log.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	VerifyReplication.Verifier Map-only comparator for 2 tables

Field Summary

Fields

Modifier and Type	Field and Description
(package private) int	batch
(package private) String	delimiter
(package private) long	endTime
(package private) String	families
(package private) boolean	includeDeletedCells
private static String	JOB_NAME_CONF_KEY
private static org.slf4j.Logger	LOG
static String	NAME
private static String	PEER_CONFIG_PREFIX
(package private) String	peerFSAddress
(package private) String	peerHBaseRootAddress
(package private) String	peerId
(package private) String	peerQuorumAddress
(package private) String	peerSnapshotName
(package private) String	peerSnapshotTmpDir
(package private) String	peerTableName
(package private) String	rowPrefixes
(package private) int	sleepMsBeforeReCompare
(package private) String	sourceSnapshotName
(package private) String	sourceSnapshotTmpDir
(package private) long	startTime
(package private) String	tableName
(package private) boolean	verbose
(package private) int	versions

Constructor Summary

Constructors

Constructor and Description
VerifyReplication()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.Job	createSubmittableJob(org.apache.hadoop.conf.Configuration conf, String[] args) Sets up the actual job.
boolean	doCommandLine(String[] args)
private static Pair<ReplicationPeerConfig,org.apache.hadoop.conf.Configuration>	getPeerQuorumConfig(org.apache.hadoop.conf.Configuration conf, String peerId)
private boolean	isPeerQuorumAddress(String cmd)
static void	main(String[] args) Main entry point.
private static void	printUsage(String errorMsg)
private void	restoreSnapshotForPeerCluster(org.apache.hadoop.conf.Configuration conf, String peerQuorumAddress)
int	run(String[] args)
private static void	setRowPrefixFilter(Scan scan, String rowPrefixes)
private static void	setStartAndStopRows(Scan scan, byte[] startPrefixRow, byte[] lastPrefixRow)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

LOG

private static final org.slf4j.Logger LOG

NAME

public static final String NAME

See Also:

Constant Field Values

PEER_CONFIG_PREFIX

private static final String PEER_CONFIG_PREFIX

See Also:

Constant Field Values

startTime

long startTime

endTime

long endTime

batch

int batch

versions

int versions

tableName

String tableName

families

String families

delimiter

String delimiter

peerId

String peerId

peerQuorumAddress

String peerQuorumAddress

rowPrefixes

String rowPrefixes

sleepMsBeforeReCompare

int sleepMsBeforeReCompare

verbose

boolean verbose

includeDeletedCells

boolean includeDeletedCells

sourceSnapshotName

String sourceSnapshotName

sourceSnapshotTmpDir

String sourceSnapshotTmpDir

peerSnapshotName

String peerSnapshotName

peerSnapshotTmpDir

String peerSnapshotTmpDir

peerFSAddress

String peerFSAddress

peerHBaseRootAddress

String peerHBaseRootAddress

peerTableName

String peerTableName

JOB_NAME_CONF_KEY

private static final String JOB_NAME_CONF_KEY

See Also:

Constant Field Values

Constructor Detail

VerifyReplication

public VerifyReplication()

Method Detail

getPeerQuorumConfig

private static Pair<ReplicationPeerConfig,org.apache.hadoop.conf.Configuration> getPeerQuorumConfig(org.apache.hadoop.conf.Configuration conf,
                                                                                                    String peerId)
                                                                                             throws IOException

Throws:

IOException

restoreSnapshotForPeerCluster

private void restoreSnapshotForPeerCluster(org.apache.hadoop.conf.Configuration conf,
                                           String peerQuorumAddress)
                                    throws IOException

Throws:

IOException

createSubmittableJob

public org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf,
                                                            String[] args)
                                                     throws IOException

Sets up the actual job.

Parameters:

conf - The current configuration.

args - The command line parameters.

Returns:

The newly created job.

Throws:

IOException - When setting up the job fails.

setRowPrefixFilter

private static void setRowPrefixFilter(Scan scan,
                                       String rowPrefixes)

setStartAndStopRows

private static void setStartAndStopRows(Scan scan,
                                        byte[] startPrefixRow,
                                        byte[] lastPrefixRow)

doCommandLine

public boolean doCommandLine(String[] args)

isPeerQuorumAddress

private boolean isPeerQuorumAddress(String cmd)

printUsage

private static void printUsage(String errorMsg)

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Main entry point.

Parameters:

args - The command line parameters.

Throws:

Exception - When running the job fails.