Package org.apache.hadoop.hbase.mapreduce

Class RowCounter

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.mapreduce.RowCounter

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

@Public
public class RowCounter
extends AbstractHBaseTool

A job with a just a map phase to count rows. Map outputs table rows IF the input row has columns that have content.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) static class	RowCounter.RowCounterCommandLineParser
(package private) static class	RowCounter.RowCounterMapper	Mapper that runs the count.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field	Description
private List<String>	columns
private boolean	countDeleteMarkers
private long	endTime
private static final String	EXPECTED_COUNT_KEY
private long	expectedCount
private org.apache.hadoop.mapreduce.Job	job
private static final String	JOB_NAME_CONF_KEY
private static final org.slf4j.Logger	LOG
(package private) static final String	NAME	Name of this 'program'.
private static final String	OPT_COUNT_DELETE_MARKERS
private static final String	OPT_END_TIME
private static final String	OPT_EXPECTED_COUNT
private static final String	OPT_RANGE
private static final String	OPT_START_TIME
private List<MultiRowRangeFilter.RowRange>	rowRangeList
private long	startTime
private String	tableName

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor	Description
RowCounter()

Method Summary

Modifier and Type	Method	Description
protected void	addOptions()	Override this to add command-line options using AbstractHBaseTool.addOptWithArg(java.lang.String, java.lang.String) and similar methods.
org.apache.hadoop.mapreduce.Job	createSubmittableJob(org.apache.hadoop.conf.Configuration conf)	Sets up the actual job.
static org.apache.hadoop.mapreduce.Job	createSubmittableJob(org.apache.hadoop.conf.Configuration conf, String[] args)	Deprecated. as of release 2.3.0.
protected int	doWork()	The "main function" of the tool
(package private) org.apache.hadoop.mapreduce.Job	getMapReduceJob()
static void	main(String[] args)	Main entry point.
protected org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLineParser	newParser()	Create the parser to use for parsing and validating the command line.
private static List<MultiRowRangeFilter.RowRange>	parseRowRangeParameter(String arg)
protected void	printUsage()
private static void	printUsage(String errorMessage)	Prints usage without error message.
protected void	printUsage(String usageStr, String usageHeader, String usageFooter)
protected void	processOldArgs(List<String> args)	For backward compatibility.
protected void	processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)	This method is called to process the options after they have been parsed.
private static void	setScanFilter(Scan scan, List<MultiRowRangeFilter.RowRange> rowRangeList, boolean countDeleteMarkers)	Sets filter FilterBase to the Scan instance.

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getConf, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, parseArgs, parseInt, parseLong, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

NAME

static final String NAME

Name of this 'program'.

See Also:

• Constant Field Values

JOB_NAME_CONF_KEY

private static final String JOB_NAME_CONF_KEY

See Also:

• Constant Field Values

EXPECTED_COUNT_KEY

private static final String EXPECTED_COUNT_KEY

OPT_START_TIME

private static final String OPT_START_TIME

See Also:

• Constant Field Values

OPT_END_TIME

private static final String OPT_END_TIME

See Also:

• Constant Field Values

OPT_RANGE

private static final String OPT_RANGE

See Also:

• Constant Field Values

OPT_EXPECTED_COUNT

private static final String OPT_EXPECTED_COUNT

See Also:

• Constant Field Values

OPT_COUNT_DELETE_MARKERS

private static final String OPT_COUNT_DELETE_MARKERS

See Also:

• Constant Field Values

tableName

private String tableName

rowRangeList

private List<MultiRowRangeFilter.RowRange> rowRangeList

startTime

private long startTime

endTime

private long endTime

expectedCount

private long expectedCount

countDeleteMarkers

private boolean countDeleteMarkers

columns

private List<String> columns

job

private org.apache.hadoop.mapreduce.Job job

Constructor Details

RowCounter

public RowCounter()

Method Details

createSubmittableJob

public org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf)
                                                     throws IOException

Sets up the actual job.

Parameters:

conf - The current configuration.

Returns:

The newly created job.

Throws:

IOException - When setting up the job fails.

createSubmittableJob

@Deprecated
public static org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf,
 String[] args)
                                                            throws IOException

Deprecated.

as of release 2.3.0. Will be removed on 4.0.0. Please use main method instead.

Sets up the actual job.

Parameters:

conf - The current configuration.

args - The command line parameters.

Returns:

The newly created job.

Throws:

IOException - When setting up the job fails.

printUsage

private static void printUsage(String errorMessage)

Prints usage without error message. Note that we don't document --expected-count, because it's intended for test.

parseRowRangeParameter

private static List<MultiRowRangeFilter.RowRange> parseRowRangeParameter(String arg)

setScanFilter

private static void setScanFilter(Scan scan,
 List<MultiRowRangeFilter.RowRange> rowRangeList,
 boolean countDeleteMarkers)

Sets filter FilterBase to the Scan instance. If provided rowRangeList contains more than one element, method sets filter which is instance of MultiRowRangeFilter. If rowRangeList contains exactly one element, startRow and stopRow are set to the scan. Also, method may apply FirstKeyOnlyFilter an KeyOnlyFilter for better performance.

printUsage

protected void printUsage()

Overrides:

printUsage in class AbstractHBaseTool

printUsage

protected void printUsage(String usageStr,
 String usageHeader,
 String usageFooter)

Overrides:

printUsage in class AbstractHBaseTool

addOptions

protected void addOptions()

Description copied from class: AbstractHBaseTool

Override this to add command-line options using AbstractHBaseTool.addOptWithArg(java.lang.String, java.lang.String) and similar methods.

Specified by:

addOptions in class AbstractHBaseTool

processOptions

protected void processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)
                       throws IllegalArgumentException

Description copied from class: AbstractHBaseTool

This method is called to process the options after they have been parsed.

Specified by:

processOptions in class AbstractHBaseTool

Throws:

IllegalArgumentException

processOldArgs

protected void processOldArgs(List<String> args)

Description copied from class: AbstractHBaseTool

For backward compatibility. DO NOT use it for new tools. We have options in existing tools which can't be ported to Apache CLI's Option. (because they don't pass validation, for e.g. "-copy-to". "-" means short name which doesn't allow '-' in name). This function is to allow tools to have, for time being, parameters which can't be parsed using Option. Overrides should consume all valid legacy arguments. If the param 'args' is not empty on return, it means there were invalid options, in which case we'll exit from the tool. Note that it's called before AbstractHBaseTool.processOptions(CommandLine), which means new options' values will override old ones'.

Overrides:

processOldArgs in class AbstractHBaseTool

doWork

protected int doWork()
              throws Exception

Description copied from class: AbstractHBaseTool

The "main function" of the tool

Specified by:

doWork in class AbstractHBaseTool

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Main entry point.

Parameters:

args - The command line parameters.

Throws:

Exception - When running the job fails.

newParser

protected org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLineParser newParser()

Description copied from class: AbstractHBaseTool

Create the parser to use for parsing and validating the command line. Since commons-cli lacks the capability to validate arbitrary combination of options, it may be helpful to bake custom logic into a specialized parser implementation. See LoadTestTool for examples.

Overrides:

newParser in class AbstractHBaseTool

Returns:

a new parser specific to the current tool

getMapReduceJob

org.apache.hadoop.mapreduce.Job getMapReduceJob()