Package org.apache.hadoop.hbase.mapreduce

Class TestRowCounter

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestRowCounter

public class TestRowCounter
extends Object

Test the rowcounter map reduce job.

Field Summary

Fields

Modifier and Type	Field	Description
static final HBaseClassTestRule	CLASS_RULE
private static final String	COL_FAM
private static final String	COL1
private static final String	COL2
private static final String	COMPOSITE_COLUMN
private static final org.slf4j.Logger	LOG
private static final int	ROWS_WITH_ONE_COL
private static final String	TABLE_NAME
private static final String	TABLE_NAME_TS_RANGE
private static final HBaseTestingUtil	TEST_UTIL
private static final int	TOTAL_ROWS

Constructor Summary

Constructors

Constructor	Description
TestRowCounter()

Method Summary

Modifier and Type	Method	Description
private void	assertUsageContent(String usage)
private void	runCreateSubmittableJobWithArgs(String[] args, int expectedCount)	Run the RowCounter map reduce job and verify the row count.
private void	runRowCount(String[] args, int expectedCount)	Run the RowCounter map reduce job and verify the row count.
static void	setUpBeforeClass()
static void	tearDownAfterClass()
void	testCreateSubmittableJobWithArgs10kRowRange()
void	testCreateSubmittableJobWithArgsColumnAndRowRange()	Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified
void	testCreateSubmittableJobWithArgsColumnWithColonInQualifier()	Test a case when the column specified in command line arguments is one for which the qualifier contains colons.
void	testCreateSubmittableJobWithArgsExclusiveColumn()	Test a case when the column specified in command line arguments is exclusive for few rows.
void	testCreateSubmittableJobWithArgsHiddenColumn()	Test a case when the column specified in command line arguments is not part of first KV for a row.
void	testCreateSubmittableJobWithArgsNoColumn()
void	testCreateSubmittableJobWithArgsRowMultiEmptyRange()	Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not
void	testCreateSubmittableJobWithArgsRowMultiRange()	Test a case when a range is specified with multiple ranges of start-end keys
void	testCreateSubmittableJobWithArgsRowMultiRangeUpperBound()	Test a case when a range is specified with two ranges where one range is with end key only
void	testCreateSubmittableJobWithArgsRowSingleRange()	Test a case when a range is specified with single range of start-end keys
void	testCreateSubmittableJobWithArgsRowSingleRangeUpperBound()	Test a case when a range is specified with single range with end key only
void	testCreateSubmittableJobWithArgsTimeRange()	Test a case when the timerange is specified with --starttime and --endtime options
void	testHelp()
void	testImportMain()	test main method.
void	testInvalidTable()
void	testRowCounter10kRowRange()
void	testRowCounterColumnAndRowRange()	Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified
void	testRowCounterColumnWithColonInQualifier()	Test a case when the column specified in command line arguments is one for which the qualifier contains colons.
void	testRowCounterExclusiveColumn()	Test a case when the column specified in command line arguments is exclusive for few rows.
void	testRowCounterHiddenColumn()	Test a case when the column specified in command line arguments is not part of first KV for a row.
void	testRowCounterNoColumn()	Test a case when no column was specified in command line arguments.
void	testRowCounterRowMultiEmptyRange()	Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not
void	testRowCounterRowMultiRange()	Test a case when a range is specified with multiple ranges of start-end keys
void	testRowCounterRowMultiRangeUpperBound()	Test a case when a range is specified with two ranges where one range is with end key only
void	testRowCounterRowSingleRange()	Test a case when a range is specified with single range of start-end keys
void	testRowCounterRowSingleRangeUpperBound()	Test a case when a range is specified with single range with end key only
void	testRowCounterTimeRange()	Test a case when the timerange is specified with --starttime and --endtime options
void	testRowCounterWithCountDeleteMarkersOption()	Step 1: Add 10 rows(row1, row2, row3, row4, row5, row6, row7, row8, row9, row10) to a table.
private void	validateCounterCounts(org.apache.hadoop.mapreduce.Counters counters, long rowCount, long rowsWithDeleteMarkersCount, long deleteCount, long deleteColumnCount, long deleteFamilyCount, long deleteFamilyVersionCount)
private static void	writeRows(org.apache.hadoop.hbase.client.Table table, int totalRows, int rowsWithOneCol)	Writes TOTAL_ROWS number of distinct rows in to the table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

LOG

private static final org.slf4j.Logger LOG

TEST_UTIL

private static final HBaseTestingUtil TEST_UTIL

TABLE_NAME

private static final String TABLE_NAME

See Also:

• Constant Field Values

TABLE_NAME_TS_RANGE

private static final String TABLE_NAME_TS_RANGE

See Also:

• Constant Field Values

COL_FAM

private static final String COL_FAM

See Also:

• Constant Field Values

COL1

private static final String COL1

See Also:

• Constant Field Values

COL2

private static final String COL2

See Also:

• Constant Field Values

COMPOSITE_COLUMN

private static final String COMPOSITE_COLUMN

See Also:

• Constant Field Values

TOTAL_ROWS

private static final int TOTAL_ROWS

See Also:

• Constant Field Values

ROWS_WITH_ONE_COL

private static final int ROWS_WITH_ONE_COL

See Also:

• Constant Field Values

Constructor Details

TestRowCounter

public TestRowCounter()

Method Details

setUpBeforeClass

public static void setUpBeforeClass()
                             throws Exception

Throws:

Exception

tearDownAfterClass

public static void tearDownAfterClass()
                               throws Exception

Throws:

Exception

testRowCounterNoColumn

public void testRowCounterNoColumn()
                            throws Exception

Test a case when no column was specified in command line arguments.

Throws:

Exception

testRowCounterExclusiveColumn

public void testRowCounterExclusiveColumn()
                                   throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows.

Throws:

Exception

testRowCounterColumnWithColonInQualifier

public void testRowCounterColumnWithColonInQualifier()
                                              throws Exception

Test a case when the column specified in command line arguments is one for which the qualifier contains colons.

Throws:

Exception

testRowCounterHiddenColumn

public void testRowCounterHiddenColumn()
                                throws Exception

Test a case when the column specified in command line arguments is not part of first KV for a row.

Throws:

Exception

testRowCounterColumnAndRowRange

public void testRowCounterColumnAndRowRange()
                                     throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified

Throws:

Exception

testRowCounterRowSingleRange

public void testRowCounterRowSingleRange()
                                  throws Exception

Test a case when a range is specified with single range of start-end keys

Throws:

Exception

testRowCounterRowSingleRangeUpperBound

public void testRowCounterRowSingleRangeUpperBound()
                                            throws Exception

Test a case when a range is specified with single range with end key only

Throws:

Exception

testRowCounterRowMultiRangeUpperBound

public void testRowCounterRowMultiRangeUpperBound()
                                           throws Exception

Test a case when a range is specified with two ranges where one range is with end key only

Throws:

Exception

testRowCounterRowMultiRange

public void testRowCounterRowMultiRange()
                                 throws Exception

Test a case when a range is specified with multiple ranges of start-end keys

Throws:

Exception

testRowCounterRowMultiEmptyRange

public void testRowCounterRowMultiEmptyRange()
                                      throws Exception

Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not

Throws:

Exception

testRowCounter10kRowRange

public void testRowCounter10kRowRange()
                               throws Exception

Throws:

Exception

testRowCounterTimeRange

public void testRowCounterTimeRange()
                             throws Exception

Test a case when the timerange is specified with --starttime and --endtime options

Throws:

Exception

runRowCount

private void runRowCount(String[] args,
 int expectedCount)
                  throws Exception

Run the RowCounter map reduce job and verify the row count.

Parameters:

args - the command line arguments to be used for rowcounter job.

expectedCount - the expected row count (result of map reduce job).

Throws:

Exception

runCreateSubmittableJobWithArgs

private void runCreateSubmittableJobWithArgs(String[] args,
 int expectedCount)
                                      throws Exception

Run the RowCounter map reduce job and verify the row count.

Parameters:

args - the command line arguments to be used for rowcounter job.

expectedCount - the expected row count (result of map reduce job).

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsNoColumn

public void testCreateSubmittableJobWithArgsNoColumn()
                                              throws Exception

Throws:

Exception

testCreateSubmittableJobWithArgsExclusiveColumn

public void testCreateSubmittableJobWithArgsExclusiveColumn()
                                                     throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows.

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsColumnWithColonInQualifier

public void testCreateSubmittableJobWithArgsColumnWithColonInQualifier()
                                                                throws Exception

Test a case when the column specified in command line arguments is one for which the qualifier contains colons.

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsHiddenColumn

public void testCreateSubmittableJobWithArgsHiddenColumn()
                                                  throws Exception

Test a case when the column specified in command line arguments is not part of first KV for a row.

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsColumnAndRowRange

public void testCreateSubmittableJobWithArgsColumnAndRowRange()
                                                       throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowSingleRange

public void testCreateSubmittableJobWithArgsRowSingleRange()
                                                    throws Exception

Test a case when a range is specified with single range of start-end keys

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowSingleRangeUpperBound

public void testCreateSubmittableJobWithArgsRowSingleRangeUpperBound()
                                                              throws Exception

Test a case when a range is specified with single range with end key only

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowMultiRangeUpperBound

public void testCreateSubmittableJobWithArgsRowMultiRangeUpperBound()
                                                             throws Exception

Test a case when a range is specified with two ranges where one range is with end key only

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowMultiRange

public void testCreateSubmittableJobWithArgsRowMultiRange()
                                                   throws Exception

Test a case when a range is specified with multiple ranges of start-end keys

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowMultiEmptyRange

public void testCreateSubmittableJobWithArgsRowMultiEmptyRange()
                                                        throws Exception

Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgs10kRowRange

public void testCreateSubmittableJobWithArgs10kRowRange()
                                                 throws Exception

Throws:

Exception

testCreateSubmittableJobWithArgsTimeRange

public void testCreateSubmittableJobWithArgsTimeRange()
                                               throws Exception

Test a case when the timerange is specified with --starttime and --endtime options

Throws:

Exception - in case of any unexpected error.

writeRows

private static void writeRows(org.apache.hadoop.hbase.client.Table table,
 int totalRows,
 int rowsWithOneCol)
                       throws IOException

Writes TOTAL_ROWS number of distinct rows in to the table. Few rows have two columns, Few have one.

Throws:

IOException

testImportMain

public void testImportMain()
                    throws Exception

test main method. Import should print help and call System.exit

Throws:

Exception

testHelp

public void testHelp()
              throws Exception

Throws:

Exception

testInvalidTable

public void testInvalidTable()
                      throws Exception

Throws:

Exception

testRowCounterWithCountDeleteMarkersOption

public void testRowCounterWithCountDeleteMarkersOption()
                                                throws Exception

Step 1: Add 10 rows(row1, row2, row3, row4, row5, row6, row7, row8, row9, row10) to a table. Each row contains 1 column family and 4 columns and values for two different timestamps - 5 & 10.

Step 2: Delete the latest version of column A for row1. --> 1 X Delete

Step 3: Delete the cell for timestamp 5 of column B for row1. --> 1 X Delete

Step 4: Delete a column family for row2 and row4. --> 2 X DeleteFamily

Step 5: Delete all versions of a specific column for row3, row5 and row6. --> 3 X DeleteColumn

Step 6: Delete all columns for timestamp 5 for row 7. --> 1 X DeleteFamilyVersion

Case 1: Run row counter without countDeleteMarkers and validate counter values.

Case 2: Run row counter with countDeleteMarkers flag and validate counter values.

Case 3: Run row counter with countDeleteMarkers flag for a row range and validate counter values.

Throws:

Exception

validateCounterCounts

private void validateCounterCounts(org.apache.hadoop.mapreduce.Counters counters,
 long rowCount,
 long rowsWithDeleteMarkersCount,
 long deleteCount,
 long deleteColumnCount,
 long deleteFamilyCount,
 long deleteFamilyVersionCount)

assertUsageContent

private void assertUsageContent(String usage)