Package org.apache.hadoop.hbase.mapreduce

Class TestRowCounter

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestRowCounter

@Tag("org.apache.hadoop.hbase.testclassification.MapReduceTests") @Tag("org.apache.hadoop.hbase.testclassification.LargeTests")
public class TestRowCounter
extends Object

Test the rowcounter map reduce job.

Field Summary

Fields

Modifier and Type	Field	Description
private static final String	COL_FAM
private static final String	COL1
private static final String	COL2
private static final String	COMPOSITE_COLUMN
private static final org.slf4j.Logger	LOG
private static final int	ROWS_WITH_ONE_COL
private static final String	TABLE_NAME
private static final String	TABLE_NAME_TS_RANGE
private static final HBaseTestingUtil	TEST_UTIL
private static final int	TOTAL_ROWS

Constructor Summary

Constructors

Constructor	Description
TestRowCounter()

Method Summary

Modifier and Type	Method	Description
private void	assertUsageContent(String usage)
private void	runCreateSubmittableJobWithArgs(String[] args, int expectedCount)	Run the RowCounter map reduce job and verify the row count.
private void	runRowCount(String[] args, int expectedCount)	Run the RowCounter map reduce job and verify the row count.
static void	setUpBeforeClass()
static void	tearDownAfterClass()
void	testCreateSubmittableJobWithArgs10kRowRange()
void	testCreateSubmittableJobWithArgsColumnAndRowRange()	Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified
void	testCreateSubmittableJobWithArgsColumnWithColonInQualifier()	Test a case when the column specified in command line arguments is one for which the qualifier contains colons.
void	testCreateSubmittableJobWithArgsExclusiveColumn()	Test a case when the column specified in command line arguments is exclusive for few rows.
void	testCreateSubmittableJobWithArgsHiddenColumn()	Test a case when the column specified in command line arguments is not part of first KV for a row.
void	testCreateSubmittableJobWithArgsNoColumn()
void	testCreateSubmittableJobWithArgsRowMultiEmptyRange()	Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not
void	testCreateSubmittableJobWithArgsRowMultiRange()	Test a case when a range is specified with multiple ranges of start-end keys
void	testCreateSubmittableJobWithArgsRowMultiRangeUpperBound()	Test a case when a range is specified with two ranges where one range is with end key only
void	testCreateSubmittableJobWithArgsRowSingleRange()	Test a case when a range is specified with single range of start-end keys
void	testCreateSubmittableJobWithArgsRowSingleRangeUpperBound()	Test a case when a range is specified with single range with end key only
void	testCreateSubmittableJobWithArgsTimeRange()	Test a case when the timerange is specified with --starttime and --endtime options
void	testHelp()
void	testImportMain()	test main method.
void	testInvalidTable()
void	testRowCounter10kRowRange()
void	testRowCounterColumnAndRowRange()	Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified
void	testRowCounterColumnWithColonInQualifier()	Test a case when the column specified in command line arguments is one for which the qualifier contains colons.
void	testRowCounterExclusiveColumn()	Test a case when the column specified in command line arguments is exclusive for few rows.
void	testRowCounterHiddenColumn()	Test a case when the column specified in command line arguments is not part of first KV for a row.
void	testRowCounterNoColumn()	Test a case when no column was specified in command line arguments.
void	testRowCounterRowMultiEmptyRange()	Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not
void	testRowCounterRowMultiRange()	Test a case when a range is specified with multiple ranges of start-end keys
void	testRowCounterRowMultiRangeUpperBound()	Test a case when a range is specified with two ranges where one range is with end key only
void	testRowCounterRowSingleRange()	Test a case when a range is specified with single range of start-end keys
void	testRowCounterRowSingleRangeUpperBound()	Test a case when a range is specified with single range with end key only
void	testRowCounterTimeRange()	Test a case when the timerange is specified with --starttime and --endtime options
void	testRowCounterWithCountDeleteMarkersOption()	Step 1: Add 10 rows(row1, row2, row3, row4, row5, row6, row7, row8, row9, row10) to a table.
private void	validateCounterCounts(org.apache.hadoop.mapreduce.Counters counters, long rowCount, long rowsWithDeleteMarkersCount, long deleteCount, long deleteColumnCount, long deleteFamilyCount, long deleteFamilyVersionCount)
private static void	writeRows(org.apache.hadoop.hbase.client.Table table, int totalRows, int rowsWithOneCol)	Writes TOTAL_ROWS number of distinct rows in to the table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

TEST_UTIL

private static final HBaseTestingUtil TEST_UTIL

TABLE_NAME

private static final String TABLE_NAME

See Also:

• Constant Field Values

TABLE_NAME_TS_RANGE

private static final String TABLE_NAME_TS_RANGE

See Also:

• Constant Field Values

COL_FAM

private static final String COL_FAM

See Also:

• Constant Field Values

COL1

private static final String COL1

See Also:

• Constant Field Values

COL2

private static final String COL2

See Also:

• Constant Field Values

COMPOSITE_COLUMN

private static final String COMPOSITE_COLUMN

See Also:

• Constant Field Values

TOTAL_ROWS

private static final int TOTAL_ROWS

See Also:

• Constant Field Values

ROWS_WITH_ONE_COL

private static final int ROWS_WITH_ONE_COL

See Also:

• Constant Field Values

Constructor Details

TestRowCounter

public TestRowCounter()

Method Details

setUpBeforeClass

@BeforeAll
public static void setUpBeforeClass()
                             throws Exception

Throws:

Exception

tearDownAfterClass

@AfterAll
public static void tearDownAfterClass()
                               throws Exception

Throws:

Exception

testRowCounterNoColumn

@Test
public void testRowCounterNoColumn()
                            throws Exception

Test a case when no column was specified in command line arguments.

Throws:

Exception

testRowCounterExclusiveColumn

@Test
public void testRowCounterExclusiveColumn()
                                   throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows.

Throws:

Exception

testRowCounterColumnWithColonInQualifier

@Test
public void testRowCounterColumnWithColonInQualifier()
                                              throws Exception

Test a case when the column specified in command line arguments is one for which the qualifier contains colons.

Throws:

Exception

testRowCounterHiddenColumn

@Test
public void testRowCounterHiddenColumn()
                                throws Exception

Test a case when the column specified in command line arguments is not part of first KV for a row.

Throws:

Exception

testRowCounterColumnAndRowRange

@Test
public void testRowCounterColumnAndRowRange()
                                     throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified

Throws:

Exception

testRowCounterRowSingleRange

@Test
public void testRowCounterRowSingleRange()
                                  throws Exception

Test a case when a range is specified with single range of start-end keys

Throws:

Exception

testRowCounterRowSingleRangeUpperBound

@Test
public void testRowCounterRowSingleRangeUpperBound()
                                            throws Exception

Test a case when a range is specified with single range with end key only

Throws:

Exception

testRowCounterRowMultiRangeUpperBound

@Test
public void testRowCounterRowMultiRangeUpperBound()
                                           throws Exception

Test a case when a range is specified with two ranges where one range is with end key only

Throws:

Exception

testRowCounterRowMultiRange

@Test
public void testRowCounterRowMultiRange()
                                 throws Exception

Test a case when a range is specified with multiple ranges of start-end keys

Throws:

Exception

testRowCounterRowMultiEmptyRange

@Test
public void testRowCounterRowMultiEmptyRange()
                                      throws Exception

Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not

Throws:

Exception

testRowCounter10kRowRange

@Test
public void testRowCounter10kRowRange()
                               throws Exception

Throws:

Exception

testRowCounterTimeRange

@Test
public void testRowCounterTimeRange()
                             throws Exception

Test a case when the timerange is specified with --starttime and --endtime options

Throws:

Exception

runRowCount

private void runRowCount(String[] args,
 int expectedCount)
                  throws Exception

Run the RowCounter map reduce job and verify the row count.

Parameters:

args - the command line arguments to be used for rowcounter job.

expectedCount - the expected row count (result of map reduce job).

Throws:

Exception

runCreateSubmittableJobWithArgs

private void runCreateSubmittableJobWithArgs(String[] args,
 int expectedCount)
                                      throws Exception

Run the RowCounter map reduce job and verify the row count.

Parameters:

args - the command line arguments to be used for rowcounter job.

expectedCount - the expected row count (result of map reduce job).

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsNoColumn

@Test
public void testCreateSubmittableJobWithArgsNoColumn()
                                              throws Exception

Throws:

Exception

testCreateSubmittableJobWithArgsExclusiveColumn

@Test
public void testCreateSubmittableJobWithArgsExclusiveColumn()
                                                     throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows.

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsColumnWithColonInQualifier

@Test
public void testCreateSubmittableJobWithArgsColumnWithColonInQualifier()
                                                                throws Exception

Test a case when the column specified in command line arguments is one for which the qualifier contains colons.

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsHiddenColumn

@Test
public void testCreateSubmittableJobWithArgsHiddenColumn()
                                                  throws Exception

Test a case when the column specified in command line arguments is not part of first KV for a row.

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsColumnAndRowRange

@Test
public void testCreateSubmittableJobWithArgsColumnAndRowRange()
                                                       throws Exception

Test a case when the column specified in command line arguments is exclusive for few rows and also a row range filter is specified

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowSingleRange

@Test
public void testCreateSubmittableJobWithArgsRowSingleRange()
                                                    throws Exception

Test a case when a range is specified with single range of start-end keys

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowSingleRangeUpperBound

@Test
public void testCreateSubmittableJobWithArgsRowSingleRangeUpperBound()
                                                              throws Exception

Test a case when a range is specified with single range with end key only

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowMultiRangeUpperBound

@Test
public void testCreateSubmittableJobWithArgsRowMultiRangeUpperBound()
                                                             throws Exception

Test a case when a range is specified with two ranges where one range is with end key only

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowMultiRange

@Test
public void testCreateSubmittableJobWithArgsRowMultiRange()
                                                   throws Exception

Test a case when a range is specified with multiple ranges of start-end keys

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgsRowMultiEmptyRange

@Test
public void testCreateSubmittableJobWithArgsRowMultiEmptyRange()
                                                        throws Exception

Test a case when a range is specified with multiple ranges of start-end keys; one range is filled, another two are not

Throws:

Exception - in case of any unexpected error.

testCreateSubmittableJobWithArgs10kRowRange

@Test
public void testCreateSubmittableJobWithArgs10kRowRange()
                                                 throws Exception

Throws:

Exception

testCreateSubmittableJobWithArgsTimeRange

@Test
public void testCreateSubmittableJobWithArgsTimeRange()
                                               throws Exception

Test a case when the timerange is specified with --starttime and --endtime options

Throws:

Exception - in case of any unexpected error.

writeRows

private static void writeRows(org.apache.hadoop.hbase.client.Table table,
 int totalRows,
 int rowsWithOneCol)
                       throws IOException

Writes TOTAL_ROWS number of distinct rows in to the table. Few rows have two columns, Few have one.

Throws:

IOException

testImportMain

@Test
public void testImportMain()
                    throws Exception

test main method. Import should print help and call System.exit

Throws:

Exception

testHelp

@Test
public void testHelp()
              throws Exception

Throws:

Exception

testInvalidTable

@Test
public void testInvalidTable()
                      throws Exception

Throws:

Exception

testRowCounterWithCountDeleteMarkersOption

@Test
public void testRowCounterWithCountDeleteMarkersOption()
                                                throws Exception

Step 1: Add 10 rows(row1, row2, row3, row4, row5, row6, row7, row8, row9, row10) to a table. Each row contains 1 column family and 4 columns and values for two different timestamps - 5 & 10.

Step 2: Delete the latest version of column A for row1. --> 1 X Delete

Step 3: Delete the cell for timestamp 5 of column B for row1. --> 1 X Delete

Step 4: Delete a column family for row2 and row4. --> 2 X DeleteFamily

Step 5: Delete all versions of a specific column for row3, row5 and row6. --> 3 X DeleteColumn

Step 6: Delete all columns for timestamp 5 for row 7. --> 1 X DeleteFamilyVersion

Case 1: Run row counter without countDeleteMarkers and validate counter values.

Case 2: Run row counter with countDeleteMarkers flag and validate counter values.

Case 3: Run row counter with countDeleteMarkers flag for a row range and validate counter values.

Throws:

Exception

validateCounterCounts

private void validateCounterCounts(org.apache.hadoop.mapreduce.Counters counters,
 long rowCount,
 long rowsWithDeleteMarkersCount,
 long deleteCount,
 long deleteColumnCount,
 long deleteFamilyCount,
 long deleteFamilyVersionCount)

assertUsageContent

private void assertUsageContent(String usage)