org.apache.hadoop.hbase.mapreduce

Class TestImportExport

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestImportExport

public class TestImportExport
extends Object

Tests the table import and table export MR job functionality

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TestImportExport.MetadataController
private static class	TestImportExport.TableWALActionListener This listens to the TestImportExport.TableWALActionListener.visitLogEntryBeforeWrite(RegionInfo, WALKey, WALEdit) to identify that an entry is written to the Write Ahead Log for the given table.

Field Summary

Fields

Modifier and Type	Field and Description
static HBaseClassTestRule	CLASS_RULE
private static String	EXPORT_BATCH_SIZE
private org.apache.hadoop.hbase.TableName	EXPORT_TABLE
private static byte[]	FAMILYA
private static String	FAMILYA_STRING
private static byte[]	FAMILYB
private static String	FAMILYB_STRING
private static String	FQ_OUTPUT_DIR
private org.apache.hadoop.hbase.TableName	IMPORT_TABLE
private static org.slf4j.Logger	LOG
org.junit.rules.TestName	name
private static long	now
private static String	OUTPUT_DIR
private static byte[]	QUAL
private static byte[]	ROW1
private static byte[]	ROW2
private static byte[]	ROW3
static String	TEST_ATTR
static String	TEST_TAG
static byte	TEST_TAG_TYPE
protected static HBaseTestingUtility	UTIL

Constructor Summary

Constructors

Constructor and Description
TestImportExport()

Method Summary

Modifier and Type	Method and Description
static void	afterClass()
void	announce()
static void	beforeClass()
private void	checkWhetherTagExists(org.apache.hadoop.hbase.TableName table, boolean tagExists)
void	cleanup()
private int	getCount(org.apache.hadoop.hbase.client.Table table, org.apache.hadoop.hbase.filter.Filter filter) Count the number of keyvalues in the specified table with the given filter
protected boolean	runExport(String[] args) Runs an export job with the specified command line args n * @return true if job completed successfully nnn
protected void	runExportMain(String[] args)
(package private) boolean	runImport(String[] args) Runs an import job with the specified command line args n * @return true if job completed successfully nnn
void	testAddFilterAndArguments() Test addFilterAndArguments method of Import This method set couple parameters into Configuration
void	testDurability()
void	testExportMain() test main method.
void	testExportScan()
void	testExportScannerBatching() Test export scanner batching
void	testImport94Table() Test import data from 0.94 exported file n
void	testImportMain() test main method.
void	testKeyValueImporter() Test map method of Importer
void	testMetaExport() Test export hbase:meta table n
void	testSimpleCase() Test simple replication case with column mapping n
void	testTagsAddition() Add cell tags to delete mutations, run export and import tool and verify that tags are present in import table also.
void	testTagsWithEmptyCodec() Set hbase.client.rpc.codec and hbase.client.default.rpc.codec both to empty string This means it will use no Codec.
void	testWithDeletes()
void	testWithFilter() Create a simple table, run an Export Job on it, Import with filtering on, verify counts, attempt with invalid values.
void	testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

LOG

private static final org.slf4j.Logger LOG

UTIL

protected static final HBaseTestingUtility UTIL

ROW1

private static final byte[] ROW1

ROW2

private static final byte[] ROW2

ROW3

private static final byte[] ROW3

FAMILYA_STRING

private static final String FAMILYA_STRING

See Also:

Constant Field Values

FAMILYB_STRING

private static final String FAMILYB_STRING

See Also:

Constant Field Values

FAMILYA

private static final byte[] FAMILYA

FAMILYB

private static final byte[] FAMILYB

QUAL

private static final byte[] QUAL

OUTPUT_DIR

private static final String OUTPUT_DIR

See Also:

Constant Field Values

FQ_OUTPUT_DIR

private static String FQ_OUTPUT_DIR

EXPORT_BATCH_SIZE

private static final String EXPORT_BATCH_SIZE

See Also:

Constant Field Values

now

private static final long now

EXPORT_TABLE

private final org.apache.hadoop.hbase.TableName EXPORT_TABLE

IMPORT_TABLE

private final org.apache.hadoop.hbase.TableName IMPORT_TABLE

TEST_TAG_TYPE

public static final byte TEST_TAG_TYPE

See Also:

Constant Field Values

TEST_ATTR

public static final String TEST_ATTR

See Also:

Constant Field Values

TEST_TAG

public static final String TEST_TAG

See Also:

Constant Field Values

name

public final org.junit.rules.TestName name

Constructor Detail

TestImportExport

public TestImportExport()

Method Detail

beforeClass

public static void beforeClass()
                        throws Throwable

Throws:

Throwable

afterClass

public static void afterClass()
                       throws Throwable

Throws:

Throwable

announce

public void announce()

cleanup

public void cleanup()
             throws Throwable

Throws:

Throwable

runExport

protected boolean runExport(String[] args)
                     throws Throwable

Runs an export job with the specified command line args n * @return true if job completed successfully nnn

Throws:

Throwable

runExportMain

protected void runExportMain(String[] args)
                      throws Throwable

Throws:

Throwable

runImport

boolean runImport(String[] args)
           throws Throwable

Runs an import job with the specified command line args n * @return true if job completed successfully nnn

Throws:

Throwable

testSimpleCase

public void testSimpleCase()
                    throws Throwable

Test simple replication case with column mapping n

Throws:

Throwable

testMetaExport

public void testMetaExport()
                    throws Throwable

Test export hbase:meta table n

Throws:

Throwable

testImport94Table

public void testImport94Table()
                       throws Throwable

Test import data from 0.94 exported file n

Throws:

Throwable

testExportScannerBatching

public void testExportScannerBatching()
                               throws Throwable

Test export scanner batching

Throws:

Throwable

testWithDeletes

public void testWithDeletes()
                     throws Throwable

Throws:

Throwable

testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily

public void testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily()
                                                            throws Throwable

Throws:

Throwable

testWithFilter

public void testWithFilter()
                    throws Throwable

Create a simple table, run an Export Job on it, Import with filtering on, verify counts, attempt with invalid values.

Throws:

Throwable

getCount

private int getCount(org.apache.hadoop.hbase.client.Table table,
                     org.apache.hadoop.hbase.filter.Filter filter)
              throws IOException

Count the number of keyvalues in the specified table with the given filter

Parameters:

table - the table to scan

Returns:

the number of keyvalues found n

Throws:

IOException

testImportMain

public void testImportMain()
                    throws Throwable

test main method. Import should print help and call System.exit

Throws:

Throwable

testExportScan

public void testExportScan()
                    throws Exception

Throws:

Exception

testExportMain

public void testExportMain()
                    throws Throwable

test main method. Export should print help and call System.exit

Throws:

Throwable

testKeyValueImporter

public void testKeyValueImporter()
                          throws Throwable

Test map method of Importer

Throws:

Throwable

testAddFilterAndArguments

public void testAddFilterAndArguments()
                               throws IOException

Test addFilterAndArguments method of Import This method set couple parameters into Configuration

Throws:

IOException

testDurability

public void testDurability()
                    throws Throwable

Throws:

Throwable

testTagsAddition

public void testTagsAddition()
                      throws Throwable

Add cell tags to delete mutations, run export and import tool and verify that tags are present in import table also.

Throws:

Throwable - throws Throwable.

checkWhetherTagExists

private void checkWhetherTagExists(org.apache.hadoop.hbase.TableName table,
                                   boolean tagExists)
                            throws IOException

Throws:

IOException

testTagsWithEmptyCodec

public void testTagsWithEmptyCodec()
                            throws Exception

Set hbase.client.rpc.codec and hbase.client.default.rpc.codec both to empty string This means it will use no Codec. Make sure that we don't return Tags in response.

Throws:

Exception - Exception