Package org.apache.hadoop.hbase.mapreduce

Class TestImportExportBase

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestImportExportBase

Direct Known Subclasses:

TestImportExport

public class TestImportExportBase
extends Object

Base class for testing Import/Export. Shared logic without @BeforeAll/@AfterAll to allow subclasses to manage their own lifecycle.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TestImportExportBase.MetadataController
private static class	TestImportExportBase.TableWALActionListener	This listens to the TestImportExportBase.TableWALActionListener.visitLogEntryBeforeWrite(RegionInfo, WALKey, WALEdit) to identify that an entry is written to the Write Ahead Log for the given table.

Field Summary

Fields

Modifier and Type	Field	Description
private static final String	EXPORT_BATCH_SIZE
protected final org.apache.hadoop.hbase.TableName	EXPORT_TABLE
private static final byte[]	FAMILYA
private static final String	FAMILYA_STRING
private static final byte[]	FAMILYB
private static final String	FAMILYB_STRING
protected static String	FQ_OUTPUT_DIR
protected final org.apache.hadoop.hbase.TableName	IMPORT_TABLE
private static final org.slf4j.Logger	LOG
protected String	name
private static final long	now
protected static final String	OUTPUT_DIR
private static final byte[]	QUAL
private static final byte[]	ROW1
private static final byte[]	ROW2
private static final byte[]	ROW3
static final String	TEST_ATTR
static final String	TEST_TAG
static final byte	TEST_TAG_TYPE
protected static final HBaseTestingUtil	UTIL

Constructor Summary

Constructors

Constructor	Description
TestImportExportBase()

Method Summary

Modifier and Type	Method	Description
void	announce(org.junit.jupiter.api.TestInfo testInfo)
private void	checkWhetherTagExists(org.apache.hadoop.hbase.TableName table, boolean tagExists)
void	cleanup()
private int	getCount(org.apache.hadoop.hbase.client.Table table, org.apache.hadoop.hbase.filter.Filter filter)	Count the number of keyvalues in the specified table with the given filter
protected boolean	runExport(String[] args)	Runs an export job with the specified command line args
protected void	runExportMain(String[] args)
(package private) boolean	runImport(String[] args)	Runs an import job with the specified command line args
protected static void	setUpBeforeClass()
void	testAddFilterAndArguments()	Test addFilterAndArguments method of Import This method set couple parameters into Configuration
void	testBulkImportAndLargeResult()	Create a simple table, run an Export Job on it, Import with bulk output and enable largeResult
void	testDurability()
void	testExportMain()	test main method.
void	testExportScan()
void	testExportScannerBatching()	Test export scanner batching
void	testImport94Table()	Test import data from 0.94 exported file
void	testImportMain()	test main method.
void	testKeyValueImporter()	Test map method of Importer
void	testMetaExport()	Test export hbase:meta table
void	testSimpleCase()	Test simple replication case with column mapping
void	testTagsAddition()	Add cell tags to delete mutations, run export and import tool and verify that tags are present in import table also.
void	testTagsWithEmptyCodec()	Set hbase.client.rpc.codec and hbase.client.default.rpc.codec both to empty string This means it will use no Codec.
void	testWithDeletes()
void	testWithFilter()	Create a simple table, run an Export Job on it, Import with filtering on, verify counts, attempt with invalid values.
void	testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

UTIL

protected static final HBaseTestingUtil UTIL

ROW1

private static final byte[] ROW1

ROW2

private static final byte[] ROW2

ROW3

private static final byte[] ROW3

FAMILYA_STRING

private static final String FAMILYA_STRING

See Also:

• Constant Field Values

FAMILYB_STRING

private static final String FAMILYB_STRING

See Also:

• Constant Field Values

FAMILYA

private static final byte[] FAMILYA

FAMILYB

private static final byte[] FAMILYB

QUAL

private static final byte[] QUAL

OUTPUT_DIR

protected static final String OUTPUT_DIR

See Also:

• Constant Field Values

FQ_OUTPUT_DIR

protected static String FQ_OUTPUT_DIR

EXPORT_BATCH_SIZE

private static final String EXPORT_BATCH_SIZE

See Also:

• Constant Field Values

now

private static final long now

EXPORT_TABLE

protected final org.apache.hadoop.hbase.TableName EXPORT_TABLE

IMPORT_TABLE

protected final org.apache.hadoop.hbase.TableName IMPORT_TABLE

TEST_TAG_TYPE

public static final byte TEST_TAG_TYPE

See Also:

• Constant Field Values

TEST_ATTR

public static final String TEST_ATTR

See Also:

• Constant Field Values

TEST_TAG

public static final String TEST_TAG

See Also:

• Constant Field Values

name

protected String name

Constructor Details

TestImportExportBase

public TestImportExportBase()

Method Details

announce

@BeforeEach
public void announce(org.junit.jupiter.api.TestInfo testInfo)

cleanup

@AfterEach
public void cleanup()
             throws Throwable

Throws:

Throwable

setUpBeforeClass

protected static void setUpBeforeClass()
                                throws Exception

Throws:

Exception

runExport

protected boolean runExport(String[] args)
                     throws Throwable

Runs an export job with the specified command line args

Returns:

true if job completed successfully

Throws:

Throwable

runExportMain

protected void runExportMain(String[] args)
                      throws Throwable

Throws:

Throwable

runImport

boolean runImport(String[] args)
           throws Throwable

Runs an import job with the specified command line args

Returns:

true if job completed successfully

Throws:

Throwable

testSimpleCase

@Test
public void testSimpleCase()
                    throws Throwable

Test simple replication case with column mapping

Throws:

Throwable

testMetaExport

@Test
public void testMetaExport()
                    throws Throwable

Test export hbase:meta table

Throws:

Throwable

testImport94Table

@Test
public void testImport94Table()
                       throws Throwable

Test import data from 0.94 exported file

Throws:

Throwable

testExportScannerBatching

@Test
public void testExportScannerBatching()
                               throws Throwable

Test export scanner batching

Throws:

Throwable

testWithDeletes

@Test
public void testWithDeletes()
                     throws Throwable

Throws:

Throwable

testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily

@Test
public void testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily()
                                                            throws Throwable

Throws:

Throwable

testWithFilter

@Test
public void testWithFilter()
                    throws Throwable

Create a simple table, run an Export Job on it, Import with filtering on, verify counts, attempt with invalid values.

Throws:

Throwable

testBulkImportAndLargeResult

@Test
public void testBulkImportAndLargeResult()
                                  throws Throwable

Create a simple table, run an Export Job on it, Import with bulk output and enable largeResult

Throws:

Throwable

getCount

private int getCount(org.apache.hadoop.hbase.client.Table table,
 org.apache.hadoop.hbase.filter.Filter filter)
              throws IOException

Count the number of keyvalues in the specified table with the given filter

Parameters:

table - the table to scan

Returns:

the number of keyvalues found

Throws:

IOException

testImportMain

@Test
public void testImportMain()
                    throws Throwable

test main method. Import should print help and call System.exit

Throws:

Throwable

testExportScan

@Test
public void testExportScan()
                    throws Exception

Throws:

Exception

testExportMain

@Test
public void testExportMain()
                    throws Throwable

test main method. Export should print help and call System.exit

Throws:

Throwable

testKeyValueImporter

@Test
public void testKeyValueImporter()
                          throws Throwable

Test map method of Importer

Throws:

Throwable

testAddFilterAndArguments

@Test
public void testAddFilterAndArguments()
                               throws IOException

Test addFilterAndArguments method of Import This method set couple parameters into Configuration

Throws:

IOException

testDurability

@Test
public void testDurability()
                    throws Throwable

Throws:

Throwable

testTagsAddition

@Test
public void testTagsAddition()
                      throws Throwable

Add cell tags to delete mutations, run export and import tool and verify that tags are present in import table also.

Throws:

Throwable - throws Throwable.

checkWhetherTagExists

private void checkWhetherTagExists(org.apache.hadoop.hbase.TableName table,
 boolean tagExists)
                            throws IOException

Throws:

IOException

testTagsWithEmptyCodec

@Test
public void testTagsWithEmptyCodec()
                            throws Exception

Set hbase.client.rpc.codec and hbase.client.default.rpc.codec both to empty string This means it will use no Codec. Make sure that we don't return Tags in response.

Throws:

Exception - Exception