org.apache.hadoop.hbase.mapreduce

Class TestImportTSVWithVisibilityLabels

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class TestImportTSVWithVisibilityLabels
extends Object
implements org.apache.hadoop.conf.Configurable

Field Summary

Fields

Modifier and Type	Field and Description
static HBaseClassTestRule	CLASS_RULE
private static org.apache.hadoop.conf.Configuration	conf
private static String	CONFIDENTIAL
protected static String	DELETE_AFTER_LOAD_CONF Delete the tmp directory after running doMROnTableTest.
private String	FAMILY
protected static String	FORCE_COMBINER_CONF Force use of combiner in doMROnTableTest.
private static org.slf4j.Logger	LOG
org.junit.rules.TestName	name
protected static String	NAME
private static String	PRIVATE
private static String	PUBLIC
private static String	SECRET
private static org.apache.hadoop.hbase.security.User	SUPERUSER
private static String	TOPSECRET
protected static HBaseTestingUtility	util

Constructor Summary

Constructors

Constructor and Description
TestImportTSVWithVisibilityLabels()

Method Summary

Modifier and Type	Method and Description
private static void	createLabels()
protected static org.apache.hadoop.util.Tool	doMROnTableTest(HBaseTestingUtility util, String family, String data, String[] args, int valueMultiplier)
protected static org.apache.hadoop.util.Tool	doMROnTableTest(HBaseTestingUtility util, String family, String data, String[] args, int valueMultiplier, int expectedKVCount) Run an ImportTsv job and perform basic validation on the results.
org.apache.hadoop.conf.Configuration	getConf()
private static int	getKVCountFromHfile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p) Method returns the total KVs in given hfile
private void	issueDeleteAndVerifyData(org.apache.hadoop.hbase.TableName tableName)
static void	provisionCluster()
static void	releaseCluster()
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	testBulkOutputWithInvalidLabels()
void	testBulkOutputWithTsvImporterTextMapper()
void	testBulkOutputWithTsvImporterTextMapperWithInvalidLabels()
void	testMROnTable()
void	testMROnTableWithBulkload()
void	testMROnTableWithDeletes()
void	testMRWithOutputFormat()
private static void	validateHFiles(org.apache.hadoop.fs.FileSystem fs, String outputPath, String family, int expectedKVCount) Confirm ImportTsv via HFiles on fs.
private static void	validateTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.TableName tableName, String family, int valueMultiplier) Confirm ImportTsv via data in online table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

LOG

private static final org.slf4j.Logger LOG

NAME

protected static final String NAME

util

protected static HBaseTestingUtility util

DELETE_AFTER_LOAD_CONF

protected static final String DELETE_AFTER_LOAD_CONF

Delete the tmp directory after running doMROnTableTest. Boolean. Default is false.

FORCE_COMBINER_CONF

protected static final String FORCE_COMBINER_CONF

Force use of combiner in doMROnTableTest. Boolean. Default is true.

FAMILY

private final String FAMILY

See Also:

Constant Field Values

TOPSECRET

private static final String TOPSECRET

See Also:

Constant Field Values

PUBLIC

private static final String PUBLIC

See Also:

Constant Field Values

PRIVATE

private static final String PRIVATE

See Also:

Constant Field Values

CONFIDENTIAL

private static final String CONFIDENTIAL

See Also:

Constant Field Values

SECRET

private static final String SECRET

See Also:

Constant Field Values

SUPERUSER

private static org.apache.hadoop.hbase.security.User SUPERUSER

conf

private static org.apache.hadoop.conf.Configuration conf

name

public org.junit.rules.TestName name

Constructor Detail

TestImportTSVWithVisibilityLabels

public TestImportTSVWithVisibilityLabels()

Method Detail

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

provisionCluster

public static void provisionCluster()
                             throws Exception

Throws:

Exception

createLabels

private static void createLabels()
                          throws IOException,
                                 InterruptedException

Throws:

IOException

InterruptedException

releaseCluster

public static void releaseCluster()
                           throws Exception

Throws:

Exception

testMROnTable

public void testMROnTable()
                   throws Exception

Throws:

Exception

testMROnTableWithDeletes

public void testMROnTableWithDeletes()
                              throws Exception

Throws:

Exception

issueDeleteAndVerifyData

private void issueDeleteAndVerifyData(org.apache.hadoop.hbase.TableName tableName)
                               throws IOException

Throws:

IOException

testMROnTableWithBulkload

public void testMROnTableWithBulkload()
                               throws Exception

Throws:

Exception

testBulkOutputWithTsvImporterTextMapper

public void testBulkOutputWithTsvImporterTextMapper()
                                             throws Exception

Throws:

Exception

testMRWithOutputFormat

public void testMRWithOutputFormat()
                            throws Exception

Throws:

Exception

testBulkOutputWithInvalidLabels

public void testBulkOutputWithInvalidLabels()
                                     throws Exception

Throws:

Exception

testBulkOutputWithTsvImporterTextMapperWithInvalidLabels

public void testBulkOutputWithTsvImporterTextMapperWithInvalidLabels()
                                                              throws Exception

Throws:

Exception

doMROnTableTest

protected static org.apache.hadoop.util.Tool doMROnTableTest(HBaseTestingUtility util,
                                                             String family,
                                                             String data,
                                                             String[] args,
                                                             int valueMultiplier)
                                                      throws Exception

Throws:

Exception

doMROnTableTest

protected static org.apache.hadoop.util.Tool doMROnTableTest(HBaseTestingUtility util,
                                                             String family,
                                                             String data,
                                                             String[] args,
                                                             int valueMultiplier,
                                                             int expectedKVCount)
                                                      throws Exception

Run an ImportTsv job and perform basic validation on the results. Returns the ImportTsv Tool instance so that other tests can inspect it for further validation as necessary. This method is static to insure non-reliance on instance's util/conf facilities.

Parameters:

args - Any arguments to pass BEFORE inputFile path is appended.

expectedKVCount - Expected KV count. pass -1 to skip the kvcount check

Returns:

The Tool instance used to run the test.

Throws:

Exception

validateHFiles

private static void validateHFiles(org.apache.hadoop.fs.FileSystem fs,
                                   String outputPath,
                                   String family,
                                   int expectedKVCount)
                            throws IOException

Confirm ImportTsv via HFiles on fs.

Throws:

IOException

validateTable

private static void validateTable(org.apache.hadoop.conf.Configuration conf,
                                  org.apache.hadoop.hbase.TableName tableName,
                                  String family,
                                  int valueMultiplier)
                           throws IOException

Confirm ImportTsv via data in online table.

Throws:

IOException

getKVCountFromHfile

private static int getKVCountFromHfile(org.apache.hadoop.fs.FileSystem fs,
                                       org.apache.hadoop.fs.Path p)
                                throws IOException

Method returns the total KVs in given hfile

Parameters:

fs - File System

p - HFile path

Returns:

KV count in the given hfile

Throws:

IOException