Package org.apache.hadoop.hbase.mapreduce

Class TestImportTSVWithVisibilityLabels

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

@Tag("org.apache.hadoop.hbase.testclassification.MapReduceTests") @Tag("org.apache.hadoop.hbase.testclassification.LargeTests")
public class TestImportTSVWithVisibilityLabels
extends Object
implements org.apache.hadoop.conf.Configurable

Field Summary

Fields

Modifier and Type	Field	Description
private static org.apache.hadoop.conf.Configuration	conf
private static final String	CONFIDENTIAL
protected static final String	DELETE_AFTER_LOAD_CONF	Delete the tmp directory after running doMROnTableTest.
private final String	FAMILY
protected static final String	FORCE_COMBINER_CONF	Force use of combiner in doMROnTableTest.
private static final org.slf4j.Logger	LOG
protected static final String	NAME
private static final String	PRIVATE
private static final String	PUBLIC
private static final String	SECRET
private static org.apache.hadoop.hbase.security.User	SUPERUSER
private static final String	TOPSECRET
protected static HBaseTestingUtil	util

Constructor Summary

Constructors

Constructor	Description
TestImportTSVWithVisibilityLabels()

Method Summary

Modifier and Type	Method	Description
private static void	createLabels()
protected static org.apache.hadoop.util.Tool	doMROnTableTest(HBaseTestingUtil util, String family, String data, String[] args, int valueMultiplier)
protected static org.apache.hadoop.util.Tool	doMROnTableTest(HBaseTestingUtil util, String family, String data, String[] args, int valueMultiplier, int expectedKVCount)	Run an ImportTsv job and perform basic validation on the results.
org.apache.hadoop.conf.Configuration	getConf()
private static int	getKVCountFromHfile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p)	Method returns the total KVs in given hfile
private void	issueDeleteAndVerifyData(org.apache.hadoop.hbase.TableName tableName)
static void	provisionCluster()
static void	releaseCluster()
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	testBulkOutputWithInvalidLabels(org.junit.jupiter.api.TestInfo testInfo)
void	testBulkOutputWithTsvImporterTextMapper(org.junit.jupiter.api.TestInfo testInfo)
void	testBulkOutputWithTsvImporterTextMapperWithInvalidLabels(org.junit.jupiter.api.TestInfo testInfo)
void	testMROnTable(org.junit.jupiter.api.TestInfo testInfo)
void	testMROnTableWithBulkload(org.junit.jupiter.api.TestInfo testInfo)
void	testMROnTableWithDeletes(org.junit.jupiter.api.TestInfo testInfo)
void	testMRWithOutputFormat(org.junit.jupiter.api.TestInfo testInfo)
private static void	validateHFiles(org.apache.hadoop.fs.FileSystem fs, String outputPath, String family, int expectedKVCount)	Confirm ImportTsv via HFiles on fs.
private static void	validateTable(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.TableName tableName, String family, int valueMultiplier)	Confirm ImportTsv via data in online table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

NAME

protected static final String NAME

util

protected static HBaseTestingUtil util

DELETE_AFTER_LOAD_CONF

protected static final String DELETE_AFTER_LOAD_CONF

Delete the tmp directory after running doMROnTableTest. Boolean. Default is false.

FORCE_COMBINER_CONF

protected static final String FORCE_COMBINER_CONF

Force use of combiner in doMROnTableTest. Boolean. Default is true.

FAMILY

private final String FAMILY

See Also:

• Constant Field Values

TOPSECRET

private static final String TOPSECRET

See Also:

• Constant Field Values

PUBLIC

private static final String PUBLIC

See Also:

• Constant Field Values

PRIVATE

private static final String PRIVATE

See Also:

• Constant Field Values

CONFIDENTIAL

private static final String CONFIDENTIAL

See Also:

• Constant Field Values

SECRET

private static final String SECRET

See Also:

• Constant Field Values

SUPERUSER

private static org.apache.hadoop.hbase.security.User SUPERUSER

conf

private static org.apache.hadoop.conf.Configuration conf

Constructor Details

TestImportTSVWithVisibilityLabels

public TestImportTSVWithVisibilityLabels()

Method Details

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

provisionCluster

@BeforeAll
public static void provisionCluster()
                             throws Exception

Throws:

Exception

createLabels

private static void createLabels()
                          throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

releaseCluster

@AfterAll
public static void releaseCluster()
                           throws Exception

Throws:

Exception

testMROnTable

@Test
public void testMROnTable(org.junit.jupiter.api.TestInfo testInfo)
                   throws Exception

Throws:

Exception

testMROnTableWithDeletes

@Test
public void testMROnTableWithDeletes(org.junit.jupiter.api.TestInfo testInfo)
                              throws Exception

Throws:

Exception

issueDeleteAndVerifyData

private void issueDeleteAndVerifyData(org.apache.hadoop.hbase.TableName tableName)
                               throws IOException

Throws:

IOException

testMROnTableWithBulkload

@Test
public void testMROnTableWithBulkload(org.junit.jupiter.api.TestInfo testInfo)
                               throws Exception

Throws:

Exception

testBulkOutputWithTsvImporterTextMapper

@Test
public void testBulkOutputWithTsvImporterTextMapper(org.junit.jupiter.api.TestInfo testInfo)
                                             throws Exception

Throws:

Exception

testMRWithOutputFormat

@Test
public void testMRWithOutputFormat(org.junit.jupiter.api.TestInfo testInfo)
                            throws Exception

Throws:

Exception

testBulkOutputWithInvalidLabels

@Test
public void testBulkOutputWithInvalidLabels(org.junit.jupiter.api.TestInfo testInfo)
                                     throws Exception

Throws:

Exception

testBulkOutputWithTsvImporterTextMapperWithInvalidLabels

@Test
public void testBulkOutputWithTsvImporterTextMapperWithInvalidLabels(org.junit.jupiter.api.TestInfo testInfo)
                                                              throws Exception

Throws:

Exception

doMROnTableTest

protected static org.apache.hadoop.util.Tool doMROnTableTest(HBaseTestingUtil util,
 String family,
 String data,
 String[] args,
 int valueMultiplier)
                                                      throws Exception

Throws:

Exception

doMROnTableTest

protected static org.apache.hadoop.util.Tool doMROnTableTest(HBaseTestingUtil util,
 String family,
 String data,
 String[] args,
 int valueMultiplier,
 int expectedKVCount)
                                                      throws Exception

Run an ImportTsv job and perform basic validation on the results. Returns the ImportTsv Tool instance so that other tests can inspect it for further validation as necessary. This method is static to insure non-reliance on instance's util/conf facilities. Any arguments to pass BEFORE inputFile path is appended.

Parameters:

expectedKVCount - Expected KV count. pass -1 to skip the kvcount check

Returns:

The Tool instance used to run the test.

Throws:

Exception

validateHFiles

private static void validateHFiles(org.apache.hadoop.fs.FileSystem fs,
 String outputPath,
 String family,
 int expectedKVCount)
                            throws IOException

Confirm ImportTsv via HFiles on fs.

Throws:

IOException

validateTable

private static void validateTable(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.TableName tableName,
 String family,
 int valueMultiplier)
                           throws IOException

Confirm ImportTsv via data in online table.

Throws:

IOException

getKVCountFromHfile

private static int getKVCountFromHfile(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path p)
                                throws IOException

Method returns the total KVs in given hfile

Parameters:

fs - File System

p - HFile path

Returns:

KV count in the given hfile

Throws:

IOException