Package org.apache.hadoop.hbase.test

Class IntegrationTestWithCellVisibilityLoadAndVerify

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify

org.apache.hadoop.hbase.test.IntegrationTestWithCellVisibilityLoadAndVerify

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class IntegrationTestWithCellVisibilityLoadAndVerify
extends IntegrationTestLoadAndVerify

A large test which loads a lot of data with cell visibility, and verifies the data. Test adds 2 users with different sets of visibility labels authenticated for them. Every row (so cells in that) added with visibility expressions. In load step, 200 map tasks are launched, which in turn write loadmapper.num_to_write (default 100K) rows to an hbase table. Rows are written in blocks, for a total of 100 blocks. Verify step scans the table as both users with Authorizations. This step asserts that user can see only those rows (and so cells) with visibility for which they have label auth. This class can be run as a unit test, as an integration test, or from the command line. Originally taken from Apache Bigtop. Issue user names as comma seperated list. ./hbase IntegrationTestWithCellVisibilityLoadAndVerify -u usera,userb

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static enum	IntegrationTestWithCellVisibilityLoadAndVerify.Counters
static class	IntegrationTestWithCellVisibilityLoadAndVerify.LoadWithCellVisibilityMapper
static class	IntegrationTestWithCellVisibilityLoadAndVerify.VerifyMapper

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify

IntegrationTestLoadAndVerify.LoadMapper, IntegrationTestLoadAndVerify.VerifyReducer, IntegrationTestLoadAndVerify.WALSearcher

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field	Description
private static final char	AND
private static final String	CONFIDENTIAL
private static final String	ERROR_STR
private static final String[]	LABELS
private static final char	NOT
private static final long	NUM_TO_WRITE_DEFAULT
private static final String	NUM_TO_WRITE_KEY
private long	numRowsLoadedWithExp1
private long	numRowsLoadedWithExp2
private long	numRowsLoadWithExp3
private long	numRowsLoadWithExp4
private long	numRowsReadWithExp1
private long	numRowsReadWithExp2
private long	numRowsReadWithExp3
private long	numRowsReadWithExp4
private static final char	OR
private static final String	PRIVATE
private static final String	PUBLIC
private static final int	SCANNER_CACHING
private static final String	SECRET
private static final byte[]	TEST_FAMILY
private static final String	TEST_NAME
private static final byte[]	TEST_QUALIFIER
private static final String	TOPSECRET
private static String	USER_OPT
private static org.apache.hadoop.hbase.security.User	USER1
private static org.apache.hadoop.hbase.security.User	USER2
private static String	userNames
private static final String[]	VISIBILITY_EXPS
private static final int	VISIBILITY_EXPS_COUNT

Fields inherited from class org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify

FOUND_GROUP_KEY, SEARCHER_INPUTDIR_KEY

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor	Description
IntegrationTestWithCellVisibilityLoadAndVerify()

Method Summary

Modifier and Type	Method	Description
private void	addLabelsAndAuths()
protected void	addOptions()
protected org.apache.hadoop.mapreduce.Job	doLoad(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor htd)
protected void	doVerify(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor)
private org.apache.hadoop.mapreduce.Job	doVerify(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor, String... auths)
static void	main(String[] argv)
void	printUsage()
protected void	processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)
int	runTestFromCommandLine()
private static void	setJobScannerConf(org.apache.hadoop.mapreduce.Job job)
protected void	setMapperClass(org.apache.hadoop.mapreduce.Job job)
void	setUpCluster()

Methods inherited from class org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify

cleanUpCluster, getColumnFamilies, getTablename, getTestDir, readKeysToSearch, swapLong, testLoadAndVerify

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

cleanUp, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, setUp, setUpMonkey, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

ERROR_STR

private static final String ERROR_STR

See Also:

• Constant Field Values

NOT

private static final char NOT

See Also:

• Constant Field Values

OR

private static final char OR

See Also:

• Constant Field Values

AND

private static final char AND

See Also:

• Constant Field Values

TEST_NAME

private static final String TEST_NAME

See Also:

• Constant Field Values

CONFIDENTIAL

private static final String CONFIDENTIAL

See Also:

• Constant Field Values

TOPSECRET

private static final String TOPSECRET

See Also:

• Constant Field Values

SECRET

private static final String SECRET

See Also:

• Constant Field Values

PUBLIC

private static final String PUBLIC

See Also:

• Constant Field Values

PRIVATE

private static final String PRIVATE

See Also:

• Constant Field Values

LABELS

private static final String[] LABELS

VISIBILITY_EXPS

private static final String[] VISIBILITY_EXPS

VISIBILITY_EXPS_COUNT

private static final int VISIBILITY_EXPS_COUNT

TEST_FAMILY

private static final byte[] TEST_FAMILY

TEST_QUALIFIER

private static final byte[] TEST_QUALIFIER

NUM_TO_WRITE_KEY

private static final String NUM_TO_WRITE_KEY

See Also:

• Constant Field Values

NUM_TO_WRITE_DEFAULT

private static final long NUM_TO_WRITE_DEFAULT

See Also:

• Constant Field Values

SCANNER_CACHING

private static final int SCANNER_CACHING

See Also:

• Constant Field Values

USER_OPT

private static String USER_OPT

userNames

private static String userNames

numRowsLoadedWithExp1

private long numRowsLoadedWithExp1

numRowsLoadedWithExp2

private long numRowsLoadedWithExp2

numRowsLoadWithExp3

private long numRowsLoadWithExp3

numRowsLoadWithExp4

private long numRowsLoadWithExp4

numRowsReadWithExp1

private long numRowsReadWithExp1

numRowsReadWithExp2

private long numRowsReadWithExp2

numRowsReadWithExp3

private long numRowsReadWithExp3

numRowsReadWithExp4

private long numRowsReadWithExp4

USER1

private static org.apache.hadoop.hbase.security.User USER1

USER2

private static org.apache.hadoop.hbase.security.User USER2

Constructor Details

IntegrationTestWithCellVisibilityLoadAndVerify

public IntegrationTestWithCellVisibilityLoadAndVerify()

Method Details

setUpCluster

public void setUpCluster()
                  throws Exception

Overrides:

setUpCluster in class IntegrationTestLoadAndVerify

Throws:

Exception

addOptions

protected void addOptions()

Overrides:

addOptions in class IntegrationTestBase

addLabelsAndAuths

private void addLabelsAndAuths()
                        throws Exception

Throws:

Exception

doLoad

protected org.apache.hadoop.mapreduce.Job doLoad(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor htd)
                                          throws Exception

Overrides:

doLoad in class IntegrationTestLoadAndVerify

Throws:

Exception

setMapperClass

protected void setMapperClass(org.apache.hadoop.mapreduce.Job job)

Overrides:

setMapperClass in class IntegrationTestLoadAndVerify

doVerify

protected void doVerify(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor)
                 throws Exception

Overrides:

doVerify in class IntegrationTestLoadAndVerify

Throws:

Exception

doVerify

private org.apache.hadoop.mapreduce.Job doVerify(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor,
 String... auths)
                                          throws IOException,
InterruptedException,
ClassNotFoundException

Throws:

IOException

InterruptedException

ClassNotFoundException

setJobScannerConf

private static void setJobScannerConf(org.apache.hadoop.mapreduce.Job job)

printUsage

public void printUsage()

Overrides:

printUsage in class IntegrationTestLoadAndVerify

runTestFromCommandLine

public int runTestFromCommandLine()
                           throws Exception

Overrides:

runTestFromCommandLine in class IntegrationTestLoadAndVerify

Throws:

Exception

processOptions

protected void processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)

Overrides:

processOptions in class IntegrationTestLoadAndVerify

main

public static void main(String[] argv)
                 throws Exception

Throws:

Exception