org.apache.hadoop.hbase.test

Class IntegrationTestBigLinkedList.Generator

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.Generator

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

IntegrationTestBigLinkedListWithVisibility.VisibilityGenerator

Enclosing class:

IntegrationTestBigLinkedList

static class IntegrationTestBigLinkedList.Generator
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

A Map only job that generates random linked list and stores them.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	IntegrationTestBigLinkedList.Generator.Counts
(package private) static class	IntegrationTestBigLinkedList.Generator.GeneratorInputFormat
(package private) static class	IntegrationTestBigLinkedList.Generator.GeneratorMapper Some ASCII art time:
(package private) static class	IntegrationTestBigLinkedList.Generator.OneFilePerMapperSFIF<K,V> Ensure output files from prev-job go to map inputs for current job

Field Summary

Fields

Modifier and Type	Field and Description
static String	BIG_FAMILY_VALUE_SIZE_KEY Set this configuration if you want to scale up the size of test data quickly.
org.apache.hadoop.mapreduce.Job	job
private static org.slf4j.Logger	LOG
static String	MULTIPLE_UNEVEN_COLUMNFAMILIES_KEY Set this configuration if you want to test single-column family flush works.
static String	USAGE

Constructor Summary

Constructors

Constructor and Description
Generator()

Method Summary

Modifier and Type	Method and Description
protected void	createSchema()
protected boolean	jobCompletion(org.apache.hadoop.mapreduce.Job job)
int	run(int numMappers, long numNodes, org.apache.hadoop.fs.Path tmpOutput, Integer width, Integer wrapMultiplier, Integer numWalkers)
int	run(String[] args)
int	runGenerator(int numMappers, long numNodes, org.apache.hadoop.fs.Path tmpOutput, Integer width, Integer wrapMultiplier, Integer numWalkers)
int	runRandomInputGenerator(int numMappers, long numNodes, org.apache.hadoop.fs.Path tmpOutput, Integer width, Integer wrapMultiplier, Integer numWalkers)
protected void	setMapperForGenerator(org.apache.hadoop.mapreduce.Job job)
boolean	verify()

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

LOG

private static final org.slf4j.Logger LOG

MULTIPLE_UNEVEN_COLUMNFAMILIES_KEY

public static final String MULTIPLE_UNEVEN_COLUMNFAMILIES_KEY

Set this configuration if you want to test single-column family flush works. If set, we will add a big column family and a small column family on either side of the usual ITBLL 'meta' column family. When we write out the ITBLL, we will also add to the big column family a value bigger than that for ITBLL and for small, something way smaller. The idea is that when flush-by-column family rather than by region is enabled, we can see if ITBLL is broke in any way. Here is how you would pass it:

$ ./bin/hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList -Dgenerator.multiple.columnfamilies=true generator 1 10 g

See Also:

Constant Field Values

BIG_FAMILY_VALUE_SIZE_KEY

public static final String BIG_FAMILY_VALUE_SIZE_KEY

Set this configuration if you want to scale up the size of test data quickly.

$ ./bin/hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList -Dgenerator.big.family.value.size=1024 generator 1 10 output

See Also:

Constant Field Values

USAGE

public static final String USAGE

job

public org.apache.hadoop.mapreduce.Job job

Constructor Detail

Generator

Generator()

Method Detail

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

createSchema

protected void createSchema()
                     throws IOException

Throws:

IOException

runRandomInputGenerator

public int runRandomInputGenerator(int numMappers,
                                   long numNodes,
                                   org.apache.hadoop.fs.Path tmpOutput,
                                   Integer width,
                                   Integer wrapMultiplier,
                                   Integer numWalkers)
                            throws Exception

Throws:

Exception

runGenerator

public int runGenerator(int numMappers,
                        long numNodes,
                        org.apache.hadoop.fs.Path tmpOutput,
                        Integer width,
                        Integer wrapMultiplier,
                        Integer numWalkers)
                 throws Exception

Throws:

Exception

jobCompletion

protected boolean jobCompletion(org.apache.hadoop.mapreduce.Job job)
                         throws IOException,
                                InterruptedException,
                                ClassNotFoundException

Throws:

IOException

InterruptedException

ClassNotFoundException

setMapperForGenerator

protected void setMapperForGenerator(org.apache.hadoop.mapreduce.Job job)

run

public int run(int numMappers,
               long numNodes,
               org.apache.hadoop.fs.Path tmpOutput,
               Integer width,
               Integer wrapMultiplier,
               Integer numWalkers)
        throws Exception

Throws:

Exception

verify

public boolean verify()