Package org.apache.hadoop.hbase.mapreduce

Class HFileOutputFormat2TestBase

java.lang.Object

org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2TestBase

Direct Known Subclasses:

MRIncrementalLoadTestBase, TestHFileOutputFormat2

public abstract class HFileOutputFormat2TestBase
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	HFileOutputFormat2TestBase.RandomKVGeneratingMapper	Simple mapper that makes KeyValue output.
protected static class	HFileOutputFormat2TestBase.RandomPutGeneratingMapper	Simple mapper that makes Put output.

Field Summary

Fields

Modifier and Type	Field	Description
protected static final int	DEFAULT_VALUE_LENGTH
protected static final byte[][]	FAMILIES
static final byte[]	FAMILY_NAME
protected static final int	ROWSPERSPLIT
protected static final org.apache.hadoop.hbase.TableName[]	TABLE_NAMES
protected static HBaseTestingUtil	UTIL

Constructor Summary

Constructors

Constructor	Description
HFileOutputFormat2TestBase()

Method Summary

Modifier and Type	Method	Description
protected static byte[]	generateData(Random r, int length)	This method takes some time and is done inline uploading data.
protected static byte[][]	generateRandomSplitKeys(int numKeys)
protected static byte[][]	generateRandomStartKeys(int numKeys)
protected static void	runIncrementalPELoad(org.apache.hadoop.conf.Configuration conf, List<org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.TableInfo> tableInfo, org.apache.hadoop.fs.Path outDir, boolean putSortReducer)
protected static void	setupRandomGeneratorMapper(org.apache.hadoop.mapreduce.Job job, boolean putSortReducer)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

ROWSPERSPLIT

protected static final int ROWSPERSPLIT

See Also:

• Constant Field Values

DEFAULT_VALUE_LENGTH

protected static final int DEFAULT_VALUE_LENGTH

See Also:

• Constant Field Values

FAMILY_NAME

public static final byte[] FAMILY_NAME

FAMILIES

protected static final byte[][] FAMILIES

TABLE_NAMES

protected static final org.apache.hadoop.hbase.TableName[] TABLE_NAMES

UTIL

protected static HBaseTestingUtil UTIL

Constructor Details

HFileOutputFormat2TestBase

public HFileOutputFormat2TestBase()

Method Details

setupRandomGeneratorMapper

protected static void setupRandomGeneratorMapper(org.apache.hadoop.mapreduce.Job job,
 boolean putSortReducer)

generateRandomStartKeys

protected static byte[][] generateRandomStartKeys(int numKeys)

generateData

protected static byte[] generateData(Random r,
 int length)

This method takes some time and is done inline uploading data. For example, doing the mapfile test, generation of the key and value consumes about 30% of CPU time.

Returns:

Generated random value to insert into a table cell.

generateRandomSplitKeys

protected static byte[][] generateRandomSplitKeys(int numKeys)

runIncrementalPELoad

protected static void runIncrementalPELoad(org.apache.hadoop.conf.Configuration conf,
 List<org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.TableInfo> tableInfo,
 org.apache.hadoop.fs.Path outDir,
 boolean putSortReducer)
                                    throws IOException,
InterruptedException,
ClassNotFoundException

Throws:

IOException

InterruptedException

ClassNotFoundException