Package org.apache.hadoop.hbase.util.bulkdatagenerator

Class BulkDataGeneratorMapper

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>

org.apache.hadoop.hbase.util.bulkdatagenerator.BulkDataGeneratorMapper

public class BulkDataGeneratorMapper
extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	BulkDataGeneratorMapper.Counters	Counter enumeration to count number of rows generated.

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

org.apache.hadoop.mapreduce.Mapper.Context

Field Summary

Fields

Modifier and Type	Field	Description
(package private) static final byte[]	COLUMN_FAMILY_BYTES
private static final List<String>	LOCATION_KEYS
private static final Map<String,org.apache.commons.math3.util.Pair<BigDecimal,BigDecimal>>	LOCATIONS
private static final int	MAX_EVENT_ID
private static final int	MAX_SPEED_KPH
private static final int	MAX_VEHICLE_ID
private static final int	NUM_LOCATIONS
private static final String	ORG_ID
private static final Random	random
static final String	SPLIT_COUNT_KEY
private static int	splitCount

Constructor Summary

Constructors

Constructor	Description
BulkDataGeneratorMapper()

Method Summary

Modifier and Type	Method	Description
private void	addKeyValue(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context, org.apache.hadoop.hbase.io.ImmutableBytesWritable key, Utility.TableColumnNames columnName, String value)
protected void	map(org.apache.hadoop.io.Text key, org.apache.hadoop.io.NullWritable value, org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context)	Generates a single record based on value set to the key by BulkDataGeneratorRecordReader.getCurrentKey().
protected void	setup(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

SPLIT_COUNT_KEY

public static final String SPLIT_COUNT_KEY

ORG_ID

private static final String ORG_ID

See Also:

• Constant Field Values

MAX_EVENT_ID

private static final int MAX_EVENT_ID

See Also:

• Constant Field Values

MAX_VEHICLE_ID

private static final int MAX_VEHICLE_ID

See Also:

• Constant Field Values

MAX_SPEED_KPH

private static final int MAX_SPEED_KPH

See Also:

• Constant Field Values

NUM_LOCATIONS

private static final int NUM_LOCATIONS

See Also:

• Constant Field Values

splitCount

private static int splitCount

random

private static final Random random

LOCATIONS

private static final Map<String,org.apache.commons.math3.util.Pair<BigDecimal,BigDecimal>> LOCATIONS

LOCATION_KEYS

private static final List<String> LOCATION_KEYS

COLUMN_FAMILY_BYTES

static final byte[] COLUMN_FAMILY_BYTES

Constructor Details

BulkDataGeneratorMapper

public BulkDataGeneratorMapper()

Method Details

setup

protected void setup(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context)
              throws IOException,
InterruptedException

Overrides:

setup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>

Throws:

IOException

InterruptedException

map

protected void map(org.apache.hadoop.io.Text key,
 org.apache.hadoop.io.NullWritable value,
 org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context)
            throws IOException,
InterruptedException

Generates a single record based on value set to the key by BulkDataGeneratorRecordReader.getCurrentKey(). Utility.TableColumnNames.TOOL_EVENT_ID is first part of row key. Keeping first Utility.SPLIT_PREFIX_LENGTH characters as index of the record to be generated ensures that records are equally distributed across all regions of the table since region boundaries are generated in similar fashion. Check Utility.createTable(Admin, String, int, Map) method for region split info.

Overrides:

map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>

Parameters:

key - - The key having index of next record to be generated

value - - Value associated with the key (not used)

context - - Context of the mapper container

Throws:

IOException

InterruptedException

addKeyValue

private void addKeyValue(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context,
 org.apache.hadoop.hbase.io.ImmutableBytesWritable key,
 Utility.TableColumnNames columnName,
 String value)
                  throws IOException,
InterruptedException

Throws:

IOException

InterruptedException