Class BulkDataGeneratorMapper
java.lang.Object
org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>
org.apache.hadoop.hbase.util.bulkdatagenerator.BulkDataGeneratorMapper
public class BulkDataGeneratorMapper
extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.KeyValue>
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic enum
Counter enumeration to count number of rows generated.Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper
org.apache.hadoop.mapreduce.Mapper.Context
-
Field Summary
Modifier and TypeFieldDescription(package private) static final byte[]
private static final Map<String,
org.apache.commons.math3.util.Pair<BigDecimal, BigDecimal>> private static final int
private static final int
private static final int
private static final int
private static final String
private static final Random
static final String
private static int
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionprivate void
addKeyValue
(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text, org.apache.hadoop.io.NullWritable, org.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context, org.apache.hadoop.hbase.io.ImmutableBytesWritable key, Utility.TableColumnNames columnName, String value) protected void
map
(org.apache.hadoop.io.Text key, org.apache.hadoop.io.NullWritable value, org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text, org.apache.hadoop.io.NullWritable, org.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context) Generates a single record based on value set to the key byBulkDataGeneratorRecordReader.getCurrentKey()
.protected void
setup
(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text, org.apache.hadoop.io.NullWritable, org.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context) Methods inherited from class org.apache.hadoop.mapreduce.Mapper
cleanup, run
-
Field Details
-
SPLIT_COUNT_KEY
-
ORG_ID
- See Also:
-
MAX_EVENT_ID
- See Also:
-
MAX_VEHICLE_ID
- See Also:
-
MAX_SPEED_KPH
- See Also:
-
NUM_LOCATIONS
- See Also:
-
splitCount
-
random
-
LOCATIONS
private static final Map<String,org.apache.commons.math3.util.Pair<BigDecimal, LOCATIONSBigDecimal>> -
LOCATION_KEYS
-
COLUMN_FAMILY_BYTES
-
-
Constructor Details
-
BulkDataGeneratorMapper
public BulkDataGeneratorMapper()
-
-
Method Details
-
setup
protected void setup(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text, org.apache.hadoop.io.NullWritable, throws IOException, InterruptedExceptionorg.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context) - Overrides:
setup
in classorg.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,
org.apache.hadoop.io.NullWritable, org.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue> - Throws:
IOException
InterruptedException
-
map
protected void map(org.apache.hadoop.io.Text key, org.apache.hadoop.io.NullWritable value, org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text, org.apache.hadoop.io.NullWritable, throws IOException, InterruptedExceptionorg.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context) Generates a single record based on value set to the key byBulkDataGeneratorRecordReader.getCurrentKey()
.Utility.TableColumnNames.TOOL_EVENT_ID
is first part of row key. Keeping firstUtility.SPLIT_PREFIX_LENGTH
characters as index of the record to be generated ensures that records are equally distributed across all regions of the table since region boundaries are generated in similar fashion. CheckUtility.createTable(Admin, String, int, Map)
method for region split info.- Overrides:
map
in classorg.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text,
org.apache.hadoop.io.NullWritable, org.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue> - Parameters:
key
- - The key having index of next record to be generatedvalue
- - Value associated with the key (not used)context
- - Context of the mapper container- Throws:
IOException
InterruptedException
-
addKeyValue
private void addKeyValue(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.Text, org.apache.hadoop.io.NullWritable, throws IOException, InterruptedExceptionorg.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.KeyValue>.org.apache.hadoop.mapreduce.Mapper.Context context, org.apache.hadoop.hbase.io.ImmutableBytesWritable key, Utility.TableColumnNames columnName, String value) - Throws:
IOException
InterruptedException
-