org.apache.hadoop.hbase.mapreduce

Class TextSortReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<ImmutableBytesWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,KeyValue>

org.apache.hadoop.hbase.mapreduce.TextSortReducer

@InterfaceAudience.Public
public class TextSortReducer
extends org.apache.hadoop.mapreduce.Reducer<ImmutableBytesWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,KeyValue>

Emits Sorted KeyValues. Parse the passed text and creates KeyValues. Sorts them before emit.

See Also:

HFileOutputFormat2, CellSortReducer, PutSortReducer

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.mapreduce.Counter	badLineCount
private String	cellVisibilityExpr Cell visibility expr
private CellCreator	kvCreator
private ImportTsv.TsvParser	parser
private String	separator Column seperator
private boolean	skipBadLines Should skip bad lines
private long	ts Timestamp for all inserted rows
private long	ttl Cell TTL

Constructor Summary

Constructors

Constructor and Description
TextSortReducer()

Method Summary

Modifier and Type	Method and Description
protected void	doSetup(org.apache.hadoop.mapreduce.Reducer.Context context, org.apache.hadoop.conf.Configuration conf) Handles common parameter initialization that a subclass might want to leverage.
org.apache.hadoop.mapreduce.Counter	getBadLineCount()
boolean	getSkipBadLines()
long	getTs()
void	incrementBadLineCount(int count)
protected void	reduce(ImmutableBytesWritable rowKey, Iterable<org.apache.hadoop.io.Text> lines, org.apache.hadoop.mapreduce.Reducer.Context context)
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context) Handles initializing this class with objects specific to it (i.e., the parser).

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ts

private long ts

Timestamp for all inserted rows

separator

private String separator

Column seperator

skipBadLines

private boolean skipBadLines

Should skip bad lines

badLineCount

private org.apache.hadoop.mapreduce.Counter badLineCount

parser

private ImportTsv.TsvParser parser

cellVisibilityExpr

private String cellVisibilityExpr

Cell visibility expr

ttl

private long ttl

Cell TTL

kvCreator

private CellCreator kvCreator

Constructor Detail

TextSortReducer

public TextSortReducer()

Method Detail

getTs

public long getTs()

getSkipBadLines

public boolean getSkipBadLines()

getBadLineCount

public org.apache.hadoop.mapreduce.Counter getBadLineCount()

incrementBadLineCount

public void incrementBadLineCount(int count)

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)

Handles initializing this class with objects specific to it (i.e., the parser). Common initialization that might be leveraged by a subsclass is done in doSetup. Hence a subclass may choose to override this method and call doSetup as well before handling it's own custom params.

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<ImmutableBytesWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,KeyValue>

Parameters:

context -

doSetup

protected void doSetup(org.apache.hadoop.mapreduce.Reducer.Context context,
                       org.apache.hadoop.conf.Configuration conf)

Handles common parameter initialization that a subclass might want to leverage.

Parameters:

context -

conf -

reduce

protected void reduce(ImmutableBytesWritable rowKey,
                      Iterable<org.apache.hadoop.io.Text> lines,
                      org.apache.hadoop.mapreduce.Reducer.Context context)
               throws IOException,
                      InterruptedException

Overrides:

reduce in class org.apache.hadoop.mapreduce.Reducer<ImmutableBytesWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,KeyValue>

Throws:

IOException

InterruptedException