Package org.apache.hadoop.hbase.mapreduce

Class TsvImporterMapper

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>

org.apache.hadoop.hbase.mapreduce.TsvImporterMapper

@Public
public class TsvImporterMapper
extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>

Write table content out to files in hdfs.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

org.apache.hadoop.mapreduce.Mapper.Context

Field Summary

Fields

Modifier and Type	Field	Description
private org.apache.hadoop.mapreduce.Counter	badLineCount
protected String	cellVisibilityExpr
protected org.apache.hadoop.conf.Configuration	conf
private String	hfileOutPath
protected CellCreator	kvCreator
private static final org.slf4j.Logger	LOG
private boolean	logBadLines
protected ImportTsv.TsvParser	parser
private String	separator	Column seperator
private boolean	skipBadLines	Should skip bad lines
private boolean	skipEmptyColumns	Should skip empty columns
private List<Tag>	tags	List of cell tags
protected long	ts	Timestamp for all inserted rows
protected long	ttl

Constructor Summary

Constructors

Constructor	Description
TsvImporterMapper()

Method Summary

Modifier and Type	Method	Description
protected void	doSetup(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>.org.apache.hadoop.mapreduce.Mapper.Context context)	Handles common parameter initialization that a subclass might want to leverage.
org.apache.hadoop.mapreduce.Counter	getBadLineCount()
boolean	getSkipBadLines()
long	getTs()
void	incrementBadLineCount(int count)
void	map(org.apache.hadoop.io.LongWritable offset, org.apache.hadoop.io.Text value, org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>.org.apache.hadoop.mapreduce.Mapper.Context context)	Convert a line of TSV text into an HBase table row.
protected void	populatePut(byte[] lineBytes, ImportTsv.TsvParser.ParsedLine parsed, Put put, int i)
protected void	setup(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>.org.apache.hadoop.mapreduce.Mapper.Context context)	Handles initializing this class with objects specific to it (i.e., the parser).

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

ts

protected long ts

Timestamp for all inserted rows

separator

private String separator

Column seperator

skipBadLines

private boolean skipBadLines

Should skip bad lines

skipEmptyColumns

private boolean skipEmptyColumns

Should skip empty columns

badLineCount

private org.apache.hadoop.mapreduce.Counter badLineCount

logBadLines

private boolean logBadLines

parser

protected ImportTsv.TsvParser parser

conf

protected org.apache.hadoop.conf.Configuration conf

cellVisibilityExpr

protected String cellVisibilityExpr

ttl

protected long ttl

kvCreator

protected CellCreator kvCreator

hfileOutPath

private String hfileOutPath

tags

private List<Tag> tags

List of cell tags

Constructor Details

TsvImporterMapper

public TsvImporterMapper()

Method Details

getTs

public long getTs()

getSkipBadLines

public boolean getSkipBadLines()

getBadLineCount

public org.apache.hadoop.mapreduce.Counter getBadLineCount()

incrementBadLineCount

public void incrementBadLineCount(int count)

setup

protected void setup(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>.org.apache.hadoop.mapreduce.Mapper.Context context)

Handles initializing this class with objects specific to it (i.e., the parser). Common initialization that might be leveraged by a subsclass is done in doSetup. Hence a subclass may choose to override this method and call doSetup as well before handling it's own custom params.

Overrides:

setup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>

doSetup

protected void doSetup(org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>.org.apache.hadoop.mapreduce.Mapper.Context context)

Handles common parameter initialization that a subclass might want to leverage.

map

public void map(org.apache.hadoop.io.LongWritable offset,
 org.apache.hadoop.io.Text value,
 org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>.org.apache.hadoop.mapreduce.Mapper.Context context)
         throws IOException

Convert a line of TSV text into an HBase table row.

Overrides:

map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,ImmutableBytesWritable,Put>

Throws:

IOException

populatePut

protected void populatePut(byte[] lineBytes,
 ImportTsv.TsvParser.ParsedLine parsed,
 Put put,
 int i)
                    throws ImportTsv.TsvParser.BadTsvLineException,
IOException

Throws:

ImportTsv.TsvParser.BadTsvLineException

IOException