org.apache.hadoop.hbase.mapreduce

Class TableOutputFormat<KEY>

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

org.apache.hadoop.hbase.mapreduce.TableOutputFormat<KEY>

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

@InterfaceAudience.Public
public class TableOutputFormat<KEY>
extends org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>
implements org.apache.hadoop.conf.Configurable

Convert Map/Reduce output and write it to an HBase table. The KEY is ignored while the output value must be either a Put or a Delete instance.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	TableOutputFormat.TableRecordWriter Writes the reducer output to an HBase table.

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf The configuration.
private static org.slf4j.Logger	LOG
static String	OUTPUT_CONF_PREFIX Prefix for configuration property overrides to apply in setConf(Configuration).
static String	OUTPUT_TABLE Job parameter that specifies the output table.
static String	QUORUM_ADDRESS Optional job parameter to specify a peer cluster.
static String	QUORUM_PORT Optional job parameter to specify peer cluster's ZK client port
static String	REGION_SERVER_CLASS Optional specification of the rs class name of the peer cluster
static String	REGION_SERVER_IMPL Optional specification of the rs impl name of the peer cluster

Constructor Summary

Constructors

Constructor and Description
TableOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context) Checks if the output table exists and is enabled.
org.apache.hadoop.conf.Configuration	getConf()
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Returns the output committer.
org.apache.hadoop.mapreduce.RecordWriter<KEY,Mutation>	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Creates a new record writer.
void	setConf(org.apache.hadoop.conf.Configuration otherConf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

OUTPUT_TABLE

public static final String OUTPUT_TABLE

Job parameter that specifies the output table.

See Also:

Constant Field Values

OUTPUT_CONF_PREFIX

public static final String OUTPUT_CONF_PREFIX

Prefix for configuration property overrides to apply in setConf(Configuration). For keys matching this prefix, the prefix is stripped, and the value is set in the configuration with the resulting key, ie. the entry "hbase.mapred.output.key1 = value1" would be set in the configuration as "key1 = value1". Use this to set properties which should only be applied to the TableOutputFormat configuration and not the input configuration.

See Also:

Constant Field Values

QUORUM_ADDRESS

public static final String QUORUM_ADDRESS

Optional job parameter to specify a peer cluster. Used specifying remote cluster when copying between hbase clusters (the source is picked up from hbase-site.xml).

See Also:

TableMapReduceUtil.initTableReducerJob(String, Class, org.apache.hadoop.mapreduce.Job, Class, String, String, String), Constant Field Values

QUORUM_PORT

public static final String QUORUM_PORT

Optional job parameter to specify peer cluster's ZK client port

See Also:

Constant Field Values

REGION_SERVER_CLASS

public static final String REGION_SERVER_CLASS

Optional specification of the rs class name of the peer cluster

See Also:

Constant Field Values

REGION_SERVER_IMPL

public static final String REGION_SERVER_IMPL

Optional specification of the rs impl name of the peer cluster

See Also:

Constant Field Values

conf

private org.apache.hadoop.conf.Configuration conf

The configuration.

Constructor Detail

TableOutputFormat

public TableOutputFormat()

Method Detail

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<KEY,Mutation> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                       throws IOException,
                                                                              InterruptedException

Creates a new record writer. Be aware that the baseline javadoc gives the impression that there is a single RecordWriter per job but in HBase, it is more natural if we give you a new RecordWriter per call of this method. You must close the returned RecordWriter when done. Failure to do so will drop writes.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

Parameters:

context - The current task context.

Returns:

The newly created writer instance.

Throws:

IOException - When creating the writer fails.

InterruptedException - When the jobs is cancelled.

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws IOException,
                             InterruptedException

Checks if the output table exists and is enabled.

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

Parameters:

context - The current context.

Throws:

IOException - When the check fails.

InterruptedException - When the job is aborted.

See Also:

OutputFormat.checkOutputSpecs(JobContext)

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws IOException,
                                                                      InterruptedException

Returns the output committer.

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

Parameters:

context - The current context.

Returns:

The committer.

Throws:

IOException - When creating the committer fails.

InterruptedException - When the job is aborted.

See Also:

OutputFormat.getOutputCommitter(TaskAttemptContext)

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration otherConf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable