Package org.apache.hadoop.hbase.mapreduce

Class TableOutputFormat<KEY>

java.lang.Object
org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>
org.apache.hadoop.hbase.mapreduce.TableOutputFormat<KEY>
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
@Public public class TableOutputFormat<KEY> extends org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation> implements org.apache.hadoop.conf.Configurable
Convert Map/Reduce output and write it to an HBase table. The KEY is ignored while the output value must be either a Put or a Delete instance.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    protected class 
    TableOutputFormat.TableRecordWriter
    Writes the reducer output to an HBase table.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private org.apache.hadoop.conf.Configuration
    conf
    The configuration.
    private static final org.slf4j.Logger
    LOG
     
    static final String
    OUTPUT_CLUSTER
    Optional job parameter to specify a peer cluster.
    static final String
    OUTPUT_COMMITTER_CLASS
    The configuration key for specifying a custom OutputCommitter implementation to be used by TableOutputFormat.
    static final String
    OUTPUT_CONF_PREFIX
    Deprecated.
    Since 3.0.0, will be removed in 4.0.0.
    static final String
    OUTPUT_TABLE
    Job parameter that specifies the output table.
    static final String
    QUORUM_ADDRESS
    Deprecated.
    Since 3.0.0, will be removed in 4.0.0.
    static final String
    QUORUM_PORT
    Deprecated.
    Since 3.0.0, will be removed in 4.0.0.
    static final String
    REGION_SERVER_CLASS
    Deprecated.
    Since 2.5.9, 2.6.1 and 2.7.0, will be removed in 4.0.0.
    static final String
    REGION_SERVER_IMPL
    Deprecated.
    Since 2.5.9, 2.6.1 and 2.7.0, will be removed in 4.0.0.
    static final boolean
    WAL_OFF
    Property value to disable write-ahead logging
    static final boolean
    WAL_ON
    Property value to use write-ahead logging
    static final String
    WAL_PROPERTY
    Set this to WAL_OFF to turn off write-ahead logging (WAL)

  • Constructor Summary

    Constructors
    Constructor
    Description
    TableOutputFormat()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
    Checks if the output table exists and is enabled.
    private static Connection
    createConnection(org.apache.hadoop.conf.Configuration conf)
     
    org.apache.hadoop.conf.Configuration
    getConf()
     
    org.apache.hadoop.mapreduce.OutputCommitter
    getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
    Returns the output committer.
    org.apache.hadoop.mapreduce.RecordWriter<KEY,Mutation>
    getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
    Creates a new record writer.
    void
    setConf(org.apache.hadoop.conf.Configuration otherConf)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

  • Method Details

    • createConnection

      private static Connection createConnection(org.apache.hadoop.conf.Configuration conf) throws IOException
      Throws:
      IOException

    • getRecordWriter

      public org.apache.hadoop.mapreduce.RecordWriter<KEY,Mutation> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException, InterruptedException
      Creates a new record writer. Be aware that the baseline javadoc gives the impression that there is a single RecordWriter per job but in HBase, it is more natural if we give you a new RecordWriter per call of this method. You must close the returned RecordWriter when done. Failure to do so will drop writes.
      Specified by:
      getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>
      Parameters:
      context - The current task context.
      Returns:
      The newly created writer instance.
      Throws:
      IOException - When creating the writer fails.
      InterruptedException - When the job is cancelled.

    • checkOutputSpecs

      public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context) throws IOException, InterruptedException
      Checks if the output table exists and is enabled.
      Specified by:
      checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>
      Parameters:
      context - The current context.
      Throws:
      IOException - When the check fails.
      InterruptedException - When the job is aborted.
      See Also:
      • OutputFormat.checkOutputSpecs(JobContext)

    • getOutputCommitter

      public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException, InterruptedException
      Returns the output committer.
      Specified by:
      getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>
      Parameters:
      context - The current context.
      Returns:
      The committer.
      Throws:
      IOException - When creating the committer fails.
      InterruptedException - When the job is aborted.
      See Also:
      • OutputFormat.getOutputCommitter(TaskAttemptContext)

    • getConf

      public org.apache.hadoop.conf.Configuration getConf()
      Specified by:
      getConf in interface org.apache.hadoop.conf.Configurable

    • setConf

      public void setConf(org.apache.hadoop.conf.Configuration otherConf)
      Specified by:
      setConf in interface org.apache.hadoop.conf.Configurable