Package org.apache.hadoop.hbase.backup.mapreduce

Class BulkLoadCollectorJob.DedupReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable>

org.apache.hadoop.hbase.backup.mapreduce.BulkLoadCollectorJob.DedupReducer

Enclosing class:

BulkLoadCollectorJob

public static class BulkLoadCollectorJob.DedupReducer
extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable>

Reducer that deduplicates full-path keys emitted by the mappers. It writes each unique key exactly once. Reduce input: (Text fullPathString, Iterable) Reduce output: (Text fullPathString, NullWritable)

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Constructor Summary

Constructors

Constructor	Description
DedupReducer()

Method Summary

Modifier and Type	Method	Description
protected void	reduce(org.apache.hadoop.io.Text key, Iterable<org.apache.hadoop.io.NullWritable> values, org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable>.org.apache.hadoop.mapreduce.Reducer.Context ctx)

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

cleanup, run, setup

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DedupReducer

public DedupReducer()

Method Details

reduce

protected void reduce(org.apache.hadoop.io.Text key,
 Iterable<org.apache.hadoop.io.NullWritable> values,
 org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable>.org.apache.hadoop.mapreduce.Reducer.Context ctx)
               throws IOException,
InterruptedException

Overrides:

reduce in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable>

Throws:

IOException

InterruptedException