org.apache.hadoop.hbase.mapreduce

Class HRegionPartitioner<KEY,VALUE>

java.lang.Object

org.apache.hadoop.mapreduce.Partitioner<ImmutableBytesWritable,VALUE>

org.apache.hadoop.hbase.mapreduce.HRegionPartitioner<KEY,VALUE>

Type Parameters:

KEY - The type of the key.

VALUE - The type of the value.

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

@InterfaceAudience.Public
public class HRegionPartitioner<KEY,VALUE>
extends org.apache.hadoop.mapreduce.Partitioner<ImmutableBytesWritable,VALUE>
implements org.apache.hadoop.conf.Configurable

This is used to partition the output keys into groups of keys. Keys are grouped according to the regions that currently exist so that each reducer fills a single region so load is distributed.

This class is not suitable as partitioner creating hfiles for incremental bulk loads as region spread will likely change between time of hfile creation and load time. See LoadIncrementalHFiles and Bulk Load.

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf
private Connection	connection
private RegionLocator	locator
private static org.slf4j.Logger	LOG
private byte[][]	startKeys

Constructor Summary

Constructors

Constructor and Description
HRegionPartitioner()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.conf.Configuration	getConf() Returns the current configuration.
int	getPartition(ImmutableBytesWritable key, VALUE value, int numPartitions) Gets the partition number for a given key (hence record) given the total number of partitions i.e.
void	setConf(org.apache.hadoop.conf.Configuration configuration) Sets the configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

conf

private org.apache.hadoop.conf.Configuration conf

connection

private Connection connection

locator

private RegionLocator locator

startKeys

private byte[][] startKeys

Constructor Detail

HRegionPartitioner

public HRegionPartitioner()

Method Detail

getPartition

public int getPartition(ImmutableBytesWritable key,
                        VALUE value,
                        int numPartitions)

Gets the partition number for a given key (hence record) given the total number of partitions i.e. number of reduce-tasks for the job.

Typically a hash function on a all or a subset of the key.

Specified by:

getPartition in class org.apache.hadoop.mapreduce.Partitioner<ImmutableBytesWritable,VALUE>

Parameters:

key - The key to be partitioned.

value - The entry value.

numPartitions - The total number of partitions.

Returns:

The partition number for the key.

See Also:

Partitioner.getPartition( java.lang.Object, java.lang.Object, int)

getConf

public org.apache.hadoop.conf.Configuration getConf()

Returns the current configuration.

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

Returns:

The current configuration.

See Also:

Configurable.getConf()

setConf

public void setConf(org.apache.hadoop.conf.Configuration configuration)

Sets the configuration. This is used to determine the start keys for the given table.

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Parameters:

configuration - The configuration to set.

See Also:

Configurable.setConf( org.apache.hadoop.conf.Configuration)