org.apache.hadoop.hbase.test

Class IntegrationTestBigLinkedList.Verify.VerifyReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.Verify.VerifyReducer

Enclosing class:

IntegrationTestBigLinkedList.Verify

public static class IntegrationTestBigLinkedList.Verify.VerifyReducer
extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Per reducer, we output problem rows as byte arrasy so can be used as input for subsequent investigative mapreduce jobs. Each emitted value is prefaced by a one byte flag saying what sort of emission it is. Flag is the Count enum ordinal as a short.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.hbase.client.Connection	connection
private org.apache.hadoop.io.BytesWritable	LOSTFAM
private ArrayList<byte[]>	refs
private AtomicInteger	rows
private org.apache.hadoop.io.BytesWritable	UNREF

Constructor Summary

Constructors

Constructor and Description
VerifyReducer()

Method Summary

Modifier and Type	Method and Description
static byte[]	addPrefixFlag(int ordinal, byte[] r)
protected void	cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
private StringBuilder	dumpExtraInfoOnRefs(org.apache.hadoop.io.BytesWritable key, org.apache.hadoop.mapreduce.Reducer.Context context, List<byte[]> refs) Dump out extra info around references if there are any.
static byte[]	getRowOnly(org.apache.hadoop.io.BytesWritable bw)
void	reduce(org.apache.hadoop.io.BytesWritable key, Iterable<org.apache.hadoop.io.BytesWritable> values, org.apache.hadoop.mapreduce.Reducer.Context context)
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context)
static IntegrationTestBigLinkedList.Verify.Counts	whichType(byte[] bs)

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

refs

private ArrayList<byte[]> refs

UNREF

private final org.apache.hadoop.io.BytesWritable UNREF

LOSTFAM

private final org.apache.hadoop.io.BytesWritable LOSTFAM

rows

private AtomicInteger rows

connection

private org.apache.hadoop.hbase.client.Connection connection

Constructor Detail

VerifyReducer

public VerifyReducer()

Method Detail

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
                     InterruptedException

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

InterruptedException

cleanup

protected void cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
                throws IOException,
                       InterruptedException

Overrides:

cleanup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

InterruptedException

addPrefixFlag

public static byte[] addPrefixFlag(int ordinal,
                                   byte[] r)

Parameters:

ordinal -

r -

Returns:

Return new byte array that has ordinal as prefix on front taking up Bytes.SIZEOF_SHORT bytes followed by r

whichType

public static IntegrationTestBigLinkedList.Verify.Counts whichType(byte[] bs)

Parameters:

bs -

Returns:

Type from the Counts enum of this row. Reads prefix added by addPrefixFlag(int, byte[])

getRowOnly

public static byte[] getRowOnly(org.apache.hadoop.io.BytesWritable bw)

Parameters:

bw -

Returns:

Row bytes minus the type flag.

reduce

public void reduce(org.apache.hadoop.io.BytesWritable key,
                   Iterable<org.apache.hadoop.io.BytesWritable> values,
                   org.apache.hadoop.mapreduce.Reducer.Context context)
            throws IOException,
                   InterruptedException

Overrides:

reduce in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

InterruptedException

dumpExtraInfoOnRefs

private StringBuilder dumpExtraInfoOnRefs(org.apache.hadoop.io.BytesWritable key,
                                          org.apache.hadoop.mapreduce.Reducer.Context context,
                                          List<byte[]> refs)
                                   throws IOException

Dump out extra info around references if there are any. Helps debugging.

Returns:

StringBuilder filled with references if any.

Throws:

IOException