org.apache.hadoop.hbase.mapreduce

Class TableRecordReaderImpl

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl

@InterfaceAudience.Public
public class TableRecordReaderImpl
extends Object

Iterate over an HBase table data, return (ImmutableBytesWritable, Result) pairs.

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.mapreduce.TaskAttemptContext	context
private Scan	currentScan
(package private) static String	HBASE_COUNTER_GROUP_NAME
private Table	htable
private ImmutableBytesWritable	key
private byte[]	lastSuccessfulRow
private static org.slf4j.Logger	LOG
static String	LOG_PER_ROW_COUNT
private int	logPerRowCount
private boolean	logScannerActivity
private long	numRestarts
private long	numStale
private int	rowcount
private Scan	scan
private ResultScanner	scanner
private long	timestamp
private Result	value

Constructor Summary

Constructors

Constructor and Description
TableRecordReaderImpl()

Method Summary

Modifier and Type	Method and Description
void	close() Closes the split.
ImmutableBytesWritable	getCurrentKey() Returns the current key.
Result	getCurrentValue() Returns the current value.
float	getProgress() The current progress of the record reader through its data.
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputsplit, org.apache.hadoop.mapreduce.TaskAttemptContext context) Build the scanner.
boolean	nextKeyValue() Positions the record reader to the next record.
void	restart(byte[] firstRow) Restart from survivable exceptions by creating a new scanner.
protected static Method	retrieveGetCounterWithStringsParams(org.apache.hadoop.mapreduce.TaskAttemptContext context) Deprecated. since 2.4.0 and 2.3.2, will be removed in 4.0.0
void	setHTable(Table htable) Sets the HBase table.
void	setScan(Scan scan) Sets the scan defining the actual details like columns etc.
private void	updateCounters() If hbase runs on new version of mapreduce, RecordReader has access to counters thus can update counters based on scanMetrics.
protected static void	updateCounters(ScanMetrics scanMetrics, long numScannerRestarts, Method getCounter, org.apache.hadoop.mapreduce.TaskAttemptContext context, long numStale) Deprecated. since 2.4.0 and 2.3.2, will be removed in 4.0.0 Use updateCounters(ScanMetrics, long, TaskAttemptContext, long) instead.
protected static void	updateCounters(ScanMetrics scanMetrics, long numScannerRestarts, org.apache.hadoop.mapreduce.TaskAttemptContext context, long numStale)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG_PER_ROW_COUNT

public static final String LOG_PER_ROW_COUNT

See Also:

Constant Field Values

LOG

private static final org.slf4j.Logger LOG

HBASE_COUNTER_GROUP_NAME

@InterfaceAudience.Private
static final String HBASE_COUNTER_GROUP_NAME

See Also:

Constant Field Values

scanner

private ResultScanner scanner

scan

private Scan scan

currentScan

private Scan currentScan

htable

private Table htable

lastSuccessfulRow

private byte[] lastSuccessfulRow

key

private ImmutableBytesWritable key

value

private Result value

context

private org.apache.hadoop.mapreduce.TaskAttemptContext context

numRestarts

private long numRestarts

numStale

private long numStale

timestamp

private long timestamp

rowcount

private int rowcount

logScannerActivity

private boolean logScannerActivity

logPerRowCount

private int logPerRowCount

Constructor Detail

TableRecordReaderImpl

public TableRecordReaderImpl()

Method Detail

restart

public void restart(byte[] firstRow)
             throws IOException

Restart from survivable exceptions by creating a new scanner.

Parameters:

firstRow - The first row to start at.

Throws:

IOException - When restarting fails.

retrieveGetCounterWithStringsParams

@Deprecated
protected static Method retrieveGetCounterWithStringsParams(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                 throws IOException

Deprecated. since 2.4.0 and 2.3.2, will be removed in 4.0.0

In new mapreduce APIs, TaskAttemptContext has two getCounter methods Check if getCounter(String, String) method is available.

Returns:

The getCounter method or null if not available.

Throws:

IOException

setHTable

public void setHTable(Table htable)

Sets the HBase table.

Parameters:

htable - The HTableDescriptor to scan.

setScan

public void setScan(Scan scan)

Sets the scan defining the actual details like columns etc.

Parameters:

scan - The scan to set.

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit inputsplit,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws IOException,
                       InterruptedException

Build the scanner. Not done in constructor to allow for extension.

Throws:

IOException

InterruptedException

close

public void close()

Closes the split.

getCurrentKey

public ImmutableBytesWritable getCurrentKey()
                                     throws IOException,
                                            InterruptedException

Returns the current key.

Returns:

The current key.

Throws:

InterruptedException - When the job is aborted.

IOException

getCurrentValue

public Result getCurrentValue()
                       throws IOException,
                              InterruptedException

Returns the current value.

Returns:

The current value.

Throws:

IOException - When the value is faulty.

InterruptedException - When the job is aborted.

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException

Positions the record reader to the next record.

Returns:

true if there was another record.

Throws:

IOException - When reading the record failed.

InterruptedException - When the job was aborted.

updateCounters

private void updateCounters()

If hbase runs on new version of mapreduce, RecordReader has access to counters thus can update counters based on scanMetrics. If hbase runs on old version of mapreduce, it won't be able to get access to counters and TableRecorderReader can't update counter values.

updateCounters

@Deprecated
protected static void updateCounters(ScanMetrics scanMetrics,
                                                 long numScannerRestarts,
                                                 Method getCounter,
                                                 org.apache.hadoop.mapreduce.TaskAttemptContext context,
                                                 long numStale)

Deprecated. since 2.4.0 and 2.3.2, will be removed in 4.0.0 Use updateCounters(ScanMetrics, long, TaskAttemptContext, long) instead.

updateCounters

protected static void updateCounters(ScanMetrics scanMetrics,
                                     long numScannerRestarts,
                                     org.apache.hadoop.mapreduce.TaskAttemptContext context,
                                     long numStale)

getProgress

public float getProgress()

The current progress of the record reader through its data.

Returns:

A number between 0.0 and 1.0, the fraction of the data read.