org.apache.hadoop.hbase.io.encoding

Class DiffKeyDeltaEncoder

java.lang.Object

org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder

org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder

org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder

All Implemented Interfaces:

DataBlockEncoder

@InterfaceAudience.Private
public class DiffKeyDeltaEncoder
extends BufferedDataBlockEncoder

Compress using: - store size of common prefix - save column family once, it is same within HFile - use integer compression for key, value and prefix (7-bit encoding) - use bits to avoid duplication key length, value length and type if it same as previous - store in 3 bits length of timestamp field - allow diff in timestamp instead of actual value Format: - 1 byte: flag - 1-5 bytes: key length (only if FLAG_SAME_KEY_LENGTH is not set in flag) - 1-5 bytes: value length (only if FLAG_SAME_VALUE_LENGTH is not set in flag) - 1-5 bytes: prefix length - ... bytes: rest of the row (if prefix length is small enough) - ... bytes: qualifier (or suffix depending on prefix length) - 1-8 bytes: timestamp or diff - 1 byte: type (only if FLAG_SAME_TYPE is not set in the flag) - ... bytes: value

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	DiffKeyDeltaEncoder.DiffCompressionState
protected static class	DiffKeyDeltaEncoder.DiffSeekerState

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder

BufferedDataBlockEncoder.BufferedEncodedSeeker<STATE extends BufferedDataBlockEncoder.SeekerState>, BufferedDataBlockEncoder.OffheapDecodedExtendedCell, BufferedDataBlockEncoder.OnheapDecodedCell, BufferedDataBlockEncoder.SeekerState

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder

AbstractDataBlockEncoder.AbstractEncodedSeeker

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.encoding.DataBlockEncoder

DataBlockEncoder.EncodedSeeker

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static int	FLAG_SAME_KEY_LENGTH
(package private) static int	FLAG_SAME_TYPE
(package private) static int	FLAG_SAME_VALUE_LENGTH
(package private) static int	FLAG_TIMESTAMP_IS_DIFF
(package private) static int	FLAG_TIMESTAMP_SIGN
(package private) static int	MASK_TIMESTAMP_LENGTH
(package private) static int	SHIFT_TIMESTAMP_LENGTH

Constructor Summary

Constructors

Constructor and Description
DiffKeyDeltaEncoder()

Method Summary

Modifier and Type	Method and Description
private int	compressSingleKeyValue(DataOutputStream out, Cell cell, Cell prevCell)
DataBlockEncoder.EncodedSeeker	createSeeker(CellComparator comparator, HFileBlockDecodingContext decodingCtx) Create a HFileBlock seeker which find KeyValues within a block.
Cell	getFirstKeyCellInBlock(ByteBuff block) Return first key in block as a cell.
protected ByteBuffer	internalDecodeKeyValues(DataInputStream source, int allocateHeaderLength, int skipLastBytes, HFileBlockDefaultDecodingContext decodingCtx)
int	internalEncode(Cell cell, HFileBlockDefaultEncodingContext encodingContext, DataOutputStream out)
String	toString()
private void	uncompressSingleKeyValue(DataInputStream source, ByteBuffer buffer, DiffKeyDeltaEncoder.DiffCompressionState state)

Methods inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder

afterDecodingKeyValue, afterEncodingKeyValue, compareCommonFamilyPrefix, compareCommonQualifierPrefix, compareCommonRowPrefix, decodeKeyValues, encode, endBlockEncoding, ensureSpace, startBlockEncoding

Methods inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder

createFirstKeyCell, newDataBlockDecodingContext, newDataBlockEncodingContext, postEncoding

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

FLAG_SAME_KEY_LENGTH

static final int FLAG_SAME_KEY_LENGTH

See Also:

Constant Field Values

FLAG_SAME_VALUE_LENGTH

static final int FLAG_SAME_VALUE_LENGTH

See Also:

Constant Field Values

FLAG_SAME_TYPE

static final int FLAG_SAME_TYPE

See Also:

Constant Field Values

FLAG_TIMESTAMP_IS_DIFF

static final int FLAG_TIMESTAMP_IS_DIFF

See Also:

Constant Field Values

MASK_TIMESTAMP_LENGTH

static final int MASK_TIMESTAMP_LENGTH

See Also:

Constant Field Values

SHIFT_TIMESTAMP_LENGTH

static final int SHIFT_TIMESTAMP_LENGTH

See Also:

Constant Field Values

FLAG_TIMESTAMP_SIGN

static final int FLAG_TIMESTAMP_SIGN

See Also:

Constant Field Values

Constructor Detail

DiffKeyDeltaEncoder

public DiffKeyDeltaEncoder()

Method Detail

uncompressSingleKeyValue

private void uncompressSingleKeyValue(DataInputStream source,
                                      ByteBuffer buffer,
                                      DiffKeyDeltaEncoder.DiffCompressionState state)
                               throws IOException,
                                      EncoderBufferTooSmallException

Throws:

IOException

EncoderBufferTooSmallException

internalEncode

public int internalEncode(Cell cell,
                          HFileBlockDefaultEncodingContext encodingContext,
                          DataOutputStream out)
                   throws IOException

Specified by:

internalEncode in class BufferedDataBlockEncoder

Throws:

IOException

compressSingleKeyValue

private int compressSingleKeyValue(DataOutputStream out,
                                   Cell cell,
                                   Cell prevCell)
                            throws IOException

Throws:

IOException

getFirstKeyCellInBlock

public Cell getFirstKeyCellInBlock(ByteBuff block)

Description copied from interface: DataBlockEncoder

Return first key in block as a cell. Useful for indexing. Typically does not make a deep copy but returns a buffer wrapping a segment of the actual block's byte array. This is because the first key in block is usually stored unencoded.

Parameters:

block - encoded block we want index, the position will not change

Returns:

First key in block as a cell.

toString

public String toString()

Overrides:

toString in class Object

createSeeker

public DataBlockEncoder.EncodedSeeker createSeeker(CellComparator comparator,
                                                   HFileBlockDecodingContext decodingCtx)

Description copied from interface: DataBlockEncoder

Create a HFileBlock seeker which find KeyValues within a block.

Parameters:

comparator - what kind of comparison should be used

Returns:

A newly created seeker.

internalDecodeKeyValues

protected ByteBuffer internalDecodeKeyValues(DataInputStream source,
                                             int allocateHeaderLength,
                                             int skipLastBytes,
                                             HFileBlockDefaultDecodingContext decodingCtx)
                                      throws IOException

Specified by:

internalDecodeKeyValues in class BufferedDataBlockEncoder

Throws:

IOException