org.apache.hadoop.hbase.io.encoding

Class FastDiffDeltaEncoder

java.lang.Object

org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder

org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder

org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder

All Implemented Interfaces:

DataBlockEncoder

@InterfaceAudience.Private
public class FastDiffDeltaEncoder
extends BufferedDataBlockEncoder

Encoder similar to DiffKeyDeltaEncoder but supposedly faster. Compress using: - store size of common prefix - save column family once in the first KeyValue - use integer compression for key, value and prefix (7-bit encoding) - use bits to avoid duplication key length, value length and type if it same as previous - store in 3 bits length of prefix timestamp with previous KeyValue's timestamp - one bit which allow to omit value if it is the same Format: - 1 byte: flag - 1-5 bytes: key length (only if FLAG_SAME_KEY_LENGTH is not set in flag) - 1-5 bytes: value length (only if FLAG_SAME_VALUE_LENGTH is not set in flag) - 1-5 bytes: prefix length - ... bytes: rest of the row (if prefix length is small enough) - ... bytes: qualifier (or suffix depending on prefix length) - 1-8 bytes: timestamp suffix - 1 byte: type (only if FLAG_SAME_TYPE is not set in the flag) - ... bytes: value (only if FLAG_SAME_VALUE is not set in the flag)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	FastDiffDeltaEncoder.FastDiffCompressionState
protected static class	FastDiffDeltaEncoder.FastDiffSeekerState

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder

BufferedDataBlockEncoder.BufferedEncodedSeeker<STATE extends BufferedDataBlockEncoder.SeekerState>, BufferedDataBlockEncoder.OffheapDecodedExtendedCell, BufferedDataBlockEncoder.OnheapDecodedCell, BufferedDataBlockEncoder.SeekerState

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder

AbstractDataBlockEncoder.AbstractEncodedSeeker

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.encoding.DataBlockEncoder

DataBlockEncoder.EncodedSeeker

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static int	FLAG_SAME_KEY_LENGTH
(package private) static int	FLAG_SAME_TYPE
(package private) static int	FLAG_SAME_VALUE
(package private) static int	FLAG_SAME_VALUE_LENGTH
(package private) static int	MASK_TIMESTAMP_LENGTH
(package private) static int	SHIFT_TIMESTAMP_LENGTH

Constructor Summary

Constructors

Constructor and Description
FastDiffDeltaEncoder()

Method Summary

Modifier and Type	Method and Description
private int	compressSingleKeyValue(DataOutputStream out, Cell cell, Cell prevCell)
DataBlockEncoder.EncodedSeeker	createSeeker(CellComparator comparator, HFileBlockDecodingContext decodingCtx) Create a HFileBlock seeker which find KeyValues within a block.
private int	findCommonTimestampPrefix(byte[] curTsBuf, byte[] prevTsBuf)
Cell	getFirstKeyCellInBlock(ByteBuff block) Return first key in block as a cell.
protected ByteBuffer	internalDecodeKeyValues(DataInputStream source, int allocateHeaderLength, int skipLastBytes, HFileBlockDefaultDecodingContext decodingCtx)
int	internalEncode(Cell cell, HFileBlockDefaultEncodingContext encodingContext, DataOutputStream out)
String	toString()
private void	uncompressSingleKeyValue(DataInputStream source, ByteBuffer out, FastDiffDeltaEncoder.FastDiffCompressionState state)

Methods inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder

afterDecodingKeyValue, afterEncodingKeyValue, compareCommonFamilyPrefix, compareCommonQualifierPrefix, compareCommonRowPrefix, decodeKeyValues, encode, endBlockEncoding, ensureSpace, startBlockEncoding

Methods inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder

createFirstKeyCell, newDataBlockDecodingContext, newDataBlockEncodingContext, postEncoding

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MASK_TIMESTAMP_LENGTH

static final int MASK_TIMESTAMP_LENGTH

See Also:

Constant Field Values

SHIFT_TIMESTAMP_LENGTH

static final int SHIFT_TIMESTAMP_LENGTH

See Also:

Constant Field Values

FLAG_SAME_KEY_LENGTH

static final int FLAG_SAME_KEY_LENGTH

See Also:

Constant Field Values

FLAG_SAME_VALUE_LENGTH

static final int FLAG_SAME_VALUE_LENGTH

See Also:

Constant Field Values

FLAG_SAME_TYPE

static final int FLAG_SAME_TYPE

See Also:

Constant Field Values

FLAG_SAME_VALUE

static final int FLAG_SAME_VALUE

See Also:

Constant Field Values

Constructor Detail

FastDiffDeltaEncoder

public FastDiffDeltaEncoder()

Method Detail

findCommonTimestampPrefix

private int findCommonTimestampPrefix(byte[] curTsBuf,
                                      byte[] prevTsBuf)

uncompressSingleKeyValue

private void uncompressSingleKeyValue(DataInputStream source,
                                      ByteBuffer out,
                                      FastDiffDeltaEncoder.FastDiffCompressionState state)
                               throws IOException,
                                      EncoderBufferTooSmallException

Throws:

IOException

EncoderBufferTooSmallException

internalEncode

public int internalEncode(Cell cell,
                          HFileBlockDefaultEncodingContext encodingContext,
                          DataOutputStream out)
                   throws IOException

Specified by:

internalEncode in class BufferedDataBlockEncoder

Throws:

IOException

compressSingleKeyValue

private int compressSingleKeyValue(DataOutputStream out,
                                   Cell cell,
                                   Cell prevCell)
                            throws IOException

Throws:

IOException

internalDecodeKeyValues

protected ByteBuffer internalDecodeKeyValues(DataInputStream source,
                                             int allocateHeaderLength,
                                             int skipLastBytes,
                                             HFileBlockDefaultDecodingContext decodingCtx)
                                      throws IOException

Specified by:

internalDecodeKeyValues in class BufferedDataBlockEncoder

Throws:

IOException

getFirstKeyCellInBlock

public Cell getFirstKeyCellInBlock(ByteBuff block)

Description copied from interface: DataBlockEncoder

Return first key in block as a cell. Useful for indexing. Typically does not make a deep copy but returns a buffer wrapping a segment of the actual block's byte array. This is because the first key in block is usually stored unencoded.

Parameters:

block - encoded block we want index, the position will not change

Returns:

First key in block as a cell.

toString

public String toString()

Overrides:

toString in class Object

createSeeker

public DataBlockEncoder.EncodedSeeker createSeeker(CellComparator comparator,
                                                   HFileBlockDecodingContext decodingCtx)

Description copied from interface: DataBlockEncoder

Create a HFileBlock seeker which find KeyValues within a block.

Parameters:

comparator - what kind of comparison should be used

Returns:

A newly created seeker.