Class FastDiffDeltaEncoder
java.lang.Object
org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder
org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder
org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder
- All Implemented Interfaces:
DataBlockEncoder
Encoder similar to
DiffKeyDeltaEncoder
but supposedly faster. Compress using: - store
size of common prefix - save column family once in the first KeyValue - use integer compression
for key, value and prefix (7-bit encoding) - use bits to avoid duplication key length, value
length and type if it same as previous - store in 3 bits length of prefix timestamp with previous
KeyValue's timestamp - one bit which allow to omit value if it is the same Format: - 1 byte: flag
- 1-5 bytes: key length (only if FLAG_SAME_KEY_LENGTH is not set in flag) - 1-5 bytes: value
length (only if FLAG_SAME_VALUE_LENGTH is not set in flag) - 1-5 bytes: prefix length - ...
bytes: rest of the row (if prefix length is small enough) - ... bytes: qualifier (or suffix
depending on prefix length) - 1-8 bytes: timestamp suffix - 1 byte: type (only if FLAG_SAME_TYPE
is not set in the flag) - ... bytes: value (only if FLAG_SAME_VALUE is not set in the flag)-
Nested Class Summary
Modifier and TypeClassDescriptionprivate static class
protected static class
private static class
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder
BufferedDataBlockEncoder.BufferedEncodedSeeker<STATE extends BufferedDataBlockEncoder.SeekerState>, BufferedDataBlockEncoder.OffheapDecodedExtendedCell, BufferedDataBlockEncoder.OnheapDecodedCell, BufferedDataBlockEncoder.SeekerState
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder
AbstractDataBlockEncoder.AbstractEncodedSeeker
Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
DataBlockEncoder.EncodedSeeker
-
Field Summary
Modifier and TypeFieldDescription(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionprivate int
compressSingleKeyValue
(DataOutputStream out, ExtendedCell cell, ExtendedCell prevCell) createSeeker
(HFileBlockDecodingContext decodingCtx) Create a HFileBlock seeker which find KeyValues within a block.private int
findCommonTimestampPrefix
(byte[] curTsBuf, byte[] prevTsBuf) getFirstKeyCellInBlock
(ByteBuff block) Return first key in block as a cell.protected ByteBuffer
internalDecodeKeyValues
(DataInputStream source, int allocateHeaderLength, int skipLastBytes, HFileBlockDefaultDecodingContext decodingCtx) int
internalEncode
(ExtendedCell cell, HFileBlockDefaultEncodingContext encodingContext, DataOutputStream out) toString()
private void
uncompressSingleKeyValue
(DataInputStream source, ByteBuffer out, FastDiffDeltaEncoder.FastDiffCompressionState state) Methods inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder
afterDecodingKeyValue, afterEncodingKeyValue, compareCommonFamilyPrefix, compareCommonQualifierPrefix, compareCommonRowPrefix, decodeKeyValues, encode, endBlockEncoding, ensureSpace, startBlockEncoding
Methods inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder
createFirstKeyCell, newDataBlockDecodingContext, newDataBlockEncodingContext, postEncoding
-
Field Details
-
MASK_TIMESTAMP_LENGTH
- See Also:
-
SHIFT_TIMESTAMP_LENGTH
- See Also:
-
FLAG_SAME_KEY_LENGTH
- See Also:
-
FLAG_SAME_VALUE_LENGTH
- See Also:
-
FLAG_SAME_TYPE
- See Also:
-
FLAG_SAME_VALUE
- See Also:
-
-
Constructor Details
-
FastDiffDeltaEncoder
public FastDiffDeltaEncoder()
-
-
Method Details
-
findCommonTimestampPrefix
-
uncompressSingleKeyValue
private void uncompressSingleKeyValue(DataInputStream source, ByteBuffer out, FastDiffDeltaEncoder.FastDiffCompressionState state) throws IOException, EncoderBufferTooSmallException -
internalEncode
public int internalEncode(ExtendedCell cell, HFileBlockDefaultEncodingContext encodingContext, DataOutputStream out) throws IOException - Specified by:
internalEncode
in classBufferedDataBlockEncoder
- Throws:
IOException
-
compressSingleKeyValue
private int compressSingleKeyValue(DataOutputStream out, ExtendedCell cell, ExtendedCell prevCell) throws IOException - Throws:
IOException
-
internalDecodeKeyValues
protected ByteBuffer internalDecodeKeyValues(DataInputStream source, int allocateHeaderLength, int skipLastBytes, HFileBlockDefaultDecodingContext decodingCtx) throws IOException - Specified by:
internalDecodeKeyValues
in classBufferedDataBlockEncoder
- Throws:
IOException
-
getFirstKeyCellInBlock
Description copied from interface:DataBlockEncoder
Return first key in block as a cell. Useful for indexing. Typically does not make a deep copy but returns a buffer wrapping a segment of the actual block's byte array. This is because the first key in block is usually stored unencoded.- Parameters:
block
- encoded block we want index, the position will not change- Returns:
- First key in block as a cell.
-
toString
-
createSeeker
Description copied from interface:DataBlockEncoder
Create a HFileBlock seeker which find KeyValues within a block.- Returns:
- A newly created seeker.
-