Class DiffKeyDeltaEncoder
java.lang.Object
org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder
org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder
org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder
- All Implemented Interfaces:
DataBlockEncoder
Compress using: - store size of common prefix - save column family once, it is same within HFile
- use integer compression for key, value and prefix (7-bit encoding) - use bits to avoid
duplication key length, value length and type if it same as previous - store in 3 bits length of
timestamp field - allow diff in timestamp instead of actual value Format: - 1 byte: flag - 1-5
bytes: key length (only if FLAG_SAME_KEY_LENGTH is not set in flag) - 1-5 bytes: value length
(only if FLAG_SAME_VALUE_LENGTH is not set in flag) - 1-5 bytes: prefix length - ... bytes: rest
of the row (if prefix length is small enough) - ... bytes: qualifier (or suffix depending on
prefix length) - 1-8 bytes: timestamp or diff - 1 byte: type (only if FLAG_SAME_TYPE is not set
in the flag) - ... bytes: value
-
Nested Class Summary
Modifier and TypeClassDescriptionprotected static class
protected static class
private static class
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder
BufferedDataBlockEncoder.BufferedEncodedSeeker<STATE extends BufferedDataBlockEncoder.SeekerState>, BufferedDataBlockEncoder.OffheapDecodedExtendedCell, BufferedDataBlockEncoder.OnheapDecodedCell, BufferedDataBlockEncoder.SeekerState
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder
AbstractDataBlockEncoder.AbstractEncodedSeeker
Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
DataBlockEncoder.EncodedSeeker
-
Field Summary
Modifier and TypeFieldDescription(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
(package private) static final int
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionprivate int
compressSingleKeyValue
(DataOutputStream out, ExtendedCell cell, ExtendedCell prevCell) createSeeker
(HFileBlockDecodingContext decodingCtx) Create a HFileBlock seeker which find KeyValues within a block.getFirstKeyCellInBlock
(ByteBuff block) Return first key in block as a cell.protected ByteBuffer
internalDecodeKeyValues
(DataInputStream source, int allocateHeaderLength, int skipLastBytes, HFileBlockDefaultDecodingContext decodingCtx) int
internalEncode
(ExtendedCell cell, HFileBlockDefaultEncodingContext encodingContext, DataOutputStream out) toString()
private void
uncompressSingleKeyValue
(DataInputStream source, ByteBuffer buffer, DiffKeyDeltaEncoder.DiffCompressionState state) Methods inherited from class org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder
afterDecodingKeyValue, afterEncodingKeyValue, compareCommonFamilyPrefix, compareCommonQualifierPrefix, compareCommonRowPrefix, decodeKeyValues, encode, endBlockEncoding, ensureSpace, startBlockEncoding
Methods inherited from class org.apache.hadoop.hbase.io.encoding.AbstractDataBlockEncoder
createFirstKeyCell, newDataBlockDecodingContext, newDataBlockEncodingContext, postEncoding
-
Field Details
-
FLAG_SAME_KEY_LENGTH
- See Also:
-
FLAG_SAME_VALUE_LENGTH
- See Also:
-
FLAG_SAME_TYPE
- See Also:
-
FLAG_TIMESTAMP_IS_DIFF
- See Also:
-
MASK_TIMESTAMP_LENGTH
- See Also:
-
SHIFT_TIMESTAMP_LENGTH
- See Also:
-
FLAG_TIMESTAMP_SIGN
- See Also:
-
-
Constructor Details
-
DiffKeyDeltaEncoder
public DiffKeyDeltaEncoder()
-
-
Method Details
-
uncompressSingleKeyValue
private void uncompressSingleKeyValue(DataInputStream source, ByteBuffer buffer, DiffKeyDeltaEncoder.DiffCompressionState state) throws IOException, EncoderBufferTooSmallException -
internalEncode
public int internalEncode(ExtendedCell cell, HFileBlockDefaultEncodingContext encodingContext, DataOutputStream out) throws IOException - Specified by:
internalEncode
in classBufferedDataBlockEncoder
- Throws:
IOException
-
compressSingleKeyValue
private int compressSingleKeyValue(DataOutputStream out, ExtendedCell cell, ExtendedCell prevCell) throws IOException - Throws:
IOException
-
getFirstKeyCellInBlock
Description copied from interface:DataBlockEncoder
Return first key in block as a cell. Useful for indexing. Typically does not make a deep copy but returns a buffer wrapping a segment of the actual block's byte array. This is because the first key in block is usually stored unencoded.- Parameters:
block
- encoded block we want index, the position will not change- Returns:
- First key in block as a cell.
-
toString
-
createSeeker
Description copied from interface:DataBlockEncoder
Create a HFileBlock seeker which find KeyValues within a block.- Returns:
- A newly created seeker.
-
internalDecodeKeyValues
protected ByteBuffer internalDecodeKeyValues(DataInputStream source, int allocateHeaderLength, int skipLastBytes, HFileBlockDefaultDecodingContext decodingCtx) throws IOException - Specified by:
internalDecodeKeyValues
in classBufferedDataBlockEncoder
- Throws:
IOException
-