Package org.apache.hadoop.hbase

Class KeyValue

java.lang.Object
org.apache.hadoop.hbase.KeyValue
All Implemented Interfaces:
Cloneable, Cell, ExtendedCell, HeapSize, RawCell
Direct Known Subclasses:
KeyValue.KeyOnlyKeyValue, NoTagsKeyValue, SizeCachedKeyValue
@Private public class KeyValue extends Object implements ExtendedCell, Cloneable
An HBase Key/Value. This is the fundamental HBase Type.

HBase applications and users should use the Cell interface and avoid directly using KeyValue and member functions not defined in Cell.

If being used client-side, the primary methods to access individual fields are getRowArray(), getFamilyArray(), getQualifierArray(), getTimestamp(), and getValueArray(). These methods allocate new byte arrays and return copies. Avoid their use server-side.

Instances of this class are immutable. They do not implement Comparable but Comparators are provided. Comparators change with context, whether user table or a catalog table comparison. Its critical you use the appropriate comparator. There are Comparators for normal HFiles, Meta's Hfiles, and bloom filter keys.

KeyValue wraps a byte array and takes offsets and lengths into passed array at where to start interpreting the content as KeyValue. The KeyValue format inside a byte array is: <keylength> <valuelength> <key> <value> Key is further decomposed as: <rowlength> <row> <columnfamilylength> <columnfamily> <columnqualifier> <timestamp> <keytype> The rowlength maximum is Short.MAX_SIZE, column family length maximum is Byte.MAX_SIZE, and column qualifier + key length must be < Integer.MAX_SIZE. The column does not contain the family/qualifier delimiter, COLUMN_FAMILY_DELIMITER
KeyValue can optionally contain Tags. When it contains tags, it is added in the byte array after the value part. The format for this part is: <tagslength><tagsbytes>. tagslength maximum is Short.MAX_SIZE. The tagsbytes contain one or more tags where as each tag is of the form <taglength><tagtype><tagbytes>. tagtype is one byte and taglength maximum is Short.MAX_SIZE and it includes 1 byte type length and actual tag bytes length.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    KeyValue.KeyOnlyKeyValue
    A simple form of KeyValue that creates a keyvalue with only the key part of the byte[] Mainly used in places where we need to compare two cells.
    static interface 
    KeyValue.SamePrefixComparator<T>
    Avoids redundant comparisons for better performance.
    static enum 
    KeyValue.Type
    Key type.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected byte[]
    bytes
     
    static final byte[]
    COLUMN_FAMILY_DELIM_ARRAY
     
    static final char
    COLUMN_FAMILY_DELIMITER
    Colon character in UTF-8
    static final int
    FAMILY_LENGTH_SIZE
    Size of the family length field in bytes
    static final int
    FIXED_OVERHEAD
     
    static final int
    KEY_INFRASTRUCTURE_SIZE
     
    static final int
    KEY_LENGTH_SIZE
    Size of the key length field in bytes
    static final int
    KEYVALUE_INFRASTRUCTURE_SIZE
     
    static final int
    KEYVALUE_WITH_TAGS_INFRASTRUCTURE_SIZE
     
    protected int
    length
     
    private static final org.slf4j.Logger
    LOG
     
    static final KeyValue
    LOWESTKEY
    Lowest possible key.
    protected int
    offset
     
    static final int
    ROW_KEY_OFFSET
     
    static final int
    ROW_LENGTH_SIZE
    Size of the row length field in bytes
    static final int
    ROW_OFFSET
     
    private long
    seqId
     
    static final int
    TAGS_LENGTH_SIZE
    Size of the tags length field in bytes
    static final int
    TIMESTAMP_SIZE
    Size of the timestamp field in bytes
    static final int
    TIMESTAMP_TYPE_SIZE
     
    static final int
    TYPE_SIZE
    Size of the key type field in bytes

    Fields inherited from interface org.apache.hadoop.hbase.ExtendedCell

    CELL_NOT_BASED_ON_CHUNK

    Fields inherited from interface org.apache.hadoop.hbase.RawCell

    MAX_TAGS_LENGTH

  • Constructor Summary

    Constructors
    Constructor
    Description
    KeyValue()
    Writable Constructor -- DO NOT USE
    KeyValue(byte[] bytes)
    Creates a KeyValue from the start of the specified byte array.
    KeyValue(byte[] row, byte[] family, byte[] qualifier)
    Constructs KeyValue structure filled with null value.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, byte[] value)
    Constructs KeyValue structure as a put filled with specified values and LATEST_TIMESTAMP.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, byte[] tags)
     
    KeyValue(byte[] row, byte[] family, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, List<Tag> tags)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, byte[] value)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, byte[] value, List<Tag> tags)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, byte[] value, Tag[] tags)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type, byte[] value)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type, byte[] value, byte[] tags)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type, byte[] value, List<Tag> tags)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] bytes, int offset)
    Creates a KeyValue from the specified byte array and offset.
    KeyValue(byte[] buffer, int boffset, byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, Tag[] tags)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] bytes, int offset, int length)
    Creates a KeyValue from the specified byte array, starting at offset, and for length length.
    KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, byte[] tags, int tagsOffset, int tagsLength)
     
    KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, List<Tag> tags)
    Constructs KeyValue structure filled with specified values.
    KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, ByteBuffer qualifier, long ts, KeyValue.Type type, ByteBuffer value, List<Tag> tags)
     
    KeyValue(byte[] bytes, int offset, int length, long ts)
    Creates a KeyValue from the specified byte array, starting at offset, and for length length.
    KeyValue(byte[] row, long timestamp)
    Constructs KeyValue structure filled with null value.
    KeyValue(byte[] row, long timestamp, KeyValue.Type type)
    Constructs KeyValue structure filled with null value.
    KeyValue(int rlength, int flength, int qlength, long timestamp, KeyValue.Type type, int vlength)
    Constructs an empty KeyValue structure, with specified sizes.
    KeyValue(int rlength, int flength, int qlength, long timestamp, KeyValue.Type type, int vlength, int tagsLength)
    Constructs an empty KeyValue structure, with specified sizes.
    KeyValue(ExtendedCell c)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    private int
    calculateHashForKey(ExtendedCell cell)
     
    (package private) static void
    checkParameters(byte[] row, int rlength, byte[] family, int flength, int qlength, int vlength)
    Checks the parameters passed to a constructor.
    KeyValue
    clone()
    Clones a KeyValue.
    static KeyValue
    create(int length, DataInput in)
    Create a KeyValue reading length from in
    static KeyValue
    create(DataInput in)
    Create a KeyValue reading from in
    private static byte[]
    createByteArray(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, byte[] tags, int tagsOffset, int tagsLength)
    Write KeyValue format into a byte array.
    private static byte[]
    createByteArray(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, Object qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, Object value, int voffset, int vlength, List<Tag> tags)
     
    private static byte[]
    createEmptyByteArray(int rlength, int flength, int qlength, long timestamp, KeyValue.Type type, int vlength, int tagsLength)
    Create an empty byte[] representing a KeyValue All lengths are preset and can be filled in later.
    KeyValue
    createKeyOnly(boolean lenAsVal)
    Creates a new KeyValue that only contains the key portion (the value is set to be null).
    ExtendedCell
    deepClone()
    Does a deep copy of the contents to a new memory area and returns it as a new cell.
    boolean
    equals(Object other)
    Needed doing 'contains' on List.
    byte[]
    getBuffer()
    To be used only in tests where the Cells are clearly assumed to be of type KeyValue and that we need access to the backing array to do some test case related assertions.
    static int
    getDelimiter(byte[] b, int offset, int length, int delimiter)
    Find index of passed delimiter walking from start of buffer forwards.
    static int
    getDelimiterInReverse(byte[] b, int offset, int length, int delimiter)
    Find index of passed delimiter walking from end of buffer backwards.
    byte[]
    getFamilyArray()
    Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
    byte
    getFamilyLength()
    Returns Family length
    byte
    getFamilyLength(int famLenPos)
    Returns Family length
    (package private) int
    getFamilyLengthPosition(int rowLength)
     
    int
    getFamilyOffset()
    Returns Family offset
    (package private) int
    getFamilyOffset(int familyLenPosition)
    Returns Family offset
    byte[]
    getKey()
    Do not use unless you have to.
    static long
    getKeyDataStructureSize(int rlength, int flength, int qlength)
    Computes the number of bytes that a KeyValue instance with the provided characteristics would take up in its underlying data structure for the key.
    int
    getKeyLength()
    Returns Length of key portion.
    int
    getKeyOffset()
    Returns Key offset in backing buffer..
    String
    getKeyString()
     
    static long
    getKeyValueDataStructureSize(int klength, int vlength, int tagsLength)
    Computes the number of bytes that a KeyValue instance with the provided characteristics would take up for its underlying data structure.
    static long
    getKeyValueDataStructureSize(int rlength, int flength, int qlength, int vlength)
    Computes the number of bytes that a KeyValue instance with the provided characteristics would take up for its underlying data structure.
    static long
    getKeyValueDataStructureSize(int rlength, int flength, int qlength, int vlength, int tagsLength)
    Computes the number of bytes that a KeyValue instance with the provided characteristics would take up for its underlying data structure.
    int
    getLength()
    Returns Length of bytes this KeyValue occupies in getBuffer().
    private static int
    getLength(byte[] bytes, int offset)
    Determines the total length of the KeyValue stored in the specified byte array and offset.
    int
    getOffset()
    Returns Offset into getBuffer() at which this KeyValue starts.
    byte[]
    getQualifierArray()
    Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
    int
    getQualifierLength()
    Returns Qualifier length
    private int
    getQualifierLength(int rlength, int flength)
    Returns Qualifier length
    (package private) int
    getQualifierLength(int keyLength, int rlength, int flength)
    Returns Qualifier length
    int
    getQualifierOffset()
    Returns Qualifier offset
    private int
    getQualifierOffset(int foffset)
    Returns Qualifier offset
    (package private) int
    getQualifierOffset(int foffset, int flength)
    Returns Qualifier offset
    byte[]
    getRowArray()
    Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
    short
    getRowLength()
    Returns Row length
    int
    getRowOffset()
    Returns Row offset
    long
    getSequenceId()
    used to achieve atomic operations in the memstore.
    int
    getSerializedSize()
    Returns Serialized size (defaults to include tag length).
    int
    getSerializedSize(boolean withTags)
    KeyValue format
    byte[]
    getTagsArray()
    Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
    int
    getTagsLength()
    Return the total length of the tag bytes
    int
    getTagsOffset()
    Return the offset where the tag data starts.
    long
    getTimestamp()
    Return the timestamp.
    (package private) long
    getTimestamp(int keylength)
    Return the timestamp.
    int
    getTimestampOffset()
    Returns Timestamp offset
    private int
    getTimestampOffset(int keylength)
    Return the timestamp offset
    byte
    getTypeByte()
    Returns KeyValue.TYPE byte representation
    (package private) byte
    getTypeByte(int keyLength)
    Return the KeyValue.TYPE byte representation
    byte[]
    getValueArray()
    Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
    int
    getValueLength()
    Returns Value length
    int
    getValueOffset()
    Returns the value offset
    int
    hashCode()
    In line with equals(Object), only uses the key portion, not the value.
    long
    heapSize()
    HeapSize implementation
    static String
    humanReadableTimestamp(long timestamp)
     
    boolean
    isLatestTimestamp()
    Returns True if this KeyValue has a LATEST_TIMESTAMP timestamp.
    static String
    keyToString(byte[] k)
    Return key as a String, empty string if k is null.
    static String
    keyToString(byte[] b, int o, int l)
    Use for logging.
    void
    setSequenceId(long seqId)
    Sets with the given seqId.
    void
    setTimestamp(byte[] ts)
    Sets with the given timestamp.
    void
    setTimestamp(long ts)
    Sets with the given timestamp.
    KeyValue
    shallowCopy()
    Creates a shallow copy of this KeyValue, reusing the data byte buffer.
    String
    toString()
     
    Map<String,Object>
    toStringMap()
    Produces a string map for this key/value pair.
    boolean
    updateLatestStamp(byte[] now)
    Update the timestamp.
    int
    write(OutputStream out, boolean withTags)
    Write this cell to an OutputStream in a KeyValue format.
    void
    write(ByteBuffer buf, int offset)
    Write this Cell into the given buf's offset in a KeyValue format.
    static long
    write(KeyValue kv, DataOutput out)
    Write out a KeyValue in the manner in which we used to when KeyValue was a Writable.
    static int
    writeByteArray(byte[] buffer, int boffset, byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, Tag[] tags)
    Write KeyValue format into the provided byte array.

    Methods inherited from class java.lang.Object

    finalize, getClass, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.hbase.ExtendedCell

    getChunkId, getType

    Methods inherited from interface org.apache.hadoop.hbase.RawCell

    cloneTags, getTag, getTags

  • Field Details

  • Constructor Details

    • KeyValue

      public KeyValue()
      Writable Constructor -- DO NOT USE

    • KeyValue

      public KeyValue(byte[] bytes)
      Creates a KeyValue from the start of the specified byte array. Presumes bytes content is formatted as a KeyValue blob.
      Parameters:
      bytes - byte array

    • KeyValue

      public KeyValue(byte[] bytes, int offset)
      Creates a KeyValue from the specified byte array and offset. Presumes bytes content starting at offset is formatted as a KeyValue blob.
      Parameters:
      bytes - byte array
      offset - offset to start of KeyValue

    • KeyValue

      public KeyValue(byte[] bytes, int offset, int length)
      Creates a KeyValue from the specified byte array, starting at offset, and for length length.
      Parameters:
      bytes - byte array
      offset - offset to start of the KeyValue
      length - length of the KeyValue

    • KeyValue

      public KeyValue(byte[] bytes, int offset, int length, long ts)
      Creates a KeyValue from the specified byte array, starting at offset, and for length length.
      Parameters:
      bytes - byte array
      offset - offset to start of the KeyValue
      length - length of the KeyValue
      ts - timestamp

    • KeyValue

      public KeyValue(byte[] row, long timestamp)
      Constructs KeyValue structure filled with null value. Sets type to KeyValue.Type.Maximum
      Parameters:
      row - - row key (arbitrary byte array)
      timestamp - version timestamp

    • KeyValue

      public KeyValue(byte[] row, long timestamp, KeyValue.Type type)
      Constructs KeyValue structure filled with null value.
      Parameters:
      row - - row key (arbitrary byte array)
      timestamp - version timestamp

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier)
      Constructs KeyValue structure filled with null value. Sets type to KeyValue.Type.Maximum
      Parameters:
      row - - row key (arbitrary byte array)
      family - family name
      qualifier - column qualifier

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, byte[] value)
      Constructs KeyValue structure as a put filled with specified values and LATEST_TIMESTAMP.
      Parameters:
      row - - row key (arbitrary byte array)
      family - family name
      qualifier - column qualifier

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type)
      Constructs KeyValue structure filled with specified values.
      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      timestamp - version timestamp
      type - key type
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, byte[] value)
      Constructs KeyValue structure filled with specified values.
      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      timestamp - version timestamp
      value - column value
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, byte[] value, Tag[] tags)
      Constructs KeyValue structure filled with specified values.
      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      timestamp - version timestamp
      value - column value
      tags - tags
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, byte[] value, List<Tag> tags)
      Constructs KeyValue structure filled with specified values.
      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      timestamp - version timestamp
      value - column value
      tags - tags non-empty list of tags or null
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type, byte[] value)
      Constructs KeyValue structure filled with specified values.
      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      timestamp - version timestamp
      type - key type
      value - column value
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type, byte[] value, List<Tag> tags)
      Constructs KeyValue structure filled with specified values.

      Column is split into two fields, family and qualifier.

      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      timestamp - version timestamp
      type - key type
      value - column value
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, long timestamp, KeyValue.Type type, byte[] value, byte[] tags)
      Constructs KeyValue structure filled with specified values.
      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      timestamp - version timestamp
      type - key type
      value - column value
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, List<Tag> tags)
      Constructs KeyValue structure filled with specified values.
      Parameters:
      row - row key
      family - family name
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      value - column value
      voffset - value offset
      vlength - value length
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, byte[] family, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, byte[] tags)
      Parameters:
      row - row key
      family - family name
      qualifier - qualifier name
      qoffset - qualifier offset
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      value - column value
      voffset - value offset
      vlength - value length
      tags - tags

    • KeyValue

      public KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength)
      Constructs KeyValue structure filled with specified values.

      Column is split into two fields, family and qualifier.

      Parameters:
      row - row key
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] buffer, int boffset, byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, Tag[] tags)
      Constructs KeyValue structure filled with specified values. Uses the provided buffer as the data buffer.

      Column is split into two fields, family and qualifier.

      Parameters:
      buffer - the bytes buffer to use
      boffset - buffer offset
      row - row key
      roffset - row offset
      rlength - row length
      family - family name
      foffset - family offset
      flength - family length
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      value - column value
      voffset - value offset
      vlength - value length
      tags - non-empty list of tags or null
      Throws:
      IllegalArgumentException - an illegal value was passed or there is insufficient space remaining in the buffer

    • KeyValue

      public KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, List<Tag> tags)
      Constructs KeyValue structure filled with specified values.

      Column is split into two fields, family and qualifier.

      Parameters:
      row - row key
      roffset - row offset
      rlength - row length
      family - family name
      foffset - family offset
      flength - family length
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      value - column value
      voffset - value offset
      vlength - value length
      tags - tags
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, byte[] tags, int tagsOffset, int tagsLength)
      Parameters:
      row - row key
      roffset - row offset
      rlength - row length
      family - family name
      foffset - fammily offset
      flength - family length
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      value - column value
      voffset - value offset
      vlength - value length
      tags - input tags

    • KeyValue

      public KeyValue(int rlength, int flength, int qlength, long timestamp, KeyValue.Type type, int vlength)
      Constructs an empty KeyValue structure, with specified sizes. This can be used to partially fill up KeyValues.

      Column is split into two fields, family and qualifier.

      Parameters:
      rlength - row length
      flength - family length
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      vlength - value length
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(int rlength, int flength, int qlength, long timestamp, KeyValue.Type type, int vlength, int tagsLength)
      Constructs an empty KeyValue structure, with specified sizes. This can be used to partially fill up KeyValues.

      Column is split into two fields, family and qualifier.

      Parameters:
      rlength - row length
      flength - family length
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      vlength - value length
      tagsLength - length of the tags
      Throws:
      IllegalArgumentException - an illegal value was passed

    • KeyValue

      public KeyValue(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, ByteBuffer qualifier, long ts, KeyValue.Type type, ByteBuffer value, List<Tag> tags)

    • KeyValue

      public KeyValue(ExtendedCell c)

  • Method Details

    • getKeyValueDataStructureSize

      public static long getKeyValueDataStructureSize(int rlength, int flength, int qlength, int vlength)
      Computes the number of bytes that a KeyValue instance with the provided characteristics would take up for its underlying data structure.
      Parameters:
      rlength - row length
      flength - family length
      qlength - qualifier length
      vlength - value length
      Returns:
      the KeyValue data structure length

    • getKeyValueDataStructureSize

      public static long getKeyValueDataStructureSize(int rlength, int flength, int qlength, int vlength, int tagsLength)
      Computes the number of bytes that a KeyValue instance with the provided characteristics would take up for its underlying data structure.
      Parameters:
      rlength - row length
      flength - family length
      qlength - qualifier length
      vlength - value length
      tagsLength - total length of the tags
      Returns:
      the KeyValue data structure length

    • getKeyValueDataStructureSize

      public static long getKeyValueDataStructureSize(int klength, int vlength, int tagsLength)
      Computes the number of bytes that a KeyValue instance with the provided characteristics would take up for its underlying data structure.
      Parameters:
      klength - key length
      vlength - value length
      tagsLength - total length of the tags
      Returns:
      the KeyValue data structure length

    • getKeyDataStructureSize

      public static long getKeyDataStructureSize(int rlength, int flength, int qlength)
      Computes the number of bytes that a KeyValue instance with the provided characteristics would take up in its underlying data structure for the key.
      Parameters:
      rlength - row length
      flength - family length
      qlength - qualifier length
      Returns:
      the key data structure length

    • getSequenceId

      public long getSequenceId()
      used to achieve atomic operations in the memstore.
      Specified by:
      getSequenceId in interface ExtendedCell
      Returns:
      seqId (always > 0 if exists), or 0 if it no longer exists

    • setSequenceId

      public void setSequenceId(long seqId)
      Description copied from interface: ExtendedCell
      Sets with the given seqId.
      Specified by:
      setSequenceId in interface ExtendedCell
      Parameters:
      seqId - sequence ID

    • createEmptyByteArray

      private static byte[] createEmptyByteArray(int rlength, int flength, int qlength, long timestamp, KeyValue.Type type, int vlength, int tagsLength)
      Create an empty byte[] representing a KeyValue All lengths are preset and can be filled in later.
      Parameters:
      rlength - row length
      flength - family length
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      vlength - value length
      Returns:
      The newly created byte array.

    • checkParameters

      static void checkParameters(byte[] row, int rlength, byte[] family, int flength, int qlength, int vlength) throws IllegalArgumentException
      Checks the parameters passed to a constructor.
      Parameters:
      row - row key
      rlength - row length
      family - family name
      flength - family length
      qlength - qualifier length
      vlength - value length
      Throws:
      IllegalArgumentException - an illegal value was passed

    • writeByteArray

      public static int writeByteArray(byte[] buffer, int boffset, byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, Tag[] tags)
      Write KeyValue format into the provided byte array.
      Parameters:
      buffer - the bytes buffer to use
      boffset - buffer offset
      row - row key
      roffset - row offset
      rlength - row length
      family - family name
      foffset - family offset
      flength - family length
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      value - column value
      voffset - value offset
      vlength - value length
      Returns:
      The number of useful bytes in the buffer.
      Throws:
      IllegalArgumentException - an illegal value was passed or there is insufficient space remaining in the buffer

    • createByteArray

      private static byte[] createByteArray(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, byte[] value, int voffset, int vlength, byte[] tags, int tagsOffset, int tagsLength)
      Write KeyValue format into a byte array.
      Parameters:
      row - row key
      roffset - row offset
      rlength - row length
      family - family name
      foffset - family offset
      flength - family length
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      timestamp - version timestamp
      type - key type
      value - column value
      voffset - value offset
      vlength - value length
      Returns:
      The newly created byte array.

    • createByteArray

      private static byte[] createByteArray(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, Object qualifier, int qoffset, int qlength, long timestamp, KeyValue.Type type, Object value, int voffset, int vlength, List<Tag> tags)
      Parameters:
      qualifier - can be a ByteBuffer or a byte[], or null.
      value - can be a ByteBuffer or a byte[], or null.

    • equals

      public boolean equals(Object other)
      Needed doing 'contains' on List. Only compares the key portion, not the value.
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      In line with equals(Object), only uses the key portion, not the value.
      Overrides:
      hashCode in class Object

    • calculateHashForKey

      private int calculateHashForKey(ExtendedCell cell)

    • clone

      public KeyValue clone() throws CloneNotSupportedException
      Clones a KeyValue. This creates a copy, re-allocating the buffer.
      Overrides:
      clone in class Object
      Returns:
      Fully copied clone of this KeyValue
      Throws:
      CloneNotSupportedException - if cloning of keyValue is not supported

    • shallowCopy

      public KeyValue shallowCopy()
      Creates a shallow copy of this KeyValue, reusing the data byte buffer. http://en.wikipedia.org/wiki/Object_copy
      Returns:
      Shallow copy of this KeyValue

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • keyToString

      public static String keyToString(byte[] k)
      Return key as a String, empty string if k is null.

    • toStringMap

      public Map<String,Object> toStringMap()
      Produces a string map for this key/value pair. Useful for programmatic use and manipulation of the data stored in an WALKey, for example, printing as JSON. Values are left out due to their tendency to be large. If needed, they can be added manually.
      Returns:
      the Map<String,?> containing data from this key

    • keyToString

      public static String keyToString(byte[] b, int o, int l)
      Use for logging.
      Parameters:
      b - Key portion of a KeyValue.
      o - Offset to start of key
      l - Length of key.
      Returns:
      Key as a String.

    • humanReadableTimestamp

      public static String humanReadableTimestamp(long timestamp)

    • getBuffer

      public byte[] getBuffer()
      To be used only in tests where the Cells are clearly assumed to be of type KeyValue and that we need access to the backing array to do some test case related assertions.
      Returns:
      The byte array backing this KeyValue.

    • getOffset

      public int getOffset()
      Returns Offset into getBuffer() at which this KeyValue starts.

    • getLength

      public int getLength()
      Returns Length of bytes this KeyValue occupies in getBuffer().

    • getLength

      private static int getLength(byte[] bytes, int offset)
      Determines the total length of the KeyValue stored in the specified byte array and offset. Includes all headers.
      Parameters:
      bytes - byte array
      offset - offset to start of the KeyValue
      Returns:
      length of entire KeyValue, in bytes

    • getKeyOffset

      public int getKeyOffset()
      Returns Key offset in backing buffer..

    • getKeyString

      public String getKeyString()

    • getKeyLength

      public int getKeyLength()
      Returns Length of key portion.

    • getValueArray

      public byte[] getValueArray()
      Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
      Specified by:
      getValueArray in interface Cell
      Returns:
      The array containing the value bytes.

    • getValueOffset

      public int getValueOffset()
      Returns the value offset
      Specified by:
      getValueOffset in interface Cell

    • getValueLength

      public int getValueLength()
      Returns Value length
      Specified by:
      getValueLength in interface Cell

    • getRowArray

      public byte[] getRowArray()
      Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
      Specified by:
      getRowArray in interface Cell
      Returns:
      The array containing the row bytes.

    • getRowOffset

      public int getRowOffset()
      Returns Row offset
      Specified by:
      getRowOffset in interface Cell

    • getRowLength

      public short getRowLength()
      Returns Row length
      Specified by:
      getRowLength in interface Cell

    • getFamilyArray

      public byte[] getFamilyArray()
      Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
      Specified by:
      getFamilyArray in interface Cell
      Returns:
      the array containing the family bytes.

    • getFamilyOffset

      public int getFamilyOffset()
      Returns Family offset
      Specified by:
      getFamilyOffset in interface Cell

    • getFamilyOffset

      int getFamilyOffset(int familyLenPosition)
      Returns Family offset

    • getFamilyLength

      public byte getFamilyLength()
      Returns Family length
      Specified by:
      getFamilyLength in interface Cell

    • getFamilyLength

      public byte getFamilyLength(int famLenPos)
      Returns Family length

    • getFamilyLengthPosition

      int getFamilyLengthPosition(int rowLength)

    • getQualifierArray

      public byte[] getQualifierArray()
      Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
      Specified by:
      getQualifierArray in interface Cell
      Returns:
      The array containing the qualifier bytes.

    • getQualifierOffset

      public int getQualifierOffset()
      Returns Qualifier offset
      Specified by:
      getQualifierOffset in interface Cell

    • getQualifierOffset

      private int getQualifierOffset(int foffset)
      Returns Qualifier offset

    • getQualifierOffset

      int getQualifierOffset(int foffset, int flength)
      Returns Qualifier offset

    • getQualifierLength

      public int getQualifierLength()
      Returns Qualifier length
      Specified by:
      getQualifierLength in interface Cell

    • getQualifierLength

      private int getQualifierLength(int rlength, int flength)
      Returns Qualifier length

    • getQualifierLength

      int getQualifierLength(int keyLength, int rlength, int flength)
      Returns Qualifier length

    • getTimestampOffset

      public int getTimestampOffset()
      Returns Timestamp offset

    • getTimestampOffset

      private int getTimestampOffset(int keylength)
      Return the timestamp offset

    • isLatestTimestamp

      public boolean isLatestTimestamp()
      Returns True if this KeyValue has a LATEST_TIMESTAMP timestamp.

    • updateLatestStamp

      public boolean updateLatestStamp(byte[] now)
      Update the timestamp.
      Parameters:
      now - Time to set into this IFF timestamp == HConstants.LATEST_TIMESTAMP (else, its a noop).
      Returns:
      True is we modified this.

    • setTimestamp

      public void setTimestamp(long ts)
      Description copied from interface: ExtendedCell
      Sets with the given timestamp.
      Specified by:
      setTimestamp in interface ExtendedCell
      Parameters:
      ts - timestamp

    • setTimestamp

      public void setTimestamp(byte[] ts)
      Description copied from interface: ExtendedCell
      Sets with the given timestamp.
      Specified by:
      setTimestamp in interface ExtendedCell
      Parameters:
      ts - buffer containing the timestamp value

    • getKey

      public byte[] getKey()
      Do not use unless you have to. Used internally for compacting and testing. Use getRowArray(), getFamilyArray(), getQualifierArray(), and getValueArray() if accessing a KeyValue client-side.
      Returns:
      Copy of the key portion only.

    • getTimestamp

      public long getTimestamp()
      Return the timestamp.
      Specified by:
      getTimestamp in interface Cell

    • getTimestamp

      long getTimestamp(int keylength)
      Return the timestamp.

    • getTypeByte

      public byte getTypeByte()
      Returns KeyValue.TYPE byte representation
      Specified by:
      getTypeByte in interface ExtendedCell

    • getTypeByte

      byte getTypeByte(int keyLength)
      Return the KeyValue.TYPE byte representation

    • getTagsOffset

      public int getTagsOffset()
      Return the offset where the tag data starts.
      Specified by:
      getTagsOffset in interface RawCell

    • getTagsLength

      public int getTagsLength()
      Return the total length of the tag bytes
      Specified by:
      getTagsLength in interface RawCell
      Returns:
      the total length of the tags in the Cell.

    • getTagsArray

      public byte[] getTagsArray()
      Returns the backing array of the entire KeyValue (all KeyValue fields are in a single array)
      Specified by:
      getTagsArray in interface RawCell
      Returns:
      the tags byte array

    • createKeyOnly

      public KeyValue createKeyOnly(boolean lenAsVal)
      Creates a new KeyValue that only contains the key portion (the value is set to be null). TODO only used by KeyOnlyFilter -- move there.
      Parameters:
      lenAsVal - replace value with the actual value length (false=empty)

    • getDelimiter

      public static int getDelimiter(byte[] b, int offset, int length, int delimiter)
      Find index of passed delimiter walking from start of buffer forwards.
      Parameters:
      b - the kv serialized byte[] to process
      delimiter - input delimeter to fetch index from start
      Returns:
      Index of delimiter having started from start of b moving rightward.

    • getDelimiterInReverse

      public static int getDelimiterInReverse(byte[] b, int offset, int length, int delimiter)
      Find index of passed delimiter walking from end of buffer backwards.
      Parameters:
      b - the kv serialized byte[] to process
      offset - the offset in the byte[]
      length - the length in the byte[]
      delimiter - input delimeter to fetch index from end
      Returns:
      Index of delimiter

    • create

      public static KeyValue create(DataInput in) throws IOException
      Create a KeyValue reading from in
      Parameters:
      in - Where to read bytes from. Creates a byte array to hold the KeyValue backing bytes copied from the steam.
      Returns:
      KeyValue created by deserializing from in OR if we find a length of zero, we will return null which can be useful marking a stream as done.
      Throws:
      IOException - if any IO error happen

    • create

      public static KeyValue create(int length, DataInput in) throws IOException
      Create a KeyValue reading length from in
      Parameters:
      length - length of the Key
      in - Input to read from
      Returns:
      Created KeyValue OR if we find a length of zero, we will return null which can be useful marking a stream as done.
      Throws:
      IOException - if any IO error happen

    • write

      public static long write(KeyValue kv, DataOutput out) throws IOException
      Write out a KeyValue in the manner in which we used to when KeyValue was a Writable.
      Parameters:
      kv - the KeyValue on which write is being requested
      out - OutputStream to write keyValue to
      Returns:
      Length written on stream
      Throws:
      IOException - if any IO error happen
      See Also:

    • write

      public int write(OutputStream out, boolean withTags) throws IOException
      Description copied from interface: ExtendedCell
      Write this cell to an OutputStream in a KeyValue format.
      KeyValue format
      <4 bytes keylength> <4 bytes valuelength> <2 bytes rowlength> <row> <1 byte columnfamilylength> <columnfamily> <columnqualifier> <8 bytes timestamp> <1 byte keytype> <value> <2 bytes tagslength> <tags>
      Specified by:
      write in interface ExtendedCell
      Parameters:
      out - Stream to which cell has to be written
      withTags - Whether to write tags.
      Returns:
      how many bytes are written.
      Throws:
      IOException

    • getSerializedSize

      public int getSerializedSize(boolean withTags)
      Description copied from interface: ExtendedCell
      KeyValue format

      <4 bytes keylength> <4 bytes valuelength> <2 bytes rowlength> <row> <1 byte columnfamilylength> <columnfamily> <columnqualifier> <8 bytes timestamp> <1 byte keytype> <value> <2 bytes tagslength> <tags>

      Specified by:
      getSerializedSize in interface ExtendedCell
      Parameters:
      withTags - Whether to write tags.
      Returns:
      Bytes count required to serialize this Cell in a KeyValue format.

    • getSerializedSize

      public int getSerializedSize()
      Description copied from interface: ExtendedCell
      Returns Serialized size (defaults to include tag length).
      Specified by:
      getSerializedSize in interface Cell
      Specified by:
      getSerializedSize in interface ExtendedCell

    • write

      public void write(ByteBuffer buf, int offset)
      Description copied from interface: ExtendedCell
      Write this Cell into the given buf's offset in a KeyValue format.
      Specified by:
      write in interface ExtendedCell
      Parameters:
      buf - The buffer where to write the Cell.
      offset - The offset within buffer, to write the Cell.

    • heapSize

      public long heapSize()
      HeapSize implementation

      We do not count the bytes in the rowCache because it should be empty for a KeyValue in the MemStore.

      Specified by:
      heapSize in interface HeapSize

    • deepClone

      public ExtendedCell deepClone()
      Description copied from interface: ExtendedCell
      Does a deep copy of the contents to a new memory area and returns it as a new cell.
      Specified by:
      deepClone in interface ExtendedCell
      Returns:
      The deep cloned cell