org.apache.hadoop.hbase

Class KeyValueUtil

java.lang.Object

org.apache.hadoop.hbase.KeyValueUtil

@InterfaceAudience.Private
public class KeyValueUtil
extends Object

static convenience methods for dealing with KeyValues and collections of KeyValues

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
KeyValueUtil()

Method Summary

Modifier and Type	Method and Description
static int	appendKeyTo(Cell cell, byte[] output, int offset)
static int	appendKeyTo(Cell cell, ByteBuffer buf, int offset)
static int	appendTo(Cell cell, ByteBuffer buf, int offset, boolean withTags) Copy the Cell content into the passed buf in KeyValue serialization format.
static int	appendToByteArray(Cell cell, byte[] output, int offset, boolean withTags) copy key and value
static void	appendToByteBuffer(ByteBuffer bb, KeyValue kv, boolean includeMvccVersion)
(package private) static String	bytesToHex(byte[] buf, int offset, int length)
(package private) static void	checkKeyValueBytes(byte[] buf, int offset, int length, boolean withTags)
private static int	checkKeyValueTagBytes(byte[] buf, int offset, int length, int pos, int endOffset)
static ByteBuffer	copyKeyToNewByteBuffer(Cell cell) The position will be set to the beginning of the new ByteBuffer
static byte[]	copyToNewByteArray(Cell cell)
static KeyValue	copyToNewKeyValue(Cell cell) copy the cell to create a new keyvalue
static KeyValue	create(DataInput in)
static KeyValue	create(int length, DataInput in) Create a KeyValue reading length from in
static KeyValue	createFirstOnRow(byte[] row) Create a KeyValue that is smaller than all other possible KeyValues for the given row.
static KeyValue	createFirstOnRow(byte[] row, byte[] family, byte[] qualifier) Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row,family,qualifier.
static KeyValue	createFirstOnRow(byte[] buffer, byte[] row, byte[] family, byte[] qualifier) Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier.
static KeyValue	createFirstOnRow(byte[] row, byte[] f, byte[] q, long ts)
static KeyValue	createFirstOnRow(byte[] buffer, int boffset, byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength) Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier.
static KeyValue	createFirstOnRow(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength) Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier.
static KeyValue	createFirstOnRow(byte[] row, int roffset, short rlength) Create a KeyValue that is smaller than all other possible KeyValues for the given row.
static KeyValue	createFirstOnRow(byte[] row, long ts) Creates a KeyValue that is smaller than all other KeyValues that are older than the passed timestamp.
static KeyValue	createKeyValueFromInputStream(InputStream in, boolean withTags) Create a KeyValue reading from the raw InputStream.
static KeyValue	createKeyValueFromKey(byte[] b)
static KeyValue	createKeyValueFromKey(byte[] b, int o, int l)
static KeyValue	createKeyValueFromKey(ByteBuffer bb)
static KeyValue	createLastOnRow(byte[] row) Creates a KeyValue that is last on the specified row id.
static KeyValue	createLastOnRow(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength) Create a KeyValue for the specified row, family and qualifier that would be larger than or equal to all other possible KeyValues that have the same row, family, qualifier.
static KeyValue	ensureKeyValue(Cell cell) Deprecated. without any replacement.
static List<KeyValue>	ensureKeyValues(List<Cell> cells) Deprecated.
static int	getSerializedSize(Cell cell, boolean withTags)
static int	keyLength(Cell cell) Returns number of bytes this cell's key part would have been used if serialized as in KeyValue.
private static int	keyLength(short rlen, byte flen, int qlen)
static int	length(Cell cell) Returns number of bytes this cell would have been used if serialized as in KeyValue
static int	length(short rlen, byte flen, int qlen, int vlen, int tlen, boolean withTags)
static int	lengthWithMvccVersion(KeyValue kv, boolean includeMvccVersion)
static KeyValue	nextShallowCopy(ByteBuffer bb, boolean includesMvccVersion, boolean includesTags) Creates a new KeyValue object positioned in the supplied ByteBuffer and sets the ByteBuffer's position to the start of the next KeyValue.
static int	oswrite(Cell cell, OutputStream out, boolean withTags)
static KeyValue	previousKey(KeyValue in) Decrement the timestamp.
static KeyValue	toNewKeyCell(Cell cell) Copies the key to a new KeyValue
static int	totalLengthWithMvccVersion(Iterable<? extends KeyValue> kvs, boolean includeMvccVersion)
static long	write(KeyValue kv, DataOutput out) Write out a KeyValue in the manner in which we used to when KeyValue was a Writable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

Constructor Detail

KeyValueUtil

public KeyValueUtil()

Method Detail

length

public static int length(Cell cell)

Returns number of bytes this cell would have been used if serialized as in KeyValue

Parameters:

cell -

Returns:

the length

length

public static int length(short rlen,
                         byte flen,
                         int qlen,
                         int vlen,
                         int tlen,
                         boolean withTags)

keyLength

public static int keyLength(Cell cell)

Returns number of bytes this cell's key part would have been used if serialized as in KeyValue. Key includes rowkey, family, qualifier, timestamp and type.

Parameters:

cell -

Returns:

the key length

keyLength

private static int keyLength(short rlen,
                             byte flen,
                             int qlen)

lengthWithMvccVersion

public static int lengthWithMvccVersion(KeyValue kv,
                                        boolean includeMvccVersion)

totalLengthWithMvccVersion

public static int totalLengthWithMvccVersion(Iterable<? extends KeyValue> kvs,
                                             boolean includeMvccVersion)

copyToNewKeyValue

public static KeyValue copyToNewKeyValue(Cell cell)

copy the cell to create a new keyvalue

copyKeyToNewByteBuffer

public static ByteBuffer copyKeyToNewByteBuffer(Cell cell)

The position will be set to the beginning of the new ByteBuffer

Parameters:

cell -

Returns:

the Bytebuffer containing the key part of the cell

toNewKeyCell

public static KeyValue toNewKeyCell(Cell cell)

Copies the key to a new KeyValue

Parameters:

cell -

Returns:

the KeyValue that consists only the key part of the incoming cell

copyToNewByteArray

public static byte[] copyToNewByteArray(Cell cell)

appendKeyTo

public static int appendKeyTo(Cell cell,
                              byte[] output,
                              int offset)

appendToByteArray

public static int appendToByteArray(Cell cell,
                                    byte[] output,
                                    int offset,
                                    boolean withTags)

copy key and value

appendTo

public static int appendTo(Cell cell,
                           ByteBuffer buf,
                           int offset,
                           boolean withTags)

Copy the Cell content into the passed buf in KeyValue serialization format.

appendKeyTo

public static int appendKeyTo(Cell cell,
                              ByteBuffer buf,
                              int offset)

appendToByteBuffer

public static void appendToByteBuffer(ByteBuffer bb,
                                      KeyValue kv,
                                      boolean includeMvccVersion)

nextShallowCopy

public static KeyValue nextShallowCopy(ByteBuffer bb,
                                       boolean includesMvccVersion,
                                       boolean includesTags)

Creates a new KeyValue object positioned in the supplied ByteBuffer and sets the ByteBuffer's position to the start of the next KeyValue. Does not allocate a new array or copy data.

Parameters:

bb -

includesMvccVersion -

includesTags -

previousKey

public static KeyValue previousKey(KeyValue in)

Decrement the timestamp. For tests (currently wasteful) Remember timestamps are sorted reverse chronologically.

Parameters:

in -

Returns:

previous key

createLastOnRow

public static KeyValue createLastOnRow(byte[] row,
                                       int roffset,
                                       int rlength,
                                       byte[] family,
                                       int foffset,
                                       int flength,
                                       byte[] qualifier,
                                       int qoffset,
                                       int qlength)

Create a KeyValue for the specified row, family and qualifier that would be larger than or equal to all other possible KeyValues that have the same row, family, qualifier. Used for reseeking. Should NEVER be returned to a client.

Parameters:

row - row key

roffset - row offset

rlength - row length

family - family name

foffset - family offset

flength - family length

qualifier - column qualifier

qoffset - qualifier offset

qlength - qualifier length

Returns:

Last possible key on passed row, family, qualifier.

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] row,
                                        int roffset,
                                        short rlength)

Create a KeyValue that is smaller than all other possible KeyValues for the given row. That is any (valid) KeyValue on 'row' would sort _after_ the result.

Parameters:

row - - row key (arbitrary byte array)

Returns:

First possible KeyValue on passed row

createLastOnRow

public static KeyValue createLastOnRow(byte[] row)

Creates a KeyValue that is last on the specified row id. That is, every other possible KeyValue for the given row would compareTo() less than the result of this call.

Parameters:

row - row key

Returns:

Last possible KeyValue on passed row

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] row)

Create a KeyValue that is smaller than all other possible KeyValues for the given row. That is any (valid) KeyValue on 'row' would sort _after_ the result.

Parameters:

row - - row key (arbitrary byte array)

Returns:

First possible KeyValue on passed row

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] row,
                                        long ts)

Creates a KeyValue that is smaller than all other KeyValues that are older than the passed timestamp.

Parameters:

row - - row key (arbitrary byte array)

ts - - timestamp

Returns:

First possible key on passed row and timestamp.

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] row,
                                        byte[] family,
                                        byte[] qualifier)

Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row,family,qualifier. Used for seeking.

Parameters:

row - - row key (arbitrary byte array)

family - - family name

qualifier - - column qualifier

Returns:

First possible key on passed row, and column.

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] row,
                                        byte[] f,
                                        byte[] q,
                                        long ts)

Parameters:

row - - row key (arbitrary byte array)

f - - family name

q - - column qualifier

ts - - timestamp

Returns:

First possible key on passed row, column and timestamp

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] row,
                                        int roffset,
                                        int rlength,
                                        byte[] family,
                                        int foffset,
                                        int flength,
                                        byte[] qualifier,
                                        int qoffset,
                                        int qlength)

Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier. Used for seeking.

Parameters:

row - row key

roffset - row offset

rlength - row length

family - family name

foffset - family offset

flength - family length

qualifier - column qualifier

qoffset - qualifier offset

qlength - qualifier length

Returns:

First possible key on passed Row, Family, Qualifier.

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] buffer,
                                        byte[] row,
                                        byte[] family,
                                        byte[] qualifier)
                                 throws IllegalArgumentException

Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier. Used for seeking.

Parameters:

buffer - the buffer to use for the new KeyValue object

row - the value key

family - family name

qualifier - column qualifier

Returns:

First possible key on passed Row, Family, Qualifier.

Throws:

IllegalArgumentException - The resulting KeyValue object would be larger than the provided buffer or than Integer.MAX_VALUE

createFirstOnRow

public static KeyValue createFirstOnRow(byte[] buffer,
                                        int boffset,
                                        byte[] row,
                                        int roffset,
                                        int rlength,
                                        byte[] family,
                                        int foffset,
                                        int flength,
                                        byte[] qualifier,
                                        int qoffset,
                                        int qlength)
                                 throws IllegalArgumentException

Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier. Used for seeking.

Parameters:

buffer - the buffer to use for the new KeyValue object

boffset - buffer offset

row - the value key

roffset - row offset

rlength - row length

family - family name

foffset - family offset

flength - family length

qualifier - column qualifier

qoffset - qualifier offset

qlength - qualifier length

Returns:

First possible key on passed Row, Family, Qualifier.

Throws:

IllegalArgumentException - The resulting KeyValue object would be larger than the provided buffer or than Integer.MAX_VALUE

ensureKeyValue

@Deprecated
public static KeyValue ensureKeyValue(Cell cell)

Deprecated. without any replacement.

Parameters:

cell -

Returns:

cell if it is an object of class KeyValue else we will return a new KeyValue instance made from cell Note: Even if the cell is an object of any of the subclass of KeyValue, we will create a new KeyValue object wrapping same buffer. This API is used only with MR based tools which expect the type to be exactly KeyValue. That is the reason for doing this way.

ensureKeyValues

@Deprecated
public static List<KeyValue> ensureKeyValues(List<Cell> cells)

Deprecated.

write

public static long write(KeyValue kv,
                         DataOutput out)
                  throws IOException

Write out a KeyValue in the manner in which we used to when KeyValue was a Writable.

Parameters:

kv -

out -

Returns:

Length written on stream

Throws:

IOException

See Also:

for the inverse function

bytesToHex

static String bytesToHex(byte[] buf,
                         int offset,
                         int length)

checkKeyValueBytes

static void checkKeyValueBytes(byte[] buf,
                               int offset,
                               int length,
                               boolean withTags)

checkKeyValueTagBytes

private static int checkKeyValueTagBytes(byte[] buf,
                                         int offset,
                                         int length,
                                         int pos,
                                         int endOffset)

createKeyValueFromInputStream

public static KeyValue createKeyValueFromInputStream(InputStream in,
                                                     boolean withTags)
                                              throws IOException

Create a KeyValue reading from the raw InputStream. Named createKeyValueFromInputStream so doesn't clash with create(DataInput)

Parameters:

in - inputStream to read.

withTags - whether the keyvalue should include tags are not

Returns:

Created KeyValue OR if we find a length of zero, we will return null which can be useful marking a stream as done.

Throws:

IOException

createKeyValueFromKey

public static KeyValue createKeyValueFromKey(byte[] b)

Parameters:

b -

Returns:

A KeyValue made of a byte array that holds the key-only part. Needed to convert hfile index members to KeyValues.

createKeyValueFromKey

public static KeyValue createKeyValueFromKey(ByteBuffer bb)

Parameters:

bb -

Returns:

A KeyValue made of a byte buffer that holds the key-only part. Needed to convert hfile index members to KeyValues.

createKeyValueFromKey

public static KeyValue createKeyValueFromKey(byte[] b,
                                             int o,
                                             int l)

Parameters:

b -

o -

l -

Returns:

A KeyValue made of a byte array that holds the key-only part. Needed to convert hfile index members to KeyValues.

create

public static KeyValue create(DataInput in)
                       throws IOException

Parameters:

in - Where to read bytes from. Creates a byte array to hold the KeyValue backing bytes copied from the steam.

Returns:

KeyValue created by deserializing from in OR if we find a length of zero, we will return null which can be useful marking a stream as done.

Throws:

IOException

create

public static KeyValue create(int length,
                              DataInput in)
                       throws IOException

Create a KeyValue reading length from in

Parameters:

length -

in -

Returns:

Created KeyValue OR if we find a length of zero, we will return null which can be useful marking a stream as done.

Throws:

IOException

getSerializedSize

public static int getSerializedSize(Cell cell,
                                    boolean withTags)

oswrite

public static int oswrite(Cell cell,
                          OutputStream out,
                          boolean withTags)
                   throws IOException

Throws:

IOException