Package org.apache.hadoop.hbase.util

Class Bytes

java.lang.Object
org.apache.hadoop.hbase.util.Bytes
All Implemented Interfaces:
Comparable<Bytes>
@Public public class Bytes extends Object implements Comparable<Bytes>
Utility class that handles byte arrays, conversions to/from other types, comparisons, hash code generation, manufacturing keys for HashMaps or HashSets, and can be used as key in maps or trees.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    Bytes.ByteArrayComparator
    Byte array comparator class.
    (package private) static class 
    Bytes.CommonPrefixer
     
    (package private) static class 
    Bytes.CommonPrefixerHolder
     
    (package private) static interface 
    Bytes.Comparer<T>
     
    (package private) static class 
    Bytes.Converter
     
    (package private) static class 
    Bytes.ConverterHolder
     
    (package private) static class 
    Bytes.LexicographicalComparerHolder
    Provides a lexicographical comparer implementation; either a Java implementation or a faster implementation based on Unsafe.
    static class 
    Bytes.RowEndKeyComparator
    A Bytes.ByteArrayComparator that treats the empty array as the largest value.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private byte[]
    bytes
     
    static final Comparator<byte[]>
    BYTES_COMPARATOR
    Pass this to TreeMaps where byte [] are keys.
    static final org.apache.hadoop.io.RawComparator<byte[]>
    BYTES_RAWCOMPARATOR
    Use comparing byte arrays, byte-by-byte
    private static final byte[]
    EMPTY_BYTE_ARRAY
     
    static final int
    ESTIMATED_HEAP_TAX
    Estimate of size cost to pay beyond payload in jvm for instance of byte [].
    private static final char[]
    HEX_CHARS
     
    private static final char[]
    HEX_CHARS_UPPER
     
    private int
    length
     
    private static final org.slf4j.Logger
    LOG
     
    static final long
    MASK_FOR_LOWER_INT_IN_LONG
    Mask to apply to a long to reveal the lower int only.
    private int
    offset
     
    private static final Random
    RNG
     
    private static final SecureRandom
    SECURE_RNG
     
    static final int
    SIZEOF_BOOLEAN
    Size of boolean in bytes
    static final int
    SIZEOF_BYTE
    Size of byte in bytes
    static final int
    SIZEOF_CHAR
    Size of char in bytes
    static final int
    SIZEOF_DOUBLE
    Size of double in bytes
    static final int
    SIZEOF_FLOAT
    Size of float in bytes
    static final int
    SIZEOF_INT
    Size of int in bytes
    static final int
    SIZEOF_LONG
    Size of long in bytes
    static final int
    SIZEOF_SHORT
    Size of short in bytes
    (package private) static final boolean
    UNSAFE_UNALIGNED
     
    private static final String
    UTF8_CSN
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    Bytes()
    Create a zero-size sequence.
    Bytes(byte[] bytes)
    Create a Bytes using the byte array as the initial value.
    Bytes(byte[] bytes, int offset, int length)
    Set the value to a given byte range
    Bytes(Bytes ibw)
    Set the new Bytes to the contents of the passed ibw.

  • Method Summary

    Modifier and Type
    Method
    Description
    static byte[]
    add(byte[][] arrays)
    Concatenate byte arrays.
    static byte[]
    add(byte[] a, byte[] b)
    Concatenate byte arrays.
    static byte[]
    add(byte[] a, byte[] b, byte[] c)
    Concatenate byte arrays.
    private static byte[]
    binaryIncrementNeg(byte[] value, long amount)
     
    private static byte[]
    binaryIncrementPos(byte[] value, long amount)
     
    static int
    binarySearch(byte[][] arr, byte[] key, int offset, int length)
    Binary search for keys in indexes using Bytes.BYTES_RAWCOMPARATOR.
    static int
    binarySearch(Cell[] arr, Cell key, CellComparator comparator)
    Binary search for keys in indexes.
    static long
    bytesToVint(byte[] buffer)
    Reads a zero-compressed encoded long from input buffer and returns it.
    int
    compareTo(byte[] that)
    Compares the bytes in this object to the specified byte array
    static int
    compareTo(byte[] left, byte[] right)
    Lexicographically compare two arrays.
    static int
    compareTo(byte[] buffer1, int offset1, int length1, byte[] buffer2, int offset2, int length2)
    Lexicographically compare two arrays.
    int
    compareTo(Bytes that)
    Define the sort order of the Bytes.
    static boolean
    contains(byte[] array, byte target)
    Return true if target is present as an element anywhere in the given array.
    static boolean
    contains(byte[] array, byte[] target)
    Return true if target is present as an element anywhere in the given array.
    static byte[]
    copy(byte[] bytes)
    Copy the byte array given in parameter and return an instance of a new byte array with the same length and the same content.
    static byte[]
    copy(byte[] bytes, int offset, int length)
    Copy the byte array given in parameter and return an instance of a new byte array with the same length and the same content.
    byte[]
    copyBytes()
    Returns a copy of the bytes referred to by this writable
    static byte[]
    createMaxByteArray(int maxByteCount)
    Create a max byte array with the specified max byte count
    static boolean
    equals(byte[] left, byte[] right)
    Lexicographically determine the equality of two arrays.
    static boolean
    equals(byte[] left, int leftOffset, int leftLen, byte[] right, int rightOffset, int rightLen)
    Lexicographically determine the equality of two arrays.
    static boolean
    equals(byte[] a, ByteBuffer buf)
    Lexicographically determine the equality of two byte[], one as ByteBuffer.
    boolean
    equals(Object right_obj)
     
    static boolean
    equals(List<byte[]> a, List<byte[]> b)
     
    private static IllegalArgumentException
    explainWrongLengthOrOffset(byte[] bytes, int offset, int length, int expectedLength)
     
    static int
    findCommonPrefix(byte[] left, byte[] right, int leftLength, int rightLength, int leftOffset, int rightOffset)
     
    static byte[]
    fromHex(String hex)
    Create a byte array from a string of hash digits.
    byte[]
    get()
    Get the data from the Bytes.
    static byte[]
    getBytes(ByteBuffer buf)
    Returns a new byte array, copied from the given buf, from the position (inclusive) to the limit (exclusive).
    int
    getLength()
    Returns the number of valid bytes in the buffer
    int
    getOffset()
    Return the offset into the buffer.
    static List<byte[]>
    getUtf8ByteArrays(List<String> strings)
     
    int
    hashCode()
     
    static int
    hashCode(byte[] b)
    Calculate a hash code from a given byte array.
    static int
    hashCode(byte[] b, int length)
    Calculate a hash code from a given byte array.
    static int
    hashCode(byte[] bytes, int offset, int length)
    Calculate the hash code for a given range of bytes.
    static byte[]
    head(byte[] a, int length)
    Make a new byte array from a subset of bytes at the head of another.
    private static byte
    hexCharsToByte(char c1, char c2)
     
    private static int
    hexCharToNibble(char ch)
     
    static byte[]
    incrementBytes(byte[] value, long amount)
    Bytewise binary increment/deincrement of long contained in byte array on given amount.
    static int
    indexOf(byte[] array, byte target)
    Returns the index of the first appearance of the value target in array.
    static int
    indexOf(byte[] array, byte[] target)
    Returns the start position of the first occurrence of the specified target within array, or -1 if there is no such occurrence.
    private static boolean
    isHexDigit(char c)
     
    static boolean
    isSorted(Collection<byte[]> arrays)
     
    static Iterable<byte[]>
    iterateOnSplits(byte[] a, byte[] b, boolean inclusive, int num)
    Iterate over keys within the passed range.
    static Iterable<byte[]>
    iterateOnSplits(byte[] a, byte[] b, int num)
    Iterate over keys within the passed range, splitting at an [a,b) boundary.
    static final int
    len(byte[] b)
    Returns length of the byte array, returning 0 if the array is null.
    (package private) static Bytes.Comparer<byte[]>
    lexicographicalComparerJavaImpl()
     
    static Integer
    mapKey(byte[] b)
    Calculate a hash code from a given byte array suitable for use as a key in maps.
    static Integer
    mapKey(byte[] b, int length)
    Calculate a hash code from a given byte array suitable for use as a key in maps.
    static byte[]
    multiple(byte[] srcBytes, int multiNum)
    Create a byte array which is multiple given bytes
    static byte[]
    padHead(byte[] a, int length)
    Make a new byte array from a subset of bytes at the head of another, zero padded as desired.
    static byte[]
    padTail(byte[] a, int length)
    Make a new byte array from a subset of bytes at the tail of another, zero padded as desired.
    static int
    putAsShort(byte[] bytes, int offset, int val)
    Put an int value as short out to the specified byte array position.
    static int
    putBigDecimal(byte[] bytes, int offset, BigDecimal val)
    Put a BigDecimal value out to the specified byte array position.
    static int
    putByte(byte[] bytes, int offset, byte b)
    Write a single byte out to the specified byte array position.
    static int
    putByteBuffer(byte[] bytes, int offset, ByteBuffer buf)
    Add the whole content of the ByteBuffer to the bytes arrays.
    static int
    putBytes(byte[] tgtBytes, int tgtOffset, byte[] srcBytes, int srcOffset, int srcLength)
    Put bytes at the specified byte array position.
    static int
    putDouble(byte[] bytes, int offset, double d)
    Put a double value out to the specified byte array position as the IEEE 754 double format.
    static int
    putFloat(byte[] bytes, int offset, float f)
    Put a float value out to the specified byte array position.
    static int
    putInt(byte[] bytes, int offset, int val)
    Put an int value out to the specified byte array position.
    static int
    putLong(byte[] bytes, int offset, long val)
    Put a long value out to the specified byte array position.
    static int
    putShort(byte[] bytes, int offset, short val)
    Put a short value out to the specified byte array position.
    static void
    random(byte[] b)
    Fill given array with random bytes.
    static void
    random(byte[] b, int offset, int length)
    Fill given array with random bytes at the specified position.
    static int
    readAsInt(byte[] bytes, int offset, int length)
    Converts a byte array to an int value
    static long
    readAsVLong(byte[] buffer, int offset)
    Reads a zero-compressed encoded long from input buffer and returns it.
    static byte[]
    readByteArray(DataInput in)
    Read byte-array written with a WritableableUtils.vint prefix.
    static byte[]
    readByteArrayThrowsRuntime(DataInput in)
    Read byte-array written with a WritableableUtils.vint prefix.
    private static byte[]
    readBytes(ByteBuffer buf)
     
    static String
    readStringFixedSize(DataInput in, int size)
    Reads a fixed-size field and interprets it as a string padded with zeros.
    static int
    searchDelimiterIndex(byte[] b, int offset, int length, int delimiter)
    Find index of passed delimiter.
    static int
    searchDelimiterIndexInReverse(byte[] b, int offset, int length, int delimiter)
    Find index of passed delimiter walking from end of buffer backwards.
    static void
    secureRandom(byte[] b)
    Fill given array with random bytes using a strong random number generator.
    static void
    secureRandom(byte[] b, int offset, int length)
    Fill given array with random bytes at the specified position using a strong random number generator.
    void
    set(byte[] b)
    Use passed bytes as backing array for this instance.
    void
    set(byte[] b, int offset, int length)
    Use passed bytes as backing array for this instance.
    static byte[][]
    split(byte[] a, byte[] b, boolean inclusive, int num)
    Split passed range.
    static byte[][]
    split(byte[] a, byte[] b, int num)
    Split passed range.
    static boolean
    startsWith(byte[] bytes, byte[] prefix)
    Return true if the byte array on the right is a prefix of the byte array on the left.
    static byte[]
    tail(byte[] a, int length)
    Make a new byte array from a subset of bytes at the tail of another.
    static byte[][]
    toArray(List<byte[]> array)
    Convert a list of byte[] to an array
    static BigDecimal
    toBigDecimal(byte[] bytes)
    Converts a byte array to a BigDecimal
    static BigDecimal
    toBigDecimal(byte[] bytes, int offset, int length)
    Converts a byte array to a BigDecimal value
    static byte[][]
    toBinaryByteArrays(String[] t)
    Create an array of byte[] given an array of String.
    static byte
    toBinaryFromHex(byte ch)
    Takes a ASCII digit in the range A-F0-9 and returns the corresponding integer/ordinal value.
    static boolean
    toBoolean(byte[] b)
    Reverses toBytes(boolean)
    static byte[][]
    toByteArrays(byte[] column)
    Create a byte[][] where first and only entry is column
    static byte[][]
    toByteArrays(String column)
    Create a byte[][] where first and only entry is column
    static byte[][]
    toByteArrays(String[] t)
    Create an array of byte[] given an array of String.
    static byte[]
    toBytes(boolean b)
    Convert a boolean to a byte array.
    static byte[]
    toBytes(double d)
    Serialize a double as the IEEE 754 double format output.
    static byte[]
    toBytes(float f)
    Return the float represented as byte[]
    static byte[]
    toBytes(int val)
    Convert an int value to a byte array.
    static byte[]
    toBytes(long val)
    Convert a long value to a byte array using big-endian.
    static byte[]
    toBytes(short val)
    Convert a short value to a byte array of SIZEOF_SHORT bytes long.
    static byte[]
    toBytes(String s)
    Converts a string to a UTF-8 byte array.
    static byte[]
    toBytes(BigDecimal val)
    Convert a BigDecimal value to a byte array
    static byte[]
    toBytes(ByteBuffer buf)
    Returns a new byte array, copied from the given buf, from the index 0 (inclusive) to the limit (exclusive), regardless of the current position.
    static byte[]
    toBytesBinary(String in)
     
    static double
    toDouble(byte[] bytes)
    Return double made from passed bytes.
    static double
    toDouble(byte[] bytes, int offset)
    Return double made from passed bytes.
    static float
    toFloat(byte[] bytes)
    Put a float value out to the specified byte array position.
    static float
    toFloat(byte[] bytes, int offset)
    Put a float value out to the specified byte array position.
    static String
    toHex(byte[] b)
    Convert a byte array into a hex string
    static String
    toHex(byte[] b, int offset, int length)
    Convert a byte range into a hex string
    static int
    toInt(byte[] bytes)
    Converts a byte array to an int value
    static int
    toInt(byte[] bytes, int offset)
    Converts a byte array to an int value
    static int
    toInt(byte[] bytes, int offset, int length)
    Converts a byte array to an int value
    static long
    toLong(byte[] bytes)
    Converts a byte array to a long value.
    static long
    toLong(byte[] bytes, int offset)
    Converts a byte array to a long value.
    static long
    toLong(byte[] bytes, int offset, int length)
    Converts a byte array to a long value.
    static short
    toShort(byte[] bytes)
    Converts a byte array to a short value
    static short
    toShort(byte[] bytes, int offset)
    Converts a byte array to a short value
    static short
    toShort(byte[] bytes, int offset, int length)
    Converts a byte array to a short value
    String
    toString()
     
    static String
    toString(byte[] b)
    Convert a byte[] into a string.
    static String
    toString(byte[] b, int off)
    This method will convert utf8 encoded bytes into a string.
    static String
    toString(byte[] b, int off, int len)
    This method will convert utf8 encoded bytes into a string.
    static String
    toString(byte[] b1, String sep, byte[] b2)
    Joins two byte arrays together using a separator.
    static String
    toStringBinary(byte[] b)
    Write a printable representation of a byte array.
    static String
    toStringBinary(byte[] b, int off, int len)
    Write a printable representation of a byte array.
    static String
    toStringBinary(ByteBuffer buf)
    Converts the given byte buffer to a printable representation, from the index 0 (inclusive) to the limit (exclusive), regardless of the current position.
    static int
    unsignedBinarySearch(byte[] a, int fromIndex, int toIndex, byte key)
    Search sorted array "a" for byte "key".
    static byte[]
    unsignedCopyAndIncrement(byte[] input)
    Treat the byte[] as an unsigned series of bytes, most significant bits first.
    static byte[]
    vintToBytes(long vint)
    Encode a long value as a variable length integer.
    static int
    writeByteArray(byte[] tgt, int tgtOffset, byte[] src, int srcOffset, int srcLength)
    Write byte-array from src to tgt with a vint length prefix.
    static void
    writeByteArray(DataOutput out, byte[] b)
    Write byte-array with a WritableableUtils.vint prefix.
    static void
    writeByteArray(DataOutput out, byte[] b, int offset, int length)
    Write byte-array to out with a vint length prefix.
    static void
    writeStringFixedSize(DataOutput out, String s, int size)
    Writes a string as a fixed-size field, padded with zeros.
    static void
    zero(byte[] b)
    Fill given array with zeros.
    static void
    zero(byte[] b, int offset, int length)
    Fill given array with zeros at the specified position.

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

  • Field Details

  • Constructor Details

    • Bytes

      public Bytes()
      Create a zero-size sequence.

    • Bytes

      public Bytes(byte[] bytes)
      Create a Bytes using the byte array as the initial value.
      Parameters:
      bytes - This array becomes the backing storage for the object.

    • Bytes

      public Bytes(Bytes ibw)
      Set the new Bytes to the contents of the passed ibw.
      Parameters:
      ibw - the value to set this Bytes to.

    • Bytes

      public Bytes(byte[] bytes, int offset, int length)
      Set the value to a given byte range
      Parameters:
      bytes - the new byte range to set to
      offset - the offset in newData to start at
      length - the number of bytes in the range

  • Method Details

    • len

      public static final int len(byte[] b)
      Returns length of the byte array, returning 0 if the array is null. Useful for calculating sizes.
      Parameters:
      b - byte array, which can be null
      Returns:
      0 if b is null, otherwise returns length

    • get

      public byte[] get()
      Get the data from the Bytes.
      Returns:
      The data is only valid between offset and offset+length.

    • set

      public void set(byte[] b)
      Use passed bytes as backing array for this instance.

    • set

      public void set(byte[] b, int offset, int length)
      Use passed bytes as backing array for this instance.

    • getLength

      public int getLength()
      Returns the number of valid bytes in the buffer

    • getOffset

      public int getOffset()
      Return the offset into the buffer.

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • compareTo

      public int compareTo(Bytes that)
      Define the sort order of the Bytes.
      Specified by:
      compareTo in interface Comparable<Bytes>
      Parameters:
      that - The other bytes writable
      Returns:
      Positive if left is bigger than right, 0 if they are equal, and negative if left is smaller than right.

    • compareTo

      public int compareTo(byte[] that)
      Compares the bytes in this object to the specified byte array
      Returns:
      Positive if left is bigger than right, 0 if they are equal, and negative if left is smaller than right.

    • equals

      public boolean equals(Object right_obj)
      Overrides:
      equals in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • toArray

      public static byte[][] toArray(List<byte[]> array)
      Convert a list of byte[] to an array
      Parameters:
      array - List of byte [].
      Returns:
      Array of byte [].

    • copyBytes

      public byte[] copyBytes()
      Returns a copy of the bytes referred to by this writable

    • readByteArray

      public static byte[] readByteArray(DataInput in) throws IOException
      Read byte-array written with a WritableableUtils.vint prefix.
      Parameters:
      in - Input to read from.
      Returns:
      byte array read off in
      Throws:
      IOException - e

    • readByteArrayThrowsRuntime

      public static byte[] readByteArrayThrowsRuntime(DataInput in)
      Read byte-array written with a WritableableUtils.vint prefix. IOException is converted to a RuntimeException.
      Parameters:
      in - Input to read from.
      Returns:
      byte array read off in

    • writeByteArray

      public static void writeByteArray(DataOutput out, byte[] b) throws IOException
      Write byte-array with a WritableableUtils.vint prefix.
      Parameters:
      out - output stream to be written to
      b - array to write
      Throws:
      IOException - e

    • writeByteArray

      public static void writeByteArray(DataOutput out, byte[] b, int offset, int length) throws IOException
      Write byte-array to out with a vint length prefix.
      Parameters:
      out - output stream
      b - array
      offset - offset into array
      length - length past offset
      Throws:
      IOException - e

    • writeByteArray

      public static int writeByteArray(byte[] tgt, int tgtOffset, byte[] src, int srcOffset, int srcLength)
      Write byte-array from src to tgt with a vint length prefix.
      Parameters:
      tgt - target array
      tgtOffset - offset into target array
      src - source array
      srcOffset - source offset
      srcLength - source length
      Returns:
      New offset in src array.

    • putBytes

      public static int putBytes(byte[] tgtBytes, int tgtOffset, byte[] srcBytes, int srcOffset, int srcLength)
      Put bytes at the specified byte array position.
      Parameters:
      tgtBytes - the byte array
      tgtOffset - position in the array
      srcBytes - array to write out
      srcOffset - source offset
      srcLength - source length
      Returns:
      incremented offset

    • putByte

      public static int putByte(byte[] bytes, int offset, byte b)
      Write a single byte out to the specified byte array position.
      Parameters:
      bytes - the byte array
      offset - position in the array
      b - byte to write out
      Returns:
      incremented offset

    • putByteBuffer

      public static int putByteBuffer(byte[] bytes, int offset, ByteBuffer buf)
      Add the whole content of the ByteBuffer to the bytes arrays. The ByteBuffer is modified.
      Parameters:
      bytes - the byte array
      offset - position in the array
      buf - ByteBuffer to write out
      Returns:
      incremented offset

    • toBytes

      public static byte[] toBytes(ByteBuffer buf)
      Returns a new byte array, copied from the given buf, from the index 0 (inclusive) to the limit (exclusive), regardless of the current position. The position and the other index parameters are not changed.
      Parameters:
      buf - a byte buffer
      Returns:
      the byte array
      See Also:

    • readBytes

      private static byte[] readBytes(ByteBuffer buf)

    • toString

      public static String toString(byte[] b)
      Convert a byte[] into a string. Charset is assumed to be UTF-8.
      Parameters:
      b - Presumed UTF-8 encoded byte array.
      Returns:
      String made from b

    • toString

      public static String toString(byte[] b1, String sep, byte[] b2)
      Joins two byte arrays together using a separator.
      Parameters:
      b1 - The first byte array.
      sep - The separator to use.
      b2 - The second byte array.

    • toString

      public static String toString(byte[] b, int off)
      This method will convert utf8 encoded bytes into a string. If the given byte array is null, this method will return null.
      Parameters:
      b - Presumed UTF-8 encoded byte array.
      off - offset into array
      Returns:
      String made from b or null

    • toString

      public static String toString(byte[] b, int off, int len)
      This method will convert utf8 encoded bytes into a string. If the given byte array is null, this method will return null.
      Parameters:
      b - Presumed UTF-8 encoded byte array.
      off - offset into array
      len - length of utf-8 sequence
      Returns:
      String made from b or null

    • toStringBinary

      public static String toStringBinary(byte[] b)
      Write a printable representation of a byte array.
      Parameters:
      b - byte array
      See Also:

    • toStringBinary

      public static String toStringBinary(ByteBuffer buf)
      Converts the given byte buffer to a printable representation, from the index 0 (inclusive) to the limit (exclusive), regardless of the current position. The position and the other index parameters are not changed.
      Parameters:
      buf - a byte buffer
      Returns:
      a string representation of the buffer's binary contents
      See Also:

    • toStringBinary

      public static String toStringBinary(byte[] b, int off, int len)
      Write a printable representation of a byte array. Non-printable characters are hex escaped in the format \\x%02X, eg: \x00 \x05 etc
      Parameters:
      b - array to write out
      off - offset to start at
      len - length to write
      Returns:
      string output

    • isHexDigit

      private static boolean isHexDigit(char c)

    • toBinaryFromHex

      public static byte toBinaryFromHex(byte ch)
      Takes a ASCII digit in the range A-F0-9 and returns the corresponding integer/ordinal value.
      Parameters:
      ch - The hex digit.
      Returns:
      The converted hex value as a byte.

    • toBytesBinary

      public static byte[] toBytesBinary(String in)

    • toBytes

      public static byte[] toBytes(String s)
      Converts a string to a UTF-8 byte array.
      Parameters:
      s - string
      Returns:
      the byte array

    • toBytes

      public static byte[] toBytes(boolean b)
      Convert a boolean to a byte array. True becomes -1 and false becomes 0.
      Parameters:
      b - value
      Returns:
      b encoded in a byte array.

    • toBoolean

      public static boolean toBoolean(byte[] b)
      Reverses toBytes(boolean)
      Parameters:
      b - array
      Returns:
      True or false.

    • toBytes

      public static byte[] toBytes(long val)
      Convert a long value to a byte array using big-endian.
      Parameters:
      val - value to convert
      Returns:
      the byte array

    • toLong

      public static long toLong(byte[] bytes)
      Converts a byte array to a long value. Reverses toBytes(long)
      Parameters:
      bytes - array
      Returns:
      the long value

    • toLong

      public static long toLong(byte[] bytes, int offset)
      Converts a byte array to a long value. Assumes there will be SIZEOF_LONG bytes available.
      Parameters:
      bytes - bytes
      offset - offset
      Returns:
      the long value

    • toLong

      public static long toLong(byte[] bytes, int offset, int length)
      Converts a byte array to a long value.
      Parameters:
      bytes - array of bytes
      offset - offset into array
      length - length of data (must be SIZEOF_LONG)
      Returns:
      the long value
      Throws:
      IllegalArgumentException - if length is not SIZEOF_LONG or if there's not enough room in the array at the offset indicated.

    • explainWrongLengthOrOffset

      private static IllegalArgumentException explainWrongLengthOrOffset(byte[] bytes, int offset, int length, int expectedLength)

    • putLong

      public static int putLong(byte[] bytes, int offset, long val)
      Put a long value out to the specified byte array position.
      Parameters:
      bytes - the byte array
      offset - position in the array
      val - long to write out
      Returns:
      incremented offset
      Throws:
      IllegalArgumentException - if the byte array given doesn't have enough room at the offset specified.

    • toFloat

      public static float toFloat(byte[] bytes)
      Put a float value out to the specified byte array position. Presumes float encoded as IEEE 754 floating-point "single format"
      Parameters:
      bytes - byte array
      Returns:
      Float made from passed byte array.

    • toFloat

      public static float toFloat(byte[] bytes, int offset)
      Put a float value out to the specified byte array position. Presumes float encoded as IEEE 754 floating-point "single format"
      Parameters:
      bytes - array to convert
      offset - offset into array
      Returns:
      Float made from passed byte array.

    • putFloat

      public static int putFloat(byte[] bytes, int offset, float f)
      Put a float value out to the specified byte array position.
      Parameters:
      bytes - byte array
      offset - offset to write to
      f - float value
      Returns:
      New offset in bytes

    • toBytes

      public static byte[] toBytes(float f)
      Return the float represented as byte[]

    • toDouble

      public static double toDouble(byte[] bytes)
      Return double made from passed bytes.

    • toDouble

      public static double toDouble(byte[] bytes, int offset)
      Return double made from passed bytes.

    • putDouble

      public static int putDouble(byte[] bytes, int offset, double d)
      Put a double value out to the specified byte array position as the IEEE 754 double format.
      Parameters:
      bytes - byte array
      offset - offset to write to
      d - value
      Returns:
      New offset into array bytes

    • toBytes

      public static byte[] toBytes(double d)
      Serialize a double as the IEEE 754 double format output. The resultant array will be 8 bytes long.
      Parameters:
      d - value
      Returns:
      the double represented as byte []

    • toBytes

      public static byte[] toBytes(int val)
      Convert an int value to a byte array. Big-endian. Same as what DataOutputStream.writeInt does.
      Parameters:
      val - value
      Returns:
      the byte array

    • toInt

      public static int toInt(byte[] bytes)
      Converts a byte array to an int value
      Parameters:
      bytes - byte array
      Returns:
      the int value

    • toInt

      public static int toInt(byte[] bytes, int offset)
      Converts a byte array to an int value
      Parameters:
      bytes - byte array
      offset - offset into array
      Returns:
      the int value

    • toInt

      public static int toInt(byte[] bytes, int offset, int length)
      Converts a byte array to an int value
      Parameters:
      bytes - byte array
      offset - offset into array
      length - length of int (has to be SIZEOF_INT)
      Returns:
      the int value
      Throws:
      IllegalArgumentException - if length is not SIZEOF_INT or if there's not enough room in the array at the offset indicated.

    • readAsInt

      public static int readAsInt(byte[] bytes, int offset, int length)
      Converts a byte array to an int value
      Parameters:
      bytes - byte array
      offset - offset into array
      length - how many bytes should be considered for creating int
      Returns:
      the int value
      Throws:
      IllegalArgumentException - if there's not enough room in the array at the offset indicated.

    • putInt

      public static int putInt(byte[] bytes, int offset, int val)
      Put an int value out to the specified byte array position.
      Parameters:
      bytes - the byte array
      offset - position in the array
      val - int to write out
      Returns:
      incremented offset
      Throws:
      IllegalArgumentException - if the byte array given doesn't have enough room at the offset specified.

    • toBytes

      public static byte[] toBytes(short val)
      Convert a short value to a byte array of SIZEOF_SHORT bytes long.
      Parameters:
      val - value
      Returns:
      the byte array

    • toShort

      public static short toShort(byte[] bytes)
      Converts a byte array to a short value
      Parameters:
      bytes - byte array
      Returns:
      the short value

    • toShort

      public static short toShort(byte[] bytes, int offset)
      Converts a byte array to a short value
      Parameters:
      bytes - byte array
      offset - offset into array
      Returns:
      the short value

    • toShort

      public static short toShort(byte[] bytes, int offset, int length)
      Converts a byte array to a short value
      Parameters:
      bytes - byte array
      offset - offset into array
      length - length, has to be SIZEOF_SHORT
      Returns:
      the short value
      Throws:
      IllegalArgumentException - if length is not SIZEOF_SHORT or if there's not enough room in the array at the offset indicated.

    • getBytes

      public static byte[] getBytes(ByteBuffer buf)
      Returns a new byte array, copied from the given buf, from the position (inclusive) to the limit (exclusive). The position and the other index parameters are not changed.
      Parameters:
      buf - a byte buffer
      Returns:
      the byte array
      See Also:

    • putShort

      public static int putShort(byte[] bytes, int offset, short val)
      Put a short value out to the specified byte array position.
      Parameters:
      bytes - the byte array
      offset - position in the array
      val - short to write out
      Returns:
      incremented offset
      Throws:
      IllegalArgumentException - if the byte array given doesn't have enough room at the offset specified.

    • putAsShort

      public static int putAsShort(byte[] bytes, int offset, int val)
      Put an int value as short out to the specified byte array position. Only the lower 2 bytes of the short will be put into the array. The caller of the API need to make sure they will not loose the value by doing so. This is useful to store an unsigned short which is represented as int in other parts.
      Parameters:
      bytes - the byte array
      offset - position in the array
      val - value to write out
      Returns:
      incremented offset
      Throws:
      IllegalArgumentException - if the byte array given doesn't have enough room at the offset specified.

    • toBytes

      public static byte[] toBytes(BigDecimal val)
      Convert a BigDecimal value to a byte array

    • toBigDecimal

      public static BigDecimal toBigDecimal(byte[] bytes)
      Converts a byte array to a BigDecimal

    • toBigDecimal

      public static BigDecimal toBigDecimal(byte[] bytes, int offset, int length)
      Converts a byte array to a BigDecimal value

    • putBigDecimal

      public static int putBigDecimal(byte[] bytes, int offset, BigDecimal val)
      Put a BigDecimal value out to the specified byte array position.
      Parameters:
      bytes - the byte array
      offset - position in the array
      val - BigDecimal to write out
      Returns:
      incremented offset

    • vintToBytes

      public static byte[] vintToBytes(long vint)
      Encode a long value as a variable length integer.
      Parameters:
      vint - Integer to make a vint of.
      Returns:
      Vint as bytes array.

    • bytesToVint

      public static long bytesToVint(byte[] buffer)
      Reads a zero-compressed encoded long from input buffer and returns it.
      Parameters:
      buffer - buffer to convert
      Returns:
      vint bytes as an integer.

    • readAsVLong

      public static long readAsVLong(byte[] buffer, int offset)
      Reads a zero-compressed encoded long from input buffer and returns it.
      Parameters:
      buffer - Binary array
      offset - Offset into array at which vint begins.
      Returns:
      deserialized long from buffer.

    • compareTo

      public static int compareTo(byte[] left, byte[] right)
      Lexicographically compare two arrays.
      Parameters:
      left - left operand
      right - right operand
      Returns:
      0 if equal, < 0 if left is less than right, etc.

    • compareTo

      public static int compareTo(byte[] buffer1, int offset1, int length1, byte[] buffer2, int offset2, int length2)
      Lexicographically compare two arrays.
      Parameters:
      buffer1 - left operand
      buffer2 - right operand
      offset1 - Where to start comparing in the left buffer
      offset2 - Where to start comparing in the right buffer
      length1 - How much to compare from the left buffer
      length2 - How much to compare from the right buffer
      Returns:
      0 if equal, < 0 if left is less than right, etc.

    • lexicographicalComparerJavaImpl

      static Bytes.Comparer<byte[]> lexicographicalComparerJavaImpl()

    • equals

      public static boolean equals(byte[] left, byte[] right)
      Lexicographically determine the equality of two arrays.
      Parameters:
      left - left operand
      right - right operand
      Returns:
      True if equal

    • equals

      public static boolean equals(byte[] left, int leftOffset, int leftLen, byte[] right, int rightOffset, int rightLen)
      Lexicographically determine the equality of two arrays.
      Parameters:
      left - left operand
      leftOffset - offset into left operand
      leftLen - length of left operand
      right - right operand
      rightOffset - offset into right operand
      rightLen - length of right operand
      Returns:
      True if equal

    • equals

      public static boolean equals(byte[] a, ByteBuffer buf)
      Lexicographically determine the equality of two byte[], one as ByteBuffer.
      Parameters:
      a - left operand
      buf - right operand
      Returns:
      True if equal

    • startsWith

      public static boolean startsWith(byte[] bytes, byte[] prefix)
      Return true if the byte array on the right is a prefix of the byte array on the left.

    • hashCode

      public static int hashCode(byte[] b)
      Calculate a hash code from a given byte array.
      Parameters:
      b - bytes to hash
      Returns:
      Runs WritableComparator.hashBytes(byte[], int) on the passed in array. This method is what Text use calculating hash code.

    • hashCode

      public static int hashCode(byte[] b, int length)
      Calculate a hash code from a given byte array.
      Parameters:
      b - value
      length - length of the value
      Returns:
      Runs WritableComparator.hashBytes(byte[], int) on the passed in array. This method is what Text use calculating hash code.

    • mapKey

      public static Integer mapKey(byte[] b)
      Calculate a hash code from a given byte array suitable for use as a key in maps.
      Parameters:
      b - bytes to hash
      Returns:
      A hash of b as an Integer that can be used as key in Maps.

    • mapKey

      public static Integer mapKey(byte[] b, int length)
      Calculate a hash code from a given byte array suitable for use as a key in maps.
      Parameters:
      b - bytes to hash
      length - length to hash
      Returns:
      A hash of b as an Integer that can be used as key in Maps.

    • add

      public static byte[] add(byte[] a, byte[] b)
      Concatenate byte arrays.
      Parameters:
      a - lower half
      b - upper half
      Returns:
      New array that has a in lower half and b in upper half.

    • add

      public static byte[] add(byte[] a, byte[] b, byte[] c)
      Concatenate byte arrays.
      Parameters:
      a - first third
      b - second third
      c - third third
      Returns:
      New array made from a, b and c

    • add

      public static byte[] add(byte[][] arrays)
      Concatenate byte arrays.
      Parameters:
      arrays - all the arrays to concatenate together.
      Returns:
      New array made from the concatenation of the given arrays.

    • head

      public static byte[] head(byte[] a, int length)
      Make a new byte array from a subset of bytes at the head of another.
      Parameters:
      a - array
      length - amount of bytes to grab
      Returns:
      First length bytes from a

    • tail

      public static byte[] tail(byte[] a, int length)
      Make a new byte array from a subset of bytes at the tail of another.
      Parameters:
      a - array
      length - amount of bytes to snarf
      Returns:
      Last length bytes from a

    • padHead

      public static byte[] padHead(byte[] a, int length)
      Make a new byte array from a subset of bytes at the head of another, zero padded as desired.
      Parameters:
      a - array
      length - new array size
      Returns:
      Value in a plus length prepended 0 bytes

    • padTail

      public static byte[] padTail(byte[] a, int length)
      Make a new byte array from a subset of bytes at the tail of another, zero padded as desired.
      Parameters:
      a - array
      length - new array size
      Returns:
      Value in a plus length appended 0 bytes

    • split

      public static byte[][] split(byte[] a, byte[] b, int num)
      Split passed range. Expensive operation relatively. Uses BigInteger math. Useful splitting ranges for MapReduce jobs.
      Parameters:
      a - Beginning of range
      b - End of range
      num - Number of times to split range. Pass 1 if you want to split the range in two; i.e. one split.
      Returns:
      Array of dividing values

    • split

      public static byte[][] split(byte[] a, byte[] b, boolean inclusive, int num)
      Split passed range. Expensive operation relatively. Uses BigInteger math. Useful splitting ranges for MapReduce jobs.
      Parameters:
      a - Beginning of range
      b - End of range
      inclusive - Whether the end of range is prefix-inclusive or is considered an exclusive boundary. Automatic splits are generally exclusive and manual splits with an explicit range utilize an inclusive end of range.
      num - Number of times to split range. Pass 1 if you want to split the range in two; i.e. one split.
      Returns:
      Array of dividing values

    • iterateOnSplits

      public static Iterable<byte[]> iterateOnSplits(byte[] a, byte[] b, int num)
      Iterate over keys within the passed range, splitting at an [a,b) boundary.

    • iterateOnSplits

      public static Iterable<byte[]> iterateOnSplits(byte[] a, byte[] b, boolean inclusive, int num)
      Iterate over keys within the passed range.

    • hashCode

      public static int hashCode(byte[] bytes, int offset, int length)
      Calculate the hash code for a given range of bytes.
      Parameters:
      bytes - array to hash
      offset - offset to start from
      length - length to hash

    • toByteArrays

      public static byte[][] toByteArrays(String[] t)
      Create an array of byte[] given an array of String.
      Parameters:
      t - operands
      Returns:
      Array of byte arrays made from passed array of Text

    • toBinaryByteArrays

      public static byte[][] toBinaryByteArrays(String[] t)
      Create an array of byte[] given an array of String.
      Parameters:
      t - operands
      Returns:
      Array of binary byte arrays made from passed array of binary strings

    • toByteArrays

      public static byte[][] toByteArrays(String column)
      Create a byte[][] where first and only entry is column
      Parameters:
      column - operand
      Returns:
      A byte array of a byte array where first and only entry is column

    • toByteArrays

      public static byte[][] toByteArrays(byte[] column)
      Create a byte[][] where first and only entry is column
      Parameters:
      column - operand
      Returns:
      A byte array of a byte array where first and only entry is column

    • binarySearch

      public static int binarySearch(byte[][] arr, byte[] key, int offset, int length)
      Binary search for keys in indexes using Bytes.BYTES_RAWCOMPARATOR.
      Parameters:
      arr - array of byte arrays to search for
      key - the key you want to find
      offset - the offset in the key you want to find
      length - the length of the key
      Returns:
      zero-based index of the key, if the key is present in the array. Otherwise, a value -(i + 1) such that the key is between arr[i - 1] and arr[i] non-inclusively, where i is in [0, i], if we define arr[-1] = -Inf and arr[N] = Inf for an N-element array. The above means that this function can return 2N + 1 different values ranging from -(N + 1) to N - 1.

    • binarySearch

      public static int binarySearch(Cell[] arr, Cell key, CellComparator comparator)
      Binary search for keys in indexes.
      Parameters:
      arr - array of byte arrays to search for
      key - the key you want to find
      comparator - a comparator to compare.
      Returns:
      zero-based index of the key, if the key is present in the array. Otherwise, a value -(i + 1) such that the key is between arr[i - 1] and arr[i] non-inclusively, where i is in [0, i], if we define arr[-1] = -Inf and arr[N] = Inf for an N-element array. The above means that this function can return 2N + 1 different values ranging from -(N + 1) to N - 1.

    • incrementBytes

      public static byte[] incrementBytes(byte[] value, long amount)
      Bytewise binary increment/deincrement of long contained in byte array on given amount.
      Parameters:
      value - - array of bytes containing long (length <= SIZEOF_LONG)
      amount - value will be incremented on (deincremented if negative)
      Returns:
      array of bytes containing incremented long (length == SIZEOF_LONG)

    • binaryIncrementPos

      private static byte[] binaryIncrementPos(byte[] value, long amount)

    • binaryIncrementNeg

      private static byte[] binaryIncrementNeg(byte[] value, long amount)

    • writeStringFixedSize

      public static void writeStringFixedSize(DataOutput out, String s, int size) throws IOException
      Writes a string as a fixed-size field, padded with zeros.
      Throws:
      IOException

    • readStringFixedSize

      public static String readStringFixedSize(DataInput in, int size) throws IOException
      Reads a fixed-size field and interprets it as a string padded with zeros.
      Throws:
      IOException

    • copy

      public static byte[] copy(byte[] bytes)
      Copy the byte array given in parameter and return an instance of a new byte array with the same length and the same content.
      Parameters:
      bytes - the byte array to duplicate
      Returns:
      a copy of the given byte array

    • copy

      public static byte[] copy(byte[] bytes, int offset, int length)
      Copy the byte array given in parameter and return an instance of a new byte array with the same length and the same content.
      Parameters:
      bytes - the byte array to copy from
      Returns:
      a copy of the given designated byte array

    • unsignedBinarySearch

      public static int unsignedBinarySearch(byte[] a, int fromIndex, int toIndex, byte key)
      Search sorted array "a" for byte "key". I can't remember if I wrote this or copied it from somewhere. (mcorgan)
      Parameters:
      a - Array to search. Entries must be sorted and unique.
      fromIndex - First index inclusive of "a" to include in the search.
      toIndex - Last index exclusive of "a" to include in the search.
      key - The byte to search for.
      Returns:
      The index of key if found. If not found, return -(index + 1), where negative indicates "not found" and the "index + 1" handles the "-0" case.

    • unsignedCopyAndIncrement

      public static byte[] unsignedCopyAndIncrement(byte[] input)
      Treat the byte[] as an unsigned series of bytes, most significant bits first. Start by adding 1 to the rightmost bit/byte and carry over all overflows to the more significant bits/bytes.
      Parameters:
      input - The byte[] to increment.
      Returns:
      The incremented copy of "in". May be same length or 1 byte longer.

    • equals

      public static boolean equals(List<byte[]> a, List<byte[]> b)

    • isSorted

      public static boolean isSorted(Collection<byte[]> arrays)

    • getUtf8ByteArrays

      public static List<byte[]> getUtf8ByteArrays(List<String> strings)

    • indexOf

      public static int indexOf(byte[] array, byte target)
      Returns the index of the first appearance of the value target in array.
      Parameters:
      array - an array of byte values, possibly empty
      target - a primitive byte value
      Returns:
      the least index i for which array[i] == target, or -1 if no such index exists.

    • indexOf

      public static int indexOf(byte[] array, byte[] target)
      Returns the start position of the first occurrence of the specified target within array, or -1 if there is no such occurrence.

      More formally, returns the lowest index i such that java.util.Arrays.copyOfRange(array, i, i + target.length) contains exactly the same elements as target.

      Parameters:
      array - the array to search for the sequence target
      target - the array to search for as a sub-sequence of array

    • contains

      public static boolean contains(byte[] array, byte target)
      Return true if target is present as an element anywhere in the given array.
      Parameters:
      array - an array of byte values, possibly empty
      target - a primitive byte value
      Returns:
      true if target is present as an element anywhere in array.

    • contains

      public static boolean contains(byte[] array, byte[] target)
      Return true if target is present as an element anywhere in the given array.
      Parameters:
      array - an array of byte values, possibly empty
      target - an array of byte
      Returns:
      true if target is present anywhere in array

    • zero

      public static void zero(byte[] b)
      Fill given array with zeros.
      Parameters:
      b - array which needs to be filled with zeros

    • zero

      public static void zero(byte[] b, int offset, int length)
      Fill given array with zeros at the specified position.

    • random

      public static void random(byte[] b)
      Fill given array with random bytes.
      Parameters:
      b - array which needs to be filled with random bytes

      If you want random bytes generated by a strong source of randomness use secureRandom(byte[]).

      b - array which needs to be filled with random bytes

    • random

      public static void random(byte[] b, int offset, int length)
      Fill given array with random bytes at the specified position.

      If you want random bytes generated by a strong source of randomness use secureRandom(byte[], int, int).

      Parameters:
      b - array which needs to be filled with random bytes
      offset - staring offset in array
      length - number of bytes to fill

    • secureRandom

      public static void secureRandom(byte[] b)
      Fill given array with random bytes using a strong random number generator.
      Parameters:
      b - array which needs to be filled with random bytes

    • secureRandom

      public static void secureRandom(byte[] b, int offset, int length)
      Fill given array with random bytes at the specified position using a strong random number generator.
      Parameters:
      b - array which needs to be filled with random bytes
      offset - staring offset in array
      length - number of bytes to fill

    • createMaxByteArray

      public static byte[] createMaxByteArray(int maxByteCount)
      Create a max byte array with the specified max byte count
      Parameters:
      maxByteCount - the length of returned byte array
      Returns:
      the created max byte array

    • multiple

      public static byte[] multiple(byte[] srcBytes, int multiNum)
      Create a byte array which is multiple given bytes
      Returns:
      byte array

    • toHex

      public static String toHex(byte[] b, int offset, int length)
      Convert a byte range into a hex string

    • toHex

      public static String toHex(byte[] b)
      Convert a byte array into a hex string

    • hexCharToNibble

      private static int hexCharToNibble(char ch)

    • hexCharsToByte

      private static byte hexCharsToByte(char c1, char c2)

    • fromHex

      public static byte[] fromHex(String hex)
      Create a byte array from a string of hash digits. The length of the string must be a multiple of 2

    • searchDelimiterIndex

      public static int searchDelimiterIndex(byte[] b, int offset, int length, int delimiter)
      Find index of passed delimiter.
      Returns:
      Index of delimiter having started from start of b moving rightward.

    • searchDelimiterIndexInReverse

      public static int searchDelimiterIndexInReverse(byte[] b, int offset, int length, int delimiter)
      Find index of passed delimiter walking from end of buffer backwards.
      Returns:
      Index of delimiter

    • findCommonPrefix

      public static int findCommonPrefix(byte[] left, byte[] right, int leftLength, int rightLength, int leftOffset, int rightOffset)