org.apache.hadoop.hbase.filter

Class FuzzyRowFilter

java.lang.Object

org.apache.hadoop.hbase.filter.Filter

org.apache.hadoop.hbase.filter.FilterBase

org.apache.hadoop.hbase.filter.FuzzyRowFilter

@InterfaceAudience.Public
public class FuzzyRowFilter
extends FilterBase

This is optimized version of a standard FuzzyRowFilter Filters data based on fuzzy row key. Performs fast-forwards during scanning. It takes pairs (row key, fuzzy info) to match row keys. Where fuzzy info is a byte array with 0 or 1 as its values:

• 0 - means that this byte in provided row key is fixed, i.e. row key's byte at same position must match
• 1 - means that this byte in provided row key is NOT fixed, i.e. row key's byte at this position can be different from the one in provided row key

Example: Let's assume row key format is userId_actionId_year_month. Length of userId is fixed and is 4, length of actionId is 2 and year and month are 4 and 2 bytes long respectively. Let's assume that we need to fetch all users that performed certain action (encoded as "99") in Jan of any year. Then the pair (row key, fuzzy info) would be the following: row key = "????_99_????_01" (one can use any value instead of "?") fuzzy info = "\x01\x01\x01\x01\x00\x00\x00\x00\x01\x01\x01\x01\x00\x00\x00" I.e. fuzzy info tells the matching mask is "????_99_????_01", where at ? can be any value.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	FuzzyRowFilter.Order Abstracts directional comparisons based on scan direction.
private class	FuzzyRowFilter.RowTracker If we have multiple fuzzy keys, row tracker should improve overall performance.
(package private) static class	FuzzyRowFilter.SatisfiesCode

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.filter.Filter

Filter.ReturnCode

Field Summary

Fields

Modifier and Type	Field and Description
private boolean	done
private List<Pair<byte[],byte[]>>	fuzzyKeysData
private int	lastFoundIndex The index of a last successfully found matching fuzzy string (in fuzzyKeysData).
private FuzzyRowFilter.RowTracker	tracker Row tracker (keeps all next rows after SEEK_NEXT_USING_HINT was returned)
private static boolean	UNSAFE_UNALIGNED

Fields inherited from class org.apache.hadoop.hbase.filter.Filter

reversed

Constructor Summary

Constructors

Constructor and Description
FuzzyRowFilter(List<Pair<byte[],byte[]>> fuzzyKeysData)

Method Summary

Modifier and Type	Method and Description
(package private) boolean	areSerializedFieldsEqual(Filter o) Default implementation so that writers of custom filters aren't forced to implement.
boolean	equals(Object obj)
boolean	filterAllRemaining() Filters that never filter all remaining can inherit this implementation that never stops the filter early.
Filter.ReturnCode	filterCell(Cell c) A way to filter based on the column family, column qualifier and/or the column value.
Filter.ReturnCode	filterKeyValue(Cell c) Deprecated.
Cell	getNextCellHint(Cell currentCell) Filters that are not sure which key must be next seeked to, can inherit this implementation that, by default, returns a null Cell.
(package private) static byte[]	getNextForFuzzyRule(boolean reverse, byte[] row, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta)
(package private) static byte[]	getNextForFuzzyRule(boolean reverse, byte[] row, int offset, int length, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta)
(package private) static byte[]	getNextForFuzzyRule(byte[] row, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta)
int	hashCode()
private boolean	isPreprocessedMask(byte[] mask)
static FuzzyRowFilter	parseFrom(byte[] pbBytes)
private byte[]	preprocessMask(byte[] mask) We need to preprocess mask array, as since we treat 2's as unfixed positions and -1 (0xff) as fixed positions
private void	preprocessSearchKey(Pair<byte[],byte[]> p)
(package private) static FuzzyRowFilter.SatisfiesCode	satisfies(boolean reverse, byte[] row, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta)
(package private) static FuzzyRowFilter.SatisfiesCode	satisfies(boolean reverse, byte[] row, int offset, int length, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta)
(package private) static FuzzyRowFilter.SatisfiesCode	satisfies(byte[] row, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta)
(package private) static FuzzyRowFilter.SatisfiesCode	satisfiesNoUnsafe(boolean reverse, byte[] row, int offset, int length, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta)
byte[]	toByteArray() Return length 0 byte array for Filters that don't require special serialization
String	toString() Return filter's info for debugging and logging purpose.
private static byte[]	trimTrailingZeroes(byte[] result, byte[] fuzzyKeyMeta, int toInc) For forward scanner, next cell hint should not contain any trailing zeroes unless they are part of fuzzyKeyMeta hint = '\x01\x01\x01\x00\x00' will skip valid row '\x01\x01\x01'

Methods inherited from class org.apache.hadoop.hbase.filter.FilterBase

createFilterFromArguments, filterRow, filterRowCells, filterRowKey, filterRowKey, hasFilterRow, isFamilyEssential, reset, transformCell

Methods inherited from class org.apache.hadoop.hbase.filter.Filter

isReversed, setReversed

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

UNSAFE_UNALIGNED

private static final boolean UNSAFE_UNALIGNED

fuzzyKeysData

private List<Pair<byte[],byte[]>> fuzzyKeysData

done

private boolean done

lastFoundIndex

private int lastFoundIndex

The index of a last successfully found matching fuzzy string (in fuzzyKeysData). We will start matching next KV with this one. If they do not match then we will return back to the one-by-one iteration over fuzzyKeysData.

tracker

private FuzzyRowFilter.RowTracker tracker

Row tracker (keeps all next rows after SEEK_NEXT_USING_HINT was returned)

Constructor Detail

FuzzyRowFilter

public FuzzyRowFilter(List<Pair<byte[],byte[]>> fuzzyKeysData)

Method Detail

preprocessSearchKey

private void preprocessSearchKey(Pair<byte[],byte[]> p)

preprocessMask

private byte[] preprocessMask(byte[] mask)

We need to preprocess mask array, as since we treat 2's as unfixed positions and -1 (0xff) as fixed positions

Parameters:

mask -

Returns:

mask array

isPreprocessedMask

private boolean isPreprocessedMask(byte[] mask)

filterKeyValue

@Deprecated
public Filter.ReturnCode filterKeyValue(Cell c)

Deprecated.

Description copied from class: Filter

A way to filter based on the column family, column qualifier and/or the column value. Return code is described below. This allows filters to filter only certain number of columns, then terminate without matching ever column. If filterRowKey returns true, filterKeyValue needs to be consistent with it. filterKeyValue can assume that filterRowKey has already been called for the row. If your filter returns ReturnCode.NEXT_ROW, it should return ReturnCode.NEXT_ROW until Filter.reset() is called just in case the caller calls for the next row. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterKeyValue in class Filter

Parameters:

c - the Cell in question

Returns:

code as described below, Filter.ReturnCode.INCLUDE by default

See Also:

Filter.ReturnCode

filterCell

public Filter.ReturnCode filterCell(Cell c)

Description copied from class: Filter

A way to filter based on the column family, column qualifier and/or the column value. Return code is described below. This allows filters to filter only certain number of columns, then terminate without matching ever column. If filterRowKey returns true, filterCell needs to be consistent with it. filterCell can assume that filterRowKey has already been called for the row. If your filter returns ReturnCode.NEXT_ROW, it should return ReturnCode.NEXT_ROW until Filter.reset() is called just in case the caller calls for the next row. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterCell in class Filter

Parameters:

c - the Cell in question

Returns:

code as described below

See Also:

Filter.ReturnCode

getNextCellHint

public Cell getNextCellHint(Cell currentCell)

Description copied from class: FilterBase

Filters that are not sure which key must be next seeked to, can inherit this implementation that, by default, returns a null Cell. If the filter returns the match code SEEK_NEXT_USING_HINT, then it should also tell which is the next key it must seek to. After receiving the match code SEEK_NEXT_USING_HINT, the QueryMatcher would call this function to find out which key it must next seek to. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

getNextCellHint in class FilterBase

Returns:

KeyValue which must be next seeked. return null if the filter is not sure which key to seek to next.

filterAllRemaining

public boolean filterAllRemaining()

Description copied from class: FilterBase

Filters that never filter all remaining can inherit this implementation that never stops the filter early. If this returns true, the scan will terminate. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterAllRemaining in class FilterBase

Returns:

true to end scan, false to continue.

toByteArray

public byte[] toByteArray()

Description copied from class: FilterBase

Return length 0 byte array for Filters that don't require special serialization

Overrides:

toByteArray in class FilterBase

Returns:

The filter serialized using pb

parseFrom

public static FuzzyRowFilter parseFrom(byte[] pbBytes)
                                throws DeserializationException

Parameters:

pbBytes - A pb serialized FuzzyRowFilter instance

Returns:

An instance of FuzzyRowFilter made from bytes

Throws:

DeserializationException

See Also:

toByteArray()

toString

public String toString()

Description copied from class: FilterBase

Return filter's info for debugging and logging purpose.

Overrides:

toString in class FilterBase

satisfies

static FuzzyRowFilter.SatisfiesCode satisfies(byte[] row,
                                              byte[] fuzzyKeyBytes,
                                              byte[] fuzzyKeyMeta)

satisfies

static FuzzyRowFilter.SatisfiesCode satisfies(boolean reverse,
                                              byte[] row,
                                              byte[] fuzzyKeyBytes,
                                              byte[] fuzzyKeyMeta)

satisfies

static FuzzyRowFilter.SatisfiesCode satisfies(boolean reverse,
                                              byte[] row,
                                              int offset,
                                              int length,
                                              byte[] fuzzyKeyBytes,
                                              byte[] fuzzyKeyMeta)

satisfiesNoUnsafe

static FuzzyRowFilter.SatisfiesCode satisfiesNoUnsafe(boolean reverse,
                                                      byte[] row,
                                                      int offset,
                                                      int length,
                                                      byte[] fuzzyKeyBytes,
                                                      byte[] fuzzyKeyMeta)

getNextForFuzzyRule

static byte[] getNextForFuzzyRule(byte[] row,
                                  byte[] fuzzyKeyBytes,
                                  byte[] fuzzyKeyMeta)

getNextForFuzzyRule

static byte[] getNextForFuzzyRule(boolean reverse,
                                  byte[] row,
                                  byte[] fuzzyKeyBytes,
                                  byte[] fuzzyKeyMeta)

getNextForFuzzyRule

static byte[] getNextForFuzzyRule(boolean reverse,
                                  byte[] row,
                                  int offset,
                                  int length,
                                  byte[] fuzzyKeyBytes,
                                  byte[] fuzzyKeyMeta)

Returns:

greater byte array than given (row) which satisfies the fuzzy rule if it exists, null otherwise

trimTrailingZeroes

private static byte[] trimTrailingZeroes(byte[] result,
                                         byte[] fuzzyKeyMeta,
                                         int toInc)

For forward scanner, next cell hint should not contain any trailing zeroes unless they are part of fuzzyKeyMeta hint = '\x01\x01\x01\x00\x00' will skip valid row '\x01\x01\x01'

Parameters:

result -

fuzzyKeyMeta -

toInc - - position of incremented byte

Returns:

trimmed version of result

areSerializedFieldsEqual

boolean areSerializedFieldsEqual(Filter o)

Description copied from class: FilterBase

Default implementation so that writers of custom filters aren't forced to implement.

Overrides:

areSerializedFieldsEqual in class FilterBase

Returns:

true if and only if the fields of the filter that are serialized are equal to the corresponding fields in other. Used for testing.

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object