org.apache.hadoop.hbase.filter

Class FilterListWithOR

java.lang.Object

org.apache.hadoop.hbase.filter.Filter

org.apache.hadoop.hbase.filter.FilterBase

org.apache.hadoop.hbase.filter.FilterListBase

org.apache.hadoop.hbase.filter.FilterListWithOR

@InterfaceAudience.Private
public class FilterListWithOR
extends FilterListBase

FilterListWithOR represents an ordered list of filters which will be evaluated with an OR operator.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.filter.Filter

Filter.ReturnCode

Field Summary

Fields

Modifier and Type	Field and Description
private List<Cell>	prevCellList
private List<Filter.ReturnCode>	prevFilterRCList Save previous return code and previous cell for every filter in filter list.

Fields inherited from class org.apache.hadoop.hbase.filter.FilterListBase

filters, subFiltersIncludedCell

Fields inherited from class org.apache.hadoop.hbase.filter.Filter

reversed

Constructor Summary

Constructors

Constructor and Description
FilterListWithOR(List<Filter> filters)

Method Summary

Modifier and Type	Method and Description
void	addFilterLists(List<Filter> filters)
private Filter.ReturnCode	calculateReturnCodeByPrevCellAndRC(Filter subFilter, Cell currentCell, Cell prevCell, Filter.ReturnCode prevCode) For MUST_PASS_ONE, we cannot make sure that when filter-A in filter list return NEXT_COL then the next cell passing to filterList will be the first cell in next column, because if filter-B in filter list return SKIP, then the filter list will return SKIP.
boolean	equals(Object obj)
boolean	filterAllRemaining() Filters that never filter all remaining can inherit this implementation that never stops the filter early.
Filter.ReturnCode	filterCell(Cell c) A way to filter based on the column family, column qualifier and/or the column value.
boolean	filterRow() Filters that never filter by rows based on previously gathered state from Filter.filterCell(Cell) can inherit this implementation that never filters a row.
boolean	filterRowKey(byte[] rowKey, int offset, int length) Filters that do not filter by row key can inherit this implementation that never filters anything.
boolean	filterRowKey(Cell firstRowCell) Filters a row based on the row key.
protected String	formatLogFilters(List<Filter> logFilters)
Cell	getNextCellHint(Cell currentCell) Filters that are not sure which key must be next seeked to, can inherit this implementation that, by default, returns a null Cell.
int	hashCode()
private Filter.ReturnCode	mergeReturnCode(Filter.ReturnCode rc, Filter.ReturnCode localRC) FilterList with MUST_PASS_ONE choose the minimal forward step among sub-filter in filter list.
void	reset() Filters that are purely stateless and do nothing in their reset() methods can inherit this null/empty implementation.
private void	updatePrevCellList(int index, Cell currentCell, Filter.ReturnCode currentRC)
private void	updatePrevFilterRCList(int index, Filter.ReturnCode currentRC)

Methods inherited from class org.apache.hadoop.hbase.filter.FilterListBase

checkAndGetReversed, compareCell, filterKeyValue, filterRowCells, getFilters, hasFilterRow, isEmpty, isFamilyEssential, isInReturnCodes, setReversed, size, toString, transformCell

Methods inherited from class org.apache.hadoop.hbase.filter.FilterBase

areSerializedFieldsEqual, createFilterFromArguments, toByteArray

Methods inherited from class org.apache.hadoop.hbase.filter.Filter

isReversed, parseFrom

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

prevFilterRCList

private List<Filter.ReturnCode> prevFilterRCList

Save previous return code and previous cell for every filter in filter list. For MUST_PASS_ONE, we use the previous return code to decide whether we should pass current cell encountered to the filter. For MUST_PASS_ALL, the two list are meaningless.

prevCellList

private List<Cell> prevCellList

Constructor Detail

FilterListWithOR

public FilterListWithOR(List<Filter> filters)

Method Detail

addFilterLists

public void addFilterLists(List<Filter> filters)

Specified by:

addFilterLists in class FilterListBase

formatLogFilters

protected String formatLogFilters(List<Filter> logFilters)

Specified by:

formatLogFilters in class FilterListBase

calculateReturnCodeByPrevCellAndRC

private Filter.ReturnCode calculateReturnCodeByPrevCellAndRC(Filter subFilter,
                                                             Cell currentCell,
                                                             Cell prevCell,
                                                             Filter.ReturnCode prevCode)
                                                      throws IOException

For MUST_PASS_ONE, we cannot make sure that when filter-A in filter list return NEXT_COL then the next cell passing to filterList will be the first cell in next column, because if filter-B in filter list return SKIP, then the filter list will return SKIP. In this case, we should pass the cell following the previous cell, and it's possible that the next cell has the same column as the previous cell even if filter-A has NEXT_COL returned for the previous cell. So we should save the previous cell and the return code list when checking previous cell for every filter in filter list, and verify if currentCell fit the previous return code, if fit then pass the currentCell to the corresponding filter. (HBASE-17678)
Note that: In StoreScanner level, NEXT_ROW will skip to the next row in current family, and in RegionScanner level, NEXT_ROW will skip to the next row in current family and switch to the next family for RegionScanner, INCLUDE_AND_NEXT_ROW is the same. so we should pass current cell to the filter, if row mismatch or row match but column family mismatch. (HBASE-18368)

Parameters:

subFilter - which sub-filter to calculate the return code by using previous cell and previous return code.

prevCell - the previous cell passed to given sub-filter.

currentCell - the current cell which will pass to given sub-filter.

prevCode - the previous return code for given sub-filter.

Returns:

return code calculated by using previous cell and previous return code. null means can not decide which return code should return, so we will pass the currentCell to subFilter for getting currentCell's return code, and it won't impact the sub-filter's internal states.

Throws:

IOException

See Also:

Filter.ReturnCode

mergeReturnCode

private Filter.ReturnCode mergeReturnCode(Filter.ReturnCode rc,
                                          Filter.ReturnCode localRC)

FilterList with MUST_PASS_ONE choose the minimal forward step among sub-filter in filter list. Let's call it: The Minimal Step Rule. So if filter-A in filter list return INCLUDE and filter-B in filter list return INCLUDE_AND_NEXT_COL, then the filter list should return INCLUDE. For SEEK_NEXT_USING_HINT, it's more special, because we do not know how far it will forward, so we use SKIP by default.

The jump step will be:

 INCLUDE < SKIP < INCLUDE_AND_NEXT_COL < NEXT_COL < INCLUDE_AND_SEEK_NEXT_ROW < NEXT_ROW < SEEK_NEXT_USING_HINT
 

Here, we have the following map to describe The Minimal Step Rule. if current return code (for previous sub-filters in filter list) is ReturnCode, and current filter returns localRC, then we should return map[ReturnCode][localRC] for the merged result, according to The Minimal Step Rule.

 LocalCode\ReturnCode       INCLUDE INCLUDE_AND_NEXT_COL     INCLUDE_AND_SEEK_NEXT_ROW  SKIP      NEXT_COL              NEXT_ROW                  SEEK_NEXT_USING_HINT
 INCLUDE                    INCLUDE INCLUDE                  INCLUDE                    INCLUDE   INCLUDE               INCLUDE                   INCLUDE
 INCLUDE_AND_NEXT_COL       INCLUDE INCLUDE_AND_NEXT_COL     INCLUDE_AND_NEXT_COL       INCLUDE   INCLUDE_AND_NEXT_COL  INCLUDE_AND_NEXT_COL      INCLUDE
 INCLUDE_AND_SEEK_NEXT_ROW  INCLUDE INCLUDE_AND_NEXT_COL     INCLUDE_AND_SEEK_NEXT_ROW  INCLUDE   INCLUDE_AND_NEXT_COL  INCLUDE_AND_SEEK_NEXT_ROW INCLUDE
 SKIP                       INCLUDE INCLUDE                  INCLUDE                    SKIP      SKIP                  SKIP                      SKIP
 NEXT_COL                   INCLUDE INCLUDE_AND_NEXT_COL     INCLUDE_AND_NEXT_COL       SKIP      NEXT_COL              NEXT_COL                  SKIP
 NEXT_ROW                   INCLUDE INCLUDE_AND_NEXT_COL     INCLUDE_AND_SEEK_NEXT_ROW  SKIP      NEXT_COL              NEXT_ROW                  SKIP
 SEEK_NEXT_USING_HINT       INCLUDE INCLUDE                  INCLUDE                    SKIP      SKIP                  SKIP                      SEEK_NEXT_USING_HINT
 

Parameters:

rc - Return code which is calculated by previous sub-filter(s) in filter list.

localRC - Return code of the current sub-filter in filter list.

Returns:

Return code which is merged by the return code of previous sub-filter(s) and the return code of current sub-filter.

updatePrevFilterRCList

private void updatePrevFilterRCList(int index,
                                    Filter.ReturnCode currentRC)

updatePrevCellList

private void updatePrevCellList(int index,
                                Cell currentCell,
                                Filter.ReturnCode currentRC)

filterCell

public Filter.ReturnCode filterCell(Cell c)
                             throws IOException

Description copied from class: Filter

A way to filter based on the column family, column qualifier and/or the column value. Return code is described below. This allows filters to filter only certain number of columns, then terminate without matching ever column. If filterRowKey returns true, filterCell needs to be consistent with it. filterCell can assume that filterRowKey has already been called for the row. If your filter returns ReturnCode.NEXT_ROW, it should return ReturnCode.NEXT_ROW until Filter.reset() is called just in case the caller calls for the next row. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterCell in class Filter

Parameters:

c - the Cell in question

Returns:

code as described below

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

See Also:

Filter.ReturnCode

reset

public void reset()
           throws IOException

Description copied from class: FilterBase

Filters that are purely stateless and do nothing in their reset() methods can inherit this null/empty implementation. Reset the state of the filter between rows. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

reset in class FilterBase

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

filterRowKey

public boolean filterRowKey(byte[] rowKey,
                            int offset,
                            int length)
                     throws IOException

Description copied from class: FilterBase

Filters that do not filter by row key can inherit this implementation that never filters anything. (ie: returns false). Filters a row based on the row key. If this returns true, the entire row will be excluded. If false, each KeyValue in the row will be passed to Filter.filterCell(Cell) below. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterRowKey in class FilterBase

Parameters:

rowKey - buffer containing row key

offset - offset into buffer where row key starts

length - length of the row key

Returns:

true, remove entire row, false, include the row (maybe).

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

filterRowKey

public boolean filterRowKey(Cell firstRowCell)
                     throws IOException

Description copied from class: Filter

Filters a row based on the row key. If this returns true, the entire row will be excluded. If false, each KeyValue in the row will be passed to Filter.filterCell(Cell) below. If Filter.filterAllRemaining() returns true, then Filter.filterRowKey(Cell) should also return true. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterRowKey in class FilterBase

Parameters:

firstRowCell - The first cell coming in the new row

Returns:

true, remove entire row, false, include the row (maybe).

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

filterAllRemaining

public boolean filterAllRemaining()
                           throws IOException

Description copied from class: FilterBase

Filters that never filter all remaining can inherit this implementation that never stops the filter early. If this returns true, the scan will terminate. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterAllRemaining in class FilterBase

Returns:

true to end scan, false to continue.

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

filterRow

public boolean filterRow()
                  throws IOException

Description copied from class: FilterBase

Filters that never filter by rows based on previously gathered state from Filter.filterCell(Cell) can inherit this implementation that never filters a row. Last chance to veto row based on previous Filter.filterCell(Cell) calls. The filter needs to retain state then return a particular value for this call if they wish to exclude a row if a certain column is missing (for example). Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterRow in class FilterBase

Returns:

true to exclude row, false to include row.

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

getNextCellHint

public Cell getNextCellHint(Cell currentCell)
                     throws IOException

Description copied from class: FilterBase

Filters that are not sure which key must be next seeked to, can inherit this implementation that, by default, returns a null Cell. If the filter returns the match code SEEK_NEXT_USING_HINT, then it should also tell which is the next key it must seek to. After receiving the match code SEEK_NEXT_USING_HINT, the QueryMatcher would call this function to find out which key it must next seek to. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

getNextCellHint in class FilterBase

Returns:

KeyValue which must be next seeked. return null if the filter is not sure which key to seek to next.

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object