org.apache.hadoop.hbase.regionserver

Class StoreFileScanner

java.lang.Object

org.apache.hadoop.hbase.regionserver.StoreFileScanner

All Implemented Interfaces:

Closeable, AutoCloseable, KeyValueScanner, Shipper

@InterfaceAudience.LimitedPrivate(value="Phoenix")
 @InterfaceStability.Evolving
public class StoreFileScanner
extends Object
implements KeyValueScanner

KeyValueScanner adaptor over the Reader. It also provides hooks into bloom filter things.

Field Summary

Fields

Modifier and Type	Field and Description
private boolean	canOptimizeForNonNullColumn
private boolean	closed
private Cell	cur
private boolean	delayedReseek
private Cell	delayedSeekKV
private boolean	enforceMVCC
private boolean	hasMVCCInfo
private HFileScanner	hfs
private StoreFileReader	reader
private long	readPt
private boolean	realSeekDone
private long	scannerOrder
private static LongAdder	seekCount
private boolean	stopSkippingKVsIfNextRow

Fields inherited from interface org.apache.hadoop.hbase.regionserver.KeyValueScanner

NO_NEXT_INDEXED_KEY

Constructor Summary

Constructors

Constructor and Description
StoreFileScanner(StoreFileReader reader, HFileScanner hfs, boolean useMVCC, boolean hasMVCC, long readPt, long scannerOrder, boolean canOptimizeForNonNullColumn) Implements a KeyValueScanner on top of the specified HFileScanner

Method Summary

Modifier and Type	Method and Description
boolean	backwardSeek(Cell key) Seek the scanner at or before the row of specified Cell, it firstly tries to seek the scanner at or after the specified Cell, return if peek KeyValue of scanner has the same row with specified Cell, otherwise seek the scanner at the first Cell of the row which is the previous row of specified KeyValue
void	close() Close the KeyValue scanner.
void	enforceSeek() Does the real seek operation in case it was skipped by seekToRowCol(KeyValue, boolean) (TODO: Whats this?).
(package private) CellComparator	getComparator()
org.apache.hadoop.fs.Path	getFilePath()
Cell	getNextIndexedKey()
(package private) StoreFileReader	getReader()
long	getScannerOrder() Get the order of this KeyValueScanner.
static List<StoreFileScanner>	getScannersForCompaction(Collection<HStoreFile> files, boolean canUseDropBehind, long readPt) Get scanners for compaction.
static List<StoreFileScanner>	getScannersForStoreFiles(Collection<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, boolean useDropBehind, long readPt) Return an array of scanners corresponding to the given set of store files.
static List<StoreFileScanner>	getScannersForStoreFiles(Collection<HStoreFile> files, boolean cacheBlocks, boolean usePread, boolean isCompaction, boolean canUseDrop, ScanQueryMatcher matcher, long readPt) Return an array of scanners corresponding to the given set of store files, And set the ScanQueryMatcher for each store file scanner for further optimization
static List<StoreFileScanner>	getScannersForStoreFiles(Collection<HStoreFile> files, boolean cacheBlocks, boolean usePread, long readPt) Return an array of scanners corresponding to the given set of store files.
(package private) static long	getSeekCount()
(package private) static void	instrument()
boolean	isFileScanner()
(package private) boolean	isPrimaryReplica()
Cell	next() Return the next Cell in this scanner, iterating the scanner
Cell	peek() Look at the next Cell in this scanner, but do not iterate scanner.
boolean	realSeekDone() We optimize our store scanners by checking the most recent store file first, so we sometimes pretend we have done a seek but delay it until the store scanner bubbles up to the top of the key-value heap.
boolean	requestSeek(Cell kv, boolean forward, boolean useBloom) Pretend we have done a seek but don't do it yet, if possible.
boolean	reseek(Cell key) Reseek the scanner at or after the specified KeyValue.
(package private) static boolean	reseekAtOrAfter(HFileScanner s, Cell k)
boolean	seek(Cell key) Seek the scanner at or after the specified KeyValue.
static boolean	seekAtOrAfter(HFileScanner s, Cell k)
boolean	seekToLastRow() Seek the scanner at the first KeyValue of last row
boolean	seekToPreviousRow(Cell originalKey) Seek the scanner at the first Cell of the row which is the previous row of specified key
protected void	setCurrentCell(Cell newVal)
void	shipped() Called after a batch of rows scanned and set to be returned to client.
boolean	shouldUseScanner(Scan scan, HStore store, long oldestUnexpiredTS) Allows to filter out scanners (both StoreFile and memstore) that we don't want to use based on criteria such as Bloom filters and timestamp ranges.
protected boolean	skipKVsNewerThanReadpoint()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

reader

private final StoreFileReader reader

hfs

private final HFileScanner hfs

cur

private Cell cur

closed

private boolean closed

realSeekDone

private boolean realSeekDone

delayedReseek

private boolean delayedReseek

delayedSeekKV

private Cell delayedSeekKV

enforceMVCC

private final boolean enforceMVCC

hasMVCCInfo

private final boolean hasMVCCInfo

stopSkippingKVsIfNextRow

private boolean stopSkippingKVsIfNextRow

seekCount

private static LongAdder seekCount

canOptimizeForNonNullColumn

private final boolean canOptimizeForNonNullColumn

readPt

private final long readPt

scannerOrder

private final long scannerOrder

Constructor Detail

StoreFileScanner

public StoreFileScanner(StoreFileReader reader,
                        HFileScanner hfs,
                        boolean useMVCC,
                        boolean hasMVCC,
                        long readPt,
                        long scannerOrder,
                        boolean canOptimizeForNonNullColumn)

Implements a KeyValueScanner on top of the specified HFileScanner

Parameters:

useMVCC - If true, scanner will filter out updates with MVCC larger than readPt.

readPt - MVCC value to use to filter out the updates newer than this scanner.

hasMVCC - Set to true if underlying store file reader has MVCC info.

scannerOrder - Order of the scanner relative to other scanners. See KeyValueScanner.getScannerOrder().

canOptimizeForNonNullColumn - true if we can make sure there is no null column, otherwise false. This is a hint for optimization.

Method Detail

isPrimaryReplica

boolean isPrimaryReplica()

getScannersForStoreFiles

public static List<StoreFileScanner> getScannersForStoreFiles(Collection<HStoreFile> files,
                                                              boolean cacheBlocks,
                                                              boolean usePread,
                                                              long readPt)
                                                       throws IOException

Return an array of scanners corresponding to the given set of store files.

Throws:

IOException

getScannersForStoreFiles

public static List<StoreFileScanner> getScannersForStoreFiles(Collection<HStoreFile> files,
                                                              boolean cacheBlocks,
                                                              boolean usePread,
                                                              boolean isCompaction,
                                                              boolean useDropBehind,
                                                              long readPt)
                                                       throws IOException

Return an array of scanners corresponding to the given set of store files.

Throws:

IOException

getScannersForStoreFiles

public static List<StoreFileScanner> getScannersForStoreFiles(Collection<HStoreFile> files,
                                                              boolean cacheBlocks,
                                                              boolean usePread,
                                                              boolean isCompaction,
                                                              boolean canUseDrop,
                                                              ScanQueryMatcher matcher,
                                                              long readPt)
                                                       throws IOException

Return an array of scanners corresponding to the given set of store files, And set the ScanQueryMatcher for each store file scanner for further optimization

Throws:

IOException

getScannersForCompaction

public static List<StoreFileScanner> getScannersForCompaction(Collection<HStoreFile> files,
                                                              boolean canUseDropBehind,
                                                              long readPt)
                                                       throws IOException

Get scanners for compaction. We will create a separated reader for each store file to avoid contention with normal read request.

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

peek

public Cell peek()

Description copied from interface: KeyValueScanner

Look at the next Cell in this scanner, but do not iterate scanner. NOTICE: The returned cell has not been passed into ScanQueryMatcher. So it may not be what the user need.

Specified by:

peek in interface KeyValueScanner

Returns:

the next Cell

next

public Cell next()
          throws IOException

Description copied from interface: KeyValueScanner

Return the next Cell in this scanner, iterating the scanner

Specified by:

next in interface KeyValueScanner

Returns:

the next Cell

Throws:

IOException

seek

public boolean seek(Cell key)
             throws IOException

Description copied from interface: KeyValueScanner

Seek the scanner at or after the specified KeyValue.

Specified by:

seek in interface KeyValueScanner

Parameters:

key - seek value

Returns:

true if scanner has values left, false if end of scanner

Throws:

IOException

reseek

public boolean reseek(Cell key)
               throws IOException

Description copied from interface: KeyValueScanner

Reseek the scanner at or after the specified KeyValue. This method is guaranteed to seek at or after the required key only if the key comes after the current position of the scanner. Should not be used to seek to a key which may come before the current position.

Specified by:

reseek in interface KeyValueScanner

Parameters:

key - seek value (should be non-null)

Returns:

true if scanner has values left, false if end of scanner

Throws:

IOException

setCurrentCell

protected void setCurrentCell(Cell newVal)
                       throws IOException

Throws:

IOException

skipKVsNewerThanReadpoint

protected boolean skipKVsNewerThanReadpoint()
                                     throws IOException

Throws:

IOException

close

public void close()

Description copied from interface: KeyValueScanner

Close the KeyValue scanner.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface KeyValueScanner

seekAtOrAfter

public static boolean seekAtOrAfter(HFileScanner s,
                                    Cell k)
                             throws IOException

Parameters:

s -

k -

Returns:

false if not found or if k is after the end.

Throws:

IOException

reseekAtOrAfter

static boolean reseekAtOrAfter(HFileScanner s,
                               Cell k)
                        throws IOException

Throws:

IOException

getScannerOrder

public long getScannerOrder()

Description copied from interface: KeyValueScanner

Get the order of this KeyValueScanner. This is only relevant for StoreFileScanners. This is required for comparing multiple files to find out which one has the latest data. StoreFileScanners are ordered from 0 (oldest) to newest in increasing order.

Specified by:

getScannerOrder in interface KeyValueScanner

See Also:

KeyValueScanner.getScannerOrder()

requestSeek

public boolean requestSeek(Cell kv,
                           boolean forward,
                           boolean useBloom)
                    throws IOException

Pretend we have done a seek but don't do it yet, if possible. The hope is that we find requested columns in more recent files and won't have to seek in older files. Creates a fake key/value with the given row/column and the highest (most recent) possible timestamp we might get from this file. When users of such "lazy scanner" need to know the next KV precisely (e.g. when this scanner is at the top of the heap), they run enforceSeek().

Note that this function does guarantee that the current KV of this scanner will be advanced to at least the given KV. Because of this, it does have to do a real seek in cases when the seek timestamp is older than the highest timestamp of the file, e.g. when we are trying to seek to the next row/column and use OLDEST_TIMESTAMP in the seek key.

Specified by:

requestSeek in interface KeyValueScanner

forward - do a forward-only "reseek" instead of a random-access seek

useBloom - whether to enable multi-column Bloom filter optimization

Throws:

IOException

getReader

StoreFileReader getReader()

getComparator

CellComparator getComparator()

realSeekDone

public boolean realSeekDone()

Description copied from interface: KeyValueScanner

We optimize our store scanners by checking the most recent store file first, so we sometimes pretend we have done a seek but delay it until the store scanner bubbles up to the top of the key-value heap. This method is then used to ensure the top store file scanner has done a seek operation.

Specified by:

realSeekDone in interface KeyValueScanner

enforceSeek

public void enforceSeek()
                 throws IOException

Description copied from interface: KeyValueScanner

Does the real seek operation in case it was skipped by seekToRowCol(KeyValue, boolean) (TODO: Whats this?). Note that this function should be never called on scanners that always do real seek operations (i.e. most of the scanners). The easiest way to achieve this is to call KeyValueScanner.realSeekDone() first.

Specified by:

enforceSeek in interface KeyValueScanner

Throws:

IOException

isFileScanner

public boolean isFileScanner()

Specified by:

isFileScanner in interface KeyValueScanner

Returns:

true if this is a file scanner. Otherwise a memory scanner is assumed.

getFilePath

public org.apache.hadoop.fs.Path getFilePath()

Specified by:

getFilePath in interface KeyValueScanner

Returns:

the file path if this is a file scanner, otherwise null.

See Also:

KeyValueScanner.isFileScanner()

getSeekCount

static final long getSeekCount()

instrument

static final void instrument()

shouldUseScanner

public boolean shouldUseScanner(Scan scan,
                                HStore store,
                                long oldestUnexpiredTS)

Description copied from interface: KeyValueScanner

Allows to filter out scanners (both StoreFile and memstore) that we don't want to use based on criteria such as Bloom filters and timestamp ranges.

Specified by:

shouldUseScanner in interface KeyValueScanner

Parameters:

scan - the scan that we are selecting scanners for

store - the store we are performing the scan on.

oldestUnexpiredTS - the oldest timestamp we are interested in for this query, based on TTL

Returns:

true if the scanner should be included in the query

seekToPreviousRow

public boolean seekToPreviousRow(Cell originalKey)
                          throws IOException

Description copied from interface: KeyValueScanner

Seek the scanner at the first Cell of the row which is the previous row of specified key

Specified by:

seekToPreviousRow in interface KeyValueScanner

Parameters:

originalKey - seek value

Returns:

true if the scanner at the first valid Cell of previous row, false if not existing such Cell

Throws:

IOException

seekToLastRow

public boolean seekToLastRow()
                      throws IOException

Description copied from interface: KeyValueScanner

Seek the scanner at the first KeyValue of last row

Specified by:

seekToLastRow in interface KeyValueScanner

Returns:

true if scanner has values left, false if the underlying data is empty

Throws:

IOException

backwardSeek

public boolean backwardSeek(Cell key)
                     throws IOException

Description copied from interface: KeyValueScanner

Seek the scanner at or before the row of specified Cell, it firstly tries to seek the scanner at or after the specified Cell, return if peek KeyValue of scanner has the same row with specified Cell, otherwise seek the scanner at the first Cell of the row which is the previous row of specified KeyValue

Specified by:

backwardSeek in interface KeyValueScanner

Parameters:

key - seek KeyValue

Returns:

true if the scanner is at the valid KeyValue, false if such KeyValue does not exist

Throws:

IOException

getNextIndexedKey

public Cell getNextIndexedKey()

Specified by:

getNextIndexedKey in interface KeyValueScanner

Returns:

the next key in the index, usually the first key of next block OR a key that falls between last key of current block and first key of next block.. see HFileWriterImpl#getMidpoint, or null if not known.

shipped

public void shipped()
             throws IOException

Description copied from interface: Shipper

Called after a batch of rows scanned and set to be returned to client. Any in between cleanup can be done here.

Specified by:

shipped in interface Shipper

Throws:

IOException