Package org.apache.hadoop.hbase.io.hfile
Class HFileReaderImpl.HFileScannerImpl
java.lang.Object
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.HFileScannerImpl
- All Implemented Interfaces:
Closeable
,AutoCloseable
,HFileScanner
,Shipper
- Direct Known Subclasses:
HFileReaderImpl.EncodedScanner
- Enclosing class:
- HFileReaderImpl
-
Field Summary
Modifier and TypeFieldDescriptionprivate ByteBuff
private ByteBufferKeyOnlyKeyValue
protected final boolean
protected HFileBlock
private int
private long
private int
private int
private int
protected final boolean
protected ExtendedCell
The next indexed key is to keep track of the indexed key of the next data block.(package private) final ObjectIntPair<ByteBuffer>
protected final boolean
protected final ArrayList<HFileBlock>
private boolean
protected final HFile.Reader
private short
-
Constructor Summary
ConstructorDescriptionHFileScannerImpl
(HFile.Reader reader, boolean cacheBlocks, boolean pread, boolean isCompaction) -
Method Summary
Modifier and TypeMethodDescriptionprivate final boolean
_next()
private void
_readMvccVersion
(int offsetFromPos) Actually do the mvcc read.protected void
protected int
Within a loaded block, seek looking for the last key that is smaller than (or equal to?) the key we are interested in.protected final boolean
checkKeyLen
(int v) Returns True if v <= 0 or v > current block buffer limit.protected final void
Check key and value lengths are wholesome.protected final boolean
checkLen
(int v) Returns True if v < 0 or v > current block buffer limit.private final void
void
close()
Close this HFile scanner and do necessary cleanup.int
compareKey
(CellComparator comparator, ExtendedCell key) getCell()
Returns Instance ofExtendedCell
.private int
protected ExtendedCell
getFirstKeyCellInBlock
(HFileBlock curBlock) getKey()
Gets the current key in the form of a cell.private int
Returns the next key in the index (the key to seek to the next block)Returns Reader that underlies this Scanner instance.getValue()
Gets a buffer view to the current value.private boolean
boolean
isSeeked()
protected int
loadBlockAndSeekToKey
(HFileBlock seekToBlock, ExtendedCell nextIndexedKey, boolean rewind, ExtendedCell key, boolean seekBefore) boolean
next()
Go to the next key/value in the block section.private boolean
Set our selves up for the next 'next' invocation, set up next block.private void
Set the position on current backing blockBuffer.protected boolean
protected void
readAndUpdateNewBlock
(long firstDataBlockOffset) protected void
protected void
readMvccVersion
(int offsetFromPos) Read mvcc.protected HFileBlock
Scans blocks in the "scanned" section of theHFile
until the next data block is found.void
recordBlockSize
(IntConsumer blockSizeConsumer) Record the size of the current block in bytes, passing as an argument to the blockSizeConsumer.protected void
releaseIfNotCurBlock
(HFileBlock block) The curBlock will be released by shipping or close method, so only need to consider releasing the block, which was read from HFile before and not referenced by curBlock.int
reseekTo
(ExtendedCell key) Reseek to or just before the passedcell
.(package private) void
reset()
private void
returnBlocks
(boolean returnAll) boolean
seekBefore
(ExtendedCell key) Consider the cell stream of all the cells in the file,c[0] ..
boolean
seekTo()
Positions this scanner at the start of the file.int
seekTo
(ExtendedCell key) SeekTo or just before the passedcell
.int
seekTo
(ExtendedCell key, boolean rewind) An internal API function.protected void
void
shipped()
Called after a batch of rows scanned and set to be returned to client.toString()
(package private) void
updateCurrBlockRef
(HFileBlock block) protected void
updateCurrentBlock
(HFileBlock newBlock) Updates the current block to be the givenHFileBlock
.
-
Field Details
-
blockBuffer
-
cacheBlocks
-
pread
-
isCompaction
-
currKeyLen
-
currValueLen
-
currMemstoreTSLen
-
currMemstoreTS
-
reader
-
currTagsLen
-
rowLen
-
bufBackedKeyOnlyKv
-
pair
-
nextIndexedKey
The next indexed key is to keep track of the indexed key of the next data block. If the nextIndexedKey is HConstants.NO_NEXT_INDEXED_KEY, it means that the current data block is the last data block. If the nextIndexedKey is null, it means the nextIndexedKey has not been loaded yet. -
curBlock
-
providedCurrentBlockSize
-
prevBlocks
-
-
Constructor Details
-
HFileScannerImpl
public HFileScannerImpl(HFile.Reader reader, boolean cacheBlocks, boolean pread, boolean isCompaction)
-
-
Method Details
-
getCurBlock
-
updateCurrBlockRef
-
reset
void reset() -
returnBlocks
-
isSeeked
- Specified by:
isSeeked
in interfaceHFileScanner
- Returns:
- True is scanner has had one of the seek calls invoked; i.e.
HFileScanner.seekBefore(ExtendedCell)
orHFileScanner.seekTo()
orHFileScanner.seekTo(ExtendedCell)
. Otherwise returns false.
-
toString
-
assertSeeked
-
getReader
Description copied from interface:HFileScanner
Returns Reader that underlies this Scanner instance.- Specified by:
getReader
in interfaceHFileScanner
-
getKVBufSize
-
close
Description copied from interface:HFileScanner
Close this HFile scanner and do necessary cleanup.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Specified by:
close
in interfaceHFileScanner
-
recordBlockSize
Description copied from interface:HFileScanner
Record the size of the current block in bytes, passing as an argument to the blockSizeConsumer. Implementations should ensure that blockSizeConsumer is only called once per block.- Specified by:
recordBlockSize
in interfaceHFileScanner
- Parameters:
blockSizeConsumer
- to be called with block size in bytes, once per block.
-
getCurCellSerializedSize
-
readKeyValueLen
-
checkTagsLen
-
readMvccVersion
Read mvcc. Does checks to see if we even need to read the mvcc at all. -
_readMvccVersion
Actually do the mvcc read. Does no checks. -
blockSeek
Within a loaded block, seek looking for the last key that is smaller than (or equal to?) the key we are interested in. A note on the seekBefore: if you have seekBefore = true, AND the first key in the block = key, then you'll get thrown exceptions. The caller has to check for that case and load the previous block as appropriate. the key to find find the key before the given key in case of exact match.- Returns:
- 0 in case of an exact key match, 1 in case of an inexact match, -2 in case of an inexact match and furthermore, the input key less than the first key of current block(e.g. using a faked index key)
-
getNextIndexedKey
Description copied from interface:HFileScanner
Returns the next key in the index (the key to seek to the next block)- Specified by:
getNextIndexedKey
in interfaceHFileScanner
-
seekTo
Description copied from interface:HFileScanner
SeekTo or just before the passedcell
. Examine the return code to figure whether we found the cell or not. Consider the cell stream of all the cells in the file,c[0] .. c[n]
, where there are n cells in the file.- Specified by:
seekTo
in interfaceHFileScanner
- Returns:
- -1, if cell < c[0], no position; 0, such that c[i] = cell and scanner is left in position i; and 1, such that c[i] < cell, and scanner is left in position i. The scanner will position itself between c[i] and c[i+1] where c[i] < cell <= c[i+1]. If there is no cell c[i+1] greater than or equal to the input cell, then the scanner will position itself at the end of the file and next() will return false when it is called.
- Throws:
IOException
-
reseekTo
Description copied from interface:HFileScanner
Reseek to or just before the passedcell
. Similar to seekTo except that this can be called even if the scanner is not at the beginning of a file. This can be used to seek only to cells which come after the current position of the scanner. Consider the cell stream of all the cells in the file,c[0] .. c[n]
, where there are n cellc in the file after current position of HFileScanner. The scanner will position itself between c[i] and c[i+1] where c[i] < cell <= c[i+1]. If there is no cell c[i+1] greater than or equal to the input cell, then the scanner will position itself at the end of the file and next() will return false when it is called.- Specified by:
reseekTo
in interfaceHFileScanner
- Parameters:
key
- Cell to find (should be non-null)- Returns:
- -1, if cell < c[0], no position; 0, such that c[i] = cell and scanner is left in position i; and 1, such that c[i] < cell, and scanner is left in position i.
- Throws:
IOException
-
seekTo
An internal API function. Seek to the given key, optionally rewinding to the first key of the block before doing the seek.- Parameters:
key
- - a cell representing the key that we need to fetchrewind
- whether to rewind to the first key of the block before doing the seek. If this is false, we are assuming we never go back, otherwise the result is undefined.- Returns:
- -1 if the key is earlier than the first key of the file, 0 if we are at the given key, 1 if we are past the given key -2 if the key is earlier than the first key of the file while using a faked index key
- Throws:
IOException
-
seekBefore
Description copied from interface:HFileScanner
Consider the cell stream of all the cells in the file,c[0] .. c[n]
, where there are n cells in the file.- Specified by:
seekBefore
in interfaceHFileScanner
- Parameters:
key
- Cell to find- Returns:
- false if cell <= c[0] or true with scanner in position 'i' such that: c[i] < cell. Furthermore: there may be a c[i+1], such that c[i] < cell <= c[i+1] but there may also NOT be a c[i+1], and next() will return false (EOF).
- Throws:
IOException
-
releaseIfNotCurBlock
The curBlock will be released by shipping or close method, so only need to consider releasing the block, which was read from HFile before and not referenced by curBlock. -
readNextDataBlock
Scans blocks in the "scanned" section of theHFile
until the next data block is found.- Returns:
- the next block, or null if there are no more data blocks
- Throws:
IOException
-
getEffectiveDataBlockEncoding
-
getCell
Description copied from interface:HFileScanner
Returns Instance ofExtendedCell
.- Specified by:
getCell
in interfaceHFileScanner
-
getKey
Description copied from interface:HFileScanner
Gets the current key in the form of a cell. You must callHFileScanner.seekTo(ExtendedCell)
before this method.- Specified by:
getKey
in interfaceHFileScanner
- Returns:
- gets the current key as a Cell.
-
getValue
Description copied from interface:HFileScanner
Gets a buffer view to the current value. You must callHFileScanner.seekTo(ExtendedCell)
before this method.- Specified by:
getValue
in interfaceHFileScanner
- Returns:
- byte buffer for the value. The limit is set to the value size, and the position is 0, the start of the buffer view.
-
setNonSeekedState
-
positionThisBlockBuffer
Set the position on current backing blockBuffer. -
positionForNextBlock
Set our selves up for the next 'next' invocation, set up next block.- Returns:
- True is more to read else false if at the end.
- Throws:
IOException
-
isNextBlock
- Throws:
IOException
-
_next
- Throws:
IOException
-
next
Go to the next key/value in the block section. Loads the next block if necessary. If successful,getKey()
andgetValue()
can be called.- Specified by:
next
in interfaceHFileScanner
- Returns:
- true if successfully navigated to the next key/value
- Throws:
IOException
-
seekTo
Positions this scanner at the start of the file.- Specified by:
seekTo
in interfaceHFileScanner
- Returns:
- false if empty file; i.e. a call to next would return false and the current key and value are undefined.
- Throws:
IOException
-
processFirstDataBlock
- Throws:
IOException
-
readAndUpdateNewBlock
- Throws:
IOException
-
loadBlockAndSeekToKey
protected int loadBlockAndSeekToKey(HFileBlock seekToBlock, ExtendedCell nextIndexedKey, boolean rewind, ExtendedCell key, boolean seekBefore) throws IOException - Throws:
IOException
-
checkKeyLen
Returns True if v <= 0 or v > current block buffer limit. -
checkLen
Returns True if v < 0 or v > current block buffer limit. -
checkKeyValueLen
Check key and value lengths are wholesome. -
updateCurrentBlock
Updates the current block to be the givenHFileBlock
. Seeks to the the first key/value pair.- Parameters:
newBlock
- the block read byHFileReaderImpl.readBlock(long, long, boolean, boolean, boolean, boolean, org.apache.hadoop.hbase.io.hfile.BlockType, org.apache.hadoop.hbase.io.encoding.DataBlockEncoding)
, it's a totally new block with new allocatedByteBuff
, so if no further reference to this block, we should release it carefully.- Throws:
IOException
-
getFirstKeyCellInBlock
-
compareKey
-
shipped
Description copied from interface:Shipper
Called after a batch of rows scanned and set to be returned to client. Any in between cleanup can be done here.- Specified by:
shipped
in interfaceShipper
- Throws:
IOException
-