org.apache.hadoop.hbase.io.hfile

Class HFileBlockIndex.BlockIndexChunk

java.lang.Object

org.apache.hadoop.hbase.io.hfile.HFileBlockIndex.BlockIndexChunk

Enclosing class:

HFileBlockIndex

static class HFileBlockIndex.BlockIndexChunk
extends Object

A single chunk of the block index in the process of writing. The data in this chunk can become a leaf-level, intermediate-level, or root index block.

Field Summary

Fields

Modifier and Type	Field and Description
private List<byte[]>	blockKeys First keys of the key range corresponding to each index entry.
private List<Long>	blockOffsets Block offset in backing stream.
private int	curTotalNonRootEntrySize The offset of the next entry to be added, relative to the end of the "secondary index" in the "non-root" format representation of this index chunk.
private int	curTotalRootSize The accumulated size of this chunk if stored in the root index format.
private List<Long>	numSubEntriesAt The cumulative number of sub-entries, i.e.
private List<Integer>	onDiskDataSizes On-disk data sizes of lower-level data or index blocks.
private List<Integer>	secondaryIndexOffsetMarks The "secondary index" used for binary search over variable-length records in a "non-root" format block.

Constructor Summary

Constructors

Constructor and Description
BlockIndexChunk()

Method Summary

Modifier and Type	Method and Description
void	add(byte[] firstKey, long blockOffset, int onDiskDataSize) The same as add(byte[], long, int, long) but does not take the key/value into account.
(package private) void	add(byte[] firstKey, long blockOffset, int onDiskDataSize, long curTotalNumSubEntries) Adds a new entry to this block index chunk.
void	clear()
byte[]	getBlockKey(int i)
long	getBlockOffset(int i)
long	getCumulativeNumKV(int i)
int	getEntryBySubEntry(long k) Finds the entry corresponding to the deeper-level index block containing the given deeper-level entry (a "sub-entry"), assuming a global 0-based ordering of sub-entries.
byte[]	getMidKeyMetadata() Used when writing the root block index of a multi-level block index.
(package private) int	getNonRootSize()
int	getNumEntries()
int	getOnDiskDataSize(int i)
(package private) int	getRootSize()
(package private) void	writeNonRoot(DataOutput out) Writes the block index chunk in the non-root index block format.
(package private) void	writeRoot(DataOutput out) Writes this chunk into the given output stream in the root block index format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

blockKeys

private final List<byte[]> blockKeys

First keys of the key range corresponding to each index entry.

blockOffsets

private final List<Long> blockOffsets

Block offset in backing stream.

onDiskDataSizes

private final List<Integer> onDiskDataSizes

On-disk data sizes of lower-level data or index blocks.

numSubEntriesAt

private final List<Long> numSubEntriesAt

The cumulative number of sub-entries, i.e. entries on deeper-level block index entries. numSubEntriesAt[i] is the number of sub-entries in the blocks corresponding to this chunk's entries #0 through #i inclusively.

curTotalNonRootEntrySize

private int curTotalNonRootEntrySize

The offset of the next entry to be added, relative to the end of the "secondary index" in the "non-root" format representation of this index chunk. This is the next value to be added to the secondary index.

curTotalRootSize

private int curTotalRootSize

The accumulated size of this chunk if stored in the root index format.

secondaryIndexOffsetMarks

private final List<Integer> secondaryIndexOffsetMarks

The "secondary index" used for binary search over variable-length records in a "non-root" format block. These offsets are relative to the end of this secondary index.

Constructor Detail

BlockIndexChunk

BlockIndexChunk()

Method Detail

add

void add(byte[] firstKey,
         long blockOffset,
         int onDiskDataSize,
         long curTotalNumSubEntries)

Adds a new entry to this block index chunk.

Parameters:

firstKey - the first key in the block pointed to by this entry

blockOffset - the offset of the next-level block pointed to by this entry

onDiskDataSize - the on-disk data of the block pointed to by this entry, including header size

curTotalNumSubEntries - if this chunk is the root index chunk under construction, this specifies the current total number of sub-entries in all leaf-level chunks, including the one corresponding to the second-level entry being added.

add

public void add(byte[] firstKey,
                long blockOffset,
                int onDiskDataSize)

The same as add(byte[], long, int, long) but does not take the key/value into account. Used for single-level indexes.

See Also:

#add(byte[], long, int, long)}

clear

public void clear()

getEntryBySubEntry

public int getEntryBySubEntry(long k)

Finds the entry corresponding to the deeper-level index block containing the given deeper-level entry (a "sub-entry"), assuming a global 0-based ordering of sub-entries.

Implementation note. We are looking for i such that numSubEntriesAt[i - 1] <= k < numSubEntriesAt[i], because a deeper-level block #i (0-based) contains sub-entries # numSubEntriesAt[i - 1]'th through numSubEntriesAt[i] - 1, assuming a global 0-based ordering of sub-entries. i is by definition the insertion point of k in numSubEntriesAt.

Parameters:

k - sub-entry index, from 0 to the total number sub-entries - 1

Returns:

the 0-based index of the entry corresponding to the given sub-entry

getMidKeyMetadata

public byte[] getMidKeyMetadata()
                         throws IOException

Used when writing the root block index of a multi-level block index. Serializes additional information allowing to efficiently identify the mid-key.

Returns:

a few serialized fields for finding the mid-key

Throws:

IOException - if could not create metadata for computing mid-key

writeNonRoot

void writeNonRoot(DataOutput out)
           throws IOException

Writes the block index chunk in the non-root index block format. This format contains the number of entries, an index of integer offsets for quick binary search on variable-length records, and tuples of block offset, on-disk block size, and the first key for each entry.

Parameters:

out -

Throws:

IOException

getNonRootSize

int getNonRootSize()

Returns:

the size of this chunk if stored in the non-root index block format

writeRoot

void writeRoot(DataOutput out)
        throws IOException

Writes this chunk into the given output stream in the root block index format. This format is similar to the HFile version 1 block index format, except that we store on-disk size of the block instead of its uncompressed size.

Parameters:

out - the data output stream to write the block index to. Typically a stream writing into an HFile block.

Throws:

IOException

getRootSize

int getRootSize()

Returns:

the size of this chunk if stored in the root index block format

getNumEntries

public int getNumEntries()

Returns:

the number of entries in this block index chunk

getBlockKey

public byte[] getBlockKey(int i)

getBlockOffset

public long getBlockOffset(int i)

getOnDiskDataSize

public int getOnDiskDataSize(int i)

getCumulativeNumKV

public long getCumulativeNumKV(int i)