Writes the block index into the output stream. Generate the tree from
bottom up. The leaf level is written to disk as a sequence of inline
blocks, if it is larger than a certain number of bytes. If the leaf level
is not large enough, we write all entries to the root level instead.
After all leaf blocks have been written, we end up with an index
referencing the resulting leaf index blocks. If that index is larger than
the allowed root index size, the writer will break it up into
reasonable-size intermediate-level index block chunks write those chunks
out, and create another index referencing those chunks. This will be
repeated until the remaining index is small enough to become the root
index. However, in most practical cases we will only have leaf-level
blocks and the root index, or just the root index.
blockWriter - the block writer to use to write index blocks
cacheConf - used to determine when and how a block should be cached-on-write.
public void setMaxChunkSize(int maxChunkSize)
public long writeIndexBlocks(org.apache.hadoop.fs.FSDataOutputStream out)
Writes the root level and intermediate levels of the block index into
the output stream, generating the tree from bottom up. Assumes that the
leaf level has been inline-written to the disk if there is enough data
for more than one leaf block. We iterate by breaking the current level
of the block index, starting with the index of all leaf-level blocks,
into chunks small enough to be written to disk, and generate its parent
level, until we end up with a level small enough to become the root
If the leaf level is not large enough, there is no inline block index
anymore, so we only write that level of block index to disk as the root
out - FSDataOutputStream
position at which we entered the root-level index.
how many block index entries there are in the root level
public int getNumLevels()
the number of levels in this block index.
public boolean shouldWriteBlock(boolean closing)
Whether there is an inline block ready to be written. In general, we
write an leaf-level index block as an inline block as soon as its size
as serialized in the non-root format reaches a certain threshold.