HFileBlock.Writer (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.io.hfile.HFileBlock.Writer

Enclosing class:

HFileBlock
```
public static class HFileBlock.Writer
extends Object
```
Unified version 2 HFile block writer. The intended usage pattern is as follows:
1. Construct an HFileBlock.Writer, providing a compression algorithm.
2. Call startWriting(org.apache.hadoop.hbase.io.hfile.BlockType) and get a data stream to write to.
3. Write your data into the stream.
4. Call writeHeaderAndData(FSDataOutputStream) as many times as you need to. store the serialized block into an external stream.
5. Repeat to write more blocks.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class HFileBlock.Writer.BufferGrabbingByteArrayOutputStream

private static class HFileBlock.Writer.State

Nested Classes
Modifier and Type	Class and Description
`static class`	`HFileBlock.Writer.BufferGrabbingByteArrayOutputStream`
`private static class`	`HFileBlock.Writer.State`

Field Summary

Fields
Modifier and Type	Field and Description
`private ByteArrayOutputStream`	`baosInMemory` The stream we use to accumulate data in uncompressed format for each block.
`private BlockType`	`blockType` Current block type.
`private HFileDataBlockEncoder`	`dataBlockEncoder` Data block encoder used for data blocks
`private HFileBlockEncodingContext`	`dataBlockEncodingCtx`
`private HFileBlockDefaultEncodingContext`	`defaultBlockEncodingCtx` block encoding context for non-data blocks
`private HFileContext`	`fileContext` Meta data that holds information about the hfileblock
`private byte[]`	`onDiskBytesWithHeader` Bytes to be written to the file system, including the header.
`private byte[]`	`onDiskChecksum` The size of the checksum data on disk.
`private long`	`prevOffset` The offset of the previous block of the same type
`private long[]`	`prevOffsetByType` Offset of previous block by block type.
`private long`	`startOffset` Current block's start offset in the `HFile`.
`private HFileBlock.Writer.State`	`state` Writer state.
`private byte[]`	`uncompressedBytesWithHeader` Valid in the READY state.
`private int`	`unencodedDataSizeWritten`
`private DataOutputStream`	`userDataStream` A stream that we write uncompressed bytes to, which compresses them and writes them to `baosInMemory`.

Constructor Summary

Constructors
Constructor and Description

HFileBlock.Writer(HFileDataBlockEncoder dataBlockEncoder, HFileContext fileContext)

Constructors
Constructor and Description
`HFileBlock.Writer(HFileDataBlockEncoder dataBlockEncoder, HFileContext fileContext)`

Method Summary

Methods
Modifier and Type	Method and Description
`int`	`blockSizeWritten()` Returns the number of bytes written into the current block so far, or zero if not writing the block at the moment.
`(package private) void`	`ensureBlockReady()` Transitions the block writer from the "writing" state to the "block ready" state.
`private void`	`expectState(HFileBlock.Writer.State expectedState)`
`private void`	`finishBlock()` An internal method that flushes the compressing stream (if using compression), serializes the header, and takes care of the separate uncompressed stream for caching on write, if applicable.
`protected void`	`finishBlockAndWriteHeaderAndData(DataOutputStream out)` Writes the header and the compressed data of this block (or uncompressed data when not using compression) into the given stream.
`HFileBlock`	`getBlockForCaching(CacheConfig cacheConf)` Creates a new HFileBlock.
`(package private) byte[]`	`getHeaderAndDataForTest()` Returns the header or the compressed data (or uncompressed data when not using compression) as a byte array.
`(package private) ByteBuffer`	`getOnDiskBufferWithHeader()` Returns the header followed by the on-disk (compressed/encoded/encrypted) data.
`(package private) int`	`getOnDiskSizeWithHeader()` Returns the on-disk size of the block.
`(package private) int`	`getOnDiskSizeWithoutHeader()` Returns the on-disk size of the data portion of the block.
`(package private) ByteBuffer`	`getUncompressedBufferWithHeader()` Returns the header followed by the uncompressed data, even if using compression.
`(package private) int`	`getUncompressedSizeWithHeader()` The uncompressed size of the block data, including header size.
`(package private) int`	`getUncompressedSizeWithoutHeader()` The uncompressed size of the block data.
`(package private) DataOutputStream`	`getUserDataStream()` Returns the stream for the user to write to.
`boolean`	`isWriting()`
`private void`	`putHeader(byte[] dest, int offset, int onDiskSize, int uncompressedSize, int onDiskDataSize)` Put the header into the given byte array at the given offset.
`void`	`release()` Releases resources used by this writer.
`DataOutputStream`	`startWriting(BlockType newBlockType)` Starts writing into the block.
`void`	`write(Cell cell)` Writes the Cell to this block
`void`	`writeBlock(HFileBlock.BlockWritable bw, org.apache.hadoop.fs.FSDataOutputStream out)` Takes the given `HFileBlock.BlockWritable` instance, creates a new block of its appropriate type, writes the writable into this block, and flushes the block into the output stream.
`void`	`writeHeaderAndData(org.apache.hadoop.fs.FSDataOutputStream out)` Similar to `writeHeaderAndData(FSDataOutputStream)`, but records the offset of this block so that it can be referenced in the next block of the same type.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - state
```
private HFileBlock.Writer.State state
```
    Writer state. Used to ensure the correct usage protocol.
  - dataBlockEncoder
```
private final HFileDataBlockEncoder dataBlockEncoder
```
    Data block encoder used for data blocks
  - dataBlockEncodingCtx
```
private HFileBlockEncodingContext dataBlockEncodingCtx
```
  - defaultBlockEncodingCtx
```
private HFileBlockDefaultEncodingContext defaultBlockEncodingCtx
```
    block encoding context for non-data blocks
  - baosInMemory
```
private ByteArrayOutputStream baosInMemory
```
    The stream we use to accumulate data in uncompressed format for each block. We reset this stream at the end of each block and reuse it. The header is written as the first HConstants.HFILEBLOCK_HEADER_SIZE bytes into this stream.
  - blockType
```
private BlockType blockType
```
    Current block type. Set in startWriting(BlockType). Could be changed in finishBlock() from BlockType.DATA to BlockType.ENCODED_DATA.
  - userDataStream
```
private DataOutputStream userDataStream
```
    A stream that we write uncompressed bytes to, which compresses them and writes them to baosInMemory.
  - unencodedDataSizeWritten
```
private int unencodedDataSizeWritten
```
  - onDiskBytesWithHeader
```
private byte[] onDiskBytesWithHeader
```
    Bytes to be written to the file system, including the header. Compressed if compression is turned on. It also includes the checksum data that immediately follows the block data. (header + data + checksums)
  - onDiskChecksum
```
private byte[] onDiskChecksum
```
    The size of the checksum data on disk. It is used only if data is not compressed. If data is compressed, then the checksums are already part of onDiskBytesWithHeader. If data is uncompressed, then this variable stores the checksum data for this block.
  - uncompressedBytesWithHeader
```
private byte[] uncompressedBytesWithHeader
```
    Valid in the READY state. Contains the header and the uncompressed (but potentially encoded, if this is a data block) bytes, so the length is HFileBlock.uncompressedSizeWithoutHeader + HConstants.HFILEBLOCK_HEADER_SIZE. Does not store checksums.
  - startOffset
```
private long startOffset
```
    Current block's start offset in the HFile. Set in writeHeaderAndData(FSDataOutputStream).
  - prevOffsetByType
```
private long[] prevOffsetByType
```
    Offset of previous block by block type. Updated when the next block is started.
  - prevOffset
```
private long prevOffset
```
    The offset of the previous block of the same type
  - fileContext
```
private HFileContext fileContext
```
    Meta data that holds information about the hfileblock
- Constructor Detail
  - HFileBlock.Writer
```
public HFileBlock.Writer(HFileDataBlockEncoder dataBlockEncoder,
                 HFileContext fileContext)
```
    Parameters:
    dataBlockEncoder - data block encoding algorithm to use
- Method Detail
  - startWriting
```
public DataOutputStream startWriting(BlockType newBlockType)
                              throws IOException
```
    Starts writing into the block. The previous block's data is discarded.
    
    Returns:
    the stream the user can write their data into
    
    Throws:
    
    IOException
  - write
```
public void write(Cell cell)
           throws IOException
```
    Writes the Cell to this block
    
    Parameters:
    cell -
    
    Throws:
    
    IOException
  - getUserDataStream
```
DataOutputStream getUserDataStream()
```
    Returns the stream for the user to write to. The block writer takes care of handling compression and buffering for caching on write. Can only be called in the "writing" state.
    
    Returns:
    the data output stream for the user to write to
  - ensureBlockReady
```
void ensureBlockReady()
                throws IOException
```
    Transitions the block writer from the "writing" state to the "block ready" state. Does nothing if a block is already finished.
    
    Throws:
    
    IOException
  - finishBlock
```
private void finishBlock()
                  throws IOException
```
    An internal method that flushes the compressing stream (if using compression), serializes the header, and takes care of the separate uncompressed stream for caching on write, if applicable. Sets block write state to "block ready".
    
    Throws:
    
    IOException
  - putHeader
```
private void putHeader(byte[] dest,
             int offset,
             int onDiskSize,
             int uncompressedSize,
             int onDiskDataSize)
```
    Put the header into the given byte array at the given offset.
    
    Parameters:
    onDiskSize - size of the block on disk header + data + checksum
    uncompressedSize - size of the block after decompression (but before optional data block decoding) including header
    onDiskDataSize - size of the block on disk with header and data but not including the checksums
  - writeHeaderAndData
```
public void writeHeaderAndData(org.apache.hadoop.fs.FSDataOutputStream out)
                        throws IOException
```
    Similar to writeHeaderAndData(FSDataOutputStream), but records the offset of this block so that it can be referenced in the next block of the same type.
    
    Parameters:
    out -
    
    Throws:
    
    IOException
  - finishBlockAndWriteHeaderAndData
```
protected void finishBlockAndWriteHeaderAndData(DataOutputStream out)
                                         throws IOException
```
    Writes the header and the compressed data of this block (or uncompressed data when not using compression) into the given stream. Can be called in the "writing" state or in the "block ready" state. If called in the "writing" state, transitions the writer to the "block ready" state.
    
    Parameters:
    out - the output stream to write the
    
    Throws:
    
    IOException
  - getHeaderAndDataForTest
```
byte[] getHeaderAndDataForTest()
                         throws IOException
```
    Returns the header or the compressed data (or uncompressed data when not using compression) as a byte array. Can be called in the "writing" state or in the "block ready" state. If called in the "writing" state, transitions the writer to the "block ready" state. This returns the header + data + checksums stored on disk.
    
    Returns:
    header and data as they would be stored on disk in a byte array
    
    Throws:
    
    IOException
  - release
```
public void release()
```
    Releases resources used by this writer.
  - getOnDiskSizeWithoutHeader
```
int getOnDiskSizeWithoutHeader()
```
    Returns the on-disk size of the data portion of the block. This is the compressed size if compression is enabled. Can only be called in the "block ready" state. Header is not compressed, and its size is not included in the return value.
    
    Returns:
    the on-disk size of the block, not including the header.
  - getOnDiskSizeWithHeader
```
int getOnDiskSizeWithHeader()
```
    Returns the on-disk size of the block. Can only be called in the "block ready" state.
    
    Returns:
    the on-disk size of the block ready to be written, including the header size, the data and the checksum data.
  - getUncompressedSizeWithoutHeader
```
int getUncompressedSizeWithoutHeader()
```
    The uncompressed size of the block data. Does not include header size.
  - getUncompressedSizeWithHeader
```
int getUncompressedSizeWithHeader()
```
    The uncompressed size of the block data, including header size.
  - isWriting
```
public boolean isWriting()
```
    Returns:
    true if a block is being written
  - blockSizeWritten
```
public int blockSizeWritten()
```
    Returns the number of bytes written into the current block so far, or zero if not writing the block at the moment. Note that this will return zero in the "block ready" state as well.
    
    Returns:
    the number of bytes written
  - getUncompressedBufferWithHeader
```
ByteBuffer getUncompressedBufferWithHeader()
```
    Returns the header followed by the uncompressed data, even if using compression. This is needed for storing uncompressed blocks in the block cache. Can be called in the "writing" state or the "block ready" state. Returns only the header and data, does not include checksum data.
    
    Returns:
    uncompressed block bytes for caching on write
  - getOnDiskBufferWithHeader
```
ByteBuffer getOnDiskBufferWithHeader()
```
    Returns the header followed by the on-disk (compressed/encoded/encrypted) data. This is needed for storing packed blocks in the block cache. Expects calling semantics identical to getUncompressedBufferWithHeader(). Returns only the header and data, Does not include checksum data.
    
    Returns:
    packed block bytes for caching on write
  - expectState
```
private void expectState(HFileBlock.Writer.State expectedState)
```
  - writeBlock
```
public void writeBlock(HFileBlock.BlockWritable bw,
              org.apache.hadoop.fs.FSDataOutputStream out)
                throws IOException
```
    Takes the given HFileBlock.BlockWritable instance, creates a new block of its appropriate type, writes the writable into this block, and flushes the block into the output stream. The writer is instructed not to buffer uncompressed bytes for cache-on-write.
    
    Parameters:
    bw - the block-writable object to write as a block
    out - the file system output stream
    
    Throws:
    
    IOException
  - getBlockForCaching
```
public HFileBlock getBlockForCaching(CacheConfig cacheConf)
```
    Creates a new HFileBlock. Checksums have already been validated, so the byte buffer passed into the constructor of this newly created block does not have checksum data even though the header minor version is MINOR_VERSION_WITH_CHECKSUM. This is indicated by setting a 0 value in bytesPerChecksum.

Class HFileBlock.Writer

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

state

dataBlockEncoder

dataBlockEncodingCtx

defaultBlockEncodingCtx

baosInMemory

blockType

userDataStream

unencodedDataSizeWritten

onDiskBytesWithHeader

onDiskChecksum

uncompressedBytesWithHeader

startOffset

prevOffsetByType

prevOffset

fileContext

Constructor Detail

HFileBlock.Writer

Method Detail

startWriting

write

getUserDataStream

ensureBlockReady

finishBlock

putHeader

writeHeaderAndData

finishBlockAndWriteHeaderAndData

getHeaderAndDataForTest

release

getOnDiskSizeWithoutHeader

getOnDiskSizeWithHeader

getUncompressedSizeWithoutHeader

getUncompressedSizeWithHeader

isWriting

blockSizeWritten

getUncompressedBufferWithHeader

getOnDiskBufferWithHeader

expectState

writeBlock

getBlockForCaching