Field Detail

• LOG

  private static final org.apache.commons.logging.Log LOG

• chunk

  private ByteBloomFilter chunk

  The current chunk being written to

• prevChunk

  private ByteBloomFilter prevChunk

  Previous chunk, so that we can create another similar chunk

• maxFold

  private int maxFold

  Maximum fold factor

• chunkByteSize

  private int chunkByteSize

  The size of individual Bloom filter chunks to create

• readyChunks

  private Queue<CompoundBloomFilterWriter.ReadyChunk> readyChunks

• firstKeyInChunk

  private byte[] firstKeyInChunk

  The first key in the current Bloom filter chunk.

• bloomBlockIndexWriter

  private HFileBlockIndex.BlockIndexWriter bloomBlockIndexWriter

• cacheOnWrite

  private boolean cacheOnWrite

  Whether to cache-on-write compound Bloom filter chunks

Constructor Detail

• CompoundBloomFilterWriter

  public CompoundBloomFilterWriter(int chunkByteSizeHint,
                           float errorRate,
                           int hashType,
                           int maxFold,
                           boolean cacheOnWrite,
                           KeyValue.KVComparator comparator)

  Parameters:

  chunkByteSizeHint - each chunk's size in bytes. The real chunk size might be different as required by the fold factor.

  errorRate - target false positive rate

  hashType - hash function type to use

  maxFold - maximum degree of folding allowed

Method Detail

• shouldWriteBlock

  public boolean shouldWriteBlock(boolean closing)

  Description copied from interface: InlineBlockWriter
  Determines whether there is a new block to be written out.

  Specified by:

  shouldWriteBlock in interface InlineBlockWriter

  Parameters:

  closing - whether the file is being closed, in which case we need to write out all available data and not wait to accumulate another block

• enqueueReadyChunk

  private void enqueueReadyChunk(boolean closing)

  Enqueue the current chunk if it is ready to be written out.

  Parameters:

  closing - true if we are closing the file, so we do not expect new keys to show up

• add

  public void add(byte[] bloomKey,
         int keyOffset,
         int keyLength)

  Adds a Bloom filter key. This key must be greater than the previous key, as defined by the comparator this compound Bloom filter is configured with. For efficiency, key monotonicity is not checked here. See StoreFile.Writer.append( org.apache.hadoop.hbase.Cell) for the details of deduplication.

  Specified by:

  add in interface BloomFilterWriter

  Parameters:

  bloomKey - data to be added to the bloom

  keyOffset - offset into the data to be added

  keyLength - length of the data to be added

• writeInlineBlock

  public void writeInlineBlock(DataOutput out)
                        throws IOException

  Description copied from interface: InlineBlockWriter
  Writes the block to the provided stream. Must not write any magic records. Called only if InlineBlockWriter.shouldWriteBlock(boolean) returned true.

  Specified by:

  writeInlineBlock in interface InlineBlockWriter

  Parameters:

  out - a stream (usually a compressing stream) to write the block to

  Throws:

  IOException

• blockWritten

  public void blockWritten(long offset,
                  int onDiskSize,
                  int uncompressedSize)

  Description copied from interface: InlineBlockWriter
  Called after a block has been written, and its offset, raw size, and compressed size have been determined. Can be used to add an entry to a block index. If this type of inline blocks needs a block index, the inline block writer is responsible for maintaining it.

  Specified by:

  blockWritten in interface InlineBlockWriter

  Parameters:

  offset - the offset of the block in the stream

  onDiskSize - the on-disk size of the block

  uncompressedSize - the uncompressed size of the block

• getInlineBlockType

  public BlockType getInlineBlockType()

  Description copied from interface: InlineBlockWriter
  The type of blocks this block writer produces.

  Specified by:

  getInlineBlockType in interface InlineBlockWriter

• getMetaWriter

  public org.apache.hadoop.io.Writable getMetaWriter()

  Description copied from interface: BloomFilterWriter
  Get a writable interface into bloom filter meta data.

  Specified by:

  getMetaWriter in interface BloomFilterWriter

  Returns:

  a writable instance that can be later written to a stream

• compactBloom

  public void compactBloom()

  Description copied from interface: BloomFilterWriter
  Compact the Bloom filter before writing metadata & data to disk.

  Specified by:

  compactBloom in interface BloomFilterWriter

• allocBloom

  public void allocBloom()

  Description copied from interface: BloomFilterWriter
  Allocate memory for the bloom filter data.

  Specified by:

  allocBloom in interface BloomFilterWriter

• getDataWriter

  public org.apache.hadoop.io.Writable getDataWriter()

  Description copied from interface: BloomFilterWriter
  Get a writable interface into bloom filter data (the actual Bloom bits). Not used for compound Bloom filters.

  Specified by:

  getDataWriter in interface BloomFilterWriter

  Returns:

  a writable instance that can be later written to a stream

• getCacheOnWrite

  public boolean getCacheOnWrite()

  Specified by:

  getCacheOnWrite in interface InlineBlockWriter

  Returns:

  true if inline blocks produced by this writer should be cached