Package org.apache.hadoop.hbase.regionserver

Class TestCompoundBloomFilter

java.lang.Object
org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter
public class TestCompoundBloomFilter extends Object
Tests writing Bloom filter blocks in the same part of the file as data blocks.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private static final int[]
    BLOCK_SIZES
     
    private org.apache.hadoop.hbase.io.hfile.BlockCache
    blockCache
     
    private static final int[]
    BLOOM_BLOCK_SIZES
    Be careful not to specify too high a Bloom filter block size, otherwise there will only be one oversized chunk and the observed false positive rate will be too low.
    private static final org.apache.hadoop.hbase.regionserver.BloomType[]
    BLOOM_TYPES
     
    private static org.apache.hadoop.hbase.io.hfile.CacheConfig
    cacheConf
     
    static final HBaseClassTestRule
    CLASS_RULE
     
    private static org.apache.hadoop.conf.Configuration
    conf
     
    private static final int
    EVALUATION_SEED
     
    private org.apache.hadoop.fs.FileSystem
    fs
     
    private static final int
    GENERATION_SEED
     
    private static final org.slf4j.Logger
    LOG
     
    private static final int[]
    NUM_KV
     
    private static final int
    NUM_TESTS
     
    private static final double[]
    TARGET_ERROR_RATES
     
    private static final HBaseTestingUtil
    TEST_UTIL
     
    private String
    testIdMsg
    A message of the form "in test#<number>:" to include in logging.
    private static final double
    TOO_HIGH_ERROR_RATE
    A false positive rate that is obviously too high.

  • Constructor Summary

    Constructors
    Constructor
    Description
    TestCompoundBloomFilter()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    private List<org.apache.hadoop.hbase.KeyValue>
    createSortedKeyValues(Random rand, int n)
     
    private boolean
    isInBloom(org.apache.hadoop.hbase.regionserver.StoreFileScanner scanner, byte[] row, byte[] qualifier)
     
    private boolean
    isInBloom(org.apache.hadoop.hbase.regionserver.StoreFileScanner scanner, byte[] row, org.apache.hadoop.hbase.regionserver.BloomType bt, Random rand)
     
    private void
    readStoreFile(int t, org.apache.hadoop.hbase.regionserver.BloomType bt, List<org.apache.hadoop.hbase.KeyValue> kvs, org.apache.hadoop.fs.Path sfPath)
     
    void
    setUp()
     
    void
    testCompoundBloomFilter()
     
    void
    testCompoundBloomSizing()
     
    void
    testCreateKey()
     
    private void
    validateFalsePosRate(double falsePosRate, int nTrials, double zValueBoundary, org.apache.hadoop.hbase.io.hfile.CompoundBloomFilter cbf, String additionalMsg)
    Validates the false positive ratio by computing its z-value and comparing it to the provided threshold.
    private org.apache.hadoop.fs.Path
    writeStoreFile(int t, org.apache.hadoop.hbase.regionserver.BloomType bt, List<org.apache.hadoop.hbase.KeyValue> kvs)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • CLASS_RULE

      public static final HBaseClassTestRule CLASS_RULE

    • TEST_UTIL

      private static final HBaseTestingUtil TEST_UTIL

    • LOG

      private static final org.slf4j.Logger LOG

    • NUM_TESTS

      private static final int NUM_TESTS
      See Also:

    • BLOOM_TYPES

      private static final org.apache.hadoop.hbase.regionserver.BloomType[] BLOOM_TYPES

    • NUM_KV

      private static final int[] NUM_KV

    • BLOCK_SIZES

      private static final int[] BLOCK_SIZES

    • BLOOM_BLOCK_SIZES

      private static final int[] BLOOM_BLOCK_SIZES
      Be careful not to specify too high a Bloom filter block size, otherwise there will only be one oversized chunk and the observed false positive rate will be too low.

    • TARGET_ERROR_RATES

      private static final double[] TARGET_ERROR_RATES

    • TOO_HIGH_ERROR_RATE

      private static final double TOO_HIGH_ERROR_RATE
      A false positive rate that is obviously too high.

    • conf

      private static org.apache.hadoop.conf.Configuration conf

    • cacheConf

      private static org.apache.hadoop.hbase.io.hfile.CacheConfig cacheConf

    • fs

      private org.apache.hadoop.fs.FileSystem fs

    • testIdMsg

      private String testIdMsg
      A message of the form "in test#<number>:" to include in logging.

    • GENERATION_SEED

      private static final int GENERATION_SEED
      See Also:

    • EVALUATION_SEED

      private static final int EVALUATION_SEED
      See Also:

    • blockCache

      private org.apache.hadoop.hbase.io.hfile.BlockCache blockCache

  • Constructor Details

  • Method Details

    • setUp

      public void setUp() throws IOException
      Throws:
      IOException

    • createSortedKeyValues

      private List<org.apache.hadoop.hbase.KeyValue> createSortedKeyValues(Random rand, int n)

    • testCompoundBloomFilter

      public void testCompoundBloomFilter() throws IOException
      Throws:
      IOException

    • validateFalsePosRate

      private void validateFalsePosRate(double falsePosRate, int nTrials, double zValueBoundary, org.apache.hadoop.hbase.io.hfile.CompoundBloomFilter cbf, String additionalMsg)
      Validates the false positive ratio by computing its z-value and comparing it to the provided threshold.
      Parameters:
      falsePosRate - experimental positive rate
      nTrials - the number of Bloom filter checks
      zValueBoundary - z-value boundary, positive for an upper bound and negative for a lower bound
      cbf - the compound Bloom filter we are using
      additionalMsg - additional message to include in log output and assertion failures

    • readStoreFile

      private void readStoreFile(int t, org.apache.hadoop.hbase.regionserver.BloomType bt, List<org.apache.hadoop.hbase.KeyValue> kvs, org.apache.hadoop.fs.Path sfPath) throws IOException
      Throws:
      IOException

    • isInBloom

      private boolean isInBloom(org.apache.hadoop.hbase.regionserver.StoreFileScanner scanner, byte[] row, org.apache.hadoop.hbase.regionserver.BloomType bt, Random rand)

    • isInBloom

      private boolean isInBloom(org.apache.hadoop.hbase.regionserver.StoreFileScanner scanner, byte[] row, byte[] qualifier)

    • writeStoreFile

      private org.apache.hadoop.fs.Path writeStoreFile(int t, org.apache.hadoop.hbase.regionserver.BloomType bt, List<org.apache.hadoop.hbase.KeyValue> kvs) throws IOException
      Throws:
      IOException

    • testCompoundBloomSizing

      public void testCompoundBloomSizing()

    • testCreateKey

      public void testCreateKey()