org.apache.hadoop.hbase.util

Class BloomFilterFactory

java.lang.Object

org.apache.hadoop.hbase.util.BloomFilterFactory

@InterfaceAudience.Private
public final class BloomFilterFactory
extends Object

Handles Bloom filter initialization based on configuration and serialized metadata in the reader and writer of HStoreFile.

Field Summary

Fields

Modifier and Type	Field and Description
static String	IO_STOREFILE_BLOOM_BLOCK_SIZE Target Bloom block size.
static String	IO_STOREFILE_BLOOM_ENABLED Master switch to enable Bloom filters
static String	IO_STOREFILE_BLOOM_ERROR_RATE Specifies the target error rate to use when selecting the number of keys per Bloom filter.
static String	IO_STOREFILE_BLOOM_MAX_FOLD Maximum folding factor allowed.
static String	IO_STOREFILE_BLOOM_MAX_KEYS For default (single-block) Bloom filters this specifies the maximum number of keys.
static String	IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED Master switch to enable Delete Family Bloom filters
private static org.slf4j.Logger	LOG
private static int	MAX_ALLOWED_FOLD_FACTOR Maximum number of times a Bloom filter can be "folded" if oversized

Constructor Summary

Constructors

Modifier	Constructor and Description
private	BloomFilterFactory() This class should not be instantiated.

Method Summary

Modifier and Type	Method and Description
static BloomFilterWriter	createDeleteBloomAtWrite(org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, int maxKeys, HFile.Writer writer) Creates a new Delete Family Bloom filter at the time of HStoreFile writing.
static BloomFilter	createFromMeta(DataInput meta, HFile.Reader reader) Instantiates the correct Bloom filter class based on the version provided in the meta block data.
static BloomFilterWriter	createGeneralBloomAtWrite(org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType bloomType, int maxKeys, HFile.Writer writer) Creates a new general (Row or RowCol) Bloom filter at the time of HStoreFile writing.
static int	getBloomBlockSize(org.apache.hadoop.conf.Configuration conf)
static float	getErrorRate(org.apache.hadoop.conf.Configuration conf)
static int	getMaxFold(org.apache.hadoop.conf.Configuration conf)
static int	getMaxKeys(org.apache.hadoop.conf.Configuration conf)
static boolean	isDeleteFamilyBloomEnabled(org.apache.hadoop.conf.Configuration conf)
static boolean	isGeneralBloomEnabled(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

IO_STOREFILE_BLOOM_ERROR_RATE

public static final String IO_STOREFILE_BLOOM_ERROR_RATE

Specifies the target error rate to use when selecting the number of keys per Bloom filter.

See Also:

Constant Field Values

IO_STOREFILE_BLOOM_MAX_FOLD

public static final String IO_STOREFILE_BLOOM_MAX_FOLD

Maximum folding factor allowed. The Bloom filter will be shrunk by the factor of up to 2 ** this times if we oversize it initially.

See Also:

Constant Field Values

IO_STOREFILE_BLOOM_MAX_KEYS

public static final String IO_STOREFILE_BLOOM_MAX_KEYS

For default (single-block) Bloom filters this specifies the maximum number of keys.

See Also:

Constant Field Values

IO_STOREFILE_BLOOM_ENABLED

public static final String IO_STOREFILE_BLOOM_ENABLED

Master switch to enable Bloom filters

See Also:

Constant Field Values

IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED

public static final String IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED

Master switch to enable Delete Family Bloom filters

See Also:

Constant Field Values

IO_STOREFILE_BLOOM_BLOCK_SIZE

public static final String IO_STOREFILE_BLOOM_BLOCK_SIZE

Target Bloom block size. Bloom filter blocks of approximately this size are interleaved with data blocks.

See Also:

Constant Field Values

MAX_ALLOWED_FOLD_FACTOR

private static final int MAX_ALLOWED_FOLD_FACTOR

Maximum number of times a Bloom filter can be "folded" if oversized

See Also:

Constant Field Values

Constructor Detail

BloomFilterFactory

private BloomFilterFactory()

This class should not be instantiated.

Method Detail

createFromMeta

public static BloomFilter createFromMeta(DataInput meta,
                                         HFile.Reader reader)
                                  throws IllegalArgumentException,
                                         IOException

Instantiates the correct Bloom filter class based on the version provided in the meta block data.

Parameters:

meta - the byte array holding the Bloom filter's metadata, including version information

reader - the HFile reader to use to lazily load Bloom filter blocks

Returns:

an instance of the correct type of Bloom filter

Throws:

IllegalArgumentException

IOException

isGeneralBloomEnabled

public static boolean isGeneralBloomEnabled(org.apache.hadoop.conf.Configuration conf)

Returns:

true if general Bloom (Row or RowCol) filters are enabled in the given configuration

isDeleteFamilyBloomEnabled

public static boolean isDeleteFamilyBloomEnabled(org.apache.hadoop.conf.Configuration conf)

Returns:

true if Delete Family Bloom filters are enabled in the given configuration

getErrorRate

public static float getErrorRate(org.apache.hadoop.conf.Configuration conf)

Returns:

the Bloom filter error rate in the given configuration

getMaxFold

public static int getMaxFold(org.apache.hadoop.conf.Configuration conf)

Returns:

the value for Bloom filter max fold in the given configuration

getBloomBlockSize

public static int getBloomBlockSize(org.apache.hadoop.conf.Configuration conf)

Returns:

the compound Bloom filter block size from the configuration

getMaxKeys

public static int getMaxKeys(org.apache.hadoop.conf.Configuration conf)

Returns:

max key for the Bloom filter from the configuration

createGeneralBloomAtWrite

public static BloomFilterWriter createGeneralBloomAtWrite(org.apache.hadoop.conf.Configuration conf,
                                                          CacheConfig cacheConf,
                                                          BloomType bloomType,
                                                          int maxKeys,
                                                          HFile.Writer writer)

Creates a new general (Row or RowCol) Bloom filter at the time of HStoreFile writing.

Parameters:

conf -

cacheConf -

bloomType -

maxKeys - an estimate of the number of keys we expect to insert. Irrelevant if compound Bloom filters are enabled.

writer - the HFile writer

Returns:

the new Bloom filter, or null in case Bloom filters are disabled or when failed to create one.

createDeleteBloomAtWrite

public static BloomFilterWriter createDeleteBloomAtWrite(org.apache.hadoop.conf.Configuration conf,
                                                         CacheConfig cacheConf,
                                                         int maxKeys,
                                                         HFile.Writer writer)

Creates a new Delete Family Bloom filter at the time of HStoreFile writing.

Parameters:

conf -

cacheConf -

maxKeys - an estimate of the number of keys we expect to insert. Irrelevant if compound Bloom filters are enabled.

writer - the HFile writer

Returns:

the new Bloom filter, or null in case Bloom filters are disabled or when failed to create one.