Package org.apache.hadoop.hbase.util

Class BloomFilterFactory

java.lang.Object

org.apache.hadoop.hbase.util.BloomFilterFactory

@Private
public final class BloomFilterFactory
extends Object

Handles Bloom filter initialization based on configuration and serialized metadata in the reader and writer of HStoreFile.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	IO_STOREFILE_BLOOM_BLOCK_SIZE	Target Bloom block size.
static final String	IO_STOREFILE_BLOOM_ENABLED	Master switch to enable Bloom filters
static final String	IO_STOREFILE_BLOOM_ERROR_RATE	Specifies the target error rate to use when selecting the number of keys per Bloom filter.
static final String	IO_STOREFILE_BLOOM_MAX_FOLD	Maximum folding factor allowed.
static final String	IO_STOREFILE_BLOOM_MAX_KEYS	For default (single-block) Bloom filters this specifies the maximum number of keys.
static final String	IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED	Master switch to enable Delete Family Bloom filters
private static final org.slf4j.Logger	LOG
private static final int	MAX_ALLOWED_FOLD_FACTOR	Maximum number of times a Bloom filter can be "folded" if oversized

Constructor Summary

Constructors

Modifier	Constructor	Description
private	BloomFilterFactory()	This class should not be instantiated.

Method Summary

Modifier and Type	Method	Description
static BloomFilterWriter	createDeleteBloomAtWrite(org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, int maxKeys, HFile.Writer writer)	Creates a new Delete Family Bloom filter at the time of HStoreFile writing.
static BloomFilter	createFromMeta(DataInput meta, HFile.Reader reader)	Instantiates the correct Bloom filter class based on the version provided in the meta block data.
static BloomFilter	createFromMeta(DataInput meta, HFile.Reader reader, BloomFilterMetrics metrics)
static BloomFilterWriter	createGeneralBloomAtWrite(org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType bloomType, int maxKeys, HFile.Writer writer)	Creates a new general (Row or RowCol) Bloom filter at the time of HStoreFile writing.
static int	getBloomBlockSize(org.apache.hadoop.conf.Configuration conf)	Returns the compound Bloom filter block size from the configuration
static float	getErrorRate(org.apache.hadoop.conf.Configuration conf)	Returns the Bloom filter error rate in the given configuration
static int	getMaxFold(org.apache.hadoop.conf.Configuration conf)	Returns the value for Bloom filter max fold in the given configuration
static int	getMaxKeys(org.apache.hadoop.conf.Configuration conf)	Returns max key for the Bloom filter from the configuration
static boolean	isDeleteFamilyBloomEnabled(org.apache.hadoop.conf.Configuration conf)	Returns true if Delete Family Bloom filters are enabled in the given configuration
static boolean	isGeneralBloomEnabled(org.apache.hadoop.conf.Configuration conf)	Returns true if general Bloom (Row or RowCol) filters are enabled in the given configuration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

IO_STOREFILE_BLOOM_ERROR_RATE

public static final String IO_STOREFILE_BLOOM_ERROR_RATE

Specifies the target error rate to use when selecting the number of keys per Bloom filter.

See Also:

• Constant Field Values

IO_STOREFILE_BLOOM_MAX_FOLD

public static final String IO_STOREFILE_BLOOM_MAX_FOLD

Maximum folding factor allowed. The Bloom filter will be shrunk by the factor of up to 2 ** this times if we oversize it initially.

See Also:

• Constant Field Values

IO_STOREFILE_BLOOM_MAX_KEYS

public static final String IO_STOREFILE_BLOOM_MAX_KEYS

For default (single-block) Bloom filters this specifies the maximum number of keys.

See Also:

• Constant Field Values

IO_STOREFILE_BLOOM_ENABLED

public static final String IO_STOREFILE_BLOOM_ENABLED

Master switch to enable Bloom filters

See Also:

• Constant Field Values

IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED

public static final String IO_STOREFILE_DELETEFAMILY_BLOOM_ENABLED

Master switch to enable Delete Family Bloom filters

See Also:

• Constant Field Values

IO_STOREFILE_BLOOM_BLOCK_SIZE

public static final String IO_STOREFILE_BLOOM_BLOCK_SIZE

Target Bloom block size. Bloom filter blocks of approximately this size are interleaved with data blocks.

See Also:

• Constant Field Values

MAX_ALLOWED_FOLD_FACTOR

private static final int MAX_ALLOWED_FOLD_FACTOR

Maximum number of times a Bloom filter can be "folded" if oversized

See Also:

• Constant Field Values

Constructor Details

BloomFilterFactory

private BloomFilterFactory()

This class should not be instantiated.

Method Details

createFromMeta

public static BloomFilter createFromMeta(DataInput meta,
 HFile.Reader reader)
                                  throws IllegalArgumentException,
IOException

Instantiates the correct Bloom filter class based on the version provided in the meta block data.

Parameters:

meta - the byte array holding the Bloom filter's metadata, including version information

reader - the HFile reader to use to lazily load Bloom filter blocks

Returns:

an instance of the correct type of Bloom filter

Throws:

IllegalArgumentException

IOException

createFromMeta

public static BloomFilter createFromMeta(DataInput meta,
 HFile.Reader reader,
 BloomFilterMetrics metrics)
                                  throws IllegalArgumentException,
IOException

Throws:

IllegalArgumentException

IOException

isGeneralBloomEnabled

public static boolean isGeneralBloomEnabled(org.apache.hadoop.conf.Configuration conf)

Returns true if general Bloom (Row or RowCol) filters are enabled in the given configuration

isDeleteFamilyBloomEnabled

public static boolean isDeleteFamilyBloomEnabled(org.apache.hadoop.conf.Configuration conf)

Returns true if Delete Family Bloom filters are enabled in the given configuration

getErrorRate

public static float getErrorRate(org.apache.hadoop.conf.Configuration conf)

Returns the Bloom filter error rate in the given configuration

getMaxFold

public static int getMaxFold(org.apache.hadoop.conf.Configuration conf)

Returns the value for Bloom filter max fold in the given configuration

getBloomBlockSize

public static int getBloomBlockSize(org.apache.hadoop.conf.Configuration conf)

Returns the compound Bloom filter block size from the configuration

getMaxKeys

public static int getMaxKeys(org.apache.hadoop.conf.Configuration conf)

Returns max key for the Bloom filter from the configuration

createGeneralBloomAtWrite

public static BloomFilterWriter createGeneralBloomAtWrite(org.apache.hadoop.conf.Configuration conf,
 CacheConfig cacheConf,
 BloomType bloomType,
 int maxKeys,
 HFile.Writer writer)

Creates a new general (Row or RowCol) Bloom filter at the time of HStoreFile writing.

Parameters:

maxKeys - an estimate of the number of keys we expect to insert. Irrelevant if compound Bloom filters are enabled.

writer - the HFile writer

Returns:

the new Bloom filter, or null in case Bloom filters are disabled or when failed to create one.

createDeleteBloomAtWrite

public static BloomFilterWriter createDeleteBloomAtWrite(org.apache.hadoop.conf.Configuration conf,
 CacheConfig cacheConf,
 int maxKeys,
 HFile.Writer writer)

Creates a new Delete Family Bloom filter at the time of HStoreFile writing.

Parameters:

maxKeys - an estimate of the number of keys we expect to insert. Irrelevant if compound Bloom filters are enabled.

writer - the HFile writer

Returns:

the new Bloom filter, or null in case Bloom filters are disabled or when failed to create one.