org.apache.hadoop.hbase.regionserver

Class StoreFileWriter

java.lang.Object

org.apache.hadoop.hbase.regionserver.StoreFileWriter

All Implemented Interfaces:

CellSink, ShipperListener

@InterfaceAudience.Private
public class StoreFileWriter
extends Object
implements CellSink, ShipperListener

A StoreFile writer. Use this to read/write HBase Store Files. It is package local because it is an implementation detail of the HBase regionserver.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StoreFileWriter.Builder

Field Summary

Fields

Modifier and Type	Field and Description
private BloomContext	bloomContext
private BloomType	bloomType
private static Pattern	dash
private BloomContext	deleteFamilyBloomContext
private BloomFilterWriter	deleteFamilyBloomFilterWriter
private long	deleteFamilyCnt
private long	earliestPutTs
private BloomFilterWriter	generalBloomFilterWriter
private static org.slf4j.Logger	LOG
private TimeRangeTracker	timeRangeTracker
protected HFile.Writer	writer

Constructor Summary

Constructors

Modifier	Constructor and Description
private	StoreFileWriter(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, CellComparator comparator, BloomType bloomType, long maxKeys, InetSocketAddress[] favoredNodes, HFileContext fileContext, boolean shouldDropCacheBehind) Creates an HFile.Writer that also write helpful meta data.

Method Summary

Modifier and Type	Method and Description
void	append(Cell cell) Append the given cell
private void	appendDeleteFamilyBloomFilter(Cell cell)
void	appendFileInfo(byte[] key, byte[] value)
private void	appendGeneralBloomfilter(Cell cell)
void	appendMetadata(long maxSequenceId, boolean majorCompaction) Writes meta data.
void	appendMetadata(long maxSequenceId, boolean majorCompaction, long mobCellsCount) Writes meta data.
void	appendTrackedTimestampsToMetadata() Add TimestampRange and earliest put timestamp to Metadata
void	beforeShipped() The action that needs to be performed before Shipper.shipped() is performed
void	close()
private boolean	closeBloomFilter(BloomFilterWriter bfw)
private boolean	closeDeleteFamilyBloomFilter()
private boolean	closeGeneralBloomFilter()
(package private) BloomFilterWriter	getGeneralBloomWriter() For unit testing only.
(package private) HFile.Writer	getHFileWriter() For use in testing.
org.apache.hadoop.fs.Path	getPath()
(package private) static org.apache.hadoop.fs.Path	getUniqueFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir)
boolean	hasGeneralBloom()
void	trackTimestamps(Cell cell) Record the earlest Put timestamp.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

dash

private static final Pattern dash

generalBloomFilterWriter

private final BloomFilterWriter generalBloomFilterWriter

deleteFamilyBloomFilterWriter

private final BloomFilterWriter deleteFamilyBloomFilterWriter

bloomType

private final BloomType bloomType

earliestPutTs

private long earliestPutTs

deleteFamilyCnt

private long deleteFamilyCnt

bloomContext

private BloomContext bloomContext

deleteFamilyBloomContext

private BloomContext deleteFamilyBloomContext

timeRangeTracker

private final TimeRangeTracker timeRangeTracker

writer

protected HFile.Writer writer

Constructor Detail

StoreFileWriter

private StoreFileWriter(org.apache.hadoop.fs.FileSystem fs,
                        org.apache.hadoop.fs.Path path,
                        org.apache.hadoop.conf.Configuration conf,
                        CacheConfig cacheConf,
                        CellComparator comparator,
                        BloomType bloomType,
                        long maxKeys,
                        InetSocketAddress[] favoredNodes,
                        HFileContext fileContext,
                        boolean shouldDropCacheBehind)
                 throws IOException

Creates an HFile.Writer that also write helpful meta data.

Parameters:

fs - file system to write to

path - file name to create

conf - user configuration

comparator - key comparator

bloomType - bloom filter setting

maxKeys - the expected maximum number of keys to be added. Was used for Bloom filter size in HFile format version 1.

favoredNodes -

fileContext - - The HFile context

shouldDropCacheBehind - Drop pages written to page cache after writing the store file.

Throws:

IOException - problem writing to FS

Method Detail

appendMetadata

public void appendMetadata(long maxSequenceId,
                           boolean majorCompaction)
                    throws IOException

Writes meta data. Call before close() since its written as meta data to this file.

Parameters:

maxSequenceId - Maximum sequence id.

majorCompaction - True if this file is product of a major compaction

Throws:

IOException - problem writing to FS

appendMetadata

public void appendMetadata(long maxSequenceId,
                           boolean majorCompaction,
                           long mobCellsCount)
                    throws IOException

Writes meta data. Call before close() since its written as meta data to this file.

Parameters:

maxSequenceId - Maximum sequence id.

majorCompaction - True if this file is product of a major compaction

mobCellsCount - The number of mob cells.

Throws:

IOException - problem writing to FS

appendTrackedTimestampsToMetadata

public void appendTrackedTimestampsToMetadata()
                                       throws IOException

Add TimestampRange and earliest put timestamp to Metadata

Throws:

IOException

trackTimestamps

public void trackTimestamps(Cell cell)

Record the earlest Put timestamp. If the timeRangeTracker is not set, update TimeRangeTracker to include the timestamp of this key

appendGeneralBloomfilter

private void appendGeneralBloomfilter(Cell cell)
                               throws IOException

Throws:

IOException

appendDeleteFamilyBloomFilter

private void appendDeleteFamilyBloomFilter(Cell cell)
                                    throws IOException

Throws:

IOException

append

public void append(Cell cell)
            throws IOException

Description copied from interface: CellSink

Append the given cell

Specified by:

append in interface CellSink

Parameters:

cell - the cell to be added

Throws:

IOException

beforeShipped

public void beforeShipped()
                   throws IOException

Description copied from interface: ShipperListener

The action that needs to be performed before Shipper.shipped() is performed

Specified by:

beforeShipped in interface ShipperListener

Throws:

IOException

getPath

public org.apache.hadoop.fs.Path getPath()

hasGeneralBloom

public boolean hasGeneralBloom()

getGeneralBloomWriter

BloomFilterWriter getGeneralBloomWriter()

For unit testing only.

Returns:

the Bloom filter used by this writer.

closeBloomFilter

private boolean closeBloomFilter(BloomFilterWriter bfw)
                          throws IOException

Throws:

IOException

closeGeneralBloomFilter

private boolean closeGeneralBloomFilter()
                                 throws IOException

Throws:

IOException

closeDeleteFamilyBloomFilter

private boolean closeDeleteFamilyBloomFilter()
                                      throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Throws:

IOException

appendFileInfo

public void appendFileInfo(byte[] key,
                           byte[] value)
                    throws IOException

Throws:

IOException

getHFileWriter

HFile.Writer getHFileWriter()

For use in testing.

getUniqueFile

static org.apache.hadoop.fs.Path getUniqueFile(org.apache.hadoop.fs.FileSystem fs,
                                               org.apache.hadoop.fs.Path dir)
                                        throws IOException

Parameters:

fs -

dir - Directory to create file in.

Returns:

random filename inside passed dir

Throws:

IOException