org.apache.hadoop.hbase.mob

Class DefaultMobStoreFlusher

java.lang.Object

org.apache.hadoop.hbase.regionserver.StoreFlusher

org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher

org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher

@InterfaceAudience.Private
public class DefaultMobStoreFlusher
extends DefaultStoreFlusher

An implementation of the StoreFlusher. It extends the DefaultStoreFlusher. If the store is not a mob store, the flusher flushes the MemStore the same with DefaultStoreFlusher, If the store is a mob store, the flusher flushes the MemStore into two places. One is the store files of HBase, the other is the mob files.

1. Cells that are not PUT type or have the delete mark will be directly flushed to HBase.
2. If the size of a cell value is larger than a threshold, it'll be flushed to a mob file, another cell with the path of this file will be flushed to HBase.
3. If the size of a cell value is smaller than or equal with a threshold, it'll be flushed to HBase directly.

Field Summary

Fields

Modifier and Type	Field and Description
private Object	flushLock
private static org.slf4j.Logger	LOG
private long	mobCellValueSizeThreshold
(package private) static ThreadLocal<Set<String>>	mobRefSet
private HMobStore	mobStore
private org.apache.hadoop.fs.Path	targetPath

Fields inherited from class org.apache.hadoop.hbase.regionserver.StoreFlusher

conf, store

Constructor Summary

Constructors

Constructor and Description
DefaultMobStoreFlusher(org.apache.hadoop.conf.Configuration conf, HStore store)

Method Summary

Modifier and Type	Method and Description
protected void	finalizeWriter(StoreFileWriter writer, long cacheFlushSeqNum, MonitoredTask status)
List<org.apache.hadoop.fs.Path>	flushSnapshot(MemStoreSnapshot snapshot, long cacheFlushId, MonitoredTask status, ThroughputController throughputController, FlushLifeCycleTracker tracker, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Flushes the snapshot of the MemStore.
private String	getStoreInfo()
protected void	performMobFlush(MemStoreSnapshot snapshot, long cacheFlushId, InternalScanner scanner, StoreFileWriter writer, MonitoredTask status, ThroughputController throughputController, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Flushes the cells in the mob store.

Methods inherited from class org.apache.hadoop.hbase.regionserver.StoreFlusher

createScanner, createWriter, performFlush

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

flushLock

private final Object flushLock

mobCellValueSizeThreshold

private long mobCellValueSizeThreshold

targetPath

private org.apache.hadoop.fs.Path targetPath

mobStore

private HMobStore mobStore

mobRefSet

static ThreadLocal<Set<String>> mobRefSet

Constructor Detail

DefaultMobStoreFlusher

public DefaultMobStoreFlusher(org.apache.hadoop.conf.Configuration conf,
                              HStore store)
                       throws IOException

Throws:

IOException

Method Detail

flushSnapshot

public List<org.apache.hadoop.fs.Path> flushSnapshot(MemStoreSnapshot snapshot,
                                                     long cacheFlushId,
                                                     MonitoredTask status,
                                                     ThroughputController throughputController,
                                                     FlushLifeCycleTracker tracker,
                                                     Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                                              throws IOException

Flushes the snapshot of the MemStore. If this store is not a mob store, flush the cells in the snapshot to store files of HBase. If the store is a mob one, the flusher flushes the MemStore into two places. One is the store files of HBase, the other is the mob files.

1. Cells that are not PUT type or have the delete mark will be directly flushed to HBase.
2. If the size of a cell value is larger than a threshold, it'll be flushed to a mob file, another cell with the path of this file will be flushed to HBase.
3. If the size of a cell value is smaller than or equal with a threshold, it'll be flushed to HBase directly.

Overrides:

flushSnapshot in class DefaultStoreFlusher

Parameters:

snapshot - Memstore snapshot.

cacheFlushId - Log cache flush sequence number.

status - Task that represents the flush operation and may be updated with status.

throughputController - A controller to avoid flush too fast

Returns:

List of files written. Can be empty; must not be null.

Throws:

IOException

performMobFlush

protected void performMobFlush(MemStoreSnapshot snapshot,
                               long cacheFlushId,
                               InternalScanner scanner,
                               StoreFileWriter writer,
                               MonitoredTask status,
                               ThroughputController throughputController,
                               Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                        throws IOException

Flushes the cells in the mob store.

In the mob store, the cells with PUT type might have or have no mob tags.
1. If a cell does not have a mob tag, flushing the cell to different files depends on the value length. If the length is larger than a threshold, it's flushed to a mob file and the mob file is flushed to a store file in HBase. Otherwise, directly flush the cell to a store file in HBase.
2. If a cell have a mob tag, its value is a mob file name, directly flush it to a store file in HBase.

Parameters:

snapshot - Memstore snapshot.

cacheFlushId - Log cache flush sequence number.

scanner - The scanner of memstore snapshot.

writer - The store file writer.

status - Task that represents the flush operation and may be updated with status.

throughputController - A controller to avoid flush too fast. n

Throws:

IOException

finalizeWriter

protected void finalizeWriter(StoreFileWriter writer,
                              long cacheFlushSeqNum,
                              MonitoredTask status)
                       throws IOException

Overrides:

finalizeWriter in class StoreFlusher

Throws:

IOException

getStoreInfo

private String getStoreInfo()