Package org.apache.hadoop.hbase.mob
Class DefaultMobStoreFlusher
java.lang.Object
org.apache.hadoop.hbase.regionserver.StoreFlusher
org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher
org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher
An implementation of the StoreFlusher. It extends the DefaultStoreFlusher. If the store is not a
mob store, the flusher flushes the MemStore the same with DefaultStoreFlusher, If the store is a
mob store, the flusher flushes the MemStore into two places. One is the store files of HBase, the
other is the mob files.
- Cells that are not PUT type or have the delete mark will be directly flushed to HBase.
- If the size of a cell value is larger than a threshold, it'll be flushed to a mob file, another cell with the path of this file will be flushed to HBase.
- If the size of a cell value is smaller than or equal with a threshold, it'll be flushed to HBase directly.
-
Field Summary
Modifier and TypeFieldDescriptionprivate final Object
private static final org.slf4j.Logger
private long
(package private) static ThreadLocal<Set<String>>
private HMobStore
private org.apache.hadoop.fs.Path
Fields inherited from class org.apache.hadoop.hbase.regionserver.StoreFlusher
conf, store
-
Constructor Summary
ConstructorDescriptionDefaultMobStoreFlusher
(org.apache.hadoop.conf.Configuration conf, HStore store) -
Method Summary
Modifier and TypeMethodDescriptionprotected void
finalizeWriter
(StoreFileWriter writer, long cacheFlushSeqNum, MonitoredTask status) List<org.apache.hadoop.fs.Path>
flushSnapshot
(MemStoreSnapshot snapshot, long cacheFlushId, MonitoredTask status, ThroughputController throughputController, FlushLifeCycleTracker tracker, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Flushes the snapshot of the MemStore.private String
protected void
performMobFlush
(MemStoreSnapshot snapshot, long cacheFlushId, InternalScanner scanner, StoreFileWriter writer, MonitoredTask status, ThroughputController throughputController, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Flushes the cells in the mob store.Methods inherited from class org.apache.hadoop.hbase.regionserver.StoreFlusher
createScanner, createWriter, performFlush
-
Field Details
-
LOG
-
flushLock
-
mobCellValueSizeThreshold
-
targetPath
-
mobStore
-
mobRefSet
-
-
Constructor Details
-
DefaultMobStoreFlusher
public DefaultMobStoreFlusher(org.apache.hadoop.conf.Configuration conf, HStore store) throws IOException - Throws:
IOException
-
-
Method Details
-
flushSnapshot
public List<org.apache.hadoop.fs.Path> flushSnapshot(MemStoreSnapshot snapshot, long cacheFlushId, MonitoredTask status, ThroughputController throughputController, FlushLifeCycleTracker tracker, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) throws IOException Flushes the snapshot of the MemStore. If this store is not a mob store, flush the cells in the snapshot to store files of HBase. If the store is a mob one, the flusher flushes the MemStore into two places. One is the store files of HBase, the other is the mob files.- Cells that are not PUT type or have the delete mark will be directly flushed to HBase.
- If the size of a cell value is larger than a threshold, it'll be flushed to a mob file, another cell with the path of this file will be flushed to HBase.
- If the size of a cell value is smaller than or equal with a threshold, it'll be flushed to HBase directly.
- Overrides:
flushSnapshot
in classDefaultStoreFlusher
- Parameters:
snapshot
- Memstore snapshot.cacheFlushId
- Log cache flush sequence number.status
- Task that represents the flush operation and may be updated with status.throughputController
- A controller to avoid flush too fast- Returns:
- List of files written. Can be empty; must not be null.
- Throws:
IOException
-
performMobFlush
protected void performMobFlush(MemStoreSnapshot snapshot, long cacheFlushId, InternalScanner scanner, StoreFileWriter writer, MonitoredTask status, ThroughputController throughputController, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) throws IOException Flushes the cells in the mob store.-
In the mob store, the cells with PUT type might have or have no mob tags.
- If a cell does not have a mob tag, flushing the cell to different files depends on the value length. If the length is larger than a threshold, it's flushed to a mob file and the mob file is flushed to a store file in HBase. Otherwise, directly flush the cell to a store file in HBase.
- If a cell have a mob tag, its value is a mob file name, directly flush it to a store file in HBase.
- Parameters:
snapshot
- Memstore snapshot.cacheFlushId
- Log cache flush sequence number.scanner
- The scanner of memstore snapshot.writer
- The store file writer.status
- Task that represents the flush operation and may be updated with status.throughputController
- A controller to avoid flush too fast.- Throws:
IOException
-
finalizeWriter
protected void finalizeWriter(StoreFileWriter writer, long cacheFlushSeqNum, MonitoredTask status) throws IOException - Overrides:
finalizeWriter
in classStoreFlusher
- Throws:
IOException
-
getStoreInfo
-