MemStoreFlusher (Apache HBase 2.2.3 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.MemStoreFlusher

All Implemented Interfaces:

FlushRequester
```
@InterfaceAudience.Private
class MemStoreFlusher
extends Object
implements FlushRequester
```
Thread that flushes cache on request NOTE: This class extends Thread rather than Chore because the sleep time can be interrupted when there is something to do, rather than the Chore sleep time which is invariant.

See Also:

FlushRequester

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`private class`	`MemStoreFlusher.FlushHandler`
`(package private) static interface`	`MemStoreFlusher.FlushQueueEntry`
`(package private) static class`	`MemStoreFlusher.FlushRegionEntry` Datastructure used in the flush queue.

Field Summary

Fields
Modifier and Type	Field and Description
`private long`	`blockingWaitTime`
`private Object`	`blockSignal`
`private org.apache.hadoop.conf.Configuration`	`conf`
`private MemStoreFlusher.FlushHandler[]`	`flushHandlers`
`private BlockingQueue<MemStoreFlusher.FlushQueueEntry>`	`flushQueue`
`private List<FlushRequestListener>`	`flushRequestListeners`
`private FlushType`	`flushType`
`private ReentrantReadWriteLock`	`lock`
`private static org.slf4j.Logger`	`LOG`
`private Map<Region,MemStoreFlusher.FlushRegionEntry>`	`regionsInQueue`
`private HRegionServer`	`server`
`private long`	`threadWakeFrequency`
`private LongAdder`	`updatesBlockedMsHighWater`
`private static MemStoreFlusher.FlushQueueEntry`	`WAKEUPFLUSH_INSTANCE` Singleton instance inserted into flush queue used for signaling.
`private AtomicBoolean`	`wakeupPending`

Constructor Summary

Constructors
Constructor and Description

MemStoreFlusher(org.apache.hadoop.conf.Configuration conf, HRegionServer server)

Constructors
Constructor and Description
`MemStoreFlusher(org.apache.hadoop.conf.Configuration conf, HRegionServer server)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`String`	`dumpQueue()`
`private boolean`	`flushOneForGlobalPressure()` The memstore across all regions has exceeded the low water mark.
`private boolean`	`flushRegion(HRegion region, boolean emergencyFlush, boolean forceFlushAllStores, FlushLifeCycleTracker tracker)` Flush a region.
`private boolean`	`flushRegion(MemStoreFlusher.FlushRegionEntry fqe)` A flushRegion that checks store file count.
`private HRegion`	`getBiggestMemStoreOfRegionReplica(SortedMap<Long,HRegion> regionsBySize, Set<HRegion> excludedRegions)`
`private HRegion`	`getBiggestMemStoreRegion(SortedMap<Long,HRegion> regionsBySize, Set<HRegion> excludedRegions, boolean checkStoreFileCount)`
`int`	`getFlushQueueSize()`
`private static long`	`getMemStoreDataSize(HRegion r)`
`private static long`	`getMemStoreHeapSize(HRegion r)`
`private static long`	`getMemStoreOffHeapSize(HRegion r)`
`private int`	`getStoreFileCount(Region region)`
`LongAdder`	`getUpdatesBlockedMsHighWater()`
`(package private) void`	`interruptIfNecessary()` Only interrupt once it's done with a run through the work loop.
`private FlushType`	`isAboveHighWaterMark()` Return true if global memory usage is above the high watermark
`private FlushType`	`isAboveLowWaterMark()` Return true if we're above the low watermark
`(package private) boolean`	`isAlive()`
`private boolean`	`isTooManyStoreFiles(Region region)`
`(package private) void`	`join()`
`private void`	`logMsg(String type, long val, long max)`
`private void`	`notifyFlushRequest(Region region, boolean emergencyFlush)`
`void`	`reclaimMemStoreMemory()` Check if the regionserver's memstore memory usage is greater than the limit.
`private boolean`	`refreshStoreFilesAndReclaimMemory(Region region)`
`void`	`registerFlushRequestListener(FlushRequestListener listener)` Register a MemstoreFlushListener
`boolean`	`requestDelayedFlush(HRegion r, long delay, boolean forceFlushAllStores)` Tell the listener the cache needs to be flushed after a delay
`boolean`	`requestFlush(HRegion r, boolean forceFlushAllStores, FlushLifeCycleTracker tracker)` Tell the listener the cache needs to be flushed.
`void`	`setFlushType(FlushType flushType)`
`void`	`setGlobalMemStoreLimit(long globalMemStoreSize)` Sets the global memstore limit to a new size.
`(package private) void`	`start(Thread.UncaughtExceptionHandler eh)`
`String`	`toString()`
`boolean`	`unregisterFlushRequestListener(FlushRequestListener listener)` Unregister the listener from MemstoreFlushListeners
`private void`	`wakeupFlushThread()`
`private void`	`wakeUpIfBlocking()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.slf4j.Logger LOG
```
  - conf
```
private org.apache.hadoop.conf.Configuration conf
```
  - flushQueue
```
private final BlockingQueue<MemStoreFlusher.FlushQueueEntry> flushQueue
```
  - regionsInQueue
```
private final Map<Region,MemStoreFlusher.FlushRegionEntry> regionsInQueue
```
  - wakeupPending
```
private AtomicBoolean wakeupPending
```
  - threadWakeFrequency
```
private final long threadWakeFrequency
```
  - server
```
private final HRegionServer server
```
  - lock
```
private final ReentrantReadWriteLock lock
```
  - blockSignal
```
private final Object blockSignal
```
  - blockingWaitTime
```
private long blockingWaitTime
```
  - updatesBlockedMsHighWater
```
private final LongAdder updatesBlockedMsHighWater
```
  - flushHandlers
```
private final MemStoreFlusher.FlushHandler[] flushHandlers
```
  - flushRequestListeners
```
private List<FlushRequestListener> flushRequestListeners
```
  - flushType
```
private FlushType flushType
```
  - WAKEUPFLUSH_INSTANCE
```
private static final MemStoreFlusher.FlushQueueEntry WAKEUPFLUSH_INSTANCE
```
    Singleton instance inserted into flush queue used for signaling.
- Constructor Detail
  - MemStoreFlusher
```
public MemStoreFlusher(org.apache.hadoop.conf.Configuration conf,
                       HRegionServer server)
```
    Parameters:
    
    conf -
    
    server -
- Method Detail
  - getUpdatesBlockedMsHighWater
```
public LongAdder getUpdatesBlockedMsHighWater()
```
  - setFlushType
```
public void setFlushType(FlushType flushType)
```
  - flushOneForGlobalPressure
```
private boolean flushOneForGlobalPressure()
```
    The memstore across all regions has exceeded the low water mark. Pick one region to flush and flush it synchronously (this is called from the flush thread)
    
    Returns:
    
    true if successful
  - getMemStoreOffHeapSize
```
private static long getMemStoreOffHeapSize(HRegion r)
```
    Returns:
    
    Return memstore offheap size or null if r is null
  - getMemStoreHeapSize
```
private static long getMemStoreHeapSize(HRegion r)
```
    Returns:
    
    Return memstore heap size or null if r is null
  - getMemStoreDataSize
```
private static long getMemStoreDataSize(HRegion r)
```
    Returns:
    
    Return memstore data size or null if r is null
  - wakeupFlushThread
```
private void wakeupFlushThread()
```
  - getBiggestMemStoreRegion
```
private HRegion getBiggestMemStoreRegion(SortedMap<Long,HRegion> regionsBySize,
                                         Set<HRegion> excludedRegions,
                                         boolean checkStoreFileCount)
```
  - getBiggestMemStoreOfRegionReplica
```
private HRegion getBiggestMemStoreOfRegionReplica(SortedMap<Long,HRegion> regionsBySize,
                                                  Set<HRegion> excludedRegions)
```
  - refreshStoreFilesAndReclaimMemory
```
private boolean refreshStoreFilesAndReclaimMemory(Region region)
```
  - isAboveHighWaterMark
```
private FlushType isAboveHighWaterMark()
```
    Return true if global memory usage is above the high watermark
  - isAboveLowWaterMark
```
private FlushType isAboveLowWaterMark()
```
    Return true if we're above the low watermark
  - requestFlush
```
public boolean requestFlush(HRegion r,
                            boolean forceFlushAllStores,
                            FlushLifeCycleTracker tracker)
```
    Description copied from interface: FlushRequester
    
    Tell the listener the cache needs to be flushed.
    
    Specified by:
    
    requestFlush in interface FlushRequester
    
    Parameters:
    
    r - the Region requesting the cache flush
    
    forceFlushAllStores - whether we want to flush all stores. e.g., when request from log rolling.
    
    Returns:
    
    true if our region is added into the queue, false otherwise
  - requestDelayedFlush
```
public boolean requestDelayedFlush(HRegion r,
                                   long delay,
                                   boolean forceFlushAllStores)
```
    Description copied from interface: FlushRequester
    
    Tell the listener the cache needs to be flushed after a delay
    
    Specified by:
    
    requestDelayedFlush in interface FlushRequester
    
    Parameters:
    
    r - the Region requesting the cache flush
    
    delay - after how much time should the flush happen
    
    forceFlushAllStores - whether we want to flush all stores. e.g., when request from log rolling.
    
    Returns:
    
    true if our region is added into the queue, false otherwise
  - getFlushQueueSize
```
public int getFlushQueueSize()
```
  - interruptIfNecessary
```
void interruptIfNecessary()
```
    Only interrupt once it's done with a run through the work loop.
  - start
```
void start(Thread.UncaughtExceptionHandler eh)
```
  - isAlive
```
boolean isAlive()
```
  - join
```
void join()
```
  - flushRegion
```
private boolean flushRegion(MemStoreFlusher.FlushRegionEntry fqe)
```
    A flushRegion that checks store file count. If too many, puts the flush on delay queue to retry later.
    
    Parameters:
    
    fqe -
    
    Returns:
    
    true if the region was successfully flushed, false otherwise. If false, there will be accompanying log messages explaining why the region was not flushed.
  - flushRegion
```
private boolean flushRegion(HRegion region,
                            boolean emergencyFlush,
                            boolean forceFlushAllStores,
                            FlushLifeCycleTracker tracker)
```
    Flush a region.
    
    Parameters:
    
    region - Region to flush.
    
    emergencyFlush - Set if we are being force flushed. If true the region needs to be removed from the flush queue. If false, when we were called from the main flusher run loop and we got the entry to flush by calling poll on the flush queue (which removed it).
    
    forceFlushAllStores - whether we want to flush all store.
    
    Returns:
    
    true if the region was successfully flushed, false otherwise. If false, there will be accompanying log messages explaining why the region was not flushed.
  - notifyFlushRequest
```
private void notifyFlushRequest(Region region,
                                boolean emergencyFlush)
```
  - wakeUpIfBlocking
```
private void wakeUpIfBlocking()
```
  - isTooManyStoreFiles
```
private boolean isTooManyStoreFiles(Region region)
```
  - getStoreFileCount
```
private int getStoreFileCount(Region region)
```
  - reclaimMemStoreMemory
```
public void reclaimMemStoreMemory()
```
    Check if the regionserver's memstore memory usage is greater than the limit. If so, flush regions with the biggest memstores until we're down to the lower limit. This method blocks callers until we're down to a safe amount of memstore consumption.
  - logMsg
```
private void logMsg(String type,
                    long val,
                    long max)
```
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - dumpQueue
```
public String dumpQueue()
```
  - registerFlushRequestListener
```
public void registerFlushRequestListener(FlushRequestListener listener)
```
    Register a MemstoreFlushListener
    
    Specified by:
    
    registerFlushRequestListener in interface FlushRequester
    
    Parameters:
    
    listener -
  - unregisterFlushRequestListener
```
public boolean unregisterFlushRequestListener(FlushRequestListener listener)
```
    Unregister the listener from MemstoreFlushListeners
    
    Specified by:
    
    unregisterFlushRequestListener in interface FlushRequester
    
    Parameters:
    
    listener -
    
    Returns:
    
    true when passed listener is unregistered successfully.
  - setGlobalMemStoreLimit
```
public void setGlobalMemStoreLimit(long globalMemStoreSize)
```
    Sets the global memstore limit to a new size.
    
    Specified by:
    
    setGlobalMemStoreLimit in interface FlushRequester
    
    Parameters:
    
    globalMemStoreSize -

Class MemStoreFlusher

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

conf

flushQueue

regionsInQueue

wakeupPending

threadWakeFrequency

server

lock

blockSignal

blockingWaitTime

updatesBlockedMsHighWater

flushHandlers

flushRequestListeners

flushType

WAKEUPFLUSH_INSTANCE

Constructor Detail

MemStoreFlusher

Method Detail

getUpdatesBlockedMsHighWater

setFlushType

flushOneForGlobalPressure

getMemStoreOffHeapSize

getMemStoreHeapSize

getMemStoreDataSize

wakeupFlushThread

getBiggestMemStoreRegion

getBiggestMemStoreOfRegionReplica

refreshStoreFilesAndReclaimMemory

isAboveHighWaterMark

isAboveLowWaterMark

requestFlush

requestDelayedFlush

getFlushQueueSize

interruptIfNecessary

start

isAlive

join

flushRegion

flushRegion

notifyFlushRequest

wakeUpIfBlocking

isTooManyStoreFiles

getStoreFileCount

reclaimMemStoreMemory

logMsg

toString

dumpQueue

registerFlushRequestListener

unregisterFlushRequestListener

setGlobalMemStoreLimit