Class StoreHotnessProtector
There are three key parameters:
1. parallelPutToStoreThreadLimitCheckMinColumnCount: If the amount of columns exceed this threshold, the HotProtector will work, 100 by default
2. parallelPutToStoreThreadLimit: The amount of concurrency allowed to write puts to a Store at the same time.
3. parallelPreparePutToStoreThreadLimit: The amount of concurrency allowed to prepare writing puts to a Store at the same time.
Notice that our writing pipeline includes three key process: MVCC acquire, writing MemStore, and WAL. Only limit the concurrency of writing puts to Store(parallelPutToStoreThreadLimit) is not enough since the actual concurrency of puts may still exceed the limit when MVCC contention or slow WAL sync happens. This is why parallelPreparePutToStoreThreadLimit is needed.
This protector is enabled by default and could be turned off by setting hbase.region.store.parallel.put.limit to 0, supporting online configuration change.
-
Field Summary
Modifier and TypeFieldDescriptionprivate static final int
private static final int
private static final int
static final long
private static final org.slf4j.Logger
private static boolean
static final String
static final String
static final String
private int
private int
private int
private final ConcurrentMap<byte[],
AtomicInteger> private final Region
-
Constructor Summary
ConstructorDescriptionStoreHotnessProtector
(Region region, org.apache.hadoop.conf.Configuration conf) -
Method Summary
Modifier and TypeMethodDescriptionvoid
(package private) Map<byte[],
AtomicInteger> void
init
(org.apache.hadoop.conf.Configuration conf) boolean
isEnable()
private static void
init(Configuration)
is called for every Store that opens on a RegionServer.void
toString()
void
update
(org.apache.hadoop.conf.Configuration conf)
-
Field Details
-
LOG
-
loggedDisableMessage
-
parallelPutToStoreThreadLimit
-
parallelPreparePutToStoreThreadLimit
-
PARALLEL_PUT_STORE_THREADS_LIMIT
- See Also:
-
PARALLEL_PREPARE_PUT_STORE_MULTIPLIER
- See Also:
-
DEFAULT_PARALLEL_PUT_STORE_THREADS_LIMIT
- See Also:
-
parallelPutToStoreThreadLimitCheckMinColumnCount
-
PARALLEL_PUT_STORE_THREADS_LIMIT_MIN_COLUMN_COUNT
- See Also:
-
DEFAULT_PARALLEL_PUT_STORE_THREADS_LIMIT_MIN_COLUMN_NUM
- See Also:
-
DEFAULT_PARALLEL_PREPARE_PUT_STORE_MULTIPLIER
- See Also:
-
preparePutToStoreMap
-
region
-
FIXED_SIZE
-
-
Constructor Details
-
StoreHotnessProtector
-
-
Method Details
-
init
-
logDisabledMessageOnce
init(Configuration)
is called for every Store that opens on a RegionServer. Here we make a lightweight attempt to log this message once per RegionServer, rather than per-Store. The goal is just to draw attention to this feature if debugging overload due to heavy writes. -
update
-
start
- Throws:
RegionTooBusyException
-
finish
-
toString
-
isEnable
-
getPreparePutToStoreMap
Map<byte[],AtomicInteger> getPreparePutToStoreMap()
-