Package org.apache.hadoop.hbase.regionserver.throttle

Class StoreHotnessProtector

java.lang.Object
org.apache.hadoop.hbase.regionserver.throttle.StoreHotnessProtector
@Private public class StoreHotnessProtector extends Object
StoreHotnessProtector is designed to help limit the concurrency of puts with dense columns, it does best-effort to avoid exhausting all RS's handlers. When a lot of clients write requests with dense (hundreds) columns to a Store at the same time, it will lead to blocking of RS because CSLM degrades when concurrency goes up. It's not a kind of throttling. Throttling is user-oriented, while StoreHotnessProtector is system-oriented, RS-self-protected mechanism.

There are three key parameters:

1. parallelPutToStoreThreadLimitCheckMinColumnCount: If the amount of columns exceed this threshold, the HotProtector will work, 100 by default

2. parallelPutToStoreThreadLimit: The amount of concurrency allowed to write puts to a Store at the same time.

3. parallelPreparePutToStoreThreadLimit: The amount of concurrency allowed to prepare writing puts to a Store at the same time.

Notice that our writing pipeline includes three key process: MVCC acquire, writing MemStore, and WAL. Only limit the concurrency of writing puts to Store(parallelPutToStoreThreadLimit) is not enough since the actual concurrency of puts may still exceed the limit when MVCC contention or slow WAL sync happens. This is why parallelPreparePutToStoreThreadLimit is needed.

This protector is enabled by default and could be turned off by setting hbase.region.store.parallel.put.limit to 0, supporting online configuration change.