CompactSplitThread (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.CompactSplitThread

All Implemented Interfaces:

ConfigurationObserver, PropagatingConfigurationObserver, CompactionRequestor
```
@InterfaceAudience.Private
public class CompactSplitThread
extends Object
implements CompactionRequestor, PropagatingConfigurationObserver
```
Compact region on request and then run split if appropriate

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`private class`	`CompactSplitThread.CompactionRunner`
`private static class`	`CompactSplitThread.Rejection` Cleanup class to use when rejecting a compaction request from the queue.

Field Summary

Fields
Modifier and Type	Field and Description
`private CompactionThroughputController`	`compactionThroughputController`
`private org.apache.hadoop.conf.Configuration`	`conf`
`static int`	`DEFAULT_REGION_SERVER_REGION_SPLIT_LIMIT`
`static String`	`LARGE_COMPACTION_THREADS`
`static int`	`LARGE_COMPACTION_THREADS_DEFAULT`
`private static org.apache.commons.logging.Log`	`LOG`
`private ThreadPoolExecutor`	`longCompactions`
`static String`	`MERGE_THREADS`
`static int`	`MERGE_THREADS_DEFAULT`
`private ThreadPoolExecutor`	`mergePool`
`static String`	`REGION_SERVER_REGION_SPLIT_LIMIT`
`private int`	`regionSplitLimit` Splitting should not take place if the total number of regions exceed this.
`private HRegionServer`	`server`
`private ThreadPoolExecutor`	`shortCompactions`
`static String`	`SMALL_COMPACTION_THREADS`
`static int`	`SMALL_COMPACTION_THREADS_DEFAULT`
`static String`	`SPLIT_THREADS`
`static int`	`SPLIT_THREADS_DEFAULT`
`private ThreadPoolExecutor`	`splits`

Constructor Summary

Constructors
Constructor and Description

CompactSplitThread(HRegionServer server)

Constructors
Constructor and Description
`CompactSplitThread(HRegionServer server)`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`deregisterChildren(ConfigurationManager manager)` Needs to be called to deregister the children from the manager.
`String`	`dumpQueue()`
`int`	`getCompactionQueueSize()` Returns the current size of the queue containing regions that are processed.
`CompactionThroughputController`	`getCompactionThroughputController()`
`int`	`getLargeCompactionQueueSize()`
`protected int`	`getLargeCompactionThreadNum()`
`protected int`	`getMergeThreadNum()`
`int`	`getRegionSplitLimit()`
`int`	`getSmallCompactionQueueSize()`
`protected int`	`getSmallCompactionThreadNum()`
`int`	`getSplitQueueSize()`
`protected int`	`getSplitThreadNum()`
`(package private) void`	`interruptIfNecessary()` Only interrupt once it's done with a run through the work loop.
`(package private) void`	`join()`
`void`	`onConfigurationChange(org.apache.hadoop.conf.Configuration newConf)` This method would be called by the `ConfigurationManager` object when the `Configuration` object is reloaded from disk.
`void`	`registerChildren(ConfigurationManager manager)` Needs to be called to register the children to the manager.
`CompactionRequest`	`requestCompaction(Region r, Store s, String why, CompactionRequest request)`
`CompactionRequest`	`requestCompaction(Region r, Store s, String why, int priority, CompactionRequest request, User user)`
`List<CompactionRequest>`	`requestCompaction(Region r, String why)`
`List<CompactionRequest>`	`requestCompaction(Region r, String why, int p, List<Pair<CompactionRequest,Store>> requests, User user)`
`List<CompactionRequest>`	`requestCompaction(Region r, String why, List<Pair<CompactionRequest,Store>> requests)`
`private CompactionRequest`	`requestCompactionInternal(Region r, Store s, String why, int priority, CompactionRequest request, boolean selectNow, User user)`
`private List<CompactionRequest>`	`requestCompactionInternal(Region r, String why, int p, List<Pair<CompactionRequest,Store>> requests, boolean selectNow, User user)`
`void`	`requestRegionsMerge(Region a, Region b, boolean forcible, long masterSystemTime, User user)`
`boolean`	`requestSplit(Region r)`
`void`	`requestSplit(Region r, byte[] midKey)`
`void`	`requestSplit(Region r, byte[] midKey, User user)`
`void`	`requestSystemCompaction(Region r, Store s, String why)`
`void`	`requestSystemCompaction(Region r, String why)`
`private CompactionContext`	`selectCompaction(Region r, Store s, int priority, CompactionRequest request, User user)`
`private boolean`	`shouldSplitRegion()`
`(package private) void`	`shutdownLongCompactions()`
`String`	`toString()`
`private void`	`waitFor(ThreadPoolExecutor t, String name)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.apache.commons.logging.Log LOG

LARGE_COMPACTION_THREADS

public static final String LARGE_COMPACTION_THREADS

See Also:: Constant Field Values

LARGE_COMPACTION_THREADS_DEFAULT
```
public static final int LARGE_COMPACTION_THREADS_DEFAULT
```
See Also:
Constant Field Values

SMALL_COMPACTION_THREADS

public static final String SMALL_COMPACTION_THREADS

See Also:: Constant Field Values

SMALL_COMPACTION_THREADS_DEFAULT
```
public static final int SMALL_COMPACTION_THREADS_DEFAULT
```
See Also:
Constant Field Values

SPLIT_THREADS

public static final String SPLIT_THREADS

See Also:: Constant Field Values

SPLIT_THREADS_DEFAULT
```
public static final int SPLIT_THREADS_DEFAULT
```
See Also:
Constant Field Values

MERGE_THREADS

public static final String MERGE_THREADS

See Also:: Constant Field Values

MERGE_THREADS_DEFAULT
```
public static final int MERGE_THREADS_DEFAULT
```
See Also:
Constant Field Values

REGION_SERVER_REGION_SPLIT_LIMIT

public static final String REGION_SERVER_REGION_SPLIT_LIMIT

See Also:: Constant Field Values

DEFAULT_REGION_SERVER_REGION_SPLIT_LIMIT
```
public static final int DEFAULT_REGION_SERVER_REGION_SPLIT_LIMIT
```
See Also:
Constant Field Values

server
```
private final HRegionServer server
```

conf

private final org.apache.hadoop.conf.Configuration conf

longCompactions

private final ThreadPoolExecutor longCompactions

shortCompactions

private final ThreadPoolExecutor shortCompactions

splits

private final ThreadPoolExecutor splits

mergePool

private final ThreadPoolExecutor mergePool

compactionThroughputController

private volatile CompactionThroughputController compactionThroughputController

regionSplitLimit
```
private int regionSplitLimit
```
Splitting should not take place if the total number of regions exceed this. This is not a hard limit to the number of regions but it is a guideline to stop splitting after number of online regions is greater than this.

Constructor Detail
- CompactSplitThread
```
CompactSplitThread(HRegionServer server)
```
  Parameters:
  server -

Method Detail

toString
```
public String toString()
```
Overrides:

toString in class Object

dumpQueue
```
public String dumpQueue()
```

requestRegionsMerge

public void requestRegionsMerge(Region a,
                       Region b,
                       boolean forcible,
                       long masterSystemTime,
                       User user)

requestSplit

public boolean requestSplit(Region r)

requestSplit

public void requestSplit(Region r,
                byte[] midKey)

requestSplit

public void requestSplit(Region r,
                byte[] midKey,
                User user)

requestCompaction
```
public List<CompactionRequest> requestCompaction(Region r,
                                        String why)
                                          throws IOException
```
Specified by:

requestCompaction in interface CompactionRequestor

Parameters:
r - Region to compact
why - Why compaction was requested -- used in debug messages

Returns:
The created CompactionRequests or an empty list if no compactions were started

Throws:

IOException

requestCompaction
```
public List<CompactionRequest> requestCompaction(Region r,
                                        String why,
                                        List<Pair<CompactionRequest,Store>> requests)
                                          throws IOException
```
Specified by:

requestCompaction in interface CompactionRequestor

Parameters:
r - Region to compact
why - Why compaction was requested -- used in debug messages
requests - custom compaction requests. Each compaction must specify the store on which it is acting. Can be null in which case a compaction will be attempted on all stores for the region.

Returns:
The created CompactionRequests or an empty list if no compactions were started

Throws:

IOException

requestCompaction
```
public CompactionRequest requestCompaction(Region r,
                                  Store s,
                                  String why,
                                  CompactionRequest request)
                                    throws IOException
```
Specified by:

requestCompaction in interface CompactionRequestor

Parameters:
r - Region to compact
s - Store within region to compact
why - Why compaction was requested -- used in debug messages
request - custom compaction request for the Region and Store. Custom request must be null or be constructed with matching region and store.

Returns:
The created CompactionRequest or null if no compaction was started.

Throws:

IOException

requestCompaction
```
public List<CompactionRequest> requestCompaction(Region r,
                                        String why,
                                        int p,
                                        List<Pair<CompactionRequest,Store>> requests,
                                        User user)
                                          throws IOException
```
Specified by:

requestCompaction in interface CompactionRequestor

Parameters:
r - Region to compact
why - Why compaction was requested -- used in debug messages
p - Priority of this compaction. minHeap. <=0 is critical
requests - custom compaction requests. Each compaction must specify the store on which it is acting. Can be null in which case a compaction will be attempted on all stores for the region.
user - the effective user

Returns:
The created CompactionRequests or an empty list if no compactions were started.

Throws:

IOException

requestCompactionInternal

private List<CompactionRequest> requestCompactionInternal(Region r,
                                                String why,
                                                int p,
                                                List<Pair<CompactionRequest,Store>> requests,
                                                boolean selectNow,
                                                User user)
                                                   throws IOException

Throws:: IOException

requestCompaction

public CompactionRequest requestCompaction(Region r,
                                  Store s,
                                  String why,
                                  int priority,
                                  CompactionRequest request,
                                  User user)
                                    throws IOException

Specified by:: requestCompaction in interface CompactionRequestor
Parameters:: r - Region to compact; s - Store within region to compact; why - Why compaction was requested -- used in debug messages; priority - Priority of this compaction. minHeap. <=0 is critical; request - custom compaction request to run. Store and Region for the request must match the region and store specified here.
Returns:: The created CompactionRequest or null if no compaction was started
Throws:: IOException

requestSystemCompaction

public void requestSystemCompaction(Region r,
                           String why)
                             throws IOException

Throws:: IOException

requestSystemCompaction

public void requestSystemCompaction(Region r,
                           Store s,
                           String why)
                             throws IOException

Throws:: IOException

requestCompactionInternal

private CompactionRequest requestCompactionInternal(Region r,
                                          Store s,
                                          String why,
                                          int priority,
                                          CompactionRequest request,
                                          boolean selectNow,
                                          User user)
                                             throws IOException

Parameters:: r - region store belongs to; s - Store to request compaction on; why - Why compaction requested -- used in debug messages; priority - override the default priority (NO_PRIORITY == decide); request - custom compaction request. Can be null in which case a simple compaction will be used.
Throws:: IOException

selectCompaction

private CompactionContext selectCompaction(Region r,
                                 Store s,
                                 int priority,
                                 CompactionRequest request,
                                 User user)
                                    throws IOException

Throws:: IOException

interruptIfNecessary
```
void interruptIfNecessary()
```
Only interrupt once it's done with a run through the work loop.

waitFor

private void waitFor(ThreadPoolExecutor t,
           String name)

join
```
void join()
```

getCompactionQueueSize
```
public int getCompactionQueueSize()
```
Returns the current size of the queue containing regions that are processed.

Returns:
The current size of the regions queue.

getLargeCompactionQueueSize

public int getLargeCompactionQueueSize()

getSmallCompactionQueueSize

public int getSmallCompactionQueueSize()

getSplitQueueSize
```
public int getSplitQueueSize()
```

shouldSplitRegion
```
private boolean shouldSplitRegion()
```

getRegionSplitLimit
```
public int getRegionSplitLimit()
```
Returns:
the regionSplitLimit

onConfigurationChange
```
public void onConfigurationChange(org.apache.hadoop.conf.Configuration newConf)
```
This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

getSmallCompactionThreadNum

protected int getSmallCompactionThreadNum()

getLargeCompactionThreadNum

protected int getLargeCompactionThreadNum()

getSplitThreadNum
```
protected int getSplitThreadNum()
```

getMergeThreadNum
```
protected int getMergeThreadNum()
```

registerChildren
```
public void registerChildren(ConfigurationManager manager)
```
Needs to be called to register the children to the manager.

Specified by:

registerChildren in interface PropagatingConfigurationObserver

Parameters:
manager - : to register to

deregisterChildren
```
public void deregisterChildren(ConfigurationManager manager)
```
Needs to be called to deregister the children from the manager.

Specified by:

deregisterChildren in interface PropagatingConfigurationObserver

Parameters:
manager - : to deregister from

getCompactionThroughputController

public CompactionThroughputController getCompactionThroughputController()

shutdownLongCompactions
```
void shutdownLongCompactions()
```

Class CompactSplitThread

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

LARGE_COMPACTION_THREADS

LARGE_COMPACTION_THREADS_DEFAULT

SMALL_COMPACTION_THREADS

SMALL_COMPACTION_THREADS_DEFAULT

SPLIT_THREADS

SPLIT_THREADS_DEFAULT

MERGE_THREADS

MERGE_THREADS_DEFAULT

REGION_SERVER_REGION_SPLIT_LIMIT

DEFAULT_REGION_SERVER_REGION_SPLIT_LIMIT

server

conf

longCompactions

shortCompactions

splits

mergePool

compactionThroughputController

regionSplitLimit

Constructor Detail

CompactSplitThread

Method Detail

toString

dumpQueue

requestRegionsMerge

requestSplit

requestSplit

requestSplit

requestCompaction

requestCompaction

requestCompaction

requestCompaction

requestCompactionInternal

requestCompaction

requestSystemCompaction

requestSystemCompaction

requestCompactionInternal

selectCompaction

interruptIfNecessary

waitFor

join

getCompactionQueueSize

getLargeCompactionQueueSize

getSmallCompactionQueueSize

getSplitQueueSize

shouldSplitRegion

getRegionSplitLimit

onConfigurationChange

getSmallCompactionThreadNum

getLargeCompactionThreadNum

getSplitThreadNum

getMergeThreadNum

registerChildren

deregisterChildren

getCompactionThroughputController

shutdownLongCompactions