MasterProcedureScheduler (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler

All Implemented Interfaces:

ProcedureRunnableSet
```
@InterfaceAudience.Private
@InterfaceStability.Evolving
public class MasterProcedureScheduler
extends Object
implements ProcedureRunnableSet
```
ProcedureRunnableSet for the Master Procedures. This RunnableSet tries to provide to the ProcedureExecutor procedures that can be executed without having to wait on a lock. Most of the master operations can be executed concurrently, if they are operating on different tables (e.g. two create table can be performed at the same, time assuming table A and table B) or against two different servers; say two servers that crashed at about the same time.
Each procedure should implement an interface providing information for this queue. for example table related procedures should implement TableProcedureInterface. each procedure will be pushed in its own queue, and based on the operation type we may take smarter decision. e.g. we can abort all the operations preceding a delete table, or similar.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`private static class`	`MasterProcedureScheduler.AvlTree`
`private static class`	`MasterProcedureScheduler.FairQueue<T extends Comparable<T>>`
`private static class`	`MasterProcedureScheduler.IterableList`
`static class`	`MasterProcedureScheduler.ProcedureEvent`
`private static class`	`MasterProcedureScheduler.Queue<TKey extends Comparable<TKey>>`
`private static class`	`MasterProcedureScheduler.QueueImpl<TKey extends Comparable<TKey>>`
`private static interface`	`MasterProcedureScheduler.QueueInterface`
`static class`	`MasterProcedureScheduler.ServerQueue`
`static class`	`MasterProcedureScheduler.TableQueue`

Field Summary

Fields
Modifier and Type	Field and Description
`private TableLockManager`	`lockManager`
`private static org.apache.commons.logging.Log`	`LOG`
`private int`	`metaTablePriority`
`private MasterProcedureScheduler.Queue<String>`	`namespaceMap`
`private long`	`nullPollCalls`
`private long`	`pollCalls`
`private int`	`queueSize`
`private ReentrantLock`	`schedLock`
`private Condition`	`schedWaitCond`
`private Object[]`	`serverBuckets`
`private MasterProcedureScheduler.FairQueue<ServerName>`	`serverRunQueue`
`private int`	`sysTablePriority`
`private MasterProcedureScheduler.Queue<TableName>`	`tableMap`
`private MasterProcedureScheduler.FairQueue<TableName>`	`tableRunQueue`
`private int`	`userTablePriority`

Constructor Summary

Constructors
Constructor and Description

MasterProcedureScheduler(org.apache.hadoop.conf.Configuration conf, TableLockManager lockManager)

Constructors
Constructor and Description
`MasterProcedureScheduler(org.apache.hadoop.conf.Configuration conf, TableLockManager lockManager)`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`addBack(Procedure proc)` Inserts the specified element at the end of this queue.
`void`	`addFront(Procedure proc)` Inserts the specified element at the front of this queue.
`private <T extends Comparable<T>> void`	`addToRunQueue(MasterProcedureScheduler.FairQueue<T> fairq, MasterProcedureScheduler.Queue<T> queue)`
`void`	`clear()` Removes all of the elements from this collection.
`private <T extends Comparable<T>> void`	`clear(MasterProcedureScheduler.Queue<T> treeMap, MasterProcedureScheduler.FairQueue<T> fairq)`
`void`	`completionCleanup(Procedure proc)` The procedure in execution completed.
`private <T extends Comparable<T>> void`	`doAdd(MasterProcedureScheduler.FairQueue<T> fairq, MasterProcedureScheduler.Queue<T> queue, Procedure proc, boolean addFront)`
`private void`	`doAdd(Procedure proc, boolean addFront)`
`private <T extends Comparable<T>> Procedure`	`doPoll(MasterProcedureScheduler.FairQueue<T> fairq)`
`private static int`	`getBucketIndex(Object[] buckets, int hashCode)`
`long`	`getNullPollCalls()`
`long`	`getPollCalls()`
`private static ServerName`	`getServerName(Procedure proc)`
`private MasterProcedureScheduler.ServerQueue`	`getServerQueue(ServerName serverName)`
`private MasterProcedureScheduler.ServerQueue`	`getServerQueueWithLock(ServerName serverName)`
`private static TableName`	`getTableName(Procedure proc)`
`private int`	`getTablePriority(TableName tableName)`
`private MasterProcedureScheduler.TableQueue`	`getTableQueue(TableName tableName)`
`private MasterProcedureScheduler.TableQueue`	`getTableQueueWithLock(TableName tableName)`
`private static <T extends Comparable<T>> MasterProcedureScheduler.Queue<T>`	`getTreeRoot(Object[] buckets, int index)`
`private static boolean`	`isServerProcedure(Procedure proc)`
`private static boolean`	`isTableProcedure(Procedure proc)`
`protected boolean`	`markTableAsDeleted(TableName table)` Tries to remove the queue and the table-lock of the specified table.
`Procedure`	`poll()` Fetch one Procedure from the queue
`(package private) Procedure`	`poll(long waitNsec)`
`void`	`releaseServerExclusiveLock(Procedure procedure, ServerName serverName)` Release the exclusive lock
`void`	`releaseServerSharedLock(Procedure procedure, ServerName serverName)` Release the shared lock taken
`void`	`releaseTableExclusiveLock(Procedure procedure, TableName table)` Release the exclusive lock taken with tryAcquireTableWrite()
`void`	`releaseTableSharedLock(Procedure procedure, TableName table)` Release the shared lock taken with tryAcquireTableRead()
`private <T extends Comparable<T>> void`	`removeFromRunQueue(MasterProcedureScheduler.FairQueue<T> fairq, MasterProcedureScheduler.Queue<T> queue)`
`private void`	`removeServerQueue(ServerName serverName)`
`private void`	`removeTableQueue(TableName tableName)`
`void`	`signalAll()` In case the class is blocking on poll() waiting for items to be added, this method should awake poll() and poll() should return.
`int`	`size()` Returns the number of elements in this collection.
`void`	`suspend(MasterProcedureScheduler.ProcedureEvent event)`
`private void`	`suspendServerQueue(MasterProcedureScheduler.ProcedureEvent event, ServerName serverName)`
`private void`	`suspendTableQueue(MasterProcedureScheduler.ProcedureEvent event, TableName tableName)`
`boolean`	`tryAcquireServerExclusiveLock(Procedure procedure, ServerName serverName)` Try to acquire the exclusive lock on the specified server.
`boolean`	`tryAcquireServerSharedLock(Procedure procedure, ServerName serverName)` Try to acquire the shared lock on the specified server.
`boolean`	`tryAcquireTableExclusiveLock(Procedure procedure, TableName table)` Try to acquire the exclusive lock on the specified table.
`private MasterProcedureScheduler.TableQueue`	`tryAcquireTableQueueSharedLock(Procedure procedure, TableName table)`
`boolean`	`tryAcquireTableSharedLock(Procedure procedure, TableName table)` Try to acquire the shared lock on the specified table.
`boolean`	`waitEvent(MasterProcedureScheduler.ProcedureEvent event, Procedure procedure)`
`boolean`	`waitEvent(MasterProcedureScheduler.ProcedureEvent event, Procedure procedure, boolean suspendQueue)`
`void`	`wake(MasterProcedureScheduler.ProcedureEvent event)`
`void`	`yield(Procedure proc)` The procedure can't run at the moment.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.apache.commons.logging.Log LOG

lockManager

private final TableLockManager lockManager

schedLock
```
private final ReentrantLock schedLock
```

schedWaitCond
```
private final Condition schedWaitCond
```

serverRunQueue

private final MasterProcedureScheduler.FairQueue<ServerName> serverRunQueue

tableRunQueue

private final MasterProcedureScheduler.FairQueue<TableName> tableRunQueue

queueSize
```
private int queueSize
```

serverBuckets
```
private final Object[] serverBuckets
```

namespaceMap

private MasterProcedureScheduler.Queue<String> namespaceMap

tableMap

private MasterProcedureScheduler.Queue<TableName> tableMap

metaTablePriority
```
private final int metaTablePriority
```

userTablePriority
```
private final int userTablePriority
```

sysTablePriority
```
private final int sysTablePriority
```

pollCalls
```
private long pollCalls
```

nullPollCalls
```
private long nullPollCalls
```

Constructor Detail

MasterProcedureScheduler

public MasterProcedureScheduler(org.apache.hadoop.conf.Configuration conf,
                        TableLockManager lockManager)

Method Detail

addFront
```
public void addFront(Procedure proc)
```
Description copied from interface: ProcedureRunnableSet

Inserts the specified element at the front of this queue.

Specified by:

addFront in interface ProcedureRunnableSet

Parameters:
proc - the Procedure to add

addBack
```
public void addBack(Procedure proc)
```
Description copied from interface: ProcedureRunnableSet

Inserts the specified element at the end of this queue.

Specified by:

addBack in interface ProcedureRunnableSet

Parameters:
proc - the Procedure to add

yield
```
public void yield(Procedure proc)
```
Description copied from interface: ProcedureRunnableSet

The procedure can't run at the moment. add it back to the queue, giving priority to someone else.

Specified by:

yield in interface ProcedureRunnableSet

Parameters:
proc - the Procedure to add back to the list

doAdd

private void doAdd(Procedure proc,
         boolean addFront)

doAdd

private <T extends Comparable<T>> void doAdd(MasterProcedureScheduler.FairQueue<T> fairq,
                                   MasterProcedureScheduler.Queue<T> queue,
                                   Procedure proc,
                                   boolean addFront)

poll
```
public Procedure poll()
```
Description copied from interface: ProcedureRunnableSet

Fetch one Procedure from the queue

Specified by:

poll in interface ProcedureRunnableSet

Returns:
the Procedure to execute, or null if nothing present.

poll
```
Procedure poll(long waitNsec)
```

doPoll

private <T extends Comparable<T>> Procedure doPoll(MasterProcedureScheduler.FairQueue<T> fairq)

clear
```
public void clear()
```
Description copied from interface: ProcedureRunnableSet

Removes all of the elements from this collection.

Specified by:

clear in interface ProcedureRunnableSet

clear

private <T extends Comparable<T>> void clear(MasterProcedureScheduler.Queue<T> treeMap,
                                   MasterProcedureScheduler.FairQueue<T> fairq)

signalAll
```
public void signalAll()
```
Description copied from interface: ProcedureRunnableSet

In case the class is blocking on poll() waiting for items to be added, this method should awake poll() and poll() should return.

Specified by:

signalAll in interface ProcedureRunnableSet

size
```
public int size()
```
Description copied from interface: ProcedureRunnableSet

Returns the number of elements in this collection.

Specified by:

size in interface ProcedureRunnableSet

Returns:
the number of elements in this collection.

completionCleanup
```
public void completionCleanup(Procedure proc)
```
Description copied from interface: ProcedureRunnableSet

The procedure in execution completed. This can be implemented to perform cleanups.

Specified by:

completionCleanup in interface ProcedureRunnableSet

Parameters:
proc - the Procedure that completed the execution.

addToRunQueue

private <T extends Comparable<T>> void addToRunQueue(MasterProcedureScheduler.FairQueue<T> fairq,
                                           MasterProcedureScheduler.Queue<T> queue)

removeFromRunQueue

private <T extends Comparable<T>> void removeFromRunQueue(MasterProcedureScheduler.FairQueue<T> fairq,
                                                MasterProcedureScheduler.Queue<T> queue)

getPollCalls
```
public long getPollCalls()
```

getNullPollCalls
```
public long getNullPollCalls()
```

waitEvent

public boolean waitEvent(MasterProcedureScheduler.ProcedureEvent event,
                Procedure procedure)

waitEvent

public boolean waitEvent(MasterProcedureScheduler.ProcedureEvent event,
                Procedure procedure,
                boolean suspendQueue)

suspendTableQueue

private void suspendTableQueue(MasterProcedureScheduler.ProcedureEvent event,
                     TableName tableName)

suspendServerQueue

private void suspendServerQueue(MasterProcedureScheduler.ProcedureEvent event,
                      ServerName serverName)

suspend

public void suspend(MasterProcedureScheduler.ProcedureEvent event)

wake

public void wake(MasterProcedureScheduler.ProcedureEvent event)

getTableQueueWithLock

private MasterProcedureScheduler.TableQueue getTableQueueWithLock(TableName tableName)

getTableQueue

private MasterProcedureScheduler.TableQueue getTableQueue(TableName tableName)

removeTableQueue

private void removeTableQueue(TableName tableName)

getTablePriority

private int getTablePriority(TableName tableName)

isTableProcedure

private static boolean isTableProcedure(Procedure proc)

getTableName

private static TableName getTableName(Procedure proc)

getServerQueueWithLock

private MasterProcedureScheduler.ServerQueue getServerQueueWithLock(ServerName serverName)

getServerQueue

private MasterProcedureScheduler.ServerQueue getServerQueue(ServerName serverName)

removeServerQueue

private void removeServerQueue(ServerName serverName)

getTreeRoot

private static <T extends Comparable<T>> MasterProcedureScheduler.Queue<T> getTreeRoot(Object[] buckets,
                                                                      int index)

getBucketIndex

private static int getBucketIndex(Object[] buckets,
                 int hashCode)

isServerProcedure

private static boolean isServerProcedure(Procedure proc)

getServerName

private static ServerName getServerName(Procedure proc)

tryAcquireTableExclusiveLock
```
public boolean tryAcquireTableExclusiveLock(Procedure procedure,
                                   TableName table)
```
Try to acquire the exclusive lock on the specified table. other operations in the table-queue will be executed after the lock is released.

Parameters:
procedure - the procedure trying to acquire the lock
table - Table to lock

Returns:
true if we were able to acquire the lock on the table, otherwise false.

releaseTableExclusiveLock
```
public void releaseTableExclusiveLock(Procedure procedure,
                             TableName table)
```
Release the exclusive lock taken with tryAcquireTableWrite()

Parameters:
procedure - the procedure releasing the lock
table - the name of the table that has the exclusive lock

tryAcquireTableSharedLock
```
public boolean tryAcquireTableSharedLock(Procedure procedure,
                                TableName table)
```
Try to acquire the shared lock on the specified table. other "read" operations in the table-queue may be executed concurrently,

Parameters:
procedure - the procedure trying to acquire the lock
table - Table to lock

Returns:
true if we were able to acquire the lock on the table, otherwise false.

tryAcquireTableQueueSharedLock

private MasterProcedureScheduler.TableQueue tryAcquireTableQueueSharedLock(Procedure procedure,
                                                                 TableName table)

releaseTableSharedLock
```
public void releaseTableSharedLock(Procedure procedure,
                          TableName table)
```
Release the shared lock taken with tryAcquireTableRead()

Parameters:
procedure - the procedure releasing the lock
table - the name of the table that has the shared lock

markTableAsDeleted
```
protected boolean markTableAsDeleted(TableName table)
```
Tries to remove the queue and the table-lock of the specified table. If there are new operations pending (e.g. a new create), the remove will not be performed.

Parameters:
table - the name of the table that should be marked as deleted

Returns:
true if deletion succeeded, false otherwise meaning that there are other new operations pending for that table (e.g. a new create).

tryAcquireServerExclusiveLock
```
public boolean tryAcquireServerExclusiveLock(Procedure procedure,
                                    ServerName serverName)
```
Try to acquire the exclusive lock on the specified server.

Parameters:
procedure - the procedure trying to acquire the lock
serverName - Server to lock

Returns:
true if we were able to acquire the lock on the server, otherwise false.
See Also:
releaseServerExclusiveLock(Procedure,ServerName)

releaseServerExclusiveLock
```
public void releaseServerExclusiveLock(Procedure procedure,
                              ServerName serverName)
```
Release the exclusive lock

Parameters:
procedure - the procedure releasing the lock
serverName - the server that has the exclusive lock
See Also:
tryAcquireServerExclusiveLock(Procedure,ServerName)

tryAcquireServerSharedLock
```
public boolean tryAcquireServerSharedLock(Procedure procedure,
                                 ServerName serverName)
```
Try to acquire the shared lock on the specified server.

Parameters:
procedure - the procedure releasing the lock
serverName - Server to lock

Returns:
true if we were able to acquire the lock on the server, otherwise false.
See Also:
releaseServerSharedLock(Procedure,ServerName)

releaseServerSharedLock
```
public void releaseServerSharedLock(Procedure procedure,
                           ServerName serverName)
```
Release the shared lock taken

Parameters:
procedure - the procedure releasing the lock
serverName - the server that has the shared lock
See Also:
tryAcquireServerSharedLock(Procedure,ServerName)

Class MasterProcedureScheduler

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

lockManager

schedLock

schedWaitCond

serverRunQueue

tableRunQueue

queueSize

serverBuckets

namespaceMap

tableMap

metaTablePriority

userTablePriority

sysTablePriority

pollCalls

nullPollCalls

Constructor Detail

MasterProcedureScheduler

Method Detail

addFront

addBack

yield

doAdd

doAdd

poll

poll

doPoll

clear

clear

signalAll

size

completionCleanup

addToRunQueue

removeFromRunQueue

getPollCalls

getNullPollCalls

waitEvent

waitEvent

suspendTableQueue

suspendServerQueue

suspend

wake

getTableQueueWithLock

getTableQueue

removeTableQueue

getTablePriority

isTableProcedure

getTableName

getServerQueueWithLock

getServerQueue

removeServerQueue

getTreeRoot

getBucketIndex

isServerProcedure

getServerName

tryAcquireTableExclusiveLock

releaseTableExclusiveLock

tryAcquireTableSharedLock

tryAcquireTableQueueSharedLock

releaseTableSharedLock

markTableAsDeleted

tryAcquireServerExclusiveLock

releaseServerExclusiveLock

tryAcquireServerSharedLock

releaseServerSharedLock