ProcedureExecutor (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.procedure2.ProcedureExecutor<TEnvironment>

```
@InterfaceAudience.Private
@InterfaceStability.Evolving
public class ProcedureExecutor<TEnvironment>
extends Object
```
Thread Pool that executes the submitted procedures. The executor has a ProcedureStore associated. Each operation is logged and on restart the pending procedures are resumed. Unless the Procedure code throws an error (e.g. invalid user input) the procedure will complete (at some point in time), On restart the pending procedures are resumed and the once failed will be rolledback. The user can add procedures to the executor via submitProcedure(proc) check for the finished state via isFinished(procId) and get the result via getResult(procId)

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`private static class`	`ProcedureExecutor.CompletedProcedureCleaner<TEnvironment>` Internal cleaner that removes the completed procedure results after a TTL.
`static interface`	`ProcedureExecutor.ProcedureExecutorListener`
`private static class`	`ProcedureExecutor.ProcedureTimeoutRetriever` Used by the TimeoutBlockingQueue to get the timeout interval of the procedure
`static class`	`ProcedureExecutor.Testing`

Field Summary

Fields
Modifier and Type	Field and Description
`private AtomicInteger`	`activeExecutorCount`
`private ConcurrentHashMap<Long,ProcedureInfo>`	`completed` Map the the procId returned by submitProcedure(), the Root-ProcID, to the ProcedureInfo.
`private org.apache.hadoop.conf.Configuration`	`conf`
`private TEnvironment`	`environment`
`private AtomicLong`	`lastProcId`
`private CopyOnWriteArrayList<ProcedureExecutor.ProcedureExecutorListener>`	`listeners`
`private static org.apache.commons.logging.Log`	`LOG`
`private ConcurrentHashMap<NonceKey,Long>`	`nonceKeysToProcIdsMap` Helper map to lookup whether the procedure already issued from the same client.
`private ConcurrentHashMap<Long,Procedure>`	`procedures` Helper map to lookup the live procedures by ID.
`private ConcurrentHashMap<Long,RootProcedureState>`	`rollbackStack` Map the the procId returned by submitProcedure(), the Root-ProcID, to the RootProcedureState.
`private ProcedureRunnableSet`	`runnables` Queue that contains runnable procedures.
`private AtomicBoolean`	`running`
`private ProcedureStore`	`store`
`private ReentrantLock`	`submitLock`
`(package private) ProcedureExecutor.Testing`	`testing`
`private Thread[]`	`threads`
`private TimeoutBlockingQueue<Procedure>`	`waitingTimeout` Timeout Queue that contains Procedures in a WAITING_TIMEOUT state or periodic procedures.

Constructor Summary

Constructors
Constructor and Description
`ProcedureExecutor(org.apache.hadoop.conf.Configuration conf, TEnvironment environment, ProcedureStore store)`
`ProcedureExecutor(org.apache.hadoop.conf.Configuration conf, TEnvironment environment, ProcedureStore store, ProcedureRunnableSet runqueue)`

Method Summary

Methods
Modifier and Type	Method and Description
`boolean`	`abort(long procId)` Send an abort notification the specified procedure.
`boolean`	`abort(long procId, boolean mayInterruptIfRunning)` Send an abort notification the specified procedure.
`NonceKey`	`createNonceKey(long nonceGroup, long nonce)` Create a NoneKey from the specified nonceGroup and nonce.
`private void`	`execLoop()` Execution loop (N threads) while the executor is in a running state, fetch a procedure from the runnables queue and start the execution.
`private void`	`execLoop(Procedure proc)`
`private void`	`execProcedure(RootProcedureState procStack, Procedure procedure)` Executes the specified procedure - calls the doExecute() of the procedure - if the procedure execution didn't fail (e.g.
`private boolean`	`executeRollback(long rootProcId, RootProcedureState procStack)` Execute the rollback of the full procedure stack.
`private boolean`	`executeRollback(Procedure proc)` Execute the rollback of the procedure step.
`int`	`getActiveExecutorCount()`
`TEnvironment`	`getEnvironment()`
`int`	`getNumThreads()`
`Procedure`	`getProcedure(long procId)`
`ProcedureInfo`	`getResult(long procId)`
`Pair<ProcedureInfo,Procedure>`	`getResultOrProcedure(long procId)`
`Map<Long,ProcedureInfo>`	`getResults()`
`private Long`	`getRootProcedureId(Procedure proc)`
`protected ProcedureRunnableSet`	`getRunnableSet()`
`ProcedureStore`	`getStore()`
`private void`	`handleInterruptedException(Procedure proc, InterruptedException e)`
`boolean`	`isFinished(long procId)` Return true if the procedure is finished.
`boolean`	`isProcedureOwner(long procId, User user)` Check if the user is this procedure's owner
`boolean`	`isRunning()`
`boolean`	`isStarted(long procId)` Return true if the procedure is started.
`void`	`join()`
`List<ProcedureInfo>`	`listProcedures()` List procedures.
`private void`	`load(boolean abortOnCorruption)`
`private void`	`loadProcedures(ProcedureStore.ProcedureIterator procIter, boolean abortOnCorruption)`
`private long`	`nextProcId()`
`private void`	`procedureFinished(Procedure proc)`
`void`	`registerListener(ProcedureExecutor.ProcedureExecutorListener listener)`
`long`	`registerNonce(NonceKey nonceKey)` Register a nonce for a procedure that is going to be submitted.
`void`	`removeResult(long procId)` Mark the specified completed procedure, as ready to remove.
`private void`	`sendProcedureAddedNotification(long procId)`
`private void`	`sendProcedureFinishedNotification(long procId)`
`private void`	`sendProcedureLoadedNotification(long procId)`
`void`	`setFailureResultForNonce(NonceKey nonceKey, String procName, User procOwner, IOException exception)` If the failure failed before submitting it, we may want to give back the same error to the requests with the same nonceKey.
`void`	`start(int numThreads, boolean abortOnCorruption)` Start the procedure executor.
`void`	`stop()`
`long`	`submitProcedure(Procedure proc)` >>>>>>> ce33cf2...
`long`	`submitProcedure(Procedure proc, NonceKey nonceKey)` Add a new root-procedure to the executor.
`private void`	`timeoutLoop()`
`boolean`	`unregisterListener(ProcedureExecutor.ProcedureExecutorListener listener)`
`void`	`unregisterNonceIfProcedureWasNotSubmitted(NonceKey nonceKey)` Remove the NonceKey if the procedure was not submitted to the executor.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.apache.commons.logging.Log LOG
```
  - testing
```
ProcedureExecutor.Testing testing
```
  - completed
```
private final ConcurrentHashMap<Long,ProcedureInfo> completed
```
    Map the the procId returned by submitProcedure(), the Root-ProcID, to the ProcedureInfo. Once a Root-Procedure completes (success or failure), the result will be added to this map. The user of ProcedureExecutor should call getResult(procId) to get the result.
  - rollbackStack
```
private final ConcurrentHashMap<Long,RootProcedureState> rollbackStack
```
    Map the the procId returned by submitProcedure(), the Root-ProcID, to the RootProcedureState. The RootProcedureState contains the execution stack of the Root-Procedure, It is added to the map by submitProcedure() and removed on procedure completion.
  - procedures
```
private final ConcurrentHashMap<Long,Procedure> procedures
```
    Helper map to lookup the live procedures by ID. This map contains every procedure. root-procedures and subprocedures.
  - nonceKeysToProcIdsMap
```
private ConcurrentHashMap<NonceKey,Long> nonceKeysToProcIdsMap
```
    Helper map to lookup whether the procedure already issued from the same client. This map contains every root procedure.
  - waitingTimeout
```
private final TimeoutBlockingQueue<Procedure> waitingTimeout
```
    Timeout Queue that contains Procedures in a WAITING_TIMEOUT state or periodic procedures.
  - runnables
```
private final ProcedureRunnableSet runnables
```
    Queue that contains runnable procedures.
  - submitLock
```
private final ReentrantLock submitLock
```
  - lastProcId
```
private final AtomicLong lastProcId
```
  - listeners
```
private final CopyOnWriteArrayList<ProcedureExecutor.ProcedureExecutorListener> listeners
```
  - activeExecutorCount
```
private final AtomicInteger activeExecutorCount
```
  - running
```
private final AtomicBoolean running
```
  - environment
```
private final TEnvironment environment
```
  - store
```
private final ProcedureStore store
```
  - conf
```
private final org.apache.hadoop.conf.Configuration conf
```
  - threads
```
private Thread[] threads
```
- Constructor Detail
  - ProcedureExecutor
```
public ProcedureExecutor(org.apache.hadoop.conf.Configuration conf,
                 TEnvironment environment,
                 ProcedureStore store)
```
  - ProcedureExecutor
```
public ProcedureExecutor(org.apache.hadoop.conf.Configuration conf,
                 TEnvironment environment,
                 ProcedureStore store,
                 ProcedureRunnableSet runqueue)
```
- Method Detail
  - load
```
private void load(boolean abortOnCorruption)
           throws IOException
```
    Throws:
    
    IOException
  - loadProcedures
```
private void loadProcedures(ProcedureStore.ProcedureIterator procIter,
                  boolean abortOnCorruption)
                     throws IOException
```
    Throws:
    
    IOException
  - start
```
public void start(int numThreads,
         boolean abortOnCorruption)
           throws IOException
```
    Start the procedure executor. It calls ProcedureStore.recoverLease() and ProcedureStore.load() to recover the lease, and ensure a single executor, and start the procedure replay to resume and recover the previous pending and in-progress perocedures.
    
    Parameters:
    numThreads - number of threads available for procedure execution.
    abortOnCorruption - true if you want to abort your service in case a corrupted procedure is found on replay. otherwise false.
    
    Throws:
    
    IOException
  - stop
```
public void stop()
```
  - join
```
public void join()
```
  - isRunning
```
public boolean isRunning()
```
  - getNumThreads
```
public int getNumThreads()
```
    Returns:
    the number of execution threads.
  - getActiveExecutorCount
```
public int getActiveExecutorCount()
```
  - getEnvironment
```
public TEnvironment getEnvironment()
```
  - getStore
```
public ProcedureStore getStore()
```
  - registerListener
```
public void registerListener(ProcedureExecutor.ProcedureExecutorListener listener)
```
  - unregisterListener
```
public boolean unregisterListener(ProcedureExecutor.ProcedureExecutorListener listener)
```
  - listProcedures
```
public List<ProcedureInfo> listProcedures()
```
    List procedures.
    
    Returns:
    the procedures in a list
  - createNonceKey
```
public NonceKey createNonceKey(long nonceGroup,
                      long nonce)
```
    Create a NoneKey from the specified nonceGroup and nonce.
    
    Parameters:
    nonceGroup -
    nonce -
    
    Returns:
    the generated NonceKey
  - registerNonce
```
public long registerNonce(NonceKey nonceKey)
```
    Register a nonce for a procedure that is going to be submitted. A procId will be reserved and on submitProcedure(), the procedure with the specified nonce will take the reserved ProcId. If someone already reserved the nonce, this method will return the procId reserved, otherwise an invalid procId will be returned. and the caller should procede and submit the procedure.
    
    Parameters:
    nonceKey - A unique identifier for this operation from the client or process.
    
    Returns:
    the procId associated with the nonce, if any otherwise an invalid procId.
  - unregisterNonceIfProcedureWasNotSubmitted
```
public void unregisterNonceIfProcedureWasNotSubmitted(NonceKey nonceKey)
```
    Remove the NonceKey if the procedure was not submitted to the executor.
    
    Parameters:
    nonceKey - A unique identifier for this operation from the client or process.
  - setFailureResultForNonce
```
public void setFailureResultForNonce(NonceKey nonceKey,
                            String procName,
                            User procOwner,
                            IOException exception)
```
    If the failure failed before submitting it, we may want to give back the same error to the requests with the same nonceKey.
    
    Parameters:
    nonceKey - A unique identifier for this operation from the client or process
    procName - name of the procedure, used to inform the user
    procOwner - name of the owner of the procedure, used to inform the user
    exception - the failure to report to the user
  - submitProcedure
```
public long submitProcedure(Procedure proc)
```
    >>>>>>> ce33cf2... HBASE-17149 Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor multiple times (Matteo Bertozzi) Add a new root-procedure to the executor.
    
    Parameters:
    proc - the new procedure to execute.
    
    Returns:
    the procedure id, that can be used to monitor the operation
  - submitProcedure
```
public long submitProcedure(Procedure proc,
                   NonceKey nonceKey)
```
    Add a new root-procedure to the executor.
    
    Parameters:
    proc - the new procedure to execute.
    nonceKey - the registered unique identifier for this operation from the client or process.
    
    Returns:
    the procedure id, that can be used to monitor the operation
  - getResult
```
public ProcedureInfo getResult(long procId)
```
  - isFinished
```
public boolean isFinished(long procId)
```
    Return true if the procedure is finished. The state may be "completed successfully" or "failed and rolledback". Use getResult() to check the state or get the result data.
    
    Parameters:
    procId - the ID of the procedure to check
    
    Returns:
    true if the procedure execution is finished, otherwise false.
  - isStarted
```
public boolean isStarted(long procId)
```
    Return true if the procedure is started.
    
    Parameters:
    procId - the ID of the procedure to check
    
    Returns:
    true if the procedure execution is started, otherwise false.
  - removeResult
```
public void removeResult(long procId)
```
    Mark the specified completed procedure, as ready to remove.
    
    Parameters:
    procId - the ID of the procedure to remove
  - abort
```
public boolean abort(long procId)
```
    Send an abort notification the specified procedure. Depending on the procedure implementation the abort can be considered or ignored.
    
    Parameters:
    procId - the procedure to abort
    
    Returns:
    true if the procedure exist and has received the abort, otherwise false.
  - abort
```
public boolean abort(long procId,
            boolean mayInterruptIfRunning)
```
    Send an abort notification the specified procedure. Depending on the procedure implementation the abort can be considered or ignored.
    
    Parameters:
    procId - the procedure to abort
    mayInterruptIfRunning - if the proc completed at least one step, should it be aborted?
    
    Returns:
    true if the procedure exist and has received the abort, otherwise false.
  - isProcedureOwner
```
public boolean isProcedureOwner(long procId,
                       User user)
```
    Check if the user is this procedure's owner
    
    Parameters:
    procId - the target procedure
    user - the user
    
    Returns:
    true if the user is the owner of the procedure, false otherwise or the owner is unknown.
  - getResults
```
public Map<Long,ProcedureInfo> getResults()
```
  - getProcedure
```
public Procedure getProcedure(long procId)
```
  - getRunnableSet
```
protected ProcedureRunnableSet getRunnableSet()
```
  - execLoop
```
private void execLoop()
```
    Execution loop (N threads) while the executor is in a running state, fetch a procedure from the runnables queue and start the execution.
  - execLoop
```
private void execLoop(Procedure proc)
```
  - timeoutLoop
```
private void timeoutLoop()
```
  - executeRollback
```
private boolean executeRollback(long rootProcId,
                      RootProcedureState procStack)
```
    Execute the rollback of the full procedure stack. Once the procedure is rolledback, the root-procedure will be visible as finished to user, and the result will be the fatal exception.
  - executeRollback
```
private boolean executeRollback(Procedure proc)
```
    Execute the rollback of the procedure step. It updates the store with the new state (stack index) or will remove completly the procedure in case it is a child.
  - execProcedure
```
private void execProcedure(RootProcedureState procStack,
                 Procedure procedure)
```
    Executes the specified procedure - calls the doExecute() of the procedure - if the procedure execution didn't fail (e.g. invalid user input) - ...and returned subprocedures - the subprocedures are initialized. - the subprocedures are added to the store - the subprocedures are added to the runnable queue - the procedure is now in a WAITING state, waiting for the subprocedures to complete - ...if there are no subprocedure - the procedure completed successfully - if there is a parent (WAITING) - the parent state will be set to RUNNABLE - in case of failure - the store is updated with the new state - the executor (caller of this method) will start the rollback of the procedure
  - handleInterruptedException
```
private void handleInterruptedException(Procedure proc,
                              InterruptedException e)
```
  - sendProcedureLoadedNotification
```
private void sendProcedureLoadedNotification(long procId)
```
  - sendProcedureAddedNotification
```
private void sendProcedureAddedNotification(long procId)
```
  - sendProcedureFinishedNotification
```
private void sendProcedureFinishedNotification(long procId)
```
  - nextProcId
```
private long nextProcId()
```
  - getRootProcedureId
```
private Long getRootProcedureId(Procedure proc)
```
  - procedureFinished
```
private void procedureFinished(Procedure proc)
```
  - getResultOrProcedure
```
public Pair<ProcedureInfo,Procedure> getResultOrProcedure(long procId)
```

Class ProcedureExecutor<TEnvironment>

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

testing

completed

rollbackStack

procedures

nonceKeysToProcIdsMap

waitingTimeout

runnables

submitLock

lastProcId

listeners

activeExecutorCount

running

environment

store

conf

threads

Constructor Detail

ProcedureExecutor

ProcedureExecutor

Method Detail

load

loadProcedures

start

stop

join

isRunning

getNumThreads

getActiveExecutorCount

getEnvironment

getStore

registerListener

unregisterListener

listProcedures

createNonceKey

registerNonce

unregisterNonceIfProcedureWasNotSubmitted

setFailureResultForNonce

submitProcedure

submitProcedure

getResult

isFinished

isStarted

removeResult

abort

abort

isProcedureOwner

getResults

getProcedure

getRunnableSet

execLoop

execLoop

timeoutLoop

executeRollback

executeRollback

execProcedure

handleInterruptedException

sendProcedureLoadedNotification

sendProcedureAddedNotification

sendProcedureFinishedNotification

nextProcId

getRootProcedureId

procedureFinished

getResultOrProcedure