ProcedureStoreTracker (Apache HBase 2.0.6 API)

java.lang.Object
- org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker

```
@InterfaceAudience.Private
public class ProcedureStoreTracker
extends Object
```
Keeps track of live procedures. It can be used by the ProcedureStore to identify which procedures are already deleted/completed to avoid the deserialization step on restart

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class ProcedureStoreTracker.DeleteState

Nested Classes
Modifier and Type	Class and Description
`static class`	`ProcedureStoreTracker.DeleteState`

Field Summary

Fields
Modifier and Type	Field and Description
`private boolean`	`keepDeletes` If true, do not remove bits corresponding to deleted procedures.
`private static org.slf4j.Logger`	`LOG`
`private TreeMap<Long,BitSetNode>`	`map`
`private long`	`maxModifiedProcId`
`private long`	`minModifiedProcId`
`(package private) boolean`	`partial` If true, it means tracker has incomplete information about the active/deleted procedures.

Constructor Summary

Constructors
Constructor and Description

ProcedureStoreTracker()

Constructors
Constructor and Description
`ProcedureStoreTracker()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`private BitSetNode`	`delete(BitSetNode node, long procId)`
`void`	`delete(long procId)`
`void`	`delete(long[] procIds)`
`void`	`dump()`
`long`	`getActiveMinProcId()`
`long[]`	`getAllActiveProcIds()` Will be used when there are too many proc wal files.
`long`	`getModifiedMaxProcId()`
`long`	`getModifiedMinProcId()`
`private BitSetNode`	`getOrCreateNode(long procId)`
`private BitSetNode`	`growNode(BitSetNode node, long procId)` Grows `node` to contain `procId` and updates the map.
`private BitSetNode`	`insert(BitSetNode node, long procId)`
`void`	`insert(long procId)`
`void`	`insert(long[] procIds)`
`void`	`insert(long procId, long[] subProcIds)`
`boolean`	`isAllModified()`
`ProcedureStoreTracker.DeleteState`	`isDeleted(long procId)` If `partial` is false, returns state from the bitmap.
`boolean`	`isEmpty()`
`boolean`	`isModified(long procId)`
`boolean`	`isPartial()`
`private BitSetNode`	`lookupClosestNode(BitSetNode node, long procId)` lookup the node containing the specified procId.
`private BitSetNode`	`mergeNodes(BitSetNode leftNode, BitSetNode rightNode)` Merges `leftNode` & `rightNode` and updates the map.
`void`	`reset()`
`void`	`resetModified()` Clears the list of updated procedure ids.
`void`	`resetTo(ProcedureStoreTracker tracker)` Resets internal state to same as given `tracker`.
`void`	`resetTo(ProcedureStoreTracker tracker, boolean resetDelete)` Resets internal state to same as given `tracker`, and change the deleted flag according to the modified flag if `resetDelete` is true.
`void`	`resetToProto(org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureStoreTracker trackerProtoBuf)`
`void`	`setDeleted(long procId, boolean isDeleted)` This method is used when restarting where we need to rebuild the ProcedureStoreTracker.
`void`	`setDeletedIfDeletedByThem(ProcedureStoreTracker tracker)` For the global tracker, we will use this method to build the holdingCleanupTracker, as the modified flags will be cleared after rolling so we only need to test the deleted flags.
`void`	`setDeletedIfModified(long... procId)` Set the given bit for the procId to delete if it was modified before.
`void`	`setDeletedIfModifiedInBoth(ProcedureStoreTracker tracker)` Similar with `setDeletedIfModified(long...)`, but here the `procId` are given by the `tracker`.
`private void`	`setDeleteIf(ProcedureStoreTracker tracker, BiFunction<BitSetNode,Long,Boolean> func)`
`void`	`setKeepDeletes(boolean keepDeletes)`
`void`	`setMinMaxModifiedProcIds(long min, long max)` Will be called when restarting where we need to rebuild the ProcedureStoreTracker.
`void`	`setPartialFlag(boolean isPartial)`
`org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureStoreTracker`	`toProto()` Builds org.apache.hadoop.hbase.protobuf.generated.ProcedureProtos.ProcedureStoreTracker protocol buffer from current state.
`private void`	`trackProcIds(long procId)`
`private BitSetNode`	`update(BitSetNode node, long procId)`
`void`	`update(long procId)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.slf4j.Logger LOG
```
  - map
```
private final TreeMap<Long,BitSetNode> map
```
  - keepDeletes
```
private boolean keepDeletes
```
    If true, do not remove bits corresponding to deleted procedures. Note that this can result in huge bitmaps overtime. Currently, it's set to true only when building tracker state from logs during recovery. During recovery, if we are sure that a procedure has been deleted, reading its old update entries can be skipped.
  - partial
```
boolean partial
```
    If true, it means tracker has incomplete information about the active/deleted procedures. It's set to true only when recovering from old logs. See isDeleted(long) docs to understand it's real use.
  - minModifiedProcId
```
private long minModifiedProcId
```
  - maxModifiedProcId
```
private long maxModifiedProcId
```
- Constructor Detail
  - ProcedureStoreTracker
```
public ProcedureStoreTracker()
```
- Method Detail
  - resetToProto
```
public void resetToProto(org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureStoreTracker trackerProtoBuf)
```
  - resetTo
```
public void resetTo(ProcedureStoreTracker tracker)
```
    Resets internal state to same as given tracker. Does deep copy of the bitmap.
  - resetTo
```
public void resetTo(ProcedureStoreTracker tracker,
                    boolean resetDelete)
```
    Resets internal state to same as given tracker, and change the deleted flag according to the modified flag if resetDelete is true. Does deep copy of the bitmap.
    The resetDelete will be set to true when building cleanup tracker, please see the comments in BitSetNode.BitSetNode(BitSetNode, boolean) to learn how we change the deleted flag if resetDelete is true.
  - insert
```
public void insert(long procId)
```
  - insert
```
public void insert(long[] procIds)
```
  - insert
```
public void insert(long procId,
                   long[] subProcIds)
```
  - insert
```
private BitSetNode insert(BitSetNode node,
                          long procId)
```
  - update
```
public void update(long procId)
```
  - update
```
private BitSetNode update(BitSetNode node,
                          long procId)
```
  - delete
```
public void delete(long procId)
```
  - delete
```
public void delete(long[] procIds)
```
  - delete
```
private BitSetNode delete(BitSetNode node,
                          long procId)
```
  - setMinMaxModifiedProcIds
```
public void setMinMaxModifiedProcIds(long min,
                                     long max)
```
    Will be called when restarting where we need to rebuild the ProcedureStoreTracker.
  - setDeleted
```
public void setDeleted(long procId,
                       boolean isDeleted)
```
    This method is used when restarting where we need to rebuild the ProcedureStoreTracker. The delete(long) method above assume that the BitSetNode exists, but when restart this is not true, as we will read the wal files in reverse order so a delete may come first.
  - setDeletedIfModified
```
public void setDeletedIfModified(long... procId)
```
    Set the given bit for the procId to delete if it was modified before.
    This method is used to test whether a procedure wal file can be safely deleted, as if all the procedures in the given procedure wal file has been modified in the new procedure wal files, then we can delete it.
  - setDeleteIf
```
private void setDeleteIf(ProcedureStoreTracker tracker,
                         BiFunction<BitSetNode,Long,Boolean> func)
```
  - setDeletedIfDeletedByThem
```
public void setDeletedIfDeletedByThem(ProcedureStoreTracker tracker)
```
    For the global tracker, we will use this method to build the holdingCleanupTracker, as the modified flags will be cleared after rolling so we only need to test the deleted flags.
    
    See Also:
    
    setDeletedIfModifiedInBoth(ProcedureStoreTracker)
  - setDeletedIfModifiedInBoth
```
public void setDeletedIfModifiedInBoth(ProcedureStoreTracker tracker)
```
    Similar with setDeletedIfModified(long...), but here the procId are given by the tracker. If a procedure is modified by us, and also by the given tracker, then we mark it as deleted.
    
    See Also:
    
    setDeletedIfModified(long...)
  - lookupClosestNode
```
private BitSetNode lookupClosestNode(BitSetNode node,
                                     long procId)
```
    lookup the node containing the specified procId.
    
    Parameters:
    
    node - cached node to check before doing a lookup
    
    procId - the procId to lookup
    
    Returns:
    
    the node that may contains the procId or null
  - trackProcIds
```
private void trackProcIds(long procId)
```
  - getModifiedMinProcId
```
public long getModifiedMinProcId()
```
  - getModifiedMaxProcId
```
public long getModifiedMaxProcId()
```
  - reset
```
public void reset()
```
  - isModified
```
public boolean isModified(long procId)
```
  - isDeleted
```
public ProcedureStoreTracker.DeleteState isDeleted(long procId)
```
    If partial is false, returns state from the bitmap. If no state is found for procId, returns YES. If partial is true, tracker doesn't have complete view of system state, so it returns MAYBE if there is no update for the procedure or if it doesn't have a state in bitmap. Otherwise, returns state from the bitmap.
  - getActiveMinProcId
```
public long getActiveMinProcId()
```
  - setKeepDeletes
```
public void setKeepDeletes(boolean keepDeletes)
```
  - isPartial
```
public boolean isPartial()
```
  - setPartialFlag
```
public void setPartialFlag(boolean isPartial)
```
  - isEmpty
```
public boolean isEmpty()
```
    Returns:
    
    true, if no procedure is active, else false.
  - isAllModified
```
public boolean isAllModified()
```
    Returns:
    
    true if all procedure was modified or deleted since last call to resetModified().
  - getAllActiveProcIds
```
public long[] getAllActiveProcIds()
```
    Will be used when there are too many proc wal files. We will rewrite the states of the active procedures in the oldest proc wal file so that we can delete it.
    
    Returns:
    
    all the active procedure ids in this tracker.
  - resetModified
```
public void resetModified()
```
    Clears the list of updated procedure ids. This doesn't affect global list of active procedure ids.
  - getOrCreateNode
```
private BitSetNode getOrCreateNode(long procId)
```
  - growNode
```
private BitSetNode growNode(BitSetNode node,
                            long procId)
```
    Grows node to contain procId and updates the map.
    
    Returns:
    
    BitSetNode instance which contains procId.
  - mergeNodes
```
private BitSetNode mergeNodes(BitSetNode leftNode,
                              BitSetNode rightNode)
```
    Merges leftNode & rightNode and updates the map.
  - dump
```
public void dump()
```
  - toProto
```
public org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos.ProcedureStoreTracker toProto()
                                                                                                throws IOException
```
    Builds org.apache.hadoop.hbase.protobuf.generated.ProcedureProtos.ProcedureStoreTracker protocol buffer from current state.
    
    Throws:
    
    IOException

Class ProcedureStoreTracker

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

map

keepDeletes

partial

minModifiedProcId

maxModifiedProcId

Constructor Detail

ProcedureStoreTracker

Method Detail

resetToProto

resetTo

resetTo

insert

insert

insert

insert

update

update

delete

delete

delete

setMinMaxModifiedProcIds

setDeleted

setDeletedIfModified

setDeleteIf

setDeletedIfDeletedByThem

setDeletedIfModifiedInBoth

lookupClosestNode

trackProcIds

getModifiedMinProcId

getModifiedMaxProcId

reset

isModified

isDeleted

getActiveMinProcId

setKeepDeletes

isPartial

setPartialFlag

isEmpty

isAllModified

getAllActiveProcIds

resetModified

getOrCreateNode

growNode

mergeNodes

dump

toProto